WO2020228768A1

WO2020228768A1 - 3d intelligent education monitoring method and system, and storage medium

Info

Publication number: WO2020228768A1
Application number: PCT/CN2020/090219
Authority: WO
Inventors: 李新福
Original assignee: 广东康云科技有限公司
Priority date: 2019-05-14
Filing date: 2020-05-14
Publication date: 2020-11-19
Also published as: CN110312121A

Abstract

Disclosed in the present invention are a 3D intelligent education monitoring method and system, and a storage medium. The method comprises the following steps: acquiring real-time video information in an educational institution; generating and storing 3D live-action data by combining the real-time video information and a pre-established on-site 3D model; and after input control information is obtained, rendering and playing a corresponding 3D monitoring picture by combining the control information and the 3D live-action data. According to the present invention, real-time video information is acquired, 3D live-action data of the education site is generated by combining the real-time video information and a preset on-site 3D model, and a 3D live-action monitoring picture of the education site is generated in combination with control information; 360-degree immersive roaming experience without dead angles is provided for parents by means of the 3D live-action monitoring picture, and the parents can watch the situation of the education site personally on the scene and can roam in the 3D live-action monitoring picture; the method can be widely applied to the field of intelligent education.

Description

A 3D intelligent education monitoring method, system and storage medium

Technical field

The invention relates to the field of intelligent education, in particular to a 3D intelligent education monitoring method, system and storage medium.

Background technique

In our country, with the improvement of the education system and the development of higher education institutions, more and more students are entering higher education institutions. Currently, many parents send their children to schools and other educational institutions for training. With the development of technology, many educational institutions can provide parents with real-time video surveillance pictures through CCTV (closed-circuit television monitoring system) and other video surveillance devices and supporting APPs, so that parents can view their children’s training in educational institutions in real time. Knowing the teacher's training courses makes parents more assured and satisfied.

However, most of the video surveillance screens provided by educational institutions are 2D video surveillance screens. The real-time video stream of the education site cannot be integrated with the 3D model of the education site to generate a 3D real-life monitoring screen of the education site, which cannot provide parents with 360 degrees. The immersive roaming experience with no dead ends needs to be further improved and improved.

Summary of the invention

In order to solve the above technical problems, the purpose of the present invention is to provide a 3D intelligent education monitoring method, system and storage medium to provide a 360-degree immersive roaming experience without dead ends.

The first technical solution adopted by the present invention is:

A 3D intelligent education monitoring method includes the following steps:

Collect real-time video information in educational institutions;

Combine real-time video information and pre-established on-site 3D model to generate and store 3D real scene data;

After obtaining the input control information, the corresponding 3D monitoring screen is rendered and played in combination with the control information and the 3D real scene data.

Furthermore, it also includes a step of establishing a 3D model on site, and the step of establishing a 3D model on site is specifically:

Scan 3D data of educational institutions;

Generate and store 3D models of the scene based on the scanned 3D data.

Further, the step of combining real-time video information and a pre-established on-site 3D model to generate and store 3D real scene data is specifically:

The real-time video information is superimposed on the preset scene 3D model, and after fusion processing, 3D real scene data is generated and stored.

Further, it also includes the following steps:

Generate and store corresponding links based on 3D real scene data.

Further, the control information includes switching information and regulation information, the switching information includes scene switching information and angle switching information, and the switching information includes scene switching information and angle switching information. After the input control information is acquired, the control information is combined The step of rendering and playing the corresponding 3D monitoring screen of information and 3D real scene data specifically includes the following steps:

After acquiring the input scene switching information, combine the scene switching information and the 3D real scene data to acquire and render the 3D monitoring screen of the corresponding scene;

After obtaining the input angle switching information, combine the angle switching information and the 3D real scene data to obtain and render 3D monitoring images of different angles in the same scene.

Further, it also includes the following steps:

After obtaining the input video playback information, obtain the corresponding 3D real scene data according to the video playback information, and render and play the corresponding 3D monitoring picture according to the 3D real scene data.

The second technical solution adopted by the present invention is:

A 3D intelligent education monitoring system, including:

Video acquisition module, used to collect real-time video information in educational institutions;

Data processing module, used to combine real-time video information and pre-established on-site 3D models to generate and store 3D real scene data;

The video playback module is used to obtain the input control information, and combine the control information and 3D real scene data to render and play the corresponding 3D monitoring picture.

Further, it also includes a building site 3D model module, which includes a scanning unit and a modeling unit;

The scanning unit is used to scan three-dimensional data of educational institutions;

The modeling unit is used to generate and store a site 3D model according to the scanned three-dimensional data.

Further, the data processing module is specifically configured to superimpose real-time video information on a preset site 3D model, and after performing fusion processing, generate and store 3D real scene data.

Further, the video playback module includes at least one of a mobile terminal, a tablet computer, a PC computer, an air screen, an LED display, an LCD display, an OLED display, and a dot matrix display.

Further, the video acquisition module adopts a CCTV video stream acquisition module.

The third technical solution adopted by the present invention is:

A storage medium in which instructions executable by a processor are stored, and the instructions executable by the processor are used to execute the above-mentioned method when executed by the processor.

The beneficial effect of the present invention is that the present invention generates 3D real-scene data of the education site by collecting real-time video information and combining the real-time video information with the preset on-site 3D model, and then combines the control information to generate the 3D real-scene monitoring picture of the education site , Provide parents with a 360-degree immersive roaming experience without blind spots through the 3D real-life monitoring screen, allowing parents to watch the education scene immersively and roam in the 3D real-life monitoring screen.

Description of the drawings

Figure 1 is a flow chart of the steps of a 3D intelligent education monitoring method of the present invention;

Figure 2 is a structural block diagram of a 3D intelligent education monitoring system of the present invention.

Detailed ways

In the following, the concept, specific structure and technical effects of the present invention will be clearly and completely described in conjunction with the embodiments and the drawings, so as to fully understand the objectives, solutions and effects of the present invention.

It should be noted that, unless otherwise specified, when a feature is called "fixed" or "connected" to another feature, it can be directly fixed and connected to another feature, or indirectly fixed or connected to another feature. One feature. In addition, the top, bottom, left, right and other descriptions used in the present disclosure are only relative to the mutual positional relationship of the components of the present disclosure in the drawings. The singular forms of "a", "said" and "the" used in the present disclosure are also intended to include plural forms, unless the context clearly indicates other meanings. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art. The terms used in this specification are only for describing specific embodiments, not for limiting the present invention. The term "and/or" as used herein includes any combination of one or more related listed items.

It should be understood that, although the terms first, second, third, etc. may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("such as", "such as", etc.) provided herein is only intended to better illustrate the embodiments of the present invention, and unless otherwise required, will not impose limitations on the scope of the present invention .

Example one

1, this embodiment provides a 3D intelligent education monitoring method, including the following steps:

S1, establish a 3D model of the scene;

S2. Collect real-time video information in educational institutions;

S3. Combine real-time video information and pre-established on-site 3D model to generate and store 3D real scene data;

S4. After obtaining the input control information, the corresponding 3D monitoring picture is rendered and played in combination with the control information and the 3D real scene data.

In the method of this embodiment, 3D modeling is performed on the training and educational institution in advance and stored in a database in the background. For example, taking a school as an example, scanning modeling is performed on the school’s teaching building, classrooms, corridors, etc. Get a 3D model of the scene. After the on-site 3D model is built, real-time video information in the school is collected in real time. The real-time video information can be a video obtained by a high-definition camera installed on the site, or a video played on site, that is, the teacher uses multimedia such as projection equipment The video played by the device. The collected real-time video information is merged with the pre-established 3D model of the scene to generate 3D real scene data, and the 3D real scene data is stored in a preset storage space. Parents can directly enter or open the URL link to access the 3D real scene data through the browser of the smart terminal, and view the real-time 3D video real scene monitoring screen in the education classroom. Parents can enter the switching information through the smart terminal to roam in the 3D video real-life monitoring screen, such as switching between different scenes or angles of the screen. For example, when the educational institution is a school, they can switch between different classrooms to find the classroom where their children are located, and pass Switch the angle of the screen so that you can see your child more clearly. Because the 3D video real-time monitoring screen is a three-dimensional visual video screen, parents have the immersive feeling of viewing it immersively, and parents and other users can enter control information to switch the screen for 360-degree roaming without dead ends, which can quickly find The location of your child; in addition, the collected video data is real-time, so parents can view their children's situation in real time and view the teacher's lectures, which meets the requirements of parents and users to view their children in real time and immersively.

Wherein, the step S1 specifically includes steps S11 to S12:

S11. Scan the three-dimensional data of educational institutions;

S12. Generate and store a 3D model of the scene according to the scanned 3D data.

Scan the three-dimensional data of the environment of the educational institution in advance. The environment of the scene includes some fixed-position equipment, such as the exterior of the teaching building, the classrooms in the teaching building, the stairs in the teaching building and the corridors outside the classroom, the sports field, the school Canteens, canteens and other corners. After processing the three-dimensional data obtained by scanning, a 3D model of the scene is generated. Specifically, the exterior of the teaching building can be scanned by aerial photography, aerial scanning equipment (handheld scanning equipment or other automatic scanning equipment), and the interior of the teaching building can be scanned by handheld scanning equipment (such as a camera with a support frame) or other automatic scanning equipment ( Such as automatic scanning robots) perform on-site scanning and obtain corresponding 3D data such as 2D pictures and depth information. After the obtained 2D pictures and 3D data such as depth information are processed in the steps of model repair, editing, cropping, surface reduction, mode reduction, compression, material processing, texture processing, lighting processing and compression rendering, the on-site 3D model is obtained. Store the obtained on-site 3D model in a preset storage space, and when the on-site 3D model needs to be called, it can be called directly.

The step S3 is specifically: superimposing the real-time video information on the preset 3D model of the scene, and performing fusion processing, and then generating and storing 3D real scene data.

Parent users can view the scene images in the classroom by inputting switching information, and can also view the playback images of multimedia devices such as classroom projection equipment. When the real-time video information is a piece of video played on-site, the video is directly integrated with the on-site 3D model to generate 3D real scene data. When the real-time video information is a video obtained by a high-definition camera installed on site, the high-definition camera may be one or multiple cameras. When there are multiple high-definition cameras, it can shoot from multiple angles.

The 3D real scene data is generated by superimposing and fusing the video information with the on-site 3D model, the 3D real scene data is coherent, and the parents can roam by inputting switching information. For example, when students run out of the classroom outside the classroom hallway, parents can realize the video to track the student's movement by switching the screen, instead of simply switching the camera to view.

The control information includes switching information and regulation information, the switching information includes scene switching information and angle switching information, and the step S4 specifically includes steps S41 to S42:

S41. After acquiring the input scene switching information, combine the scene switching information and the 3D real scene data to acquire and render a 3D monitoring screen corresponding to the scene;

S42. After obtaining the input angle switching information, combine the angle switching information and the 3D real scene data to obtain and render 3D monitoring pictures of different angles in the same scene.

The control information includes switching information and adjustment information, the switching information is used to switch the scene and angle of the screen, and the adjustment information is used to adjust the brightness, clarity, or size of the screen, etc. When parents monitor in real time through smart terminals, they can quickly switch to the classroom or training room where their children are located through scene switching information. After switching to the corresponding classroom, they can find the location of their children by entering the angle switching information . When a student is resting between classes, after the position moves, parents can track by switching the screen. For example, when a student runs from the second floor to the third floor, the parent can track to the third floor by 3D roaming. Since the viewing screen is a 3D screen, it can give parents an immersive feeling and greatly improve the viewing experience of parents.

Further as a preferred embodiment, it also includes the following steps:

Generate and store corresponding links based on 3D real scene data.

After the link is generated, the parent user can directly view the real-time monitoring scene through the link and forward the link to other users, which greatly facilitates the viewing operation of the parent user.

Further as a preferred embodiment, it also includes the following steps:

Parents can view the previous 3D real scene data by inputting the video playback information. For example, after parents discover that their child is injured in an educational institution, they can play back and view the corresponding 3D video to avoid misunderstandings, improve the safety of the child, and make parents more rest assured.

Example two

2, this embodiment provides a 3D intelligent education monitoring system, including:

In the system of this embodiment, 3D modeling is performed on training and educational institutions in advance and stored in a database in the background. For example, taking a school as an example, scanning modeling is performed on the school’s teaching buildings, classrooms, corridors, etc. Get a 3D model of the scene. After the on-site 3D model is built, real-time video information in the school is collected in real time. The real-time video information can be a video obtained by a high-definition camera installed on the site, or a video played on site, that is, the teacher uses multimedia such as projection equipment The video played by the device. The collected real-time video information is merged with the pre-established 3D model of the scene to generate 3D real scene data, and the 3D real scene data is stored in a preset storage space. Parents can directly enter or open the URL link to access the 3D real scene data through the browser of the smart terminal, and view the real-time 3D video real scene monitoring screen in the education classroom. Parents can enter the switching information through the smart terminal to roam in the 3D video real-life monitoring screen, such as switching between different scenes or angles of the screen. For example, when the educational institution is a school, they can switch between different classrooms to find the classroom where their children are located, and pass Switch the angle of the screen so that you can see your child more clearly. Because the 3D video real-time monitoring screen is a three-dimensional visual video screen, parents have the immersive feeling of viewing it immersively, and parents and other users can enter control information to switch the screen for 360-degree roaming without dead ends, which can quickly find The location of your child; in addition, the collected video data is real-time, so parents can view their children's situation in real time and view the teacher's lectures, which meets the requirements of parents and users to view their children in real time and immersively.

Further as a preferred embodiment, as a preferred real-time mode, it further includes a building site 3D model module, where the building site 3D model module includes a scanning unit and a modeling unit;

Further as a preferred embodiment, the data processing module is specifically configured to superimpose real-time video information on a preset on-site 3D model, and after performing fusion processing, generate and store 3D real scene data.

Parent users can view the scene images in the classroom by inputting switching information, and can also view the playback images of multimedia devices such as classroom projection equipment. When the real-time video information is a piece of video played on-site, the video is directly integrated with the on-site 3D model to generate 3D real scene data. When the real-time video information is a video obtained by a high-definition camera installed on the site, the high-definition camera may be one or multiple cameras. When there are multiple high-definition cameras, shooting can be performed from multiple angles.

As a further preferred embodiment, the switching information includes scene switching information and angle switching information, and the video playback module includes a switching scene playback unit and a switching angle playback unit;

The switching scene playback unit is configured to obtain and render a 3D monitoring picture of the corresponding scene in combination with the scene switching information and 3D real scene data after acquiring the input scene switching information;

The switching angle playback unit is used to obtain and render 3D monitoring images of different angles in the same scene by combining the angle switching information and the 3D real scene data after obtaining the input angle switching information.

When parents monitor in real time on the web through a browser, they can quickly switch to the classroom or training room where their children are located through the scene switching information. When switching to the corresponding classroom, they can find their children by entering the angle switching information Location. When a student is resting between classes, after the position moves, parents can track by switching the screen. For example, when a student runs from the second floor to the third floor, the parent can track to the third floor by 3D roaming. Since the viewing screen is a 3D screen, it can give parents an immersive feeling and greatly improve the viewing experience of parents.

Further as a preferred embodiment, the video playback module includes at least one of a mobile terminal, a tablet computer, a PC computer, an air screen, an LED display, an LCD display, an OLED display, and a dot matrix display.

Parent users can check their children's education through the smart terminal devices around them anytime and anywhere, which greatly facilitates parents' checking operations.

Further as a preferred embodiment, the video acquisition module adopts a CCTV video stream acquisition module.

Example three

A storage medium storing instructions executable by a processor, and the instructions executable by the processor are used to execute the method described in the embodiments when being executed by the processor.

The storage medium of this embodiment can execute a 3D intelligent education monitoring method provided in the method embodiment 1 of the present invention, and can execute any combination of implementation steps of the method embodiments, and has corresponding functions and beneficial effects of the method.

It should be realized that the embodiments of the present invention can be realized or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method can be implemented in a computer program using standard programming techniques-including a non-transitory computer readable storage medium configured with a computer program, where the storage medium so configured allows the computer to operate in a specific and predefined manner-according to the specific The methods and drawings described in the examples. Each program can be implemented in a high-level process or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

In addition, the operations of the processes described herein may be performed in any suitable order, unless otherwise indicated herein or otherwise clearly contradictory to the context. The processes (or variants and/or combinations thereof) described herein can be executed under the control of one or more computer systems configured with executable instructions, and can be used as code (for example, , Executable instructions, one or more computer programs, or one or more applications), implemented by hardware or a combination thereof. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computing platform, including but not limited to a personal computer, a mini computer, a main frame, a workstation, a network or a distributed computing environment, a separate or integrated computer Platform, or communication with charged particle tools or other imaging devices, etc. Aspects of the present invention can be implemented by machine-readable codes stored on non-transitory storage media or devices, whether removable or integrated into a computing platform, such as hard disks, optical reading and/or writing storage media, RAM, ROM, etc., so that they can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. In addition, the machine-readable code, or part thereof, can be transmitted through a wired or wireless network. When such a medium includes instructions or programs that implement the steps described above in combination with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programming according to the methods and techniques of the present invention, the present invention also includes the computer itself.

A computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data that is stored in non-volatile memory. The output information can also be applied to one or more output devices such as displays. In a preferred embodiment of the present invention, the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.

The above are only preferred embodiments of the present invention. The present invention is not limited to the above-mentioned embodiments, as long as it achieves the technical effects of the present invention by the same means, everything is done within the spirit and principle of the present invention. Any modifications, equivalent replacements, improvements, etc., shall be included in the protection scope of the present invention. Within the protection scope of the present invention, its technical solutions and/or implementations can have various modifications and changes.

Claims

A 3D intelligent education monitoring method is characterized in that it includes the following steps:

Collect real-time video information in educational institutions;

Combine real-time video information and pre-established on-site 3D model to generate and store 3D real scene data;

After obtaining the input control information, the corresponding 3D monitoring screen is rendered and played in combination with the control information and the 3D real scene data.
The 3D intelligent education monitoring method according to claim 1, characterized in that it further comprises a step of establishing an on-site 3D model, and the step of establishing an on-site 3D model is specifically:

Scan 3D data of educational institutions;

Generate and store 3D models of the scene based on the scanned 3D data.
The 3D intelligent education monitoring method according to claim 2, wherein the step of combining real-time video information and a pre-established on-site 3D model to generate and store 3D real scene data is specifically:

The real-time video information is superimposed on the preset scene 3D model, and after fusion processing, 3D real scene data is generated and stored.
A 3D intelligent education monitoring method according to claim 1, characterized in that it further comprises the following steps:

Generate and store corresponding links based on 3D real scene data.
A 3D intelligent education monitoring method according to claim 1, characterized in that it further comprises the following steps:

After obtaining the input video playback information, obtain the corresponding 3D real scene data according to the video playback information, and render and play the corresponding 3D monitoring picture according to the 3D real scene data.
A 3D intelligent education monitoring system is characterized in that it includes:

Video collection module, used to collect real-time video information in educational institutions;

Data processing module, used to combine real-time video information and pre-established on-site 3D models to generate and store 3D real scene data;

The video playback module is used to obtain the input control information, and combine the control information and 3D real scene data to render and play the corresponding 3D monitoring picture.
The 3D intelligent education monitoring system according to claim 6, characterized in that it further comprises a building site 3D model module, said building site 3D model module comprising a scanning unit and a modeling unit;

The scanning unit is used to scan three-dimensional data of educational institutions;

The modeling unit is used to generate and store a site 3D model according to the scanned three-dimensional data.
The 3D intelligent education monitoring system according to claim 7, wherein the data processing module is specifically used to superimpose real-time video information on a preset site 3D model, and after performing fusion processing, generate and store 3D Real-world data.
The 3D intelligent education monitoring system according to claim 6, wherein the video playback module includes a mobile terminal, a tablet computer, a PC computer, an air screen, an LED display, an LCD display, and an OLED display And at least one of dot matrix display screens.
A storage medium storing instructions executable by a processor, wherein the instructions executable by the processor are used to execute the method according to any one of claims 1-5 when executed by the processor.