design idea 

In response to project requirements, we will focus on building various classrooms and central platforms. We have made corresponding deployments for the application needs of school classrooms. Through high-definition recording and broadcasting hosts, achieve access management, image synthesis, and remote interaction of video and audio information; By selecting high-capacity and high-performance central management platform equipment, it meets the requirements of access, device management, audio and video stream distribution and storage for various network frontends; Through the platform management system, it meets the online communication, learning, and daily management of different user groups, and relies on the established IP network to achieve the transmission, management, and storage of audio and video resources.

System architecture diagram

Diagram: System application diagram

As shown in the above figure, this system is a fully digital, networked, and intelligent comprehensive intelligent classroom system built on an IP network.

Classroom design

The recording classroom consists of multiple parts, including a smart recording system, high-definition camera, omnidirectional pickup microphone, guidance console, guidance control keyboard, decoration, etc. Among them, the smart recording system is completed by a high-definition recording host; Camera section: composed of teacher's camera (including blackboard close-up tracking) and student's camera; Tracking and positioning are completed by two positioning cameras for image acquisition and analysis; The sound collection is completed by two ceiling microphones; The sound amplifier is completed by two active electronic conference speakers. The management control part consists of a guidance control platform and a guidance control keyboard.

Recording classroom rendering

Function Introduction

The entire system is built on the basis of classrooms, adopting the construction concept of distributed deployment and centralized management, fully meeting the daily recording and broadcasting applications in a normalized teaching environment. At the same time, the system supports multi role participation and allocation, utilizing a unified front-end device to achieve rich business applications such as campus monitoring, electronic examination rooms, and teaching equipment management. The entire system includes five major system functions: comprehensive recording of teaching, high-definition fully automatic recording, daily security and electronic inspection, teaching equipment control and management, and teaching resource platform management.

Image recognition and tracking system

The HAIMINGWEI image recognition and tracking system is an intelligent automatic tracking system. It adopts Ncast's innovative image analysis and processing technology, achieving multi region and multi strategy automatic tracking through different strategy definitions.

The HAIMINGWEI image recognition and tracking system is embedded in the recording and broadcasting host, and can achieve high-definition fully automatic image recognition and tracking without the need for additional tracking host hardware and software.

The HAIMINGWEI image recognition and tracking system adopts a unique target shape feature detection method. The tracked person does not need to wear any auxiliary equipment to lock and track the target, making the camera image centered on the locked target and controlling the camera to perform corresponding scaling strategies. The system supports multiple custom strategies, multi-level close-up modes, strong adaptability, and is not affected by strong light, sound, electromagnetic and other environments.

The HAIMINGWEI image recognition and tracking system supports innovative "wide dynamic image stabilization technology", which can ignore the movement of teachers within a custom numerical range, keep the camera stationary, ensure the smoothness and stability of the image, and avoid phenomena such as image jumping and shaking.

Teacher image recognition and tracking module: The podium and classroom can be divided into three tracking areas: VIP area, regular area, and ignored area. And it is possible to customize tracking strategies for each area, such as lens zoom factor, highest and lowest shooting areas of the camera, tracking speed, etc. The Ncast agile image recognition and tracking system supports three tracking modes: single target tracking, multi target tracking, and fixed coordinate tracking. Different tracking modes can be set for different regions.

The teacher image recognition and tracking module also supports fully automatic zoom mode for teacher tracking. When the teacher walks quickly in the podium area, the system will automatically take photos in full. When the teacher stops walking or writing on the blackboard, the camera will automatically take close-up shots. When students step onto the podium to interact with teachers, the camera will automatically zoom in and out to capture images of both students and teachers. The system can customize the camera tracking speed and close-up lens magnification through the network.

The teacher image recognition and tracking module only needs to be equipped with a pan tilt camera to achieve panoramic, mid range, and close-up functions for teachers;

The student image recognition and tracking module can achieve pure image recognition and tracking effects in standard classrooms. Students do not need to use auxiliary devices such as infrared handheld microphones or wireless transmission buttons. They only need to stand up to achieve image recognition and tracking. When a single student stands up, the camera will take close-up shots. When two or more students stand up, the camera will automatically zoom to the panoramic shooting angle, allowing both students to enter the captured scene.

Automatic broadcasting system

A、 When a teacher stands on the podium to give a lecture, the system automatically switches the close-up video signal of the teacher's camera to the main screen;

B、 When the teacher moves quickly on the podium, the system will automatically switch to the teacher's panoramic view to ensure that the teacher does not leave the shooting range;

C、 When the teacher steps down from the podium to interact with the students in the student area, the system still uses the teacher's camera image as the main screen, but at this point, the camera image becomes a pre-set panoramic image.

D、 When the teacher is on the podium and the students stand up to answer questions, switch the student close-up camera screen to the main screen.

E、 When a student stands up to answer a question, the system first switches the student's panoramic view to the main screen. After the student's close-up camera tracks it in place, it then switches from the student's panoramic view to the student's close-up view.

F、 When the teacher operates the computer, the computer screen switches to the main screen.

Note: Administrators can switch between automatic/manual modes of the tracking system and the broadcasting system in real-time through the network broadcasting platform, and use manual image click tracking or manual screen switching to operate the system.

System features

Intelligent image recognition processing technology, the tracked person does not need to wear any auxiliary equipment;

Innovative wide dynamic painting stabilization technology for smoother visuals;

Adopting a unique target shape feature detection method with high recognition accuracy;

Strong adaptability, unaffected by strong light, sound, electromagnetic and other environmental factors;

Supports three tracking modes: multi-target tracking, single target tracking, and fixed area tracking;

Automatic recognition of target position, real-time control of camera for precise positioning, and dynamic adjustment of pan tilt speed based on target movement speed;

Multiple custom strategies, supporting multi-level close-up mode;

The system can support panoramic shooting when teachers are walking, and close-up shooting when teachers stop;

When a single student stands up, the camera will take close-up shots;

The system sets multiple ignoring areas, such as staff areas, classroom entrances and exits, etc;

Support automatic tracking in conjunction with various brand cameras such as SONY and IHAIBI series;

Remote network broadcasting system

The HAIMINGWEI remote network guidance system adopts a touch controlled all-in-one machine structure and integrates an embedded operating system. It can be combined with standard definition and high-definition series recording and broadcasting all-in-one machines to easily achieve network manual guidance functions for various images.

The remote network guidance system integrates a "remote click tracking module" and a "manual network guidance module".

The HAIMINGWEI remote network guidance system allows for easy selection of camera control modes: automatic image recognition tracking mode and remote click tracking mode. Speakers and administrators can easily track the audience or speakers on the display screen through this system. The operator only needs to click on the image in the picture, and the camera pan tilt will automatically move, taking pictures centered on the clicked position.

The HAIMINGWEI remote network guidance system can also achieve automatic switching between multiple video signal sources, video/audio files, and computer screens in multiple scenes during the course recording process, effectively achieving intelligent guidance recording.

Classroom explanations, whiteboard writing, on-site Q&A, communication and interaction, classroom documents, physical displays, computer screens, and other scenarios are automatically scheduled and switched between teaching display files, without the need for any additional equipment or human operation. The activity changes in each scenario are automatically recognized, and the core part of educational activities is automatically switched to

The HAIMINGWEI single stream single screen recording method and single stream multi screen recording method can achieve fully automatic guidance mode, adopting a humanized strategy design, and can switch between fully automatic screens in the computer screens of teachers, students, or speakers.

For example:

A. When the teacher is giving a speech on stage, the system automatically switches the teacher's camera video signal to the main screen;

B. When the teacher walks on the podium, the system automatically switches to the teacher's panoramic main screen;

C. When students stand up and answer questions, the system automatically switches the student's camera video signal to the main screen;

D. When the teacher operates the computer, the system automatically switches the computer VGA signal to the main screen;

Integrated video conferencing and synchronous teaching system

Video Conference: The recording host integrates professional video conference system functions, which can directly hold remote video conferences in three venues through the recording host, including video broadcasting, voice broadcasting, document sharing and other functions.

Synchronous interactive teaching: The recording host integrates professional synchronous interactive teaching functions, which can directly conduct remote interactive synchronous teaching across three classrooms through the recording host, including teacher audio and video broadcasting, desktop sharing, and hand up questioning functions.

Rich visual style

Single screen mode

The system supports high-definition single screen recording, and in single screen recording mode, the system provides automatic or manual guidance settings. In automatic recording mode, the system intelligently switches the recording screen based on the change in the switch value of the input signal. For example, the system is recording teaching footage centered around the teacher. When the teacher switches to the PPT, the footage will automatically switch to the PPT. After the PPT remains unchanged for a period of time, the footage will automatically switch back to the teacher's teaching scene. In manual mode, users can manually switch between recorded images based on the progress of the teaching.

Multi screen mode

The system supports synchronous recording of multiple screens, and users can set the display style of the screen according to their preferences. The system defaults to one large and three small screen modes. During the recording process, the system can switch the display content of the main screen based on the changes in the switch quantity of the connected signal. For example, at this point, the main screen displays the teaching screen of the teacher. When the teacher switches to the PPT, the main screen will automatically switch to the content of the PPT. After the display content of the PPT remains unchanged for a period of time, the main screen will switch back to the teacher screen.

PIP Mode 

The system supports synchronized recording of picture in picture, and the position and size of the floating small window can be adjusted according to user needs. The system switches the display content of large and small screens based on changes in the switch value of the access signal. Users can set the preferred input signal for large and small screens. For example, the default main screen can be set to display the teacher's screen. When the teacher switches to the PPT, the main screen will switch to the content of the PPT. After the display content of the PPT remains unchanged for a period of time, the main screen will switch back to the teacher's screen.

Custom screen mode

The system supports custom screen modes. Users can customize the screen style according to the actual needs of the course, including the number of screen displays, the position and size of each screen, etc. Through customization, it can meet more school requirements and be close to actual classroom needs.

Convenient one click recording

Through the configured visual control panel, classroom teachers only need simple operations to complete the recording of the entire class. During the recording process, the playback screen can also be viewed in real-time through the control panel, and manual switching and intervention can be performed on the screen. Provide convenience for teachers to manually record through the control panel.

Real time teaching courseware association

After the recorded video is automatically uploaded to the platform, it can be associated with the corresponding teaching documents and courseware PPT on the platform for later on-demand use.

Highly integrated and low-power recording and broadcasting hosts

The recording host integrates video matrix, audio matrix, encoder, central control module, etc. All audio and video, computer, central control and other devices can be connected to the recording host

Main Equipment Introduction

The series host adopts a pure embedded system architecture, with 4K image quality, four-way interactive interaction, multi-channel resource synchronization preview and recording. A industry-specific recording host that integrates dual channel touch management, real-time storage, live streaming, on-demand, playback, editing, and downloading.

Normalized recording and broadcasting host

Built in navigation software

Main hardware features

1. High definition image quality: high-definition video input, output, encoding, decoding, recording.

2. Video processing: 4 physical video sources and 4 network video source inputs, 2 video outputs, HDMI with 4K image quality.

3. Audio processing: Supports functions such as AEC, ANS, and mixing.

4. Programmable central control: An open programmable central control that supports multiple interfaces such as RS232/485/USB/TCP/UDP.

5. HD recording: Multi task recording/live streaming/on-demand/playback/download/special effects switching/self selection of storage space.

6. Interactive terminal: Supports multi protocol hybrid remote interaction modes such as RTSP/RTMP/H.323/SIP.

7. MCU management: Single machine 4-point MCU conference, seamless integration with mainstream third-party MCUs or terminals.

8. Decoding algorithms: H.265/H.264 video compression algorithm, AAC/G.711 audio compression algorithm.

9. Streaming protocol: Supports TCP/UDP/RTSP/RTP/RTMP/ONVIF/H.323/SIP/HTTP protocol.

10. Device management: Various visual navigation management methods such as B/S, C/S, GUI touch, etc.

Main features of built-in recording software

1. The embedded recording and playback management software must be installed in the host, based on the dual management interface architecture of B/S and local HDMI/VGA video output GUI, and compatible with browsers such as IE.

2. The system includes video monitoring, video switching, pan tilt control, audio adjustment, live streaming/recording, pause control, subtitles, logo, school emblem, live streaming monitoring, guidance, on-demand, system settings, and other functions

3. Video preview display: Real time preview display of 7 high-definition videos on the GUI, allowing for SDI/HDMI/network remote mixed simultaneous input;

4. The recording method supports manual guidance switching control or seamless combination with automatic tracking system to achieve fully automatic recording; Support advance reservation for recording, and the end time can be adjusted during the recording process.

5. Real time display of recording information, which must include recording duration, video resolution, frame rate, and host IP information; And it is required to be able to display the hard disk capacity in real time;

6. Implement real-time pre monitoring function for cameras with at least 6 channels; Pan tilt control interface:

7. The recording mode supports two modes: movie mode and movie plus resource mode. The movie mode and resource mode can work simultaneously. When recording a movie in resource mode, the resource mode works in the background.

8. The broadcasting mode supports manual, semi-automatic, and fully automatic modes, and the three modes can be switched freely;

9. The manual broadcast mode supports functions such as video preview, live output monitoring, video switching, audio adjustment, and recording mode switching; Supports manual PTZ control, and both teacher and student videos support multiple preset settings.

10. Unified resource mode interface and movie mode interface; Select the channel that requires recording resources in the menu, and open the movie+resource mode recording with just one click to simultaneously record PGM and resource recordings.

11. The recording host comes with an on-demand function, which supports a maximum of 10 simultaneous recording times.

12. Support functions such as course information, subtitles, opening and ending information, and support manual and automatic subtitle machines to publish subtitles to enhance the infectivity of courseware.

13. Support multiple switching effects, including mainstream switching effects such as overlay, fade in, and fade out.

14. Support remote FTP upload/download of videos, making it convenient for users to export and copy video files from the recording host without entering the site.

15. Support TS multicast streaming and enable more users to watch live broadcasts within the local area network.

16. Require audio and video synchronization of recorded files to meet the normal application of third-party editing software.