1. Introduction
According to a study by the School of Public Health at Boston University, nearly 30% of suffocation cases of children in cars are due to children entering parked car to play, and 70% are due to children being left in the cars by adults. And it is reported that China's new car evaluation procedures guided by the child protection working group will test the vehicle’s child detention detection device in terms of its function, performance, safety and software recognition algorithm in the future [1]. Therefore, it is of great significance to design safe, reliable, stable and intelligent solutions to in-vehicle live detection.
A variety of sensors are used in in-vehicle live detection, ranging from cameras to infrared sensors. But cameras have the problems of leakage of privacy, requirement of high-quality light sources and high cost. Infrared sensors are sensitive to interference from heating devices and the sun and are easy to be obstructed malfunctioning in the short term. Millimeter-wave radars have recently received lots of attention in vehicle application for the high anti-interference ability and stability [2, 3]. For these kinds of applications, millimeter-wave radars have the advantages of reduced misreport rate, high detection sensitivity and are not sensitive to light, temperature and dust. With the miniaturization of radar system, millimeter-wave radar can be applied not only to outside of vehicle application but also to resolve the problem of in-vehicle life detection. The wavelength of 76–81 GHz millimeter-wave radar is 4 mm, so it can be used to detect very fine movement such as breathing of body and so as to detect children and other objects. It is also possible to detect heart information of human and alarm fatigue driving or remind of seat belt wearing [4-6]. Two dimensional radar images can also be used to detect vital signs, and even recognize faces based on deep learning algorithms [7-10].
2. Hardware Design
2.1 Radar Module Performance
In radar design, module performance debugging is a crucial link, which not only involves hardware printed circuit board (PCB) layout design, software parameter debugging and algorithm optimization, but also involves product structure design optimization (material, signal reflection index, etc.). The performance of the radar module Yosemite 4T8R 77/79 GHz is shown in Table 1.
Radar performance specifications
2.2 Hardware Design Architecture
The hardware design architecture is shown in Fig. 1. The advantages of this design scheme are mainly reflected in the following four points:
· Millimeter-wave radar (MMIC) of domestic vehicle gauge level high integrated in-chip antenna;
· 4 send 4 receive, signal detection ability is strong;
· Not affected by external temperature, light, dust and other environmental conditions;
· No visual image, privacy protection.
In the field of microwave, the key to hardware design lies in the design of antenna [11]. The radar used in this paper has a built-in antenna (the antenna design is inside the chip). So that the product design can be miniaturized, the design difficulty of the product is reduced, and the product generation cycle is shortened. In this technical solution, the radar module is 4-transmitter-4-receiver, and the configurable frequency range is 77–81 GHz, so the maximum bandwidth is 4GHz. The radar module is mainly responsible for detecting the occupant's legacy, and sends the detection result to the engine control unit (ECU) or vehicle controller area network (CAN) bus through the private CAN bus. The ECU module is connected to the vehicle CAN bus to monitor vehicle status, process and upload data. The ECU module communicates with the radar module privately through the CAN bus. And on the other hand, it can access the relevant vehicle intelligent platform database. Since this article focuses on radar design, the design of the ECU module will not be explained in detail.
Hardware design architecture.
2.3 Power Module
The radar, microcontroller unit (MCU), and flash modules require different voltages for power supply. The radar requires a variety of power supply voltages, such as 3.3 V, 1.5 V, and 1.8 V. A stable and reliable system power supply solution can ensure the stable running of the system. After comparing the power management chips from Maxim Integrated, Texas Instruments (TI), Onsemi, and other manufacturers, Maxim Integrated MAX20430 four-output small power management integrated chip (PMIC) is chosen as the power module.
MAX20430 is a high-efficiency, four-output DC-DC converter for automotive advanced driving assistance system (ADAS) applications. The schematic diagram of PMIC is shown in Fig. 2. OUT1 is a synchronous buck converter that converts the vehicle battery voltage to 3.3 V at current up to 2.5 A; OUT3 boosts OUT1 to 5 V at currents up to 500 mA; OUT2 and OUT4 are low-voltage synchronous buck converters that operate from OUT1, providing an output voltage range of 0.8–3.9875 V at current up to 3 A. The device offers a 2.1 MHz fixed frequency pulse width modulation (PWM) mode for all DC-DC outputs for better noise resistance and load transient response. The 2.1 MHz frequency operation allows the use of all-ceramic capacitors and minimizes external components, making it the optimal power solution for this solution.
2.4 Radar Module
The radar module in this paper is a domestic Gartland vehicle gauge level 4-transmit and 4-receive radar chip (model: ALPS MO-AIP-FC). Because the transceiver antenna is built into the module chip, it is easy to design the external circuit. Fig. 3 shows the radar module circuit schematic.
Schematic diagram of power management integrated chip.
Schematic of radar module.
3. Software Design
The overall software development framework is divided into two parts: one part is developed for RadarSoc, which is used to configure radar parameter configuration, file management, communication driver and system task assignment. The other one is developed for MCU processor, which is used for system power management, CAN bus communication, serial universal asynchronous receiver/transmitter (UART) communication and other functions, as shown in Fig. 4.
Overall software architecture of the system.
3.1 Software System Hierarchy
In the system, the software system is mainly divided into driver layer, service layer, and application layer. The main functions of the application layer are as follows:
The application layer mainly includes the status switching and logic processing modules of the functional modules at the system level or the whole machine level;
The service layer mainly includes the further analysis and processing of the data obtained by the driver layer, and provides the upper layer with more comprehensive and useful information and the combined operation of multiple driver modules;
The driver layer mainly includes the operation module of the hardware level register.
3.2 Overall Software Monitoring Process
In different states, the vehicle wakes up the MCU through the vehicle CAN bus, and the MCU parses the CAN bus message. Determine the vehicle status according to the message data and enter the corresponding detection mode. The MCU enables the radar module scanning and monitoring function through UART communication. The radar module will judge the point cloud data collected by scanning and output the monitoring result. The vehicle CAN bus analyzes the message data to determine whether it is necessary to perform actions such as sound and light alarm, skylight opening, and air conditioning. Fig. 5 is the overall monitoring flow chart of the software.
3.3 Basic Design of Radar Module Software
After the software system is initialized, the adjusted radar configuration parameters are invoked in flash, and real-time operating system (RTOS) tasks are created and invoked, including radar monitoring tasks, UART communication tasks, and user application tasks. In radar monitoring tasks, it receives radar data, executes fast Fourier transform (FFT) algorithm, direction of arrival (DOA) algorithm and Kalman filtering algorithm, analyzes point cloud data and regional position information in user application tasks, and finally outputs the results to MCU through UART communication. Fig. 6 shows the basic design flow chart of radar module software.
Overall monitoring flow chart of MCU.
Basic flowchart of radar module software.
After analog digital conversion (ADC), one period of intermediate frequency (IF) radar signal is transformed to its spectrum through FFT. And the phase of the object on the specific frequency of FFT spectrum is obtained. When there are multi-objects, the reflected signals of multi-objects are as shown in Fig. 7. Multi-objects will produce multi-IF signals. And there will be multi-peaks in IF spectrum. Each peak represents one object at specific distance. So the distance resolution of the system is limited by the bandwidth of the radar signal. The wider the bandwidth, the higher the distance resolution. The highest test speed depends on wavelength and the time interval between two chips. In practical scenarios, there will multi-peaks at the same position of distance spectrum when radar receives IF signal and the reflected signal. And radar cannot identify whether it is signal object or multi-objects. Two-dimensional FFT is used to obtain the change of phase so as to get the moving speed of different objects.
Demonstration of multi-objects detection by FMCW radar.
One transmitter and multi-antenna structure as shown in Fig. 8 is designed for DOA test. The phase change of IF signal received by neighboring antennas are used to compute DOA.
Scheme of one transmitter two receiver design.
The Kalman filtering is used to filter out the interference from the dynamic information of the target, obtain the optimal and accurate estimation value, and estimate the next value to obtain an accurate, stable, and smooth data.
3.4 Vehicle Status Software Detection Process
Another core task of software detection in this paper is the detection and judgment of vehicle state. The system tells the radar module which detection mode to enter based on the status of the vehicle. There are many different states in the status detection of each door. The different state data is sent to the MCU through the vehicle CAN bus message, and then the MCU parses the CAN message and tells the radar which state should be monitored several times. Fig. 9 shows the flow chart of vehicle status detection.
Flow chart of vehicle state detection.
3.5 Confidence Judgment of Software Monitoring Results
After the system enters different vehicle status monitoring modes, the initial value of monitoring confidence is 50%. The radar module will continuously monitor the rear position information at a frequency of 20 Hz, and output the result every 4 seconds. If there is a dynamic target, the confidence is increased by 5%, and the highest confidence is 100%. If there is no dynamic target, the confidence is reduced by 5%, the minimum is 0%; continuous monitoring for 1 minute, if the confidence level is more than 90%, it means that passengers are stranded in the car and enter the alarm process. The main monitoring comparison parameters are speed (V) and distance (R), and the comparison conditions are as follows:
V > 0.1 m/s → There are passengers in the car;
R < 0.3 m → There are passengers on the left side of the rear seat;
0.3 m < R < 0.6 m → There are passengers in the middle of the rear seat;
0.6 m < R < 1.5 m → There are passengers on the right side of the rear seat.
4. Test and Data Analysis
In order to obtain the original ADC data collected by the radar module, the designed system is interconnected with the low-voltage differential signaling (LVDS) signal interface on the data acquisition card of the radar manufacturer. Then MATLAB is used to acquire and plot the ADC data frame. Speed-dimension FFT, range-dimension FFT and DOA functions are made to process and plot the corresponding results.
Fig. 10 shows the ADC raw data obtained by the four receiving antennas of the radar module. From the mapping between the sampling point and the number of pulses, it can be noticed that the sampling points and pulse numbers of each receiving antenna are different because each receiving antenna in the radar module receives different reflected signals.
Raw ADC data obtained by the four receiving antennas of the radar module: (a) echo R1, (b) echo R2, (c) echo R3, and (d) echo R4.
Detection matrix for comprehensive analysis of speed and distance.
Fig. 11 is the detection matrix for comprehensive analysis of speed and distance based on the analysis of ADC data. It shows that the radar module receives reflected signals with the same relative speed and different strengths at different distances. For example, there are radar signals at the distance of 1 m, 11 m, and 17 m.
Then we test the performance of the radar system.
1) Distance resolution
When the radar is fixed on the test bench and the radar corner reflector is moved on the test bench with the step of 0.001 m until the radar finds a sudden change of the object distance. Then the difference between the tested object distance and the initial object distance is defined as [TeX:] $$D_{\Delta i} .$$ The maximum value of [TeX:] $$D_{\Delta i} \text { is } D_{\Delta i m a x}$$ and the minimum value of [TeX:] $$D_{\Delta i} \text { is } D_{\Delta i m i n} .$$ And the average value of them is taken as distance resolution [TeX:] $$A_d .$$
From Fig. 12, it is observed that there are two sudden changes of object when the radar corner reflector is moved between 0.60 m and 0.71 m at the position of 0.63 m and 0.67 m. And for 0.63 m, [TeX:] $$D_{\Delta i}$$= 0.64 –0.60 = 0.04 m. For 0.67 m, [TeX:] $$D_{\Delta i}$$= 0.68 – 0.64 = 0.04 m. So it is calculated that the distance resolution is 0.04 m.
Demonstration of distance resolution tested from the detected object distance by radar.
Demonstration of angel resolution tested from the detected object angle by radar.
2) Angle resolution
When the radar is working at normal mode and the radar corner reflector object is rotating on the rotating platform with the step angle of [TeX:] $$0.1^{\circ}$$ until the radar finds a sudden change of object angle. Then the difference between the tested object angle and the initial object angle is the angle resolution.
From Fig. 13, it is observed that there are three sudden changes of object when the rotating plat is rotated between [TeX:] $$50^{\circ} \text { and } 53^{\circ}$$ at the angle of [TeX:] $$50.6^{\circ}, 51.4^{\circ} \text {, and } 52.3^{\circ} \text {. }$$ And for all of them, the calculated angle resolution is [TeX:] $$0.8^{\circ}.$$
3) Speed error
When the radar is working at normal mode, and the radar corner reflector object is moved at certain speed. If the detected speed by radar is [TeX:] $$V_i^*$$ and the actual object moving speed is [TeX:] $$V_i, 1 \leq i \leq n .$$ Then we can calculate the speed error with:
When the object speed is increased from 0.05 m/s to 0.3 m/s with the step of 0.05 m/s, the measured speed by radar is shown in Table 2. Then the speed error calculated from the measured data is 0.0065 m/s.
The speed error of the system (0.0065 m/s) is much lower than the threshold of finding passengers in the car (0.1 m/s). So it can lead to the conclusion that the system is suitable for in-vehicle life detection. And the distance resolution (0.04 m) also qualify the system to detect passengers' position (0.3 m).
Actual object moving speed and the measured speed by radar (unit: m/s)
5. Conclusion
This paper has designed and presented a vital signs monitoring system based on FMCW millimeter-wave radar for in-vehicle safety applications. The preliminary results suggest the system's potential effectiveness in addressing the critical issue of forgotten occupants, offering a promising technological solution to reduce the risks of vehicle-related hyperthermia. Extensive real-world tests will be further conducted to validate the system's robustness and reliability. And seamless integration with the smart cockpit system and other sensors will achieve higher accuracy in vital sign extraction and broaden the application scenarios of the designed system.
Conflict of Interest
The authors declare that they have no competing interests.