WO2017215024A1

WO2017215024A1 - Pedestrian navigation device and method based on novel multi-sensor fusion technology

Info

Publication number: WO2017215024A1
Application number: PCT/CN2016/087281
Authority: WO
Inventors: 庄园; 杨军; 戚隆宁; 李由
Original assignee: 东南大学
Priority date: 2016-06-16
Filing date: 2016-06-27
Publication date: 2017-12-21
Also published as: CN106017454A; CN106017454B

Abstract

A pedestrian navigation apparatus and method based on a novel multi-sensor fusion technology. The apparatus comprises a handheld intelligent device (1), an observation processing unit (2) and a fusion filter (3). The method comprises: the handheld intelligent device (1) acquiring, by hardware thereof, original data of an IMU (11), a magnetometer (12), a pressure gauge (13), WiFi (14), BLE (15) and GNSS (16); the observation processing unit (2) processing the original data provided by the handheld intelligent device (1) to provide observed quantities of position and speed to the fusion filter (3); and the fusion filter (3) obtaining a pedestrian navigation result by using a kinematics model as a system model and an observation model that is established according to a processing result by the observation processing unit (2) as well as through the processing of the fusion filter (3). The method, without the assistance of other supplemental systems, overcomes the issue of rapidly accumulated navigation errors. An IMU processing unit (21) takes into account multiple modes of the handheld intelligent device (1) and surmounts a limitation whereby a conventional multi-sensor combined with an IMU needs to be fixed to a carrier, enhancing the accuracy of pedestrian navigation.

Description

Pedestrian navigation device and method based on novel multi-sensor fusion technology

Technical field

The invention relates to the field of multi-sensor fusion and pedestrian navigation, in particular to a pedestrian navigation device and method based on a novel multi-sensor fusion technology.

Background technique

With the development of the mobile Internet, indoor and outdoor pedestrian navigation applications are booming, such as indoor shopping in large shopping malls, hospital patient tracking, and supermarket traffic analysis. A number of market analysis reports at home and abroad agree that pedestrian navigation will be a research direction with a huge market. At the same time, portable smart devices, such as smartphones, tablets and smart watches, have been developing at an ultra-high speed for the past decade and have become an indispensable part of people's lives. Most of these portable devices have powerful processors, wireless transceivers, cameras, GNSS receivers for GNSS, and numerous sensors. As a result, these portable smart devices have become an ideal platform for multi-sensor fusion and pedestrian navigation related applications.

At present, single pedestrian navigation technology has different degrees of defects. Pedestrian navigation technology based on wireless systems such as WiFi and Bluetooth usually has large fluctuations in wireless signal strength in harsh environments, and cannot provide complete navigation information such as three-dimensional position, velocity and attitude. System performance is highly dependent on the distribution and number of transmitting devices. , location information is not continuous smooth and other defects. The pedestrian navigation technology based on the micro inertial unit is short-term accurate, but the navigation error accumulates faster. Camera-based visual positioning is characterized by slow calibration of visual sensors, high error rate of feature information extraction, and slow calculation of navigation information in complex environments. Therefore, multi-sensor fusion has become the mainstream solution for pedestrian navigation.

At present, the existing multi-sensor fusion technology generally includes the following steps: (1) Calculating the position, velocity and attitude of the tracking object by the inertial mechanical programming algorithm with the measurement data of the inertial unit (three-axis accelerometer and three-axis gyroscope) (2) Establish an error model corresponding to the inertial mechanical programming algorithm and use it as a system model of the fusion filter; (3) Establish other auxiliary systems (GPS, WiFi, Bluetooth, RFID, GNSS, etc.) as fusion filters Observing the model; (4) estimating the state quantity error of the system through the prediction and update process of the fusion filter; and (5) compensating the inertial unit error of the system state quantity error and the position, velocity and attitude information based on the inertial mechanical programming algorithm Final location information, speed and attitude information.

The existing multi-sensor fusion technology has the following two fatal disadvantages: (1) in the absence of other auxiliary systems, the navigation error will accumulate rapidly; (2) when the inertial unit and the carrier are not fixed, for example: in pedestrian navigation Mobile phones and pedestrians, traditional multi-sensor fusion technology can not correctly estimate the carrier information. Therefore, the existing multi-sensor fusion technology cannot provide accurate pedestrian navigation information in many scenarios.

Summary of the invention

The technical problem to be solved by the present invention is to provide a pedestrian navigation device and method based on a novel multi-sensor fusion technology, which can improve the navigation accuracy and usability of pedestrians.

In order to solve the above technical problem, the present invention provides a pedestrian navigation device and method based on a novel multi-sensor fusion technology, including: a handheld intelligent device platform, an observation measurement processing unit, and a fusion filter; the handheld intelligent device acquires an inertial measurement unit by using its own hardware. (Inertial Measurement Unit, IMU), magnetometer, pressure gauge, WiFi, Bluetooth Low Energy (BLE) and Global Navigation Satellite System (GNSS) raw data, observation processing unit handles handheld The raw data provided by the intelligent device provides the positional or velocity observation to the fusion filter, and the fusion filter uses the kinematic model as the system model, and the observation processing unit results in the observation model, and the fusion filter is processed to finally obtain the pedestrian navigation. result.

Preferably, the handheld smart device platform comprises an IMU, a magnetometer, a pressure gauge, a WiFi, a low energy Bluetooth and a GNSS, which are common to existing smart devices; the IMU provides raw data of acceleration and angular velocity; the magnetometer provides raw data of geomagnetism The pressure gauge provides raw data of atmospheric pressure; WiFi provides raw data of Received Signal Strength (RSS); BLE provides raw data of BLERSS; GNSS receiver provides raw data of GNSS; any device of intelligent equipment can provide observation Other sensors of information can be included in the proposed multi-sensor fusion algorithm.

Preferably, the observation processing unit comprises: an IMU processing unit, a magnetometer processing unit, a pressure gauge processing unit, a WiFi processing unit, a BLE processing unit, a GNSS processing unit, and the like; and the IMU processing unit processes the original acceleration and angular velocity provided by the IMU. Data to obtain IMU position information and transmitted to the fusion filter; the magnetometer processing unit processes the geomagnetic raw data provided by the magnetometer to obtain geomagnetic position information and transmits to the fusion filter; the pressure gauge processing unit processes Raw data of atmospheric pressure provided by the pressure gauge to obtain elevation information and transmitted to the fusion filter; the WiFi processing unit processes the RSS original data provided by the WiFi to obtain WiFi location information and transmit to the fusion filter; BLE The processing unit processes the RSS raw data provided by the BLE to obtain BLE location information and transmits to the fusion filter; the GNSS processing unit processes the position and velocity information provided by the GNSS receiver and transmits the information to the fusion filter. The observation processing unit also includes other processing units to process other sensors of the smart device platform to obtain position or velocity information and transmit to the fusion filter.

Preferably, the fusion filter comprises a system model and an observation model; the system model uses the kinematic model to predict the position and velocity information of the object to be measured, and transmits the information to the observation model; the observation model predicts the position of the system model, The speed information is combined with information such as position, speed, etc. of the IMU, magnetometer, pressure gauge, WiFi, BLE, and GNSS provided by the observation processing unit to update the final position and speed information of the target to be tested.

Preferably, the IMU processing unit comprises a user motion mode and a device usage pattern recognition module, a heading angle deviation estimation module, an improved dead reckoning algorithm module, a user motion mode and a device usage pattern recognition module according to the IMU of the handheld smart device platform and others The raw data provided by optional hardware (such as a magnetometer) identifies user motion patterns such as stationary, walking, running, and the device usage modes of handheld, text messaging, telephone, navigation, pocket, backpack, etc.; the heading angle deviation estimation module is based on the user The motion mode and the device use the user motion mode and device usage mode output by the pattern recognition module and the raw data provided by the IMU of the handheld smart device platform and other optional hardware (eg, a magnetometer) to estimate the heading angle deviation; The estimation algorithm module obtains IMU position information according to the heading angle deviation output by the heading angle deviation estimation module and the original data provided by the IMU of the handheld smart device platform and other optional hardware (for example, a magnetometer) and transmits the information to the fusion. filter.

Preferably, the improved dead reckoning algorithm module comprises a attitude measuring system module, a heading angle deviation compensation module, a step detecting module, a step estimating module, a dead reckoning algorithm module, and the attitude determining system module is based on the IMU of the handheld smart device platform. The original data provided by other optional magnetometers recognizes the attitude information of the handheld smart device; the heading angle deviation compensation module reads the heading angle deviation outputted by the heading angle deviation estimation module and compensates the pedestrian heading angle and outputs to the dead reckoning algorithm; The detection module algorithm module detects the step number of the pedestrian to the step estimation module according to the original data of the IMU of the handheld smart device platform; the step estimation module according to the result of the step detection module and the IMU of the handheld smart device platform The raw data estimates the step size of the pedestrian and feeds back to the dead reckoning module; the dead reckoning module estimates the step information output by the module according to the step size and the pedestrian heading angle output by the heading angle offset compensation module The information calculates the IMU position observation and feeds back to the fusion filter.

Correspondingly, the present invention also provides a pedestrian navigation method based on a novel multi-sensor fusion technology, comprising the following steps:

(1) The handheld smart device uses its own hardware to obtain raw data of IMU, magnetometer, pressure gauge, WiFi, BLE and GNSS; (2) The observation processing unit processes the raw data provided by the handheld smart device to provide position or velocity measurement. The fusion filter is used; (3) the fusion filter uses the kinematics model as the system model, and the observation processing unit is used to establish the observation model. After the fusion filter is processed, the pedestrian navigation result is finally obtained.

Preferably, the observation processing unit processes the raw data provided by the handheld smart device to provide a position or velocity observation to the fusion filter, including the following steps:

(1) The IMU processing unit processes the raw data of the acceleration and angular velocity provided by the IMU to obtain the IMU Position information is transmitted to the fusion filter; (2) the magnetometer processing unit processes the geomagnetic raw data provided by the magnetometer to obtain geomagnetic position information and transmits to the fusion filter; (3) the pressure gauge processing unit Processing raw data of atmospheric pressure provided by the pressure gauge to obtain elevation information and transmitting to the fusion filter; (4) the WiFi processing unit processes the RSS provided raw data to obtain WiFi location information and transmitting the information to the a fusion filter; (5) the BLE processing unit processes the RSS raw data provided by the BLE to obtain BLE location information and transmits the BLE location information to the fusion filter; (6) the GNSS processing unit processes the raw data of the GNSS provided by the GNSS chip The position and velocity information of the GNSS is obtained and transmitted to the fusion filter.

Preferably, the IMU processing unit processes the raw data of the acceleration and angular velocity provided by the IMU to obtain the IMU position information and transmits the information to the fusion filter, including the following steps:

(1) The user motion mode and device usage pattern recognition module identifies user motion patterns such as still, walking, running, etc. according to raw data provided by the IMU of the handheld smart device platform and other optional hardware (eg, a magnetometer). Handheld, SMS, phone, navigation, pocket, backpack and other device usage modes;

(2) The heading angle deviation estimation module is provided according to the user motion mode and the device usage mode output by the user motion mode and the device usage mode recognition module, and the IMU of the handheld smart device platform and other optional hardware (eg, a magnetometer) The raw data estimates the heading angle deviation;

(3) The improved dead reckoning algorithm module obtains IMU position information according to the heading angle deviation output by the heading angle deviation estimating module and the original data provided by the IMU of the handheld smart device platform and other optional hardware (for example, a magnetometer). And transmitted to the fusion filter.

Preferably, the improved dead reckoning algorithm module comprises the following steps:

(1) The attitude measuring system module identifies the posture information of the handheld smart device according to the original data provided by the IMU of the handheld smart device platform and other optional magnetometers;

(2) The heading angle deviation compensation module reads the heading angle deviation outputted by the heading angle deviation estimation module and compensates the pedestrian heading angle and outputs to the dead reckoning algorithm;

(3) The step detection module algorithm module detects the step number of the pedestrian feedback to the step size estimation module according to the original data of the IMU of the handheld smart device platform;

(4) The step estimation module estimates the step size of the pedestrian according to the result of the step detection module and the original data of the IMU of the handheld smart device platform, and feeds back to the dead reckoning module;

(5) The dead reckoning module calculates the IMU position observation based on the step information output by the step estimation module and the pedestrian heading angle information output by the heading angle offset compensation module, and feeds back to the fusion filter.

The invention has the beneficial effects that the invention optimizes the use method of the IMU in the traditional pedestrian navigation, liberates it from the system model of the fusion filter into an observation model, and overcomes the situation that the traditional multi-sensor is integrated without other auxiliary systems. Under the disadvantage that navigation errors will accumulate quickly. The IMU processing module in the present invention considers various modes of handheld smart devices in daily life, and breaks through the limitation that the traditional multi-sensor fusion IMU needs to be fixed with the carrier. Therefore, the present invention greatly improves the accuracy and usability of pedestrian navigation.

DRAWINGS

FIG. 1 is a schematic structural diagram of a pedestrian navigation device based on a novel multi-sensor fusion according to the present invention.

2 is a schematic structural view of an inertial unit processing module according to the present invention.

FIG. 3 is a schematic diagram of a Gaussian kernel support vector machine nonlinear classifier according to the present invention.

4 is a schematic diagram of a user motion mode and a smart device usage pattern recognition support vector machine in the present invention.

FIG. 5 is a schematic diagram of an improved pedestrian dead reckoning algorithm in the present invention.

Detailed ways

As shown in FIG. 1 , a pedestrian navigation device based on a novel multi-sensor fusion includes: a handheld smart device platform 1 , an observation measurement processing unit 2 , and a fusion filter 3 . The handheld smart device 1 utilizes its own hardware to acquire an Inertial Measurement Unit (IMU) 11, a magnetometer 12, a pressure gauge 13, a WiFi 14, a Bluetooth Low Energy (BLE) 15 and a Global Navigation Satellite System (Global Navigation). Raw data of Satellite Systems, GNSS) 16, the observation processing unit 2 processes the raw data provided by the handheld smart device 1 to provide a position or velocity observation to the fusion filter 3, and the fusion filter 3 utilizes the kinematic model as the system model 31. The result of the observation processing unit establishes the observation model 32, and the processing of the fusion filter 3 finally obtains the pedestrian navigation result. The above novel multi-sensor fusion pedestrian navigation device can be used for various handheld smart devices (including smart phones, tablet computers, smart watches, etc.), and the handheld smart device can be held by hand or fixed on a pedestrian. The new multi-sensor fusion pedestrian navigation system shown in Figure 1 subverts the use of IMU in traditional pedestrian navigation, liberating it from the system model 31 of the fusion filter into the observation model 32, overcoming the integration of traditional multi-sensor pedestrians. The disadvantage that the navigation device can accumulate rapidly without other auxiliary systems.

The above-mentioned handheld smart device platform 1 includes an IMU 11, a magnetometer 12, a pressure gauge 13, a WiFi 14, a BLE 15, and a GNSS 16 which are common to existing smart devices; the IMU 11 provides raw data of acceleration and angular velocity; and the magnetometer 12 provides geomagnetism. Raw data; the pressure gauge 13 provides raw data of atmospheric pressure; WiFi 14 provides raw data of Received Signal Strength (RSS); BLE 15 provides original BLERSS Data; GNSS 16 provides raw speed and position data for GNSS. Any other sensor that can hold the observation information of the handheld smart device 1 can be included in the proposed multi-sensor fusion algorithm.

The above-described observation processing unit 2 includes an IMU processing unit 21, a magnetometer processing unit 22, a pressure gauge processing unit 23, a WiFi processing unit 24, a BLE processing unit 25, a GNSS processing unit 26, and the like. The IMU processing unit 21 processes the raw data of the acceleration and angular velocity provided by the IMU 11 to obtain IMU position information and transmits it to the fusion filter 3; the magnetometer processing unit 22 processes the original data of the geomagnetism provided by the magnetometer 12 to The geomagnetic position information is obtained and transmitted to the fusion filter 3; the pressure gauge processing unit 23 processes the raw data of the atmospheric pressure provided by the pressure gauge 13 to obtain elevation information and transmits to the fusion filter 3; the WiFi processing unit 24 Processing the RSS raw data provided by the WiFi 14 to obtain WiFi location information and transmitting the same to the fusion filter 3; the BLE processing unit 25 processes the RSS raw data provided by the BLE 15 to obtain BLE location information and transmit the information to the fusion. Filter 3; GNSS processing unit 26 processes the GNSS 16 raw data to obtain image position information and transmits to the fusion filter 3. The observation processing unit 2 also includes other processing units to process other sensors of the handheld smart device platform 1 to obtain position or velocity information and transmit to the fusion filter 3.

The above-described fusion filter 3 includes a system model 31 and an observation model 32. The system model 31 predicts the position and velocity information of the object to be measured using the kinematic model and transmits it to the observation model 32; the observation model 32 predicts the position and velocity information of the system model 31 and the IMU based on the observation processing unit, and the magnetic force. The information such as the position, speed, and the like of the gauge 12, the pressure gauge 13, the WiFi 14, the BLE 15, and the GNSS 16 are combined to update the final position and speed information of the target to be tested.

As shown in FIG. 2, the IMU processing unit 21 includes a user motion mode and device usage mode recognition module 211, a heading angle deviation estimation module 212, a modified dead reckoning algorithm module 213, and a user motion mode and device usage pattern recognition module 211 according to the The raw data provided by the IMU 11 and other optional hardware (such as the magnetometer 12) of the handheld smart device platform recognizes user motion patterns such as still, walking, running, and device usage modes such as handheld, text messaging, telephone, navigation, pocket, and backpack. . The heading angle deviation estimation 212 module is based on the user motion mode and the device usage mode output by the user motion mode and the device usage pattern recognition module 211 and the IMU 11 and other optional hardware of the handheld smart device platform (eg, magnetometer 12) The raw data provided estimates the heading angle deviation. The improved dead reckoning algorithm module 213 obtains IMU position information based on the heading angle deviation output by the heading angle deviation estimating module 212 and the raw data provided by the handheld smart device platform 1IMU 11 and other optional hardware (eg, magnetometer 12). And transmitted to the fusion filter 3. The IMU processing unit in Figure 2 takes into account multiple movements of pedestrians Modes and multiple use modes of smart devices, designed IMU data processing methods for a variety of use scenarios, breaking through the limitations of IMU and carrier fixed in traditional algorithms, improving the usability of pedestrian navigation systems.

The user motion mode and device usage pattern recognition module 211 uses existing handheld smart device 1 related sensor outputs: IMU 11, magnetometer 12, ranging sensor (optional), light sensor (optional). The IMU 11 and magnetometer 12 update frequency is 50-200 Hz; the output of the latter two sensors is scalar and updated to trigger user behavior. The user motion mode and the device use the pattern recognition algorithm to extract sensor statistics within 1-3 seconds to make a classification decision. User motion patterns and device usage pattern recognition algorithms can be implemented in a variety of ways. The present invention uses a Gaussian kernel dual type support vector machine as an example of implementation.

As shown in Figure 3, the Gaussian kernel-based support vector machine can implicitly map eigenvectors to infinite dimensional linear spaces to achieve or exceed the effects of nonlinear classifications (such as traditional KNN). The prototype of the ¹ norm soft marginal support vector machine is as follows:

Training formula (1):

Sty _i (w ^T φ(x _i )+b)≥1-ξ _i ,

ξ _i ≥0.

Where x _i ∈R ^d , y _i , i=1, 2,..., N are the feature vectors and classification results in the training set, w∈R ^d is the weight vector, and C is a controllable normalized constant for balancing the training The fitting of the data in the set is excessive and insufficient, and φ(·) is a feature vector mapping function.

Classification formula (2):

f(x)=w ^T φ(x)+b.

Since the KKT condition is satisfied, the l ¹ norm soft marginal support vector machine duality is as follows:

Training formula (3):

St0 ≤ a _i ≤ C,

Classification formula (4):

The introduction of the Gaussian kernel as an efficient way to calculate the mapping and inner product is as follows (5):

k(x,x')=φ(x) ^T φ(x')=exp(-||xx'|| ² /2σ ² ), σ>0.

Then the dual type can be converted into the following form:

Training formula (6):

St0 ≤ a _i ≤ C,

Classification formula (7):

Unlike the traditional classifier KNN, the optimal solution a _i , i = 1, 2, ..., N of the support vector machine dual type only has a small number of non-zero values, so only a small number of training feature vectors (ie support vectors) need to be retained. Participate in online classification calculations (for example, to implement a nonlinear classifier based on randomly generated data as shown in Figure 3, KNN needs to store and use 1000 original training feature vectors, and support vector machines only need 142 support vectors) Therefore, the requirements for processor battery consumption and system memory are greatly reduced, and the application is suitable for the handheld smart device platform.

The user motion mode and device usage pattern recognition module 211 classifies the pedestrian behavior patterns into five categories: 1. stationary; 2. walking; 3. running; 4. bicycle; 5. driving a car. The identification of pedestrian behavior patterns can be used to apply zero-speed correction, to adjust the variance of the tracking filter process noise, and to adjust the correlation time of the dynamic system Markov process. The user motion mode and the device usage mode recognition module 211 classify the device usage modes into four categories: 1. the front end is flat; 2. the ear side is vertical; 3. the backpack; 4. the armband. The identification of the handset gesture mode can be used for the determination of the heading direction (coordinate transformation) and for adjusting the variance of the tracking filter process noise.

As shown in FIG. 4, a schematic diagram of a support vector machine for user motion mode and device usage pattern identification using a secondary classifier, including acceleration statistics 2111, angular velocity statistics 2112, rotation angle and tilt angle statistics 2113, light and distance statistics A quantity 2114, a speed feedback statistic 2115, a feature normalization module 2116, a principal component analysis module 2117, Support vector machine module 2118, user motion mode primary classifier 2119, and device usage mode secondary classifier 2110. The specific implementation steps include: collecting representative data sets offline, performing eigenvector normalization and principal component analysis, applying formulas (5) and (6) for training, extracting and storing support vectors, and calculating sensor output statistics online. Feature vector normalization and principal component extraction (same as the training concentration factor), application of the stored support vector, and equations (5) and (7) for secondary classification, determining user motion patterns and device usage patterns.

The heading angle deviation estimation module 212 in the present invention includes a number of different methods. When only IMU 11 is available, we use Principal Component Analysis (PCA). One of the characteristics of pedestrian movement is that the direction of pedestrian acceleration and deceleration is in the direction of travel. Therefore, the data of the accelerometer can be analyzed by PCA to obtain the traveling direction of the pedestrian. When GNSS 16 is available, the direction in which pedestrians travel can be calculated from the speed of the GNSS. When the magnetometer 12 is available, the direction in which the pedestrian travels can also be calculated by the magnetometer 12. The heading angle of the handheld smart device is derived from a nine-axis fusion or a six-axis fusion. Therefore, the heading angle deviation can be obtained by subtracting the convergence direction of the traveling direction of the pedestrian and the heading angle of the hand-held smart device obtained by various methods, and outputting to the heading angle deviation compensation module 2132.

As shown in FIG. 5, it is a schematic diagram of the improved dead reckoning algorithm module 213, including a attitude measuring system module 2131, a heading angle deviation compensation module 2132, a step detecting module 2133, a step estimating module 2134, and a dead reckoning algorithm module 2135. The attitude system module 2131 identifies the gesture information of the handheld smart device 1 based on the raw data provided by the IMU 11 of the handheld smart device platform 1 and other optional magnetometers 12. The heading angle deviation compensation module 2132 reads the heading angle deviation output by the heading angle deviation estimation module 212 and compensates the pedestrian heading angle to the dead reckoning algorithm module 2135. The step detection module 2133 detects the step number of the pedestrian from the raw data of the IMU 11 of the handheld smart device platform and feeds back to the step estimation module 2134. The step estimation module 2134 estimates the step size of the pedestrian based on the result of the step detection module and the original data of the IMU 11 of the handheld smart device platform 1 and feeds back to the dead reckoning module 2135. The dead reckoning module 2135 calculates the IMU position observation based on the step information output by the step estimating module 2134 and the heading information output by the heading angle offset compensation module 2132, and outputs the IMU position observation to the fusion filter 3.

The attitude measurement system module 2131 identifies the attitude information of the handheld smart device 1 based on the raw data provided by the IMU 11 of the handheld smart device platform 1 and other optional magnetometers 12. The attitude measurement system module 2131 uses an algorithm to select a nine-axis attitude determination algorithm or a six-axis attitude determination algorithm according to whether or not the geomagnetic information is available. Finally, the attitude measuring system module 2131 outputs the heading angle of the smart device 1 to the heading angle deviation compensation module 2132.

The heading angle deviation compensation module 2132 reads the heading angle deviation output by the heading angle deviation estimation module 212 and The compensation is given to the pedestrian heading angle and output to the dead reckoning algorithm module 2135. The specific calculation formula is as follows:

θ _p = θ _d + θ _offset . (1) where θ _p is the pedestrian heading angle, θ _d is the heading angle of the equipment, and θ _offset is the heading angle deviation.

The step detection module 2133 detects the step number of the pedestrian from the raw data of the IMU 11 of the handheld smart device platform 1 and feeds back to the step estimation module 2134. Pace detection can detect the pace by means of peak detection, zero-crossing detection, correlation detection and power spectrum detection. The present invention contemplates a variety of user motion modes and device usage modes. The step detection algorithm uses peak detection to simultaneously detect acceleration data and gyroscope data of the IMU 11.

The step estimation module 2134 estimates the step size of the pedestrian according to the result of the step detection module 2133 and the original data of the IMU 11 of the handheld smart device platform 1, and outputs the step to the dead reckoning module 2135. The step size estimation can be calculated by different methods such as acceleration integral, pendulum model, linear model, and empirical model. The present invention contemplates a variety of user motion patterns and device usage patterns, and the step size estimation uses the following linear model:

s _k-1,k =A·(f _k-1 +f _k )+B·(σ _acc,k-1 +σ _acc,k )+C (2)

Where A, B and C are constants, f _k-1 and f _k are the step frequencies at k-1 and k, σ _{acc, k-1} and σ _{acc, k} are the accelerations at k-1 and k The variance of the meter.

The dead reckoning module 2135 is based on the position [r _e,k-1 r _n,k-1 ] ^T at the k-1 time, the step information s _k-1,k output by the step estimation module 2134 _, and the heading angle offset compensation module. The heading angle information θ _k-1 outputted by 2132 derives the position [r _e,k r _n,k ] ^T at time _k . The corresponding calculation formula is as follows:

Finally, the dead reckoning module 2135 outputs an IMU position measurement to the fusion filter 3.

The fusion filter 3 includes a system model 31 and an observation model 32. The traditional multi-sensor fusion structure generally processes the IMU measurement data through an inertial mechanical orchestration algorithm and establishes a related fusion filter system model. Since there are many integral operations in the inertial mechanical arrangement, the positioning error of the conventional multi-sensor fusion structure is rapidly accumulated when there is no external auxiliary system. The present invention overcomes the shortcomings of the conventional multi-sensor fusion structure, and uses the pedestrian motion model as a system model, and the IMU related data is used as an observation model like other systems.

The fusion filter 3 can adopt Kalman Filter (KF), Adaptive Kalman Filter (AKF), UKF Lossless Kalman Filter (UKF) or particles. Filter (PF). The present invention gives a design example of KF. Other filters can refer to the KF design. The state vector implemented by the fusion filter 3 as KF is defined as follows:

x=[r _e r _n r _u v _e v _n v _u ] ^T (4)

Where re, rn and ru are three-dimensional positions (Northeast celestial coordinate system), and ve, vn and vu are corresponding three-dimensional velocity components. The KF system model 32 uses a classical kinematic model and is defined as follows:

x _k+1|k =Φ _k,k+1 x _k|k +ω _k (5)

Where x _k+1|k is the predicted state vector, x _k|k is the previous state vector at time _k, and Φ _k,k+1 is a 6×6 transition matrix:

Where Δt is the time difference between two moments. ω _k is the covariance matrix

The processing noise is defined as follows:

In the middle

with

It is the velocity noise in the east-to-day coordinate system at time k, which is modeled by random walk.

In the middle

with

It is the velocity noise of the eastward sky coordinate system at time k-1, n _e , n _n and n _u are Gaussian white noise, and Δt is the time difference between two moments.

The measurement model 31 implemented by the fusion filter 3 as KF is defined as follows:

z _k =H _k x _k|k +υ _k (9)

Where z _k is the measurement vector and H _k is the decision matrix. υ _k is the measurement noise with Gaussian white noise as the model, and its covariance matrix is

z _k and H _k vary depending on the measurement. When the observations are from IMU 11, the typical z _k and H _{k are} defined as follows:

When the observations are from magnetometer 12, the typical z _k and H _{k are} defined as follows:

z _k =[r _e r _n r _u ] ^T

When the observations are from the pressure gauge 13, the typical z _k and H _{k are} defined as follows:

z _k =r _u

H _k =[0 0 1 0 0 0] (12) When the observation is from WiFi 14, the typical z _k and H _{k are} defined as follows:

z _k =[r _e r _n r _u ] ^T

When the observations are from BLE15, the typical z _k and H _{k are} defined as follows:

z _k =[r _e r _n r _u ] ^T

When the observations are from GNSS16, the typical z _k and H _{k are} defined as follows:

z _k =[r _e r _n r _u v _e v _n v _u ] ^T

The KF process has two phases: forecasting and updating. In the prediction process, the state vector and the covariance matrix are predicted based on the system model.

During the update process, the state vector and covariance matrix are updated according to the measurement model:

Where K _{k is} called the Kalman gain.

While the invention has been shown and described with respect to the preferred embodiments of the present invention, it will be understood that

Claims

A pedestrian navigation device based on a novel multi-sensor fusion technology, comprising: a handheld smart device (1), a measurement processing unit (2), and a fusion filter (3); the handheld smart device (1) The observation raw data is acquired by its own hardware, and the observation processing unit (2) processes the raw data provided by the handheld smart device (1) to provide a position or velocity observation to the fusion filter (3), the fusion filter Using the kinematics model as the system model (32), the observation model (33) is established as a result of the observation processing unit, and the pedestrian navigation result is finally obtained through the processing of the fusion filter (3).
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 1, wherein the hardware of the handheld smart device 1 comprises: an inertial measurement unit IMU (11), a magnetometer (12), and a pressure gauge (13). ), WiFi (14), low energy Bluetooth BLE (15) and Global Navigation Satellite System GNSS (16); the IMU (11) provides raw data of acceleration and angular velocity; the magnetometer (12) provides the original geomagnetic Data; the pressure gauge (13) provides raw data of atmospheric pressure; the WiFi (14) provides raw data of WiFi received signal strength RSS; the BLE (15) provides raw data of BLERSS; the GNSS receiver ( 16) Providing raw data of the GNSS; the handheld smart device (1) may also include other sensors that provide observation information.
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 1, wherein the observation processing unit module (2) comprises: an IMU processing unit (21), a magnetometer processing unit (22), and a pressure gauge processing. a unit (23), a WiFi processing unit (24), a BLE processing unit (25), and a GNSS processing unit (26); the IMU processing unit (21) processes raw data of acceleration and angular velocity provided by the IMU (11) to Obtaining IMU position information and transmitting to the fusion filter (3); the magnetometer processing unit (22) processes the geomagnetic raw data provided by the magnetometer (12) to obtain geomagnetic position information and transmit the fusion to the fusion a filter (3); the pressure gauge processing unit (23) processes the raw data of the atmospheric pressure provided by the pressure gauge (13) to obtain elevation information and transmits to the fusion filter (3); the WiFi processing The unit processes (24) the RSS raw data provided by the WiFi (14) to obtain WiFi location information and transmits the information to the fusion filter (3); the BLE processing unit processes (25) the BLE (15) provided by the BLE (15) RSS raw data to obtain BLE location information and transmitted to the fusion filter The GNSS processing unit (26) processes the position and velocity information provided by the GNSS receiver (16) and transmits the information to the fusion filter (3); the observation processing unit module (2) further includes Other processing units obtain position or velocity information from other sensors that process the smart device platform and transmit them to the fusion filter (3).
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 1, wherein the IMU processing unit (21) comprises a user motion mode and a device usage pattern recognition module, and a heading angle deviation estimation a module, an improved dead reckoning algorithm module, the user motion mode and the device usage pattern recognition module identify user motions such as still, walking, running, etc. according to original data provided by the IMU of the handheld smart device platform and other optional hardware Mode and device usage mode of handheld, short message, telephone, navigation, pocket, backpack, etc.; the heading angle deviation estimation module according to the user motion mode and the device usage mode identification module output user motion mode and device usage mode and the handheld The raw data provided by the IMU of the smart device platform and other optional hardware estimates a heading angle deviation; the improved dead reckoning algorithm module estimates the heading angle deviation output by the heading angle deviation module and the IMU of the handheld smart device platform The raw data provided by other optional hardware is derived from the IMU location information and transmitted to the fusion filter.
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 4, wherein the improved dead reckoning algorithm module comprises a attitude measuring system module, a heading angle deviation compensation module, a step detecting module, and a step estimating module. a dead reckoning algorithm module, wherein the attitude determining system module identifies posture information of the handheld smart device according to the original data provided by the IMU of the handheld smart device platform and other optional magnetometers; the heading angle deviation compensation module reads The heading angle deviation of the heading angle deviation estimation module is compensated to the heading angle of the pedestrian and output to the dead reckoning algorithm; the step detecting module algorithm module detects the step number feedback of the pedestrian according to the original data of the IMU of the handheld smart device platform. a long estimation module; the step estimation module estimates the step size of the pedestrian according to the result of the step detection module and the original data of the IMU of the handheld smart device platform, and feeds back to the dead reckoning module; the dead reckoning module Step size information outputted by the step estimation module and the heading angle deviation compensation module input The pedestrian heading angle information is calculated to calculate the IMU position observation and fed back to the fusion filter.
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 4, wherein the user motion mode and the device usage pattern recognition module adopt a second classifier based on a Gaussian kernel dual type support vector machine. The sensor output of the handheld smart device is extracted during the periodic time interval, and the user motion mode and the mobile phone usage mode are identified and classified by using the support vector and the Gaussian kernel.
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 1, wherein: calculating an IMU, a magnetometer, an optional ranging, an output statistic of the optional photosensor, and a derived statistic thereof, It normalizes and principal component analysis for feature vectors, and intercepts the feature vectors corresponding to the feature values that retain most of the data variance to participate in the offline training of the support vector machine.
The pedestrian navigation device based on the novel multi-sensor fusion technology according to claim 1, wherein: the online classification calculation of the handheld intelligent device platform only needs to store and use a small number of support vectors after dimension reduction to perform user motion mode and device. Use the secondary classification of the pattern.
A pedestrian navigation method based on a novel multi-sensor fusion technology, comprising the following steps:

(1) Handheld smart devices use their own hardware to obtain raw data of IMU, magnetometer, pressure gauge, WiFi, BLE and GNSS;

(2) the observation processing unit processes the raw data provided by the handheld smart device to provide a position or velocity observation to the fusion filter;

(3) The fusion filter uses the kinematics model as the system model, and the observation and processing unit results to establish an observation model. After the fusion filter is processed, the pedestrian navigation result is finally obtained.
A pedestrian navigation method based on a novel multi-sensor fusion technology according to claim 9, wherein the observation processing unit processes the raw data provided by the handheld smart device to provide a position or velocity and the like to the fusion filter, including the following step:

The IMU processing unit processes raw data of acceleration and angular velocity provided by the IMU to obtain IMU position information and transmits the same to the fusion filter;

The magnetometer processing unit processes the geomagnetic raw data provided by the magnetometer to obtain geomagnetic position information and transmits the same to the fusion filter;

The pressure gauge processing unit processes raw data of atmospheric pressure provided by the pressure gauge to obtain elevation information and transmits the same to the fusion filter;

The WiFi processing unit processes the RSS original data provided by the WiFi to obtain WiFi location information and transmits the information to the fusion filter;

The BLE processing unit processes the RSS raw data provided by the BLE to obtain BLE location information and transmits the BLE location information to the fusion filter;

The GNSS processing unit processes raw data of the GNSS provided by the GNSS chip to obtain position and velocity information of the GNSS and transmits the information to the fusion filter.
A pedestrian navigation method based on a novel multi-sensor fusion technology according to claim 10, wherein the IMU processing unit processes the raw data of the acceleration and angular velocity provided by the IMU to obtain IMU position information and transmit the same to the fusion filter. Including the following steps:

The user motion mode and the device usage pattern recognition module identify user motion patterns such as still, walking, running, and the like, handheld, short message, telephone, navigation, according to original data provided by the IMU of the handheld smart device platform and other optional hardware. Pocket, backpack and other device usage modes;

The heading angle deviation estimation module outputs the output according to the user motion mode and the device usage pattern recognition module The heading angle deviation is estimated by the user motion mode and the device usage mode and raw data provided by the IMU of the handheld smart device platform and other optional hardware;

The improved dead reckoning algorithm module obtains IMU position information according to the heading angle deviation output by the heading angle deviation estimating module and the original data provided by the IMU of the handheld smart device platform and other optional hardware, and transmits the IMU position information to the fusion filter. .
A pedestrian navigation method based on a novel multi-sensor fusion technology according to claim 11, wherein the improved dead reckoning algorithm module comprises the following steps:

The attitude determining system module identifies posture information of the handheld smart device according to the original data provided by the IMU of the handheld smart device platform and other optional magnetometers;

The heading angle deviation compensation module reads the heading angle deviation outputted by the heading angle deviation estimating module and compensates the heading angle of the pedestrian to the dead reckoning algorithm;

The step detection module algorithm module detects the step number of the pedestrian feedback to the step estimation module according to the original data of the IMU of the handheld smart device platform;

The step estimation module estimates the step size of the pedestrian according to the result of the step detection module and the original data of the IMU of the handheld smart device platform, and feeds back to the dead reckoning module;

The dead reckoning module calculates the IMU position observation and feeds back to the fusion filter according to the step size information output by the step estimation module and the pedestrian heading angle information output by the heading angle offset compensation module.