CN114612523B - FPGA-based dynamic target tracking system and detection method thereof - Google Patents

FPGA-based dynamic target tracking system and detection method thereof Download PDF

Info

Publication number
CN114612523B
CN114612523B CN202210229821.7A CN202210229821A CN114612523B CN 114612523 B CN114612523 B CN 114612523B CN 202210229821 A CN202210229821 A CN 202210229821A CN 114612523 B CN114612523 B CN 114612523B
Authority
CN
China
Prior art keywords
data
module
image
steering engine
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210229821.7A
Other languages
Chinese (zh)
Other versions
CN114612523A (en
Inventor
陈龙
陈静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN202210229821.7A priority Critical patent/CN114612523B/en
Publication of CN114612523A publication Critical patent/CN114612523A/en
Application granted granted Critical
Publication of CN114612523B publication Critical patent/CN114612523B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a dynamic target tracking system based on FPGA and a detection method thereof, wherein a CMOS camera is used for collecting real-time video data, a VDMA is used for image caching, and then a programmable logic PL terminal is used for data processing: the frame difference processing module completes target detection through two adjacent frames of images, and further processes image data, and a minimum circumscribed rectangle of a moving object is drawn in the images; then establishing a mathematical model, and judging an effective frame of the data result obtained by the frame difference processing module; if the judging result is true, calculating a PWM wave duty ratio required by the steering engine with two degrees of freedom for rotating to the target position by the steering engine control module, transmitting the PWM wave duty ratio to the steering engine driving plate through serial port communication, and updating model data in the effective frame judging module; if the judgment result is negative, recalculating the next video data; the display module outputs the processed image data; the method and the device can improve the image processing speed, track the dynamic target in real time and reduce the interference influence of other objects in the picture.

Description

FPGA-based dynamic target tracking system and detection method thereof
Technical Field
The invention belongs to the field of dynamic target tracking, and particularly relates to a dynamic target detection method and a tracking system based on an FPGA.
Background
With the rapid development of computer technology, computer processing of visual information is widely focused on. Computer vision is a very intersecting discipline that incorporates image processing, computer graphics, pattern recognition, artificial intelligence, artificial neural networks, computers, psychology, physics, and mathematics. The purpose of computer vision research is to enable a computer to perceive, describe, store, identify and understand geometric information of objects in an environment, including its shape, position, pose, motion, etc. The research and application of the video target tracking method are taken as an important branch in the field of computer vision, and are increasingly widely applied to various fields of science and technology, national defense construction, aerospace, medical sanitation and national economy, so that the research target tracking technology has great practical value and wide development prospect.
The existing dynamic target tracking system has the following problems: 1) Processing speed is not ideal due to lack of parallel computing capability; 2) The real-time performance is low, and the target is easy to lose; 3) The expansibility and the portability are not ideal; 4) The identification error is larger, and when multiple objects exist in the picture, the interference processing on other objects needs to be optimized; 5) The equipment is heavy and inconvenient to move.
As a high-performance and low-power consumption Programmable chip, the Field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) is suitable for small-batch streaming application, is very suitable for processing the inference stage of deep learning, and can greatly shorten the development period. Along with the improvement of the accuracy and the instantaneity of moving object identification, the calculation complexity and the memory requirement of a deep learning algorithm are also increased sharply, and the current general processor cannot meet the calculation requirement of the deep learning algorithm, so that the real-time detection speed and the low-power consumption running state can be better achieved by using the FPGA for moving object identification, and the method is suitable for the actual application scene of moving object identification.
Disclosure of Invention
Technical problems: the invention aims to optimize the problems in the prior art, and provides a dynamic target detection method and a tracking system based on an FPGA, so as to achieve the purposes of improving the image processing speed, tracking the dynamic target in real time and reducing the interference of other objects in a picture.
The technical scheme is as follows: the invention discloses a dynamic target tracking system based on an FPGA, which comprises a data signal acquisition module, a data signal preprocessing module, a target identification module, a steering engine control module and a data signal display module;
The data signal acquisition module comprises a CMOS camera, a data stream conversion module, a VDMA data cache module and an acquisition module control unit; the CMOS camera achieves an original video data acquisition function, and the acquired video data is input into the data stream conversion module; the data stream conversion module realizes the function of converting RGB888 data into axi4 data stream format, and is convenient for the PL end transmission of the data on PYNQ-Z2 development board; the VDMA data buffer module realizes an image buffer function and can store two frames of images so as to process the subsequent images; the acquisition module control unit is realized at the PS end of the processing system, and completes the camera initialization and DDR3 memory control functions;
the data signal preprocessing module is used for extracting image data in the VDMA and converting the image data into gray image data;
The target identification module comprises a frame difference operation module and an effective frame judgment module; the frame difference operation module performs frame difference operation and binarization processing on the gray level image, and fills the hole and smoothes the boundary through closed operation; the effective frame judging module judges the result obtained by the frame difference operation module by establishing a mathematical model;
the steering engine control module comprises a duty ratio calculation module and a serial port communication module; the duty ratio calculation module calculates the duty ratio required by the steering engine according to the data judged by the effective frame, so as to realize tracking of the object; the serial port communication module is used for realizing communication with the steering engine driving plate and completing control of the steering engine with two degrees of freedom; the steering engine with two degrees of freedom can rotate in the X direction and the Y direction, and the rotating angle is controlled by the duty ratio of an input PWM wave;
The data signal display module comprises a processing module and an image conversion module which are externally connected with a rectangular frame, so that a moving area is marked by the externally connected rectangular frame, and RGB888 image data is converted into a DVI video format, thereby being convenient for intuitively knowing the position of a dynamic target and subsequent debugging;
Firstly, recording real-time video data by a data signal acquisition module, then converting a data format by a data signal preprocessing module, then finding out an effective movement area by a target recognition module, finally calculating the duty ratio of PWM waves required by steering engine rotation by a steering engine control module and transmitting the duty ratio to a steering engine driving plate, and meanwhile, outputting the data processed by the target recognition module by a data signal display module, thereby facilitating the debugging of the system.
Wherein,
The detection method comprises the following steps:
Step S1: the camera module collects real-time video data, the data stream conversion module converts RGB888 data into axi4 data stream format, the data is transmitted at the PL end of PYNQ-Z2 development board, two continuous frames of images are buffered in the VDMA data buffer module, and then frame difference processing is carried out;
step S2: expanding and then corroding the image data after the frame difference treatment to realize filling of tiny holes in the object and connecting of adjacent objects and smooth boundary functions;
Step S3: searching the boundary in the image processed in the step S2, drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object; the resulting motion region parameters (X 12,X22,X32,X42) and target parameters (X 11,X21,X31,X41);
Step S4: establishing a similarity measurement mathematical model, judging the similarity between the shape and the area of the circumscribed rectangular frame and the center and the original tracking target, if the judging result is true, continuing to step S5, if not, returning to step S1;
Step S5: updating the length and width of the external rectangular frame and the central data of the original tracked object, calculating the duty ratio of the steering engine according to the central position of the object obtained in the step S4, and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
The step S1 comprises the following specific steps:
step S11: enabling two EMIO interfaces of the FPGA development board to simulate the SCCB time sequence to control the reading and writing of an SCCB bus, and further completing the configuration of a camera register;
Step S12: the 8-bit camera input data are spliced into 16-bit RGB565 data, and VIVADO is used for converting RGB888 data into an axi4 data stream format by using a Video in to axi4-stream IP core, so that the data can be conveniently transmitted at the PL end of a PYNQ-Z2 development board;
step S13: caching two continuous frames of images in PYNQ-Z2 development board-mounted DDR 3;
Step S14: reading an Nth frame and an (n+1) th frame of images from VDMA, converting the two frames of images into gray images, subtracting the two frames of images, and performing binarization processing:
Wherein, P 1 (x, y) represents the difference between the n+1st frame and the Nth frame gray scale image; (x, y) represents the abscissa position of the pixel point; y (x, Y) represents a luminance component in the YCbCr format of the gradation image; t represents a set threshold value.
Step S2 is performed by expanding and then corroding to finish closing operation; realizing the functions of filling tiny holes in an object, connecting adjacent objects, smoothing the boundary of the object without obviously changing the area of the object and simultaneously inhibiting details smaller than structural units,
The expansion is carried out, a set A is defined as a coordinate position of a pixel point with a binarized value of 255, a structural element B is a rectangle of 3*3, an origin of the structural element B is continuously translated, if an intersection of B and A is not empty, the origin position of B is recorded, and the set is recorded as a set C;
The corrosion, defining structural element D as The origin of the structural element D is continuously translated, the set of which D is still contained in C is recorded as a set E, and the set E is the closed operation result.
The step S3 comprises the following specific steps:
Step S31: performing area detection on the result of the step S2, and removing the connected region with the area smaller than the threshold T 2 to obtain a complete movement region;
step S32: traversing the complete motion area obtained in the step S31 to obtain the total number of image pixel points;
step S33: performing horizontal projection on the image, recording the number of pixel points of each row of the image on an X axis, and finding out a left boundary and a right boundary of a motion area, and subtracting X values of the right boundary and the left boundary to obtain the length of a rectangle circumscribed by the motion area;
step S34: the number of the X-axis pixel points recorded in the step S33 is subjected to weighted accumulation, namely the number of each row of pixel points is multiplied by the corresponding X-axis coordinate and added;
step S35: calculating the horizontal center point of the motion area, dividing the weighted accumulated value of the step S34 by the total number of the pixel points of the image traversed by the step S32, and obtaining a horizontal center point coordinate x of the motion area;
step S36: performing vertical projection on the image, recording the number of pixel points of each row of the image on a Y axis, and finding out the upper boundary and the lower boundary of the motion area, wherein Y values of the upper boundary and the lower boundary are subtracted to obtain the width of a rectangle circumscribed by the motion area;
step S37: the number of the Y-axis pixel points recorded in the step S36 is subjected to weighted accumulation, namely, the number of each row of pixel points is multiplied by the corresponding Y-axis coordinates and added;
step S38: and calculating the vertical center point of the motion area, dividing the weighted accumulated value of the step S37 by the total number of the pixel points of the image traversed by the step S32, and obtaining the vertical center point coordinate y of the motion area.
The step S4 comprises the following specific steps:
Step S41: taking the cosine of the included angle between the motion area parameter (X 12,X22,X32,X42) and the target parameter (X 11,X21,X31,X41) obtained in the step S3 as a similarity measure, recording the original target variable as (X 11,X21,X31,X41), and carrying out cosine similarity calculation with the motion area variable obtained in the step S3 as (X 12,X22,X32,X42):
Wherein (X 11,X21) represents the center coordinate position of the original target, X 31 represents the length of the circumscribed rectangle of the original target, X 41 represents the width of the circumscribed rectangle of the original target, (X 12,X22) represents the center coordinate position (X, y) of the motion area obtained in the step S3, X 32 represents the length of the circumscribed rectangle of the motion area obtained in the step S3, and X 42 represents the width of the circumscribed rectangle of the motion area obtained in the step S3; x i1 represents the original target ith variable, and X i2 represents the ith variable of the motion area;
If r 12 is greater than the threshold T3, the motion area obtained in step S3 is considered to be similar to the original target, and step S42 is entered, otherwise, the result obtained in step S3 is determined to be an invalid image, and step S1 is returned;
step S42: the movement area obtained in the step S3 is circumscribed with the rectangular frame length and width calculation area, area judgment is carried out on the area of the external rectangular frame with the original target:
If r 2 is smaller than the threshold T4, the area is determined to be true, and the process proceeds to step S5, whereas the result of determination S3 is an invalid image, and the process returns to step S1.
The step S5 comprises the following specific steps:
Step S51: updating the length and width of the external rectangular frame and the central position data of the original tracking object according to the result obtained in the step S3;
Step S52: the steering engine control comprises two parts of numerical conversion and serial port communication control, the numerical conversion is completed by determining the central position of a moving target to realize the conversion from the rotation angle required by the tracking of a camera to the duty ratio of the PWM wave controlled and output by a driving plate, because the system uses the camera with smaller visual angle, the relation between the pixel position in an image and the rotation angle of the steering engine, namely the relation between the duty ratio and the horizontal and vertical coordinates of the central position is fitted by using a curve,
Wherein A 1,A2,C1,C2 is a curve parameter corresponding to the duty ratio and the pixel position, which is obtained by fitting a plurality of groups of data measured by upper computer software and a steering engine driving board;
Step S53: and (3) using serial communication to bring the calculated duty ratio conversion value calculated by the formula (4) and the time required by rotating the designated angle set by the user into the driving plate by the central coordinate position (X 12,X22) of the movement area, so as to realize the tracking of the dynamic target.
The beneficial effects are that: compared with the prior art, the invention has the following advantages:
(1) The invention is built on a PYNQ-Z2 expandable platform of Xilinx company based on ZYNQ architecture, and completes frame difference processing, effective frame judgment, image processing output, steering engine control and serial port communication at PL end, and completes CMOS camera register initialization and VDMA initialization at PS end. The method not only fully utilizes the strong parallel operation capability and logic resources of PYNQ-Z2, but also combines the advantages of ARM core in the aspects of constructing an operating system and realizing complex algorithm, improves the operation speed and reduces the system volume;
(2) According to the invention, by establishing a mathematical model and combining similarity judgment and area judgment, invalid data is eliminated, interference when a single target is tracked and a plurality of objects exist in the background is reduced, and the stability of the system is improved.
(3) The invention realizes a miniaturized moving object tracking system, and has low power consumption, small volume and portability.
(4) The hardware price of the present invention is advantageous over that of a PC.
Drawings
FIG. 1 is a hardware acceleration data flow diagram of the present invention.
FIG. 2 is a flow chart of the detection and determination of a moving object according to the present invention.
Detailed Description
For a better understanding of the invention, the invention is further described below with reference to examples and drawings.
The invention relates to a dynamic target detection method and a tracking system based on an FPGA, wherein the system comprises a PYNQ-Z2 development board, a power line, an OV5640CMOS camera, a liquid crystal display, a steering engine driving board, a two-degree-of-freedom steering engine and an HDMI line;
PYNQ-Z2 is a development platform supporting PYNQ open source framework, and can support Python to carry out APSoC programming in addition to the traditional ZYNQ development mode. PYNQ-Z2 development board takes ZYNQ XC7Z020 FPGA as core, is equipped with Ethernet, HDMI input/output, MIC input, audio output, arduino interface, raspberry group interface, 2 Pmod, on-board DDR3, user LED, button and switch.
The ASPSoC chip comprises a programmable logic PL end and a processing system PS end, wherein the PS end part consists of a dual-core ARM Cortex-A9 processor, two Neon coprocessors and the like, a large number of peripheral devices and memory controllers are integrated in the PS end part, and the PL end is a programmable logic device and realizes the functions of frame difference processing, effective frame judgment, steering engine control and the like.
The power line is used for supplying power to the PYNQ-Z2 development board;
the system hardware acceleration data flow diagram is shown in figure 1;
The OV5640CMOS camera is connected with a PMOD interface of the PYNQ-Z2 development board through a self-welded adapter board, and two EMIO interfaces simulate SCCB time sequence to control the reading and writing of an SCCB bus, so that the configuration of a camera register is completed;
the Tx and Rx interfaces of the steering engine driving board are connected with GPIO ports of the PYNQ-Z2 development board, so that serial communication between the PYNQ-Z2 development board and the steering engine driving board is realized;
The HDMI line is connected with an HDMI OUT interface of the PYNQ-Z2 development board and the liquid crystal display;
the DDR3 memory controller is connected with the ASPSoC chip and the SD Card interface;
The ASPSoC chip is connected with an HDMI OUT interface;
the PS end of the processing system is connected with the DDR3 memory controller;
the AXI data transmission bus is used for connecting the PL and PS ends, and the AXI bus realizes data communication between the ARM and the FPGA, so that the transmission speed between the PL and PS ends is effectively improved;
The programmable logic PL end comprises a VDMA data storage module, a frame difference processing module, an effective frame judging module, a steering engine control module and a display output module;
the VDMA data storage module is connected with the AXI data transmission bus and is used for accessing video stream data;
the frame difference processing module is used for processing the VDMA image data;
The effective frame judging module is used for judging the effective frame according to the result of the frame difference processing module and eliminating invalid data;
The steering engine control module is connected with the steering engine driving plate through a GPIO interface and is used for calculating the duty ratio of the steering engine and carrying out serial port communication with the steering engine driving plate;
the display output module is used for outputting the processed image, so that the system is convenient to debug;
A dynamic target detection method based on FPGA includes the following steps:
S1, a CMOS camera collects real-time video data and then performs frame difference processing. Firstly, two EMIO interfaces of the FPGA development board are enabled to simulate the SCCB time sequence to control the reading and writing of the SCCB bus, and then the configuration of the camera register is completed. In this embodiment, the OV5640 output pixel is set to 640×480, and the image frame number is 40 frames/sec;
Then splicing 8-bit camera input data into 16-bit RGB565 data, converting RGB888 data into an axi4 data stream format by using VIVADO self-contained Video in to axi4-stream IP core, facilitating the transmission of data at the PL end of a PYNQ-Z2 development board, and caching two continuous frames of images in a PYNQ-Z2 development board-mounted DDR 3. Finally, the Nth frame and the (n+1) th frame of image are read from the VDMA, and the two frames of images are converted into gray images.
Subtracting the two frames of images, and performing binarization processing:
Wherein P1 (x, y) represents the difference between the n+1st frame and the nth frame gray scale image; (x, y) represents the abscissa position of the pixel point; y (x, Y) represents a luminance component in the YCbCr format of the gradation image; t represents a set threshold (t=100).
S2, expanding and then corroding to finish the closing operation. Filling small holes in the object, connecting adjacent objects, smoothing the boundary of the object without obviously changing the area of the object, and simultaneously inhibiting details smaller than the structural units to obtain an Image1.
S3, searching the boundary in the image processed in the step S2, drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object. Firstly, performing area detection on the result Image1 in the step S2, and removing the connected area with the area smaller than the threshold T2 to obtain a complete motion area Image. And traversing the obtained complete motion area Image to obtain the total number of Image pixel points. And then horizontally projecting the image, recording the number of pixel points of each row of the image on the X axis, and finding out the left boundary and the right boundary of the motion area. And then carrying out weighted accumulation on the number of the X-axis pixel points, namely multiplying the number of the pixel points in each row by the corresponding X-axis coordinate, and adding. Finally, calculating the horizontal center point of the motion area, dividing the weighted accumulated value of the horizontal projection by the total number of the traversed image pixels, and obtaining the horizontal center point coordinate x of the motion area. And vertically projecting the image, recording the number of pixel points of each row of the image on the Y axis, and finding out the upper boundary and the lower boundary of the motion area. And then carrying out weighted accumulation on the number of the recorded Y-axis pixel points, namely multiplying the number of the pixel points in each row by the corresponding Y-axis coordinates, and adding. Finally, calculating a vertical center point of the motion area, dividing the weighted accumulated value of the vertical projection by the total number of the traversed image pixel points, and obtaining a vertical center point coordinate y of the motion area as a result;
And S4, establishing a mathematical model, and judging the similarity between the center and the original tracking target according to the shape and the area of the circumscribed rectangular frame. Firstly, taking the cosine of the included angle between the motion area parameter obtained in the step S3 and the target parameter as a similarity measure, recording the original target variable as (X 11,X21,X31,X41), obtaining the motion area variable obtained in the step S3 as (X 12,X22,X32,X42), and performing cosine similarity calculation:
Wherein (X 11,X21) represents the center coordinate position of the original target, X 31 represents the length of the circumscribed rectangle of the original target, X 41 represents the width of the circumscribed rectangle of the original target, (X 12,X22) represents the center coordinate position of the movement region obtained in the step S3, X 32 represents the length of the circumscribed rectangle of the movement region obtained in the step S3, and X 42 represents the width of the circumscribed rectangle of the movement region obtained in the step S3;
If r 12 is greater than the threshold T 3, the motion area obtained in step S3 is considered to be similar to the original target, otherwise, the result obtained in step S3 is determined to be an invalid image, and the step S1 is returned;
Then, the movement area obtained in the step S3 is circumscribed with the length and width calculation area of the rectangular frame, area judgment is carried out on the area of the external rectangular frame with the original target:
if r 2 is smaller than the threshold T 4, the area is determined to be true, and the process proceeds to step S5, otherwise, the result obtained in step S3 is determined to be an invalid image, and the process returns to step S1;
And S5, updating the length and width of the external rectangular frame and the central data of the original tracked object, calculating the duty ratio of the steering engine according to the central position of the object obtained in the step S4, and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
Firstly, updating the length and width of the circumscribed rectangular frame of the original tracking object and the central position data according to the result obtained in the step S3.
And then steering engine control is carried out, and the steering engine control mainly comprises two parts of numerical conversion and serial port communication control. The conversion from the rotation angle required by the tracking of the camera to the duty ratio of the PWM wave controlled and output by the driving plate is completed by the numerical conversion. Because the system uses the camera with smaller visual angle, the relation between the pixel position in the once curve fitting image and the steering engine rotation angle, namely the relation between the duty ratio and the central position horizontal and vertical coordinates:
wherein A 1,A2,C1,C2 is obtained by fitting a plurality of groups of data measured by upper computer software and a steering engine driving plate, in the embodiment, the fitting result is as follows:
And finally, using serial port communication, transmitting the calculated angle conversion value and the time required by rotating the designated angle to a driving plate, and realizing the tracking of the dynamic target.
The following is a specific embodiment of the system, in which the system processes the image consistently, and different functions are realized by changing devices installed at the top of the steering engine with two degrees of freedom, but the scheme of the invention is not limited to the following application, and is not described here in detail by way of example.
Example 1:
the steering engine with two degrees of freedom is controlled by PWM waves output by a steering engine driving plate, a USB camera is arranged at the top end of the steering engine, the steering engine is connected with a computer end, and a tracking result is observed by the computer;
The USB camera is LRCP1080 P_OV2710, and the photosensitive chip is 1/2.7inch; pixel Size 3um to 3um;
(1) The camera module collects real-time video data, the data stream conversion module converts RGB888 data into axi4 data stream format, the data is conveniently transmitted at the PL end of PYNQ-Z2 development board, two continuous frames of images are buffered in the VDMA data buffer module, and then frame difference processing is carried out.
(2) Expanding and then corroding the image data after the frame difference treatment, filling black holes in the object, and connecting adjacent objects with smooth boundaries;
(3) Searching the boundary in the image processed in the step (2), drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object;
(4) Establishing a mathematical model, judging the similarity between the shape and the area of the circumscribed rectangular frame and the center and the original tracking target, if the judging result is true, continuing to carry out the step (5), and if not, returning to the step (1);
(5) Updating the length, width and center data of the external rectangular frame of the original tracked object, calculating the duty ratio of the steering engine according to the center position of the object obtained in the step (4), and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
The object is always positioned in the video center recorded by the USB camera at the top end of the steering engine, and real-time monitoring of a moving target can be observed at the computer end connected with the USB camera.
Example 2:
The two-degree-of-freedom steering engine is controlled by PWM waves output by a steering engine driving plate, and a laser is arranged at the top end of the steering engine, so that a laser spot is always positioned on a moving target;
(1) The camera module collects real-time video data, the data stream conversion module converts RGB888 data into axi4 data stream format, the data is conveniently transmitted at the PL end of PYNQ-Z2 development board, two continuous frames of images are buffered in the VDMA data buffer module, and then frame difference processing is carried out.
(2) Expanding and then corroding the image data after the frame difference treatment, filling black holes in the object, and connecting adjacent objects with smooth boundaries;
(3) Searching the boundary in the image processed in the step (2), drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object;
(4) Establishing a mathematical model, judging the similarity between the shape and the area of the circumscribed rectangular frame and the center and the original tracking target, if the judging result is true, continuing to carry out the step (5), and if not, returning to the step (1);
(5) Updating the length, width and center data of the external rectangular frame of the original tracked object, calculating the duty ratio of the steering engine according to the center position of the object obtained in the step (4), and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
The laser at the top end of the steering engine always irradiates on the target no matter the moving target moves in any track, so that the real-time tracking of the moving target is realized.
Example 3:
the steering engine with two degrees of freedom is controlled by PWM waves output by a steering engine driving plate, and a pen is arranged at the top end of the steering engine, so that handwriting can be realized at intervals.
(1) The camera module collects real-time video data, tracks hand information, the data stream conversion module converts RGB888 data into axi4 data stream format, the data is conveniently transmitted at the PL end of PYNQ-Z2 development board, two continuous frames of images are buffered in the VDMA data buffer module, and then frame difference processing is carried out.
(2) Expanding and then corroding the image data after the frame difference treatment, filling black holes in the object, and connecting adjacent objects with smooth boundaries;
(3) Searching the boundary in the image processed in the step (2), drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object;
(4) Establishing a mathematical model, judging the similarity between the shape and the area of the circumscribed rectangular frame and the center and the original tracking target, if the judging result is true, continuing to carry out the step (5), and if not, returning to the step (1);
(5) Updating the length, width and center data of the external rectangular frame of the original tracked object, calculating the duty ratio of the steering engine according to the center position of the object obtained in the step (4), and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
The COMS camera tracks hand movement information, and the pen at the top end of the COMS camera is driven to rotate by the steering engine with two degrees of freedom, so that space-free handwriting is realized.

Claims (6)

1. A dynamic target detection method of a dynamic target tracking system based on FPGA is characterized by comprising the following steps: the detection method comprises the following steps:
Step S1: the camera module collects real-time video data, the data stream conversion module converts RGB888 data into axi4 data stream format, the data is transmitted at the PL end of PYNQ-Z2 development board, two continuous frames of images are buffered in the VDMA data buffer module, and then frame difference processing is carried out;
step S2: expanding and then corroding the image data after the frame difference treatment to realize filling of tiny holes in the object and connecting of adjacent objects and smooth boundary functions;
Step S3: searching the boundary in the image processed in the step S2, drawing the circumscribed rectangular frame of the moving object, and calculating the center of the moving object; the resulting motion region parameters (X 12,X22,X32,X42) and target parameters (X 11,X21,X31,X41);
Step S4: establishing a similarity measurement mathematical model, judging the similarity between the shape and the area of the circumscribed rectangular frame and the center and the original tracking target, if the judging result is true, continuing to step S5, if not, returning to step S1;
Step S41: taking the cosine of the included angle between the motion area parameter (X 12,X22,X32,X42) and the target parameter (X 11,X21,X31,X41) obtained in the step S3 as a similarity measure, recording the original target variable as (X 11,X21,X31,X41), and carrying out cosine similarity calculation with the motion area variable obtained in the step S3 as (X 12,X22,X32,X42):
Wherein (X 11,X21) represents the center coordinate position of the original target, X 31 represents the length of the circumscribed rectangle of the original target, X 41 represents the width of the circumscribed rectangle of the original target, (X 12,X22) represents the center coordinate position (X, y) of the motion area obtained in the step S3, X 32 represents the length of the circumscribed rectangle of the motion area obtained in the step S3, and X 42 represents the width of the circumscribed rectangle of the motion area obtained in the step S3; x i1 represents the original target ith variable, and X i2 represents the ith variable of the motion area;
If r 12 is greater than the threshold value T3, the motion area obtained in the step S3 is considered to be similar to the original target, and the step S42 is carried out, otherwise, the result obtained in the step S3 is judged to be an invalid image, and the step S1 is returned;
step S42: the movement area obtained in the step S3 is circumscribed with the rectangular frame length and width calculation area, area judgment is carried out on the area of the external rectangular frame with the original target:
If r 2 is smaller than the threshold T4, the area is judged to be true, and the process proceeds to step S5, otherwise, the result obtained in step S3 is judged to be an invalid image, and the process returns to step S1;
Step S5: updating the length and width of the external rectangular frame and the central data of the original tracked object, calculating the duty ratio of the steering engine according to the central position of the object obtained in the step S4, and transmitting the duty ratio into a steering engine driving plate through serial port communication to adjust the angle of the steering engine.
2. The dynamic object detection method according to claim 1, wherein: the step S1 comprises the following specific steps:
step S11: enabling two EMIO interfaces of the FPGA development board to simulate the SCCB time sequence to control the reading and writing of an SCCB bus, and further completing the configuration of a camera register;
Step S12: the 8-bit camera input data are spliced into 16-bit RGB565 data, and VIVADO is used for converting RGB888 data into an axi4 data stream format by using a Video in to axi4-stream IP core, so that the data can be conveniently transmitted at the PL end of a PYNQ-Z2 development board;
step S13: caching two continuous frames of images in PYNQ-Z2 development board-mounted DDR 3;
Step S14: reading an Nth frame and an (n+1) th frame of images from VDMA, converting the two frames of images into gray images, subtracting the two frames of images, and performing binarization processing:
Wherein, P 1 (x, y) represents the difference between the n+1st frame and the Nth frame gray scale image; (x, y) represents the abscissa position of the pixel point; y (x, Y) represents a luminance component in the YCbCr format of the gradation image; t represents a set threshold value.
3. The dynamic object detection method according to claim 1, wherein: step S2 is performed by expanding and then corroding to finish closing operation; realizing the functions of filling fine cavities of objects, connecting adjacent objects, smoothing the boundary of the objects without obviously changing the area of the objects and simultaneously inhibiting details smaller than structural units,
The expansion is carried out, a set A is defined as a coordinate position of a pixel point with a binarized value of 255, a structural element B is a rectangle of 3*3, an origin of the structural element B is continuously translated, if an intersection of B and A is not empty, the origin position of B is recorded, and the set is recorded as a set C;
The corrosion, defining structural element D as The origin of the structural element D is continuously translated, the set of which D is still contained in C is recorded as a set E, and the set E is the closed operation result.
4. The dynamic object detection method according to claim 1, wherein: the step S3 comprises the following specific steps:
Step S31: performing area detection on the result of the step S2, and removing the connected region with the area smaller than the threshold T 2 to obtain a complete movement region;
step S32: traversing the complete motion area obtained in the step S31 to obtain the total number of image pixel points;
step S33: performing horizontal projection on the image, recording the number of pixel points of each row of the image on an X axis, and finding out a left boundary and a right boundary of a motion area, and subtracting X values of the right boundary and the left boundary to obtain the length of a rectangle circumscribed by the motion area;
step S34: the number of the X-axis pixel points recorded in the step S33 is subjected to weighted accumulation, namely the number of each row of pixel points is multiplied by the corresponding X-axis coordinate and added;
step S35: calculating the horizontal center point of the motion area, dividing the weighted accumulated value of the step S34 by the total number of the pixel points of the image traversed by the step S32, and obtaining a horizontal center point coordinate x of the motion area;
step S36: performing vertical projection on the image, recording the number of pixel points of each row of the image on a Y axis, and finding out the upper boundary and the lower boundary of the motion area, wherein Y values of the upper boundary and the lower boundary are subtracted to obtain the width of a rectangle circumscribed by the motion area;
step S37: the number of the Y-axis pixel points recorded in the step S36 is subjected to weighted accumulation, namely, the number of each row of pixel points is multiplied by the corresponding Y-axis coordinates and added;
step S38: and calculating the vertical center point of the motion area, dividing the weighted accumulated value of the step S37 by the total number of the pixel points of the image traversed by the step S32, and obtaining the vertical center point coordinate y of the motion area.
5. The dynamic object detection method according to claim 1, wherein: the step S5 comprises the following specific steps:
Step S51: updating the length and width of the external rectangular frame and the central position data of the original tracking object according to the result obtained in the step S3;
Step S52: the steering engine control comprises two parts of numerical conversion and serial port communication control, the numerical conversion is completed by determining the central position of a moving target to realize the conversion from the rotation angle required by the tracking of a camera to the duty ratio of the PWM wave controlled and output by a driving plate, because the system uses the camera with smaller visual angle, the relation between the pixel position in an image and the rotation angle of the steering engine, namely the relation between the duty ratio and the horizontal and vertical coordinates of the central position is fitted by using a curve,
Wherein A 1,A2,C1,C2 is a curve parameter corresponding to the duty ratio and the pixel position, which is obtained by fitting a plurality of groups of data measured by upper computer software and a steering engine driving board;
Step S53: and (3) using serial communication to bring the calculated duty ratio conversion value calculated by the formula (4) and the time required by rotating the designated angle set by the user into the driving plate by the central coordinate position (X 12,X22) of the movement area, so as to realize the tracking of the dynamic target.
6. An FPGA-based dynamic target tracking system for use in the dynamic target detection method of claim 1, characterized by: the system comprises a data signal acquisition module, a data signal preprocessing module, a target identification module, a steering engine control module and a data signal display module;
The data signal acquisition module comprises a CMOS camera, a data stream conversion module, a VDMA data cache module and an acquisition module control unit; the CMOS camera achieves an original video data acquisition function, and the acquired video data is input into the data stream conversion module; the data stream conversion module realizes the function of converting RGB888 data into axi4 data stream format, and is convenient for the PL end transmission of the data on PYNQ-Z2 development board; the VDMA data buffer module realizes an image buffer function and can store two frames of images so as to process the subsequent images; the acquisition module control unit is realized at the PS end of the processing system, and completes the camera initialization and DDR3 memory control functions;
the data signal preprocessing module is used for extracting image data in the VDMA and converting the image data into gray image data;
The target identification module comprises a frame difference operation module and an effective frame judgment module; the frame difference operation module performs frame difference operation and binarization processing on the gray level image, and fills the hole and smoothes the boundary through closed operation; the effective frame judging module judges the result obtained by the frame difference operation module by establishing a mathematical model;
the steering engine control module comprises a duty ratio calculation module and a serial port communication module; the duty ratio calculation module calculates the duty ratio required by the steering engine according to the data judged by the effective frame, so as to realize tracking of the object; the serial port communication module is used for realizing communication with the steering engine driving plate and completing control of the steering engine with two degrees of freedom; the steering engine with two degrees of freedom can rotate in the X direction and the Y direction, and the rotating angle is controlled by the duty ratio of an input PWM wave;
The data signal display module comprises a processing module and an image conversion module which are externally connected with a rectangular frame, so that a moving area is marked by the externally connected rectangular frame, and RGB888 image data is converted into a DVI video format, thereby being convenient for intuitively knowing the position of a dynamic target and subsequent debugging;
Firstly, recording real-time video data by a data signal acquisition module, then converting a data format by a data signal preprocessing module, then finding out an effective movement area by a target recognition module, finally calculating the duty ratio of PWM waves required by steering engine rotation by a steering engine control module and transmitting the duty ratio to a steering engine driving plate, and meanwhile, outputting the data processed by the target recognition module by a data signal display module, thereby facilitating the debugging of the system.
CN202210229821.7A 2022-03-10 2022-03-10 FPGA-based dynamic target tracking system and detection method thereof Active CN114612523B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210229821.7A CN114612523B (en) 2022-03-10 2022-03-10 FPGA-based dynamic target tracking system and detection method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210229821.7A CN114612523B (en) 2022-03-10 2022-03-10 FPGA-based dynamic target tracking system and detection method thereof

Publications (2)

Publication Number Publication Date
CN114612523A CN114612523A (en) 2022-06-10
CN114612523B true CN114612523B (en) 2024-11-08

Family

ID=81860343

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210229821.7A Active CN114612523B (en) 2022-03-10 2022-03-10 FPGA-based dynamic target tracking system and detection method thereof

Country Status (1)

Country Link
CN (1) CN114612523B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115278525B (en) * 2022-08-09 2023-03-28 成都航空职业技术学院 Method and system for simplifying cluster moving object continuous space-time positioning data

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107516296A (en) * 2017-07-10 2017-12-26 昆明理工大学 A kind of moving object detection tracking system and method based on FPGA
CN113989683A (en) * 2021-09-16 2022-01-28 中国科学院空天信息创新研究院 Ship detection method for synthesizing synchronous orbit sequence optical image space-time information

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106910203B (en) * 2016-11-28 2018-02-13 江苏东大金智信息系统有限公司 The quick determination method of moving target in a kind of video surveillance

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107516296A (en) * 2017-07-10 2017-12-26 昆明理工大学 A kind of moving object detection tracking system and method based on FPGA
CN113989683A (en) * 2021-09-16 2022-01-28 中国科学院空天信息创新研究院 Ship detection method for synthesizing synchronous orbit sequence optical image space-time information

Also Published As

Publication number Publication date
CN114612523A (en) 2022-06-10

Similar Documents

Publication Publication Date Title
Nieto et al. Real-time lane tracking using Rao-Blackwellized particle filter
US20080240497A1 (en) Method for tracking objects in videos using forward and backward tracking
CN110610510A (en) Target tracking method and device, electronic equipment and storage medium
CN108427924A (en) A kind of text recurrence detection method based on rotational sensitive feature
Yang et al. Using lightweight convolutional neural network to track vibration displacement in rotating body video
CN101299233A (en) Device and method for realizing moving object identification and track based on FPGA
US20220262093A1 (en) Object detection method and system, and non-transitory computer-readable medium
Sanchez-Ferreira et al. Background subtraction algorithm for moving object detection in FPGA
CN111860483B (en) Target detection method based on Haisi platform
WO2020082936A1 (en) Object detection method and apparatus
US20220277581A1 (en) Hand pose estimation method, device and storage medium
CN110827320B (en) Target tracking method and device based on time sequence prediction
CN114612523B (en) FPGA-based dynamic target tracking system and detection method thereof
CN111008994A (en) Moving target real-time detection and tracking system and method based on MPSoC
CN111046746A (en) License plate detection method and device
CN109708662B (en) High-frame-frequency high-precision injection type star atlas simulation test platform based on target identification
CN111340841A (en) Real-time multi-moving-object detection method and system
CN105282399B (en) Zero-lag electronic image stabilization method based on digital circuit and sparse point registration
CN209897174U (en) Video image processing device based on FPGA
Zhang et al. Hierarchical and parallel pipelined heterogeneous SoC for embedded vision processing
CN112053384B (en) Target tracking method based on bounding box regression model
CN111242835B (en) Feature edge detection system and method based on ZYNQ and GPU
CN114245102A (en) Vehicle-mounted camera shake identification method and device and computer readable storage medium
Madhekar et al. Real time object detection and tracking using Raspberry Pi
CN115115605B (en) Method and system for realizing circle detection based on Hough transformation of ZYNQ

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant