TWI812102B - Method for two unmanned vehicles cooperatively navigating and system thereof - Google Patents

Abstract

A method for two unmanned vehicles cooperatively navigating is provided to solve poor efficiency on route planning by one single unmanned ground vehicle in conventional way. The method of the present invention includes the following steps. A computer receives a map image having a first resolution. By a preset resolution adjusting regulation, the computer decreases the first resolution to a second resolution. In the preset resolution adjusting regulation, the decreased second resolution is positively correlated with an image capturing height of the map image. The computer receives a start-point data and an end-point data, and then analyzes said point data and the map image to generate a route image having a route trajectory with said point data.

Description

2. Unmanned vehicle cooperative navigation method and system

The present invention relates to a navigation method and system, in particular to a cooperative navigation method and system for two unmanned vehicles.

In the conventional technology, when an unmanned vehicle is used to cross obstacles or detect the environment in a field, especially if the terrain and obstacles of the field are unknown, the unmanned vehicle will conduct detailed path exploration for the field. , to obtain the corresponding path planning and environment detection. However, the aforementioned process of path exploration requires the unmanned vehicle to actually walk through or photograph the specific site to a certain extent before it can build map data or models of the site and then perform path planning, so it will be wasted. It takes a considerable amount of time, and the effect of path planning or specific destination environment detection cannot be achieved immediately.

In view of this, the conventional autonomous vehicle navigation technology still needs to be improved.

In order to solve the above problems, the purpose of the present invention is to provide a cooperative navigation method and system for two unmanned vehicles, which can utilize the cooperative cooperation of unmanned vehicles and drones to greatly shorten the shortcomings of the unmanned vehicle's slow search for paths on the map.

The second object of the present invention is to provide a cooperative navigation method and system for two unmanned vehicles. System, by setting the computer used for obstacle recognition and path planning calculations as a cloud server, the mobility and endurance of unmanned vehicles can be improved.

Another object of the present invention is to provide a collaborative navigation method and system for two unmanned vehicles. It proposes a preset resolution rule to adjust the map image resolution according to the shooting height, which can improve the overall image processing speed and quickly produce path planning results.

Another object of the present invention is to provide a collaborative navigation method and system for two unmanned vehicles, which can merge the path planning results with the map image of the original image resolution to obtain a high-quality merged image with path planning, so as to facilitate the user to observe the plan. path or based on which the unmanned vehicle is operated.

Another object of the present invention is to provide a cooperative navigation method and system for two unmanned vehicles, which can be based on the path planning results, the current position of the unmanned vehicle and/or the maximum length and maximum width (optionally including minimum radius of gyration), generate corresponding path trajectory control instructions to achieve automatic control of the unmanned vehicle to move along the path trajectory.

The use of the quantifier "a" or "an" in the elements and components described throughout the present invention is only for convenience of use and to provide a common sense of the scope of the present invention; in the present invention, it should be interpreted as including one or at least one, and single The concept of also includes the plural unless it is obvious that something else is meant.

The term "coupling" throughout this disclosure includes direct or indirect electrical and/or signal connections, which can be selected by those with ordinary knowledge in the art according to usage requirements.

"Computer" as mentioned throughout the present invention refers to various data processing devices with specific functions and implemented by hardware or hardware and software, especially a processor to process and analyze information and/or generate correspondence Control information, such as servers, virtual machines, desktop computers, notebook computers, tablet computers or smart phones, etc., can be understood by those with ordinary knowledge in the technical field to which the present invention belongs.

The "Cloud Server (CloudServer)" described in the entire text of this invention utilizes virtual Virtual software is created to divide a physical (bare metal) server into several virtual servers for executing applications and information processing storage; users can remotely access the functions of these virtual servers through an online interface .

"Processor" as mentioned throughout the present invention refers to any electronic chip with data storage, computing and signal generation functions, or an electronic device equipped with such an electronic chip. For example, the electronic chip can be a central processing unit (CPU), a microcontroller (MCU), a digital signal processor (DSP), a field programmable gate array (FPGA) or a system on a chip (SoC); the The electronic device may be a programmable logic controller (PLC) or an Arduino UNO, and a person with ordinary knowledge in the field can select one based on computing performance, price, volume limitations or functional requirements.

The "database" mentioned in the entire text of this invention refers to a group of related electronic data collected and stored on a hard disk, a memory, or a combination of the above, and can be provided by a database management system (DBSMS). Syntax functions, such as adding, reading, searching, updating and deleting, etc., are used to process electronic data; the database management system can manage electronic data through different data structures, such as associative, hierarchical, network form or object-oriented, etc., the present invention takes a relational database management system such as MySQL as an example for the following description, but is not intended to limit the present invention.

In the collaborative path navigation method of two unmanned vehicles of the present invention, the two unmanned vehicles are an unmanned vehicle and a drone respectively. The unmanned parking space is at a predetermined location and the drone captures a map image of the predetermined location. In the state, it includes: a computer receives the map image, and the map image has a first resolution; the computer reduces the first resolution to a second resolution according to a preset resolution adjustment rule, and in the preset resolution In the degree rule, the reduced second resolution is positively correlated with the shooting height of the map image; and the computer receives a starting point information and an end point information, and performs operations based on the starting point information, the end point information and the map image. analysis to produce a path trajectory with a A path image, and the path trajectory includes the starting point information and the end point information; the starting point information and the end point information can be defined by the computer based on a current location of the unmanned vehicle and the map image, or by a user. Defined by an operation module, the operation module receives and displays the map image on a display device for the user to set the corresponding starting point information and the end point information; the computer is based on the path trajectory and the unmanned vehicle. The current position is used to generate a corresponding path trajectory control instruction to control the unmanned vehicle to move along the path trajectory.

The cooperative path navigation system for two unmanned vehicles of the present invention includes: an unmanned vehicle with a positioning unit for obtaining a current position of the unmanned vehicle; a drone with another positioning unit and a camera module, the Another positioning unit is used to obtain a current position of the drone, the camera module is used to capture a map image; and a computer is coupled to the unmanned vehicle and the drone, and performs the two unmanned operations as in the present invention. Vehicle collaborative path navigation method.

Accordingly, the collaborative path navigation method and system of two unmanned vehicles of the present invention can capture the map image through the UAV, and associate the path trajectory generated by processing the map image with the computer to the unmanned vehicle. The computer or a user controls the movement of the unmanned vehicle, which greatly reduces the shortcomings of the unmanned vehicle simply searching for a path slowly on the map, and improves the efficiency of the unmanned vehicle in crossing obstacles and/or performing tasks. In addition, through the preset resolution rules, the map image resolution can be adjusted according to the shooting height, thereby improving the overall image processing speed and quickly generating path planning results. In addition, the starting point information and the end point information can be defined, especially when the starting point information and the end point information can be set according to a predefined method, the function of the computer's automatic path planning function can be realized. In addition, the computer can automatically generate control instructions based on the path trajectory and the current position to achieve the effect of automatically controlling the operation of the unmanned vehicle (overcoming obstacles or reaching the destination).

Wherein, the first resolution of the map image can be 1280 x 960 pixels, and the preset resolution rule can be defined as: in a state where the shooting height exceeds 50 meters but is less than 60 meters, the second resolution The height is 1280 x 960 pixels; the shooting height is more than 40 meters but less than 50 meters. In a state where the shooting height exceeds 30 meters but is less than 40 meters, the second resolution is 320 x 240 pixels; in a state where the shooting height exceeds 20 meters In a state where the shooting height is more than 10 meters but less than 30 meters, the second resolution is 256 x 192 pixels; in a state where the shooting height is more than 10 meters but less than 20 meters, the second resolution is 128 x 96 pixels; in a state where the shooting height is below 10 meters, the second resolution is 128 x 96 pixels. In this way, through the default adjustment rules, the computing, processing and response speed of the computer and the overall system can be improved without affecting the accuracy of path planning.

Wherein, the computer can be a cloud server. In this way, compared with the situation where the computer is directly installed on the unmanned vehicle, the load of the unmanned vehicle can be reduced to improve the mobility and endurance of the unmanned vehicle.

Among them, the computer can remove the background image corresponding to the path other than the path trajectory in the path image, so that the path image only contains the image of the path trajectory, and align and merge the path image with the map image to obtain a merged image. image. In this way, by aligning and merging the path trajectory with the map image, the path planning results can be obtained efficiently and a high-quality merged image can be obtained.

Wherein, a display device of an operation module can accept and display the merged image and the unmanned vehicle corresponding to a real-time position in the merged image. In this way, displaying the merged image and the real-time position through the operation module can facilitate the user to observe the merged image or operate the unmanned vehicle to move accordingly.

Among them, the computer can generate a corresponding corrected path trajectory according to a corresponding maximum length and a maximum width of the unmanned vehicle. In this way, through the corresponding corrected path trajectory generated corresponding to the maximum length and the maximum width of the unmanned vehicle, the path trajectory can be optimized to avoid unexpected collisions or obstacles when the unmanned vehicle moves according to the path trajectory; and can be based on Unmanned vehicles of different specifications generate corresponding appropriate path trajectories.

Wherein, the computer generates a corresponding path trajectory control instruction based on the path trajectory and a current position, a maximum length and a maximum width of a corresponding unmanned vehicle to control the corresponding The unmanned vehicle moves along this path. In this way, through the path trajectory and the current position, maximum length, and maximum width, control instructions suitable for unmanned vehicles that meet specific specifications can be automatically generated to achieve the effect of automatically controlling the operation of unmanned vehicles.

1:Unmanned vehicle

2: Drone

10,20: Processor

11,21:Transmission module

12,22: Positioning unit

13,23:Camera module

14,24: Thermal imaging module

15,25: Gas detection module

16,26: Suspended particle detection module

3:Computer

31:Image processing module

32:Image recognition module

33: Path planning model

4: Operation module

S1: Image receiving steps

S2: Image reduction step

S21: Steps to reduce resolution

S21A: Steps to reduce chroma

S3: Path planning steps

S31: Obstacle identification steps

S32: Binarization step

S32A: Image aspect ratio adjustment steps

S33: Path trajectory generation steps

S33A: Steps to improve path image resolution

S33B: Proportional restoration step

S4: Image merging step

S41: Eliminate background step

S41A: Resolution consistency step

S42: Merge step

S5: Steps to control the movement of unmanned vehicles

[Figure 1] System architecture diagram of a preferred embodiment of the present invention.

[Figure 2] A method flow chart of a preferred embodiment of the present invention.

[Figure 3] According to Figure 2 of this case, the detailed process of the path planning steps.

In order to make the above and other objects, features and advantages of the present invention more obvious and understandable, preferred embodiments of the present invention are illustrated below and described in detail with reference to the accompanying drawings; in addition, the same symbols are used in different drawings. are considered to be the same and their description will be omitted.

Please refer to Figure 1, which is a preferred embodiment of a cooperative navigation system for two unmanned vehicles of the present invention. It includes an unmanned vehicle (UGV) 1, an unmanned aerial vehicle (UAV) 2 and a computer 3. The car 1 and the drone 2 are coupled to the computer 3 respectively. Preferably, it also includes an operation module 4 coupled to the unmanned vehicle 1, the drone 2 and the computer 3 respectively.

The unmanned vehicle 1 and the drone 2 respectively have a processor 10 and 20, a transmission module 11 and 21, a positioning unit 12 and 22, and preferably have a camera module 13 and 23, a At least one of thermal imaging modules 14 and 24, a gas detection module 15 and 25, and an aerosol detection module 16 and 26. The processors 10 and 20 are respectively used to control the actions of the unmanned vehicle 1 and the unmanned aerial vehicle 2, preferably based on the feedback of various signals received by the unmanned vehicle 1 and the unmanned aerial vehicle 2 to perform corresponding control. In addition, the unmanned vehicle 1 and the drone 2 preferably have automatic obstacle avoidance functions. The transmission modules 11 and 21 of the unmanned vehicle 1 and the drone 2 can be coupled to each other, or can be coupled to the computer 3 respectively, or can be coupled to other devices to receive, transmit or receive data. exchange, and can be used as the basis for the processors 10 and 20 to control the unmanned vehicle 1 and the unmanned aerial vehicle 2 respectively. The positioning units 12 and 22 may be a global positioning system, respectively used to obtain the current positions of the unmanned vehicle 1 and the UAV 2 . The camera module 13, 23 can be, for example, a camera for capturing surrounding environment images or map images. The thermal imaging modules 14 and 24 are used to detect/obtain the temperature of an object and/or an ambient temperature, and may use, for example, the MELEXIS product model MLX90640 infrared thermal imaging camera. The gas detection modules 15 and 25 can be used to detect carbon monoxide (CO), carbon dioxide (CO ₂ ), hydrocarbon mixture gas (LPG), ammonia (NH3), nitrogen dioxide (NO ₂ ), methane (CH ₄ ), propane (C ₃ H ₈ ), butane (C ₄ H ₁₀ ), hydrogen (H ₂ ), and at least one of alcohol gas. The suspended particulate detection modules 16 and 26 series can be used to detect PM2.5. Preferably, each unmanned vehicle also has a pre-trained face recognition model for recognizing faces from the camera modules 13 and 23 or the thermal imaging modules 14 and 24. It is better to obtain the corresponding Temperature distribution on human face. It should be noted that the above-mentioned technology and functional systems related to processors/controllers, signal exchange, automatic obstacle avoidance, image photography, thermal imaging sensing, gas detection, suspended particle detection, face recognition, etc. between unmanned vehicles It belongs to common knowledge in the technical field to which the present invention belongs and can be understood by those skilled in the art, so no further description will be given here.

The computer 3 has an image processing module 31 , and the image processing module 31 includes an image recognition module 32 and a path planning model 33 . The image processing module 31 is used to receive an image with a first resolution (Resolution) taken from the drone 2, and the image recognition module 32 is used to determine and identify at least one obstacle in the map image. , and generate a path planning result through the path planning model 33 for the unmanned vehicle 1 to operate in an area corresponding to the path planning. It should be noted that the computer 3 can be a local configuration installed on the unmanned vehicle 1 or the drone 2, or the computer 3 can be a remote configuration of a cloud/remote server to correspond to the unmanned vehicle. The data of the vehicle 1 or the drone 2 is received, processed, and/or fed back. Preferably, in order to increase the unmanned vehicle 1 and To improve the mobility and endurance of the UAV 2, the computer 3 is a cloud server and is coupled to the unmanned vehicle 1 and the UAV 2 respectively to reduce the weight of the unmanned vehicle 1 and the UAV 2.

In one example, the image recognition module 32 is an object detection model for performing corresponding image recognition functions. The object detection model is built on the Path Planning Node workstation. The Path Planning Node can regularly check from a database whether there is a need to trigger object detection, image recognition and path planning. Preferably, the Path Planning Node uses Windows 10 system, uses Anaconda to build the Python environment required for research, the Pytorch training framework required for the object detection model, and the image annotation tool for creating custom data sets, and uses Visual C++ The anchors parameters required to build Darknet to calculate a custom model.

The training set used by the custom model can come from free materials on the Internet and images taken by yourself. Adjusting the same object through different shooting angles, distances, sizes, directions, etc. can help improve the accuracy of the model after training. In a training example of a single category of obstacles/objects, the training set has a total of more than 500 images. All images are labeled with the identified objects in the image using the Labelimg tool. Each labeled image will generate a file with the coordinates of the object labeled; And based on this single category of obstacle training, it can be expanded to multiple categories of obstacle/object recognition effects.

The custom training set is divided into a training set and a validation set with 80% data set and 20% data set respectively, and two corresponding path files are generated. After generating the path file, create the yaml file and names file for training the model. The content of the yaml file is the file access location of the training set and verification set path files and the number of categories of identified objects, and the content of the names file is the category name. Using the cluster analysis function provided by Darknet, the training set is used to calculate 3 groups of anchors for different scales. To train a custom model, you need to modify the parameter configuration of the cfg file. The parameters that need to be modified are width, height, filters, and anchors. In an example, the object detection model can use 608 x 608 pixels (Pixels) resolution and a recognition category, so the width and height are set to 608, the number of filters is set to 18, and the anchors calculated by Darknet are filled in, and Training is completed after 300 epochs practice to output records of the training process. However, various image recognition/object detection technologies have been widely used in this field, and the technologies applied in the present invention are not limited to the above content.

In addition, in an example, the path planning model 33 uses the A* Search Algorithm to perform the corresponding path planning function. The A* search algorithm mainly operates on planar grid maps; in an example of using the A* search algorithm, the input binary PNG image is first converted into an occupancy grid map (OGM) to satisfy the need to perform A* search. The conditions required by the algorithm for path planning; in the grid map, a value of 1 represents that the node is occupied, and a value of 0 represents that the node can be walked. Before path planning, the coordinates of the starting point and the ending point will be obtained. The coordinates will also be adjusted appropriately according to the different resolutions of the input image; that is, if the input image resolution is scaled, the coordinates will also be adjusted. Adjust the zoom to avoid errors in the coordinates of the starting point and the end point. After the aforementioned grid map is converted and the coordinates of the starting point and end point are obtained, the A* search algorithm can be executed. A* can be moved in eight directions (up, down, left, right, up left, up right, down left, down right) or four directions (up, down, left, right), and preferably in Smoother and complete eight-way movement for path planning. After the path planning is completed, use the matplotlib tool to draw the path data into a path trajectory and output a path image. However, various path planning technologies have been widely used in this field, and the technologies applied in the present invention are not limited to the above content.

The operation module 4 is coupled to the unmanned vehicle 1, the unmanned aerial vehicle 2 and the computer 3 respectively to control the unmanned vehicle 1 and the unmanned aerial vehicle 2, and to receive the unmanned vehicle 1, the unmanned aerial vehicle 2 and/or the Computer 3 information. In other words, the operating module 4 can be a control platform, and specifically can be a device with functions such as displaying images, generating signals, and performing wireless transmission, such as a smart phone, a tablet, a laptop computer (Laptop Computer) or a desktop computer ( Desktop Computer). The data transmission type between each unit (the unmanned vehicle 1, the drone 2, the computer 3 and the operation module 4) can be direct transmission or indirect transmission, or can be based on the connection status or data processing capabilities of each unit. And corresponding changes. Take the data transmission or reception between the unmanned vehicle 1 and the operation module 4 as an example. If it is a direct transmission type, The operation module 4 can issue an instruction to the unmanned vehicle 1 so that the unmanned vehicle 1 generates a corresponding feedback (for example, controlling the unmanned vehicle to move or stop); if it is an indirect transmission type, the operation module 4 The issued command is first transmitted to the drone 2 or the computer 3, and then the drone 2 or the computer 3 issues the corresponding command to the unmanned vehicle 1, so that the unmanned vehicle 1 generates a corresponding feedback. In addition, in other possible examples, the operation module 4 can also be only coupled to an operation panel of the computer 3 to achieve the content described in the present invention.

The operation module 4 can have a display device 41 for displaying an operation interface or a corresponding image. The operation interface has a plurality of corresponding operation buttons (which can be physical buttons, virtual buttons or a combination thereof) to generate corresponding control instructions respectively. , especially for operating various functions or actions in the above-mentioned modules or units of the corresponding vehicle (unmanned vehicle 1 or drone 2). In an implementation example, the operation module 4 is a mobile device, and the operation interface is preferably an application program (APP) window of the mobile device. The application window may include a plurality of virtual buttons, and the plurality of virtual buttons correspond to The generated control instructions may include a direction control instruction, a shooting control instruction, and a positioning instruction. Specifically, the direction control command is used to control the movement of the vehicle (unmanned vehicle 1 or drone 2) selected by the user; for example, the direction control command may include a stop command, a forward command, and a backward command. , a left turn command and a right turn command to respectively control the corresponding unmanned vehicle to produce corresponding actions. The shooting control instruction is used to control the operation of the camera module 13 or 23 of the vehicle selected by the user; for example, the shooting control instruction may include a photographing instruction, a video recording instruction, a stop photographing instruction and/or a stop recording instruction, to The corresponding camera module 13 or 23 is respectively controlled to generate corresponding functions. The positioning command is used to control the operation of the positioning unit 12 or 22 of the vehicle selected by the user to obtain the current or real-time position information of the corresponding vehicle, such as quantifiable coordinates.

According to the above system of the present invention, in a specific embodiment, when the unmanned vehicle 1 is at a predetermined location, especially when the computer 3 lacks a pre-established or real-time map image corresponding to the predetermined location, it cannot be efficient. When crossing an obstacle or reaching a target location, the computer 3 Based on a current position of the unmanned vehicle 1 (based on its positioning unit 12), a support command (including the current location information of the unmanned vehicle 1) is issued to control the UAV 2 to the predetermined position, especially to make the UAV 2 The position information of the positioning unit 22 matches the current position of the unmanned vehicle 1 to capture a map image of the predetermined position; the map image is preferably a top view. In another implementation example, corresponding to the aforementioned predetermined position, the user controls the drone 2 to the predetermined position through the operation module 4 to capture the map image.

In detail, please refer to Figure 2, which shows the image processing flow based on the above system of the present invention, and the following steps are executed by the computer 3:

Image receiving step S1: Receive the map image captured by the UAV 2. The map image has a first resolution. Preferably, the first resolution may be 1280 x 960 pixels, and the map image has a default image aspect ratio of 4:3.

Image downgrading step S2: used to reduce the capacity of the received image, so as to facilitate the overall computer operating efficiency. The specific steps of the image downgrading include:

Resolution reduction step S21: reduce a resolution of the received map image; in particular, reduce the first resolution of the received map image to a second resolution.

其中，根據本發明之實驗，所述拍攝高度與降階後解析度的關係，係可在不影響路徑規劃正確性的情況下，使解析度降到最低的規則。該影像處理工具OpenCV、pillow係基於Python程式語言架構所提供的開源工具(Open Source Tool)。該平均效率係表示對應該地圖影像解析度的降低程度，而能提升後續路徑規劃步驟(包含障礙物辨識及產生路徑軌跡)的處理速度(與影像處理工具OpenCV所處理的參考模式相比)，如此，本發明所提出的預設調整規則，可在不影響路徑規劃正確性的情況下，提昇該電腦及整體系統運算、處理及反應的速度，以改善習知技術中在影像處理及生成具有路徑規劃之地圖影像時運算速度過慢的問題；特別是，該平均效率係顯根據本案上述步驟S21及下述步驟S31、S32、S33、S41、S42的優化效果。 In the preset resolution rule, the reduced resolution is positively correlated with the shooting height of the map image; preferably, the preset adjustment rule is as shown in Table 1:

Among them, according to the experiments of the present invention, the relationship between the shooting height and the reduced resolution is a rule that can minimize the resolution without affecting the accuracy of path planning. The image processing tools OpenCV and pillow are open source tools (Open Source Tool) provided based on the Python programming language architecture. The average efficiency represents the degree of reduction corresponding to the map image resolution, which can improve the processing speed of subsequent path planning steps (including obstacle identification and path trajectory generation) (compared to the reference mode processed by the image processing tool OpenCV). In this way, the preset adjustment rules proposed by the present invention can improve the computing, processing and response speed of the computer and the overall system without affecting the accuracy of path planning, so as to improve the image processing and generation capabilities of the conventional technology. The problem of too slow computing speed when using map images for path planning; in particular, the average efficiency shows the optimization effect based on the above step S21 and the following steps S31, S32, S33, S41, and S42 of this case.

Optional reducing saturation step S21A: reducing a saturation (Saturation) of the map image; in particular, reducing the saturation of the image with the second resolution.

Path planning step S3: used to analyze and process the map image to generate corresponding path trajectories. The specific steps of path planning (please refer to Figure 3) include:

Obstacle recognition step S31: Perform image recognition (especially through the image recognition module 32) on the map image processed in the previous step (which can be step S21 or S21A). If it is determined that there is an obstacle after recognition, add an obstacle. object boundary to mark the obstacle in the map image.

Binarization step S32: Binarize the image processed in the previous step (step S31).

Optional image aspect ratio adjustment step S32A: Adjust the image aspect ratio of the map image processed in the previous step (step S32) from a default image aspect ratio to 1 through a ratio normalization process: 1.

Path trajectory generation step S33: Load a predetermined starting point information/coordinates and an end point information/coordinates of the image, and perform path planning (especially through The path planning model 33) is used to generate a path image with a path trajectory. Preferably, the path trajectory includes the starting point information and the end point information.

Among them, the aforementioned starting point information/coordinates and end point information/coordinates are information that can be set by the computer 3 according to a predefined method. The starting point information can be defined as the current position of the unmanned vehicle 1; the end point information can be a vector direction between a position before the unmanned vehicle 1 moves to the current position and the current position, and the current position is directed toward the vector direction. Extending to a boundary of the corresponding map image or a position at a distance from the boundary; the end point information may also be a position symmetrically mapped to the geometric center point of the map image corresponding to the current position. Specifically, the starting point information and the end point information are defined by the computer based on the current location of the unmanned vehicle and the map image. However, various predefined methods can be adjusted according to the user's needs, and can include corresponding judgment conditions to avoid setting the starting point or the end point in unfeasible locations (such as obstacles, lakes, etc. where the unmanned vehicle 1 cannot move). position), the technology applied in the present invention is not limited to the above content. In another implementation example, the starting point information and the end point information can be set by the user through the operation module 4. For example, the operation module 4 accepts the map image to display on the display device 41, and displays it on the display device 41 through the operation module 4. The user sets the corresponding starting point information and end point information.

Optional step S33A of improving path image resolution: improving a path image resolution of the path image in the previous step (step S33).

Optional scale restoration step S33B: If the image is processed by the scale normalization process (step S32A), the aspect ratio of the route image will be adjusted to the default aspect ratio of the map image through a scale restoration step. .

Optional image merging step S4: used to merge the obtained path trajectory with the map image. The specific steps of the image merging include:

Background elimination step S41: The path image processed in the previous step (step S33, S33A or S33B) is subjected to a background removal (background elimination) process to remove the background image other than the path trajectory in the path image, so that the path image is The path image only contains the image of the path trajectory; specifically, the background image corresponding to the path trajectory is transparent.

Optional resolution unification step S41A: Improve the route image resolution of the route image processed in the previous step (step S41), so that the route image resolution is consistent with the first resolution of the map image.

Merging step S42: Align and merge the route image processed in the previous step (step S41 or S41A) with the map image having the first resolution, especially based on the same size ratio and the same reference point. Merge to obtain a merged image, so that the map image has the path trajectory in the path image.

Step S5 of controlling the movement of the unmanned vehicle: Control the movement of the unmanned vehicle 1 according to a path trajectory in any of the aforementioned steps containing the path trajectory (such as the aforementioned steps S33, S33A, S33B, S41, S41A or S42) to avoid the Obstacles in the map. In a preferred embodiment, the unmanned vehicle 1 is controlled by the computer 3 generating corresponding control instructions based on the path trajectory and a current position of the unmanned vehicle 1 to control the unmanned vehicle 1 to move along the path trajectory. ; In detail, at this time, the computer 3 will convert a current position of the unmanned vehicle 1 and the path trace into quantifiable coordinate information to generate corresponding path trajectory control instructions (especially through the built-in predefined The program code generation/conversion database can generate appropriate program instructions corresponding to different unmanned vehicle specifications), and the processor 10 of the unmanned vehicle 1 receives the path trajectory instructions to control the unmanned vehicle 1 to move accordingly. Wherein, the current position can be defined based on its positioning unit 12; specifically, the current position is the predetermined position. In another feasible example, especially in a state of acquiring the merged image (corresponding to the aforementioned step S42), the user can control the movement of the unmanned vehicle 1 according to the merged image through the operation module 4; for example, the user can control the movement of the unmanned vehicle 1 through the merged image. A display device 41 of the operation module 4 displays the merged image and the unmanned Car 1 corresponds to the real-time position in the merged image (accurate position coordinates/information can be defined based on its positioning unit 12) for the user to operate. In this way, the map image captured by the drone 2 and the result of processing the map image by the computer 3 will be associated with the unmanned vehicle 1 and thereby control the movement of the unmanned vehicle 1 .

Preferably, the generation of the path gauge trajectory is related to information such as the maximum length and maximum width (especially defined from a top view) in the specification information of the unmanned vehicle 1; more preferably, the generation of the path gauge trajectory It is associated with the maximum length, maximum width, minimum radius of gyration and other information in the specification information of the unmanned vehicle 1. In this way, the planning of the path trajectory is consistent with the specifications of the unmanned vehicle 1 to avoid unexpected collisions or obstacles when the unmanned vehicle 1 moves along the path trajectory. In a preferred implementation example, when performing the path planning process in the aforementioned step S33, the computer 3 will load the aforementioned specification information of the unmanned vehicle 1 to generate the path trajectory. In another implementation example, after the path trajectory is obtained through the aforementioned step S33, the aforementioned specification information of the unmanned vehicle 1 can be considered in subsequent steps or additional procedures to correct the path trajectory. Or, in another implementation example, the display or presentation of the path trajectory is optionally corrected or uncorrected, and the computer 3 will consider the maximum length, maximum width, and optional length of a corresponding control unmanned vehicle 1 The minimum radius of gyration is used to generate a corresponding path trajectory control instruction. In other words, the computer 3 can generate a corresponding corrected path trajectory or a corresponding path trajectory control instruction according to a maximum length and a maximum width of a corresponding unmanned vehicle 1; wherein the corresponding unmanned vehicle 1 can be the original unmanned vehicle 1 Or another unmanned vehicle 1, and the specification information of the other unmanned vehicle 1 may be different or the same as the original unmanned vehicle 1. The above-mentioned path trajectories or path trajectory control instructions that take into account the specification information of each unmanned vehicle 1 can be particularly suitable for the selection of unmanned vehicles 1 of different specifications, so that when the original unmanned vehicle 1 cannot perform the task, the computer 3 can control different The unmanned vehicle 1 of the specifications can continue to perform its original tasks without repeatedly taking map images, which can simplify the route planning operation and improve the real-time and applicability of the overall system.

According to the aforementioned system and image processing flow, the present invention can implement a two-unmanned vehicle The collaborative navigation method, especially in a state where the unmanned vehicle 1 is at a predetermined location and the drone 2 captures a map image of the predetermined location, includes the following steps:

Corresponding to the aforementioned step S1, the computer 3 receives the map image.

Corresponding to the aforementioned step S2, the computer 3 reduces a resolution of the map image according to a preset resolution adjustment rule. In the preset resolution rule, the reduced resolution is correct to the shooting height of the map image. Related. The method may include the aforementioned step S21 and optional step S21A.

Corresponding to the aforementioned step S3, the computer 3 receives a starting point information and an end point information, and analyzes the starting point information, the end point information and the map image to generate a path image having a path trajectory. Preferably, the path trajectory includes the starting point information and the end point information. The method may include the aforementioned steps S31, S32, S33 and optional steps S32A, S33A, S33B, and may further include steps S41, S42 in the aforementioned step S4 and the optional step S41A.

Corresponding to the aforementioned step S5, the computer 3 generates corresponding control instructions based on the path trajectory and a current position of the unmanned vehicle to control the unmanned vehicle 1 to move along the path trajectory. Optionally, in a state of acquiring the merged image (corresponding to the aforementioned step S32), a user can control the movement of the unmanned vehicle 1 according to the merged image through the operation module 4.

In summary, the cooperative navigation method and system of two unmanned vehicles of the present invention can use the UAV to capture map images and the path trajectories generated by computer processing of the map images, which can be associated with the unmanned vehicle and thereby control the movement of the unmanned vehicle. , which greatly shortens the shortcomings of simply slowly searching for paths on the map for unmanned vehicles, and improves the efficiency of unmanned vehicles in crossing obstacles and/or performing tasks. In addition, by setting up the computer as a cloud server, the mobility and endurance of the unmanned vehicle can be improved. In addition, by adjusting the map image resolution according to the shooting height through preset resolution rules, the overall image processing speed can be improved to quickly produce path planning results. In addition, the computer can generate corresponding path trajectory control instructions based on the path trajectory and the current position information of the unmanned vehicle, which can control the automated movement of the unmanned vehicle along the path trajectory. move. In addition, the technical means of finally restoring the image resolution to form a merged image can efficiently obtain path planning results and obtain high-quality merged images, so that users can observe the merged image or operate the autonomous vehicle accordingly. In addition, according to the maximum length, maximum width and other information associations in the unmanned vehicle specification information (optionally including the minimum turning radius), the path trajectory can be optimized to avoid unexpected collisions or obstacles when the unmanned vehicle moves along the path trajectory.

Although the present invention has been disclosed using the above-mentioned preferred embodiments, they are not intended to limit the invention. Anyone skilled in the art can make various changes and modifications to the above-described embodiments without departing from the spirit and scope of the invention. The technical scope protected by the invention, therefore, the protection scope of the invention shall include all changes within the literal and equivalent scope described in the appended patent application scope.

S1: Image receiving steps

S2: Image reduction step

S21: Steps to reduce resolution

S21A: Steps to reduce chroma

S3: Path planning steps

S4: Image merging step

S41: Eliminate background step

S41A: Resolution consistency step

S42: Merge step

S5: Steps to control the movement of unmanned vehicles

Claims (8)

A cooperative navigation method for two unmanned vehicles. The two unmanned vehicles are an unmanned vehicle and a drone respectively. In a state where the unmanned parking space is at a predetermined location and the drone captures a map image of the predetermined location, It includes: a computer receives the map image, and the map image has a first resolution; the computer reduces the first resolution to a second resolution according to a preset resolution adjustment rule, and in the preset resolution rule , the reduced second resolution is positively correlated with the shooting height of the map image; and the computer receives a starting point information and an end point information, and analyzes based on the starting point information, the end point information and the map image, to Generate a path image with a path trajectory, and the path trajectory includes the starting point information and the end point information; the starting point information and the end point information are defined by the computer based on a current location of the unmanned vehicle and the map image, or It is defined by a user through an operation module. The operation module receives and displays the map image on a display device for the user to set the corresponding starting point information and the end point information; the computer calculates the corresponding starting point information and the end point information according to the path trajectory and A current position of the unmanned vehicle generates a corresponding path trajectory control instruction to control the unmanned vehicle to move along the path trajectory. For example, the unmanned vehicle cooperative navigation method of claim 1-2, wherein the first resolution of the map image is 1280 x 960 pixels, the default resolution rule is defined as: when the shooting height exceeds 50 meters but is not In a state where the shooting height exceeds 60 meters, the second resolution is 1280 x 960 pixels; in a state where the shooting height exceeds 40 meters but is less than 50 meters, the second resolution is 640 x 480 pixels; in the In a state where the shooting height exceeds 30 meters but is less than 40 meters, the second resolution is 320 x 240 pixels; in a state where the shooting height exceeds 20 meters but is less than 30 meters, the second resolution 256 x 192 pixels; when the shooting height is more than 10 meters but less than 20 meters In a state where the shooting height is below 10 meters, the second resolution is 128 x 96 pixels. For example, the unmanned vehicle collaborative navigation method of claim 1-2, wherein the computer is a cloud server. For example, the unmanned vehicle collaborative navigation method of claim 1-2, wherein the computer removes the background images corresponding to the path image other than the path trajectory, so that the path image only contains the image of the path trajectory, and combines the path image with the image having the path trajectory. The map images are aligned and merged to obtain a merged image. For example, the unmanned vehicle collaborative navigation method of claim 4-2, wherein a display device of an operation module accepts and displays the merged image and the unmanned vehicle corresponding to a real-time position in the merged image. For example, claim 1 to 3-2 for the unmanned vehicle coordinated navigation method, wherein the computer can generate a corresponding corrected path trajectory based on a maximum length and a maximum width of a corresponding unmanned vehicle. As claimed in any one of items 1 to 3, the second unmanned vehicle collaborative navigation method, wherein the computer generates a corresponding path trajectory based on the path trajectory and a current position, a maximum length and a maximum width of a corresponding unmanned vehicle Control instructions to control the corresponding unmanned vehicle to move along the path trajectory. A cooperative navigation system for two unmanned vehicles, including: an unmanned vehicle with a positioning unit used to obtain a current position of the unmanned vehicle; a drone with another positioning unit and a camera module, the other positioning unit The unit is used to obtain a current position of the drone, the camera module is used to capture a map image; and a computer is coupled to the unmanned vehicle and the drone, and executes any one of claims 1 to 7. 2. Unmanned vehicle cooperative navigation method.

Images