WO2022139009A1 - Method and apparatus for configuring deep learning algorithm for autonomous driving - Google Patents
Method and apparatus for configuring deep learning algorithm for autonomous driving Download PDFInfo
- Publication number
- WO2022139009A1 WO2022139009A1 PCT/KR2020/018864 KR2020018864W WO2022139009A1 WO 2022139009 A1 WO2022139009 A1 WO 2022139009A1 KR 2020018864 W KR2020018864 W KR 2020018864W WO 2022139009 A1 WO2022139009 A1 WO 2022139009A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- deep learning
- information
- vehicle
- driving environment
- environment information
- Prior art date
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 174
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013136 deep learning model Methods 0.000 claims abstract description 64
- 238000004590 computer program Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000011176 pooling Methods 0.000 description 5
- 230000002194 synthesizing effect Effects 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
- G05D1/027—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means comprising intertial navigation means, e.g. azimuth detector
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0002—Automatic control, details of type of controller or control system architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0062—Adapting control system settings
- B60W2050/0075—Automatic parameter input, automatic initialising or calibrating means
- B60W2050/0083—Setting, resetting, calibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/01—Occupants other than the driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/043—Identity of occupants
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/30—Driving style
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2552/00—Input parameters relating to infrastructure
- B60W2552/05—Type of road, e.g. motorways, local streets, paved or unpaved roads
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2555/00—Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
- B60W2555/20—Ambient conditions, e.g. wind or rain
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2555/00—Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
- B60W2555/60—Traffic rules, e.g. speed limits or right of way
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/10—Historical data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/45—External transmission of data to or from the vehicle
- B60W2556/50—External transmission of data to or from the vehicle of positioning data, e.g. GPS [Global Positioning System] data
Definitions
- the present invention relates to a method and apparatus for setting a deep learning algorithm for autonomous driving. More specifically, the present invention relates to a method and apparatus for adaptively setting a deep learning algorithm for autonomous driving according to a driving environment of a vehicle.
- Autonomous driving means that a vehicle system performs vehicle operation on its own, without partial or complete driver intervention. To implement this, an algorithm that can control various situations or variables is required. Accordingly, a deep learning algorithm with an artificial neural network structure that mimics the human neural network structure that can analyze various features from a lot of data by itself is being applied to autonomous driving.
- An object of the present invention is to provide a method and apparatus for adaptively setting a deep learning algorithm for autonomous driving according to environmental information around a vehicle.
- driving environment information for determining driving environment information of a vehicle based on input information including image information outside the vehicle decision step; a deep learning model and a deep learning parameter set determining step of determining a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and a deep learning algorithm setting step of setting a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle.
- the input information includes external signal information including at least one of a global positioning system (GPS) signal, a broadcast signal related to a road on which the vehicle is traveling, and a dedicated signal related to a road on which the vehicle is traveling.
- GPS global positioning system
- the input information includes external signal information including at least one of a global positioning system (GPS) signal, a broadcast signal related to a road on which the vehicle is traveling, and a dedicated signal related to a road on which the vehicle is traveling.
- GPS global positioning system
- the determining of the driving environment information may include: inferring first driving environment information using image information outside the vehicle; obtaining second driving environment information by using the external signal information; and determining the driving environment information of the vehicle using both the first driving environment information and the second driving environment information, wherein the first detailed information of the first driving environment information and the second detailed information of the second driving environment information If the information is different, determining the first detailed information or the second detailed information as the detailed information of the driving environment information based on a comparison result of a probability value related to the first detailed information and a corresponding threshold value may include
- the threshold value may be set differently according to the type of corresponding detailed information.
- the driving environment information of the vehicle may be determined based on a deep learning algorithm using the input information.
- the determined driving environment information may include at least one or more of the following information.
- the type of road the vehicle is traveling on e.g. city center, highway, countryside, children's area, etc.
- Traffic congestion information on the road on which the vehicle is traveling eg smooth, congested, etc.
- Vehicle visibility information (eg day, evening, night, etc.)
- the determined deep learning model is determined based on a first information set of the driving environment information
- the determined deep learning parameter set is second information including the first information set among the driving environment information It may be determined based on the set.
- the first information set may include a type of road on which the vehicle is traveling.
- the step of determining the driving environment information may be performed at regular intervals or in real time.
- the deep learning model and deep learning parameter set determining step and the deep learning algorithm setting step can be performed.
- a deep learning setting apparatus for autonomous driving for solving the above-described problems, based on input information including image information outside the vehicle, determines driving environment information for determining driving environment information of a vehicle wealth; a deep learning model and a deep learning parameter set determiner configured to determine a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and a deep learning algorithm setting unit configured to set a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle.
- the driving environment information determining unit may determine the driving environment information based on a deep learning algorithm using the input information.
- the determined driving environment information may include at least one or more of the following information.
- the type of road the vehicle is traveling on e.g. city center, highway, countryside, children's area, etc.
- Traffic congestion information on the road on which the vehicle is traveling eg smooth, congested, etc.
- Vehicle visibility information (eg day, evening, night, etc.)
- a computer program according to another aspect of the present invention for solving the above-described problems may be stored in a computer-readable recording medium in combination with a computer to execute the method for setting a deep learning algorithm for autonomous driving described above. .
- a deep learning algorithm capable of exhibiting optimal performance according to the current driving environment of the vehicle may be set as a deep learning algorithm for autonomous driving. Through this, the accuracy and reliability of autonomous driving are increased, and thus driving stability is also improved.
- 1 is a diagram briefly illustrating the basic concept of an artificial neural network.
- FIG. 2 is a diagram briefly illustrating a method for setting a deep learning algorithm according to the present invention.
- FIG. 3 is a diagram briefly showing a deep learning model and a deep learning parameter set applicable to the present invention.
- FIG. 4 is a diagram briefly showing a device for setting a deep learning algorithm and a peripheral device according to the present invention.
- the present invention discloses a method of setting a deep learning algorithm for autonomous driving. More specifically, the present invention discloses a method of adaptively setting a deep learning algorithm for autonomous driving according to the driving environment of a vehicle.
- the deep learning algorithm is one of the machine learning algorithms and refers to a modeling technique developed from an artificial neural network that mimics a human neural network.
- the artificial neural network may be configured in a multi-layered hierarchical structure as shown in FIG. 1 .
- 1 is a diagram briefly illustrating the basic concept of an artificial neural network.
- an artificial neural network is a layer including an input layer, an output layer, and at least one intermediate layer (or a hidden layer) between the input layer and the output layer.
- the deep learning algorithm can derive reliable results as a result through learning that optimizes the weight of the activation function between layers based on such a multi-layer structure.
- the deep learning algorithm applicable to the present invention may include a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), and the like.
- DNN deep neural network
- CNN convolutional neural network
- RNN recurrent neural network
- the DNN is basically characterized by increasing the middle layer (or hidden layer) in the existing ANN model to improve the learning result.
- the DNN is characterized in that the learning process is performed using two or more intermediate layers. Accordingly, the computer can derive the optimal output value by repeating the process of creating a classification label by itself, distorting the space, and classifying the data.
- CNN is characterized in that it has a structure in which data features are extracted and patterns of features are identified.
- the CNN may be performed through a convolution process and a pooling process.
- the CNN may include an algorithm in which a convolution layer and a pooling layer are combined.
- a process of extracting features of data (aka, convolution process) is performed.
- the convolution process is a process of examining adjacent components of each component in the data to determine the characteristics and deriving the identified characteristics into a single sheet. As a single compression process, the number of parameters can be effectively reduced.
- pooling process a process of reducing the size of the convolutional layer (so-called pooling process) is performed.
- the pooling process may reduce the size of data, cancel noise, and provide consistent features in minute details.
- the CNN may be used in various fields such as information extraction, sentence classification, and face recognition.
- RNN is a type of artificial neural network specialized for iterative and sequential data learning, and is characterized by having a cyclic structure inside.
- the RNN uses the cyclic structure to apply weights to past learning contents and reflect them in current learning, thereby enabling a connection between current learning and past learning, and being dependent on time.
- the RNN is an algorithm that solves the limitations of the existing continuous, iterative and sequential data learning, and can be used to identify a speech waveform or identify the front and back components of a text.
- FIG. 2 is a diagram briefly illustrating a method for setting a deep learning algorithm according to the present invention.
- the deep learning algorithm setting method includes a driving environment information determination step (S210), a deep learning model and a deep learning parameter set determination step (S220), and a deep learning algorithm setting step (S230) ) may be included.
- the deep learning algorithm setting method according to the present invention is performed by a deep learning algorithm setting apparatus.
- the apparatus for setting the deep learning algorithm may be included in a vehicle system performing autonomous driving or, conversely, may include the vehicle system.
- the deep learning algorithm setting apparatus may determine driving environment information of the vehicle based on input information including image information outside the vehicle.
- the apparatus for setting the deep learning algorithm may determine driving environment information of the vehicle by using input information including image information outside the vehicle.
- the input information may include only video image information outside the vehicle.
- the input information may include at least one of a global positioning system (GPS) signal, a broadcast signal related to a road on which the vehicle is traveling, and a dedicated signal related to a road on which the vehicle is traveling in addition to the video image information outside the vehicle.
- GPS global positioning system
- the broadcast signal is a signal transmitted to the public and may include a signal broadcast from the base station to all signal receivers located within a predetermined area.
- the dedicated signal is a signal exclusively transmitted from the base station to the corresponding vehicle (or in-vehicle signal receiver) and may include a signal transmitted only to the vehicle (or in-vehicle signal receiver).
- the broadcast signal and/or the dedicated signal may include at least one or more of the following information.
- the type of road the vehicle is traveling on e.g. city center, highway, countryside, children's area, etc.
- Traffic congestion information on the road on which the vehicle is traveling eg smooth, congested, etc.
- Vehicle visibility information (eg day, evening, night, etc.)
- driving environment information may be determined/inferred in real time or at regular intervals by applying image information outside the vehicle to a separate deep learning algorithm.
- the driving environment information may be determined/inferred in real time or at regular intervals by synthesizing external signal information (eg, GPS, Internet information, etc.) as well as the deep learning algorithm.
- external signal information eg, GPS, Internet information, etc.
- the device for setting a deep learning algorithm can determine driving environment information as follows by synthesizing the result of applying the deep learning algorithm to (video) image information outside the vehicle and external signal information received from the outside. have.
- the first driving environment information is inferred using image information outside the vehicle.
- the deep learning algorithm setting apparatus may infer the first driving environment information by applying the deep learning algorithm to the image information outside the vehicle.
- the apparatus for setting a deep learning algorithm may obtain each of the detailed information described above from the external signal information.
- the driving environment information of the vehicle is determined by using both the first driving environment information and the second driving environment information.
- the first detailed information of the first driving environment information and the second detailed information of the second driving environment information (in this case, the second detailed information corresponds to the first detailed information) are different, the first detailed information Determining the first detailed information or the second detailed information as detailed information of the driving environment information based on a comparison result of a probability value related to and a corresponding threshold value
- the apparatus for setting a deep learning algorithm may compare the first driving environment information with the second driving environment information to finally determine the vehicle driving environment information. For example, when the first detailed information of the first driving environment information and the second detailed information of the second driving environment information (in this case, the second detailed information corresponds to the first detailed information) are the same, the deep learning algorithm setting The device may determine the same detailed information as detailed information of vehicle driving environment information. However, when the first detailed information of the first driving environment information and the second detailed information of the second driving environment information are different, the deep learning algorithm setting apparatus compares the probability value related to the first detailed information to a corresponding threshold value and According to the comparison result, the first detailed information or the second detailed information may be determined as detailed information of the driving environment information.
- the threshold value applicable to the present invention may be set differently according to the type of the corresponding detailed information.
- the threshold value may be set differently according to weather information around the vehicle, a type of road on which the vehicle is traveling, information about a region on which the vehicle is traveling, and the like.
- the weather information the first driving environment information inferred based on an external image of the vehicle (eg, a video image obtained from a camera installed outside the vehicle, etc.) may be more accurate than the second driving environment information based on external signal information.
- the probability may be relatively high. Accordingly, the threshold value for the weather information may be set to be relatively low.
- the second driving environment information based on external signal information is based on an external image of the vehicle (eg, a video image acquired from a camera installed outside the vehicle). It may be more likely to be more accurate than the inferred first driving environment information. Accordingly, the threshold value for the local information may be set relatively high (compared to the threshold value for the weather information).
- the driving environment information of the vehicle according to the present invention may be determined based on a deep learning algorithm using the above-described input information.
- the driving environment information may be obtained through a deep learning algorithm to which the input information is applied.
- the determined driving environment information may include at least one or more of the following.
- the type of road the vehicle is traveling on e.g. city center, highway, countryside, children's area, etc.
- Traffic congestion information on the road on which the vehicle is traveling eg smooth, congested, etc.
- Vehicle visibility information (eg day, evening, night, etc.)
- the apparatus for setting a deep learning algorithm may determine driving environment information of the vehicle using only external signal information.
- the deep learning algorithm setting apparatus may determine the driving environment information of the vehicle by using input information including the external signal information but excluding image information outside the vehicle.
- the deep learning algorithm setting device applies the detailed information included in the external signal information as it is as the driving environment information, or separately determines each detailed information of the driving environment information using the detailed information (eg, external The driving environment information may be determined by synthesizing two or more detailed information included in the signal information to determine specific detailed information of the driving environment information).
- the deep learning algorithm setting apparatus may determine a deep learning model corresponding to the driving environment information determined in step S210 and a deep learning parameter set of the deep learning model.
- FIG. 3 is a diagram briefly showing a deep learning model and a deep learning parameter set applicable to the present invention.
- the deep learning algorithm setting method according to the present invention may be implemented based on one or more deep learning models and one or more deep learning parameter sets set for each deep learning model.
- the deep learning algorithm setting method according to the present invention may be implemented using a plurality of deep learning models and one or more deep learning parameter sets set for each deep learning model.
- the apparatus for setting a deep learning algorithm may determine an appropriate deep learning model and a set of deep learning parameters according to the driving environment information determined in step S210.
- the apparatus for setting a deep learning algorithm may determine a specific deep learning model and a specific deep learning parameter set capable of providing optimal performance in a corresponding environment according to the determined driving environment information.
- the device for setting a deep learning algorithm includes information on the brightness of vision (or time information, eg, night/day), weather information (eg, sunny, rain, snow, fog, etc.) included in the driving environment information, By considering road information (e.g., city center, highway, countryside, child protection area, etc.), it is possible to determine a deep learning model and a set of deep learning parameters that can perform optimally in a given environment.
- information on the brightness of vision or time information, eg, night/day
- weather information eg, sunny, rain, snow, fog, etc.
- road information e.g., city center, highway, countryside, child protection area, etc.
- the deep learning algorithm setting apparatus determines/selects an optimal deep learning model based on the first information set among the driving environment information determined in step S210, and the first set of information among the driving environment information determined in step S210 It is possible to determine/select an optimal deep learning parameter set for the determined deep learning model based on a second information set including
- each of the first information set and the second information set may include some or all of the above-described driving environment information.
- the apparatus for setting a deep learning algorithm may determine/select a different deep learning model and a set of deep learning parameters for each of the following cases by using the determined driving environment information.
- a first deep learning model eg, EfficientDet D2 model
- a first deep learning parameter set among a plurality of deep learning parameter sets for the first deep learning model eg, optimized for night highway driving
- a set of learned parameters e.g., learned parameters
- a first deep learning model eg, EfficientDet D2 model
- a second deep learning parameter set among a plurality of deep learning parameter sets for the first deep learning model eg, optimized for interstate driving a set of learned parameters
- a second deep learning model eg, EfficientDet D3 model
- a third deep learning parameter set among a plurality of deep learning parameter sets for the second deep learning model eg, optimized for night city driving
- a set of learned parameters e.g., learned parameters
- a second deep learning model eg, EfficientDet D3 model
- a fourth deep learning parameter set among a plurality of deep learning parameter sets for the second deep learning model eg, optimized for daytime city driving
- a set of learned parameters e.g., optimized for daytime city driving
- the EfficientDet model may include an object detection model focused on efficiency to minimize model size and maximize performance.
- a faster reaction speed is required for high-speed driving, and a first deep learning model (eg, EfficientDet D2 model) that can implement this may be utilized.
- a first deep learning model eg, EfficientDet D2 model
- city driving the driving speed of the vehicle is relatively slow, but the road is complicated and there are many pedestrians, so much more objects need to be detected with high accuracy.
- the third deep learning model a larger model for city driving (eg EfficientDet D3 model) can be utilized.
- the deep learning algorithm setting device may set the deep learning algorithm to which the deep learning parameter set determined in the deep learning model determined in step S220 is applied as a deep learning algorithm for autonomous driving of the vehicle. Accordingly, the deep learning algorithm setting apparatus may apply the deep learning algorithm to which the deep learning parameter set determined in the deep learning model determined in step S220 is applied as a deep learning algorithm for autonomous driving of the vehicle. Through this, the deep learning algorithm setting apparatus may select/apply the deep learning algorithm for autonomous driving adaptively according to surrounding environment information.
- the step of determining the driving environment information may be performed at regular intervals or in real time. And, when the driving environment information of the vehicle determined through the driving environment information determination step is different from the driving environment information of the vehicle determined immediately before, the deep learning model and deep learning parameter set determining step and the deep learning algorithm setting step may be performed have.
- the apparatus for setting a deep learning algorithm may perform the above-described driving environment information determination step at regular intervals or in real time.
- the apparatus for setting the deep learning algorithm may compare the driving environment information of the vehicle determined through the driving environment information determination step with the driving environment information of the vehicle determined just before. Then, when the driving environment information of the vehicle determined through the driving environment information determination step is different from the driving environment information of the vehicle determined immediately before, the deep learning algorithm setting apparatus additionally determines a deep learning model and a deep learning parameter set and deep learning algorithm setting steps.
- the apparatus for setting a deep learning algorithm according to the present invention can set a deep learning algorithm for autonomous driving according to driving environment information more efficiently and quickly by minimizing unnecessary computational operations.
- FIG. 4 is a diagram briefly showing a device for setting a deep learning algorithm and a peripheral device according to the present invention.
- the apparatus 400 for setting a deep learning algorithm for autonomous driving may be included in an autonomous driving control system of an autonomous driving vehicle or implemented as a device separate from the autonomous driving control system.
- the deep learning algorithm setting apparatus 400 may include the autonomous driving control system.
- the deep learning algorithm setting apparatus 400 may be implemented as a part of the autonomous driving vehicle system or as a whole system device including the autonomous driving vehicle system.
- such a deep learning algorithm setting device 400 includes a driving environment information determining unit 410, a deep learning model and deep learning parameter set determining unit 420, and a deep learning algorithm setting unit ( 430) may be included.
- the driving environment information determining unit 410 may determine the driving environment information by using the input information obtained from the camera device 10 or the external information receiving device 20 as in the above-described driving environment information determining step.
- the deep learning model and deep learning parameter set determining unit 420 uses the driving environment information determined by the driving environment information determining unit 410 to make a deep learning model like the above-described deep learning model and deep learning parameter set determining step. and a set of deep learning parameters may be determined/selected.
- information on one or more deep learning models and information on one or more deep learning parameter sets for each deep learning model may be stored in a separate storage device (eg, a database, etc.).
- the storage device may be included in the deep learning algorithm setting apparatus 400 according to the present invention or located outside the deep learning algorithm setting apparatus 400 according to an embodiment.
- the deep learning algorithm setting unit 430 may set the deep learning model and the deep learning parameter set determined as in the above-described deep learning algorithm setting step as a deep learning algorithm for autonomous driving.
- the deep learning algorithm setting device 400 is connected to the camera device 10 installed in the vehicle, the external information receiving device 20, etc., and the camera device 10 and the external information receiving device 20 Relevant information can be obtained from As another example applicable to the present invention, the deep learning algorithm setting device 400 includes the camera device 10 and the external information receiving device 20, including the camera device 10 and the external information receiving device 20 You can also use the relevant information obtained through
- the deep learning algorithm setting device 400 is connected to an autonomous driving control device that controls autonomous driving in a vehicle system, and sets/selects a deep learning algorithm used by the autonomous driving control device to set/select the It can be provided as an autonomous driving control device.
- the deep learning algorithm setting device 400 provides information about the deep learning model and the deep learning parameter set determined by the autonomous driving control system, and thus uses the determined deep learning model and the deep learning parameter set for autonomous driving. It can be set as a learning algorithm.
- the deep learning algorithm setting apparatus 400 includes the autonomous driving control system
- the deep learning algorithm setting apparatus 400 causes the autonomous driving control system to determine the deep learning model and the deep learning parameter set can also be controlled to be set as a deep learning algorithm for autonomous driving.
- the deep learning algorithm setting apparatus 400 may operate according to the various deep learning algorithm setting methods described above.
- the computer program according to the present invention may be stored in a computer-readable recording medium in combination with a computer to execute the deep learning algorithm setting method for various autonomous driving described above.
- the above-described program is a computer such as C, C++, JAVA, machine language, etc. that the processor (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as the program.
- It may include code (Code) coded in the language.
- code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do.
- this code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have.
- the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.
- a software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.
- RAM random access memory
- ROM read only memory
- EPROM erasable programmable ROM
- EEPROM electrically erasable programmable ROM
- flash memory hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Aviation & Aerospace Engineering (AREA)
- Medical Informatics (AREA)
- Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Traffic Control Systems (AREA)
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
Abstract
Provided are a method and an apparatus for configuring a deep learning algorithm for autonomous driving. The method comprises: a driving environment information determination step of determining driving environment information of a vehicle on the basis of input information including image information outside the vehicle; a deep learning model and deep learning parameter set determination step of determining a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and a deep learning algorithm configuring step of configuring, as a deep learning algorithm for autonomous driving of the vehicle, a deep learning algorithm in which the determined deep learning parameter set is applied to the determined deep learning model.
Description
본 발명은 자율 주행을 위한 딥러닝 알고리즘 설정 방법 및 장치에 관한 것이다. 보다 구체적으로, 본 발명은 차량의 주행 환경에 따라 적응적으로 자율 주행을 위한 딥러닝 알고리즘을 설정하는 방법 및 장치에 관한 것이다.The present invention relates to a method and apparatus for setting a deep learning algorithm for autonomous driving. More specifically, the present invention relates to a method and apparatus for adaptively setting a deep learning algorithm for autonomous driving according to a driving environment of a vehicle.
자율 주행은 일부 또는 완전한 운전자의 개입 없이 차량 시스템이 자체적으로 차량 운행을 수행하는 것을 의미한다. 이를 구현하기 위해서는 다양한 상황 또는 변수를 제어할 수 있는 알고리즘이 필요하다. 이에, 많은 데이터로부터 다양한 특징을 스스로 분석할 수 있는 인간의 신경망 구조를 본딴 인공 신경망 구조가 적용된 딥러닝 알고리즘이 자율 주행에 적용되고 있다.Autonomous driving means that a vehicle system performs vehicle operation on its own, without partial or complete driver intervention. To implement this, an algorithm that can control various situations or variables is required. Accordingly, a deep learning algorithm with an artificial neural network structure that mimics the human neural network structure that can analyze various features from a lot of data by itself is being applied to autonomous driving.
이러한 딥러닝 알고리즘의 정확도는 차량의 주변 환경에 따라 영향을 많이 받을 수 있다. 이에, 딥러닝 알고리즘의 신뢰도를 높이기 위한 다양한 기술들이 개발되고 있으나, 아직까지 많은 딥러닝 알고리즘들이 일정 이상의 정확도를 제공하지 못하는 한계가 있다.The accuracy of these deep learning algorithms can be greatly affected by the surrounding environment of the vehicle. Accordingly, various techniques have been developed to increase the reliability of deep learning algorithms, but there is a limit in that many deep learning algorithms cannot provide accuracy above a certain level.
본 발명이 해결하고자 하는 과제는 차량 주변의 환경 정보에 따라 적응적으로 자율 주행을 위한 딥러닝 알고리즘을 설정하는 방법 및 장치를 제공하는 것이다.An object of the present invention is to provide a method and apparatus for adaptively setting a deep learning algorithm for autonomous driving according to environmental information around a vehicle.
본 발명이 해결하고자 하는 과제들은 이상에서 언급된 과제로 제한되지 않으며, 언급되지 않은 또 다른 과제들은 아래의 기재로부터 통상의 기술자에게 명확하게 이해될 수 있을 것이다.The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.
상술한 과제를 해결하기 위한 본 발명의 일 면에 따른 자율 주행을 위한 딥러닝 알고리즘 설정 방법은, 차량 외부의 이미지 정보를 포함하는 입력 정보에 기초하여, 차량의 주행 환경 정보를 결정하는 주행 환경 정보 결정 단계; 상기 결정된 주행 환경 정보에 대응하는 딥러닝 모델 및 상기 딥러닝 모델의 딥러닝 파라미터 세트를 결정하는 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계; 및 상기 결정된 딥러닝 모델에 상기 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 상기 차량의 자율 주행을 위한 딥러닝 알고리즘으로 설정하는 딥러닝 알고리즘 설정 단계를 포함할 수 있다.In a deep learning algorithm setting method for autonomous driving according to an aspect of the present invention for solving the above-mentioned problems, driving environment information for determining driving environment information of a vehicle based on input information including image information outside the vehicle decision step; a deep learning model and a deep learning parameter set determining step of determining a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and a deep learning algorithm setting step of setting a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle.
본 발명에 있어, 상기 입력 정보는, GPS (Global Positioning System) 신호, 상기 차량이 주행 중인 도로와 관련된 방송 신호, 상기 차량이 주행 중인 도로와 관련된 전용 신호 중 적어도 하나 이상을 포함하는 외부 신호 정보를 더 포함할 수 있다.In the present invention, the input information includes external signal information including at least one of a global positioning system (GPS) signal, a broadcast signal related to a road on which the vehicle is traveling, and a dedicated signal related to a road on which the vehicle is traveling. may include more.
이때, 상기 주행 환경 정보 결정 단계는, 상기 차량 외부의 이미지 정보를 이용하여 제1 주행 환경 정보를 추론하는 단계; 상기 외부 신호 정보를 이용하여 제2 주행 환경 정보를 획득하는 단계; 및 상기 제1 주행 환경 정보 및 상기 제2 주행 환경 정보를 모두 이용하여 상기 차량의 주행 환경 정보를 결정하되, 상기 제1 주행 환경 정보의 제1 세부 정보와 상기 제2 주행 환경 정보의 제2 세부 정보가 상이할 경우, 상기 제1 세부 정보와 관련된 확률 값 및 대응하는 문턱 값의 비교 결과에 기반하여 상기 제1 세부 정보 또는 상기 제2 세부 정보를 상기 주행 환경 정보의 세부 정보로 결정하는 단계를 포함할 수 있다. 여기서, 상기 문턱 값은 대응하는 세부 정보의 종류에 따라 상이하게 설정될 수 있다.In this case, the determining of the driving environment information may include: inferring first driving environment information using image information outside the vehicle; obtaining second driving environment information by using the external signal information; and determining the driving environment information of the vehicle using both the first driving environment information and the second driving environment information, wherein the first detailed information of the first driving environment information and the second detailed information of the second driving environment information If the information is different, determining the first detailed information or the second detailed information as the detailed information of the driving environment information based on a comparison result of a probability value related to the first detailed information and a corresponding threshold value may include Here, the threshold value may be set differently according to the type of corresponding detailed information.
본 발명에 있어, 상기 차량의 주행 환경 정보는 상기 입력 정보를 이용한 딥러닝 알고리즘에 기초하여 결정될 수 있다. 이때, 상기 결정된 주행 환경 정보는 다음의 정보 중 적어도 하나 이상을 포함할 수 있다.In the present invention, the driving environment information of the vehicle may be determined based on a deep learning algorithm using the input information. In this case, the determined driving environment information may include at least one or more of the following information.
- 차량이 주행 중인 위치의 날씨 정보 (예: 맑음, 비, 눈, 안개 등)- Weather information of the location where the vehicle is driving (eg sunny, rain, snow, fog, etc.)
- 차량이 주행 중인 도로의 종류 (예: 도심, 고속도로, 시골, 어린이 보호 구역 등)- the type of road the vehicle is traveling on (e.g. city center, highway, countryside, children's area, etc.)
- 차량이 주행 중인 도로의 정체 정보 (예: 원활, 정체 등)- Traffic congestion information on the road on which the vehicle is traveling (eg smooth, congested, etc.)
- 차량의 시야 밝기 정보 (예: 낮, 저녁, 밤 등)- Vehicle visibility information (eg day, evening, night, etc.)
- 태양의 방향 및 고도 정보 (예: 동쪽, 동남쪽, 북서쪽 등)- Direction and altitude information of the sun (eg east, southeast, northwest, etc.)
- 차량이 주행 중인 위치의 법규 정보 (예: 서울, 부산, LA (Los Angeles), NY (New York) 등)- Legal information of the location where the vehicle is driving (eg, Seoul, Busan, LA (Los Angeles), NY (New York), etc.)
상술한 실시예에 있어, 상기 결정된 딥러닝 모델은 상기 주행 환경 정보 중 제1 정보 세트에 기초하여 결정되고, 상기 결정된 딥러닝 파라미터 세트는 상기 주행 환경 정보 중 상기 제1 정보 세트를 포함한 제2 정보 세트에 기초하여 결정될 수 있다.In the above-described embodiment, the determined deep learning model is determined based on a first information set of the driving environment information, and the determined deep learning parameter set is second information including the first information set among the driving environment information It may be determined based on the set.
상기 실시예에 있어, 상기 제1 정보 세트는 상기 차량이 주행 중인 도로의 종류를 포함할 수 있다.In the above embodiment, the first information set may include a type of road on which the vehicle is traveling.
본 발명에 있어, 상기 주행 환경 정보 결정 단계는 일정 주기마다 수행되거나 실시간으로 수행될 수 있다. 이어, 상기 주행 환경 정보 결정 단계를 통해 결정된 상기 차량의 주행 환경 정보가 직전에 결정된 상기 차량의 주행 환경 정보와 상이한 경우, 상기 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계 및 상기 딥러닝 알고리즘 설정 단계가 수행될 수 있다.In the present invention, the step of determining the driving environment information may be performed at regular intervals or in real time. Next, when the driving environment information of the vehicle determined through the driving environment information determination step is different from the driving environment information of the vehicle determined immediately before, the deep learning model and deep learning parameter set determining step and the deep learning algorithm setting step can be performed.
상술한 과제를 해결하기 위한 본 발명의 다른 면에 따른 자율 주행을 위한 딥러닝 설정 장치는, 차량 외부의 이미지 정보를 포함하는 입력 정보에 기초하여, 차량의 주행 환경 정보를 결정하는 주행 환경 정보 결정부; 상기 결정된 주행 환경 정보에 대응하는 딥러닝 모델 및 상기 딥러닝 모델의 딥러닝 파라미터 세트를 결정하는 딥러닝 모델 및 딥러닝 파라미터 세트 결정부; 및 상기 결정된 딥러닝 모델에 상기 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 상기 차량의 자율 주행을 위한 딥러닝 알고리즘으로 설정하는 딥러닝 알고리즘 설정부를 포함할 수 있다.A deep learning setting apparatus for autonomous driving according to another aspect of the present invention for solving the above-described problems, based on input information including image information outside the vehicle, determines driving environment information for determining driving environment information of a vehicle wealth; a deep learning model and a deep learning parameter set determiner configured to determine a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and a deep learning algorithm setting unit configured to set a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle.
본 발명에 있어, 상기 주행 환경 정보 결정부는 상기 입력 정보를 이용한 딥러닝 알고리즘에 기초하여 상기 주행 환경 정보를 결정할 수 있다. 이때, 상기 결정된 주행 환경 정보는 다음의 정보 중 적어도 하나 이상을 포함할 수 있다.In the present invention, the driving environment information determining unit may determine the driving environment information based on a deep learning algorithm using the input information. In this case, the determined driving environment information may include at least one or more of the following information.
- 차량이 주행 중인 위치의 날씨 정보 (예: 맑음, 비, 눈, 안개 등)- Weather information of the location where the vehicle is driving (eg sunny, rain, snow, fog, etc.)
- 차량이 주행 중인 도로의 종류 (예: 도심, 고속도로, 시골, 어린이 보호 구역 등)- the type of road the vehicle is traveling on (e.g. city center, highway, countryside, children's area, etc.)
- 차량이 주행 중인 도로의 정체 정보 (예: 원활, 정체 등)- Traffic congestion information on the road on which the vehicle is traveling (eg smooth, congested, etc.)
- 차량의 시야 밝기 정보 (예: 낮, 저녁, 밤 등)- Vehicle visibility information (eg day, evening, night, etc.)
- 태양의 방향 및 고도 정보 (예: 동쪽, 동남쪽, 북서쪽 등)- Direction and altitude information of the sun (eg east, southeast, northwest, etc.)
- 차량이 주행 중인 위치의 법규 정보 (예: 서울, 부산, LA (Los Angeles), NY (New York) 등)- Legal information of the location where the vehicle is driving (eg, Seoul, Busan, LA (Los Angeles), NY (New York), etc.)
상술한 과제를 해결하기 위한 본 발명의 또 다른 면에 따른 컴퓨터 프로그램은, 컴퓨터와 결합하여, 앞서 상술한 자율 주행을 위한 딥러닝 알고리즘 설정 방법을 실행시키기 위하여 컴퓨터 판독가능 기록매체에 저장될 수 있다.A computer program according to another aspect of the present invention for solving the above-described problems may be stored in a computer-readable recording medium in combination with a computer to execute the method for setting a deep learning algorithm for autonomous driving described above. .
본 발명의 기타 구체적인 사항들은 상세한 설명 및 도면들에 포함되어 있다.Other specific details of the invention are included in the detailed description and drawings.
이와 같은 본 발명에 따르면, 현재 차량의 주행 환경에 따라 최적의 성능을 발휘할 수 있는 딥러닝 알고리즘을 자율 주행을 위한 딥러닝 알고리즘으로 설정할 수 있다. 이를 통해, 자율 주행의 정확도 및 신뢰도가 높아지고, 이로 인해 주행 안정성 또한 향상되는 효과가 있다.According to the present invention, a deep learning algorithm capable of exhibiting optimal performance according to the current driving environment of the vehicle may be set as a deep learning algorithm for autonomous driving. Through this, the accuracy and reliability of autonomous driving are increased, and thus driving stability is also improved.
본 발명의 효과들은 이상에서 언급된 효과로 제한되지 않으며, 언급되지 않은 또 다른 효과들은 아래의 기재로부터 통상의 기술자에게 명확하게 이해될 수 있을 것이다.Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.
도 1은 인공 신경망의 기본적인 개념을 간단히 나타낸 도면이다.1 is a diagram briefly illustrating the basic concept of an artificial neural network.
도 2는 본 발명에 따른 딥러닝 알고리즘 설정 방법을 간단히 나타낸 도면이다. 2 is a diagram briefly illustrating a method for setting a deep learning algorithm according to the present invention.
도 3은 본 발명에 적용 가능한 딥러닝 모델 및 딥러닝 파라미터 세트를 간단히 나타낸 도면이다.3 is a diagram briefly showing a deep learning model and a deep learning parameter set applicable to the present invention.
도 4는 본 발명에 따른 딥러닝 알고리즘 설정 장치 및 주변 장치를 간단히 나타낸 도면이다.4 is a diagram briefly showing a device for setting a deep learning algorithm and a peripheral device according to the present invention.
본 발명의 이점 및 특징, 그리고 그것들을 달성하는 방법은 첨부되는 도면과 함께 상세하게 후술되어 있는 실시예들을 참조하면 명확해질 것이다. 그러나, 본 발명은 이하에서 개시되는 실시예들에 제한되는 것이 아니라 서로 다른 다양한 형태로 구현될 수 있으며, 단지 본 실시예들은 본 발명의 개시가 완전하도록 하고, 본 발명이 속하는 기술 분야의 통상의 기술자에게 본 발명의 범주를 완전하게 알려주기 위해 제공되는 것이며, 본 발명은 청구항의 범주에 의해 정의될 뿐이다. Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.
본 명세서에서 사용된 용어는 실시예들을 설명하기 위한 것이며 본 발명을 제한하고자 하는 것은 아니다. 본 명세서에서, 단수형은 문구에서 특별히 언급하지 않는 한 복수형도 포함한다. 명세서에서 사용되는 "포함한다(comprises)" 및/또는 "포함하는(comprising)"은 언급된 구성요소 외에 하나 이상의 다른 구성요소의 존재 또는 추가를 배제하지 않는다. 명세서 전체에 걸쳐 동일한 도면 부호는 동일한 구성 요소를 지칭하며, "및/또는"은 언급된 구성요소들의 각각 및 하나 이상의 모든 조합을 포함한다. 비록 "제1", "제2" 등이 다양한 구성요소들을 서술하기 위해서 사용되나, 이들 구성요소들은 이들 용어에 의해 제한되지 않음은 물론이다. 이들 용어들은 단지 하나의 구성요소를 다른 구성요소와 구별하기 위하여 사용하는 것이다. 따라서, 이하에서 언급되는 제1 구성요소는 본 발명의 기술적 사상 내에서 제2 구성요소일 수도 있음은 물론이다.The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.
다른 정의가 없다면, 본 명세서에서 사용되는 모든 용어(기술 및 과학적 용어를 포함)는 본 발명이 속하는 기술분야의 통상의 기술자에게 공통적으로 이해될 수 있는 의미로 사용될 수 있을 것이다. 또한, 일반적으로 사용되는 사전에 정의되어 있는 용어들은 명백하게 특별히 정의되어 있지 않는 한 이상적으로 또는 과도하게 해석되지 않는다.Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.
이하, 첨부된 도면을 참조하여 본 발명의 실시예를 상세하게 설명한다. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
본 발명에서는 자율 주행을 위한 딥러닝 알고리즘을 설정하는 방법을 개시한다. 보다 구체적으로, 본 발명에서는 차량의 주행 환경에 따라 적응적으로 자율 주행을 위한 딥러닝 알고리즘을 설정하는 방법을 개시한다.The present invention discloses a method of setting a deep learning algorithm for autonomous driving. More specifically, the present invention discloses a method of adaptively setting a deep learning algorithm for autonomous driving according to the driving environment of a vehicle.
설명에 앞서 본 명세서에서 사용하는 용어의 의미를 간략히 설명한다. 그렇지만 용어의 설명은 본 명세서의 이해를 돕기 위한 것이므로, 명시적으로 본 발명을 한정하는 사항으로 기재하지 않은 경우에 본 발명의 기술적 사상을 한정하는 의미로 사용하는 것이 아님을 주의해야 한다.Before the description, the meaning of the terms used in this specification will be briefly described. However, it should be noted that, since the description of the term is for the purpose of helping the understanding of the present specification, it is not used in the meaning of limiting the technical idea of the present invention unless explicitly described as limiting the present invention.
먼저, 딥 러닝 (deep learning) 알고리즘은 머신 러닝 (machine learning) 알고리즘의 하나로 인간의 신경망을 본딴 인공 신경망에서 발전된 모델링 기법을 의미한다. 인공 신경망은 도 1에 도시된 바와 같이 다층 계층 구조로 구성될 수 있다.First, the deep learning algorithm is one of the machine learning algorithms and refers to a modeling technique developed from an artificial neural network that mimics a human neural network. The artificial neural network may be configured in a multi-layered hierarchical structure as shown in FIG. 1 .
도 1은 인공 신경망의 기본적인 개념을 간단히 나타낸 도면이다.1 is a diagram briefly illustrating the basic concept of an artificial neural network.
도 1에 도시된 바와 같이, 인공 신경망 (artifical neural network; ANN)은 입력 층, 출력 층, 그리고 상기 입력 층과 출력 층 사이에 적어도 하나 이상의 중간 층 (또는 은닉 층, hidden layer)을 포함하는 계층 구조로 구성될 수 있다. 딥러닝 알고리즘은, 이와 같은 다중 계층 구조에 기반하여, 층간 활성화 함수 (activation function)의 가중치를 최적화 (optimization)하는 학습을 통해 결과적으로 신뢰성 높은 결과를 도출할 수 있다.1 , an artificial neural network (ANN) is a layer including an input layer, an output layer, and at least one intermediate layer (or a hidden layer) between the input layer and the output layer. can be structured. The deep learning algorithm can derive reliable results as a result through learning that optimizes the weight of the activation function between layers based on such a multi-layer structure.
본 발명에 적용 가능한 딥러닝 알고리즘은, 심층 신경망 (deep neural network; DNN), 합성곱 신경망 (convolutional neural network; CNN), 순환 신경망 (recurrent neural network; RNN) 등을 포함할 수 있다. The deep learning algorithm applicable to the present invention may include a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), and the like.
DNN은 기본적으로 기존 ANN 모델 내 중간 층 (또는 은닉 층)을 많이 늘려서 학습의 결과를 향상시키는 것을 특징으로 한다. 일 예로, 상기 DNN은 2개 이상의 중간 층을 이용하여 학습 과정을 수행하는 것을 특징으로 한다. 이에 따라, 컴퓨터는 스스로 분류 레이블을 만들어 내고 공간을 왜곡하고 데이터를 구분짓는 과정을 반복하여 최적의 출력 값을 도출할 수 있다.DNN is basically characterized by increasing the middle layer (or hidden layer) in the existing ANN model to improve the learning result. As an example, the DNN is characterized in that the learning process is performed using two or more intermediate layers. Accordingly, the computer can derive the optimal output value by repeating the process of creating a classification label by itself, distorting the space, and classifying the data.
CNN은, 기존의 데이터에서 지식을 추출하여 학습 과정이 수행되는 기법과 달리, 데이터의 특징을 추출하여 특징들의 패턴을 파악하는 구조를 갖는 것을 특징으로 한다. 상기 CNN은 콘볼루션 (convolution) 과정과 풀링 (pooling) 과정을 통해 수행될 수 있다. 다시 말해, 상기 CNN은 콘볼루션 층과 풀링 층이 복합적으로 구성된 알고리즘을 포함할 수 있다. 여기서, 콘볼루션 층에서는 데이터의 특징을 추출하는 과정 (일명, 콘볼루션 과정)이 수행된다. 상기 콘볼루션 과정은 데이터에 각 성분의 인접 성분들을 조사해 특징을 파악하고 파악한 특징을 한장으로 도출하는 과정으로써, 하나의 압축 과정으로써 파라미터의 개수를 효과적으로 줄일 수 있다. 풀링 층에서는 콘볼루션 과정을 거친 레이어의 사이즈를 줄여주는 과정 (일명, 풀링 과정)이 수행된다. 상기 풀링 과정은 데이터의 사이즈를 줄이고 노이즈를 상쇄시키고 미세한 부분에서 일관적인 특징을 제공할 수 있다. 일 예로, 상기 CNN은 정보 추출, 문장 분류, 얼굴 인식 등 여러 분야에 활용될 수 있다.Unlike the conventional technique in which a learning process is performed by extracting knowledge from data, CNN is characterized in that it has a structure in which data features are extracted and patterns of features are identified. The CNN may be performed through a convolution process and a pooling process. In other words, the CNN may include an algorithm in which a convolution layer and a pooling layer are combined. Here, in the convolution layer, a process of extracting features of data (aka, convolution process) is performed. The convolution process is a process of examining adjacent components of each component in the data to determine the characteristics and deriving the identified characteristics into a single sheet. As a single compression process, the number of parameters can be effectively reduced. In the pooling layer, a process of reducing the size of the convolutional layer (so-called pooling process) is performed. The pooling process may reduce the size of data, cancel noise, and provide consistent features in minute details. For example, the CNN may be used in various fields such as information extraction, sentence classification, and face recognition.
RNN은 반복적이고 순차적인 데이터 (sequential data) 학습에 특화된 인공 신경망의 한 종류로써 내부에 순환 구조를 갖는 것을 특징으로 한다. 상기 RNN은 상기 순환 구조를 이용하여 과거의 학습 내용에 가중치를 적용하여 현재 학습에 반영함으로써, 현재의 학습과 과거의 학습 간 연결을 가능하게 하고 시간에 종속된다는 특징을 갖는다. 상기 RNN은 기존의 지속적이고 반복적이며 순차적인 데이터 학습의 한계를 해결한 알고리즘으로써, 음성 웨이브폼을 파악하거나 텍스트의 앞 뒤 성분을 파악하는 등에 활용될 수 있다.RNN is a type of artificial neural network specialized for iterative and sequential data learning, and is characterized by having a cyclic structure inside. The RNN uses the cyclic structure to apply weights to past learning contents and reflect them in current learning, thereby enabling a connection between current learning and past learning, and being dependent on time. The RNN is an algorithm that solves the limitations of the existing continuous, iterative and sequential data learning, and can be used to identify a speech waveform or identify the front and back components of a text.
다만, 이는 본 발명에 적용 가능한 구체적인 딥러닝 기법의 일 예시들에 불과하며, 실시예에 따라 다른 딥러닝 기법이 본 발명에 적용될 수도 있다.However, these are only examples of specific deep learning techniques applicable to the present invention, and other deep learning techniques may be applied to the present invention according to embodiments.
도 2는 본 발명에 따른 딥러닝 알고리즘 설정 방법을 간단히 나타낸 도면이다. 2 is a diagram briefly illustrating a method for setting a deep learning algorithm according to the present invention.
도 2에 도시된 바와 같이, 본 발명에 따른 딥러닝 알고리즘 설정 방법은, 주행 환경 정보 결정 단계 (S210), 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계 (S220), 및 딥러닝 알고리즘 설정 단계 (S230)를 포함할 수 있다.2, the deep learning algorithm setting method according to the present invention includes a driving environment information determination step (S210), a deep learning model and a deep learning parameter set determination step (S220), and a deep learning algorithm setting step (S230) ) may be included.
이하 설명의 편의 상, 본 발명에 따른 딥러닝 알고리즘 설정 방법은 딥러닝 알고리즘 설정 장치에 의해 수행된다고 가정한다. 이때, 실시예에 따라, 상기 딥러닝 알고리즘 설정 장치는 자율 주행을 수행하는 차량 시스템에 포함되거나, 반대로 상기 차량 시스템을 포함할 수 있다.For convenience of description below, it is assumed that the deep learning algorithm setting method according to the present invention is performed by a deep learning algorithm setting apparatus. In this case, according to an embodiment, the apparatus for setting the deep learning algorithm may be included in a vehicle system performing autonomous driving or, conversely, may include the vehicle system.
S210 단계에서, 딥러닝 알고리즘 설정 장치는, 차량 외부의 이미지 정보를 포함하는 입력 정보에 기초하여, 차량의 주행 환경 정보를 결정할 수 있다. 다시 말해, 상기 딥러닝 알고리즘 설정 장치는, 상기 차량 외부의 이미지 정보를 포함하는 입력 정보를 이용하여 상기 차량의 주행 환경 정보를 결정할 수 있다.In step S210 , the deep learning algorithm setting apparatus may determine driving environment information of the vehicle based on input information including image information outside the vehicle. In other words, the apparatus for setting the deep learning algorithm may determine driving environment information of the vehicle by using input information including image information outside the vehicle.
본 발명에 적용 가능한 일 예로, 입력 정보는 차량 외부의 영상 이미지 정보만을 포함할 수 있다. 또는, 다른 예로, 상기 입력 정보는 상기 차량 외부의 영상 이미지 정보 외 GPS (Global Positioning System) 신호, 상기 차량이 주행 중인 도로와 관련된 방송 신호, 상기 차량이 주행 중인 도로와 관련된 전용 신호 중 적어도 하나 이상을 포함하는 외부 신호 정보를 더 포함할 수 있다. 여기서, 상기 방송 신호는 공중을 대상으로 전송된 신호로서 기지국으로부터 일정 영역 내에 위치한 모든 신호 수신기로 방송된 신호를 포함할 수 있다. 또한, 상기 전용 신호는 상기 기지국으로부터 해당 차량 (또는 차량 내 신호 수신기)에 전용적으로 전송된 신호로서 상기 차량 (또는 차량 내 신호 수신기)으로만 전송된 신호를 포함할 수 있다. 이때, 상기 방송 신호 및/또는 전용 신호는 다음 중 적어도 하나 이상의 정보를 포함할 수 있다.As an example applicable to the present invention, the input information may include only video image information outside the vehicle. Alternatively, as another example, the input information may include at least one of a global positioning system (GPS) signal, a broadcast signal related to a road on which the vehicle is traveling, and a dedicated signal related to a road on which the vehicle is traveling in addition to the video image information outside the vehicle. It may further include external signal information including Here, the broadcast signal is a signal transmitted to the public and may include a signal broadcast from the base station to all signal receivers located within a predetermined area. In addition, the dedicated signal is a signal exclusively transmitted from the base station to the corresponding vehicle (or in-vehicle signal receiver) and may include a signal transmitted only to the vehicle (or in-vehicle signal receiver). In this case, the broadcast signal and/or the dedicated signal may include at least one or more of the following information.
- 차량이 주행 중인 위치의 날씨 정보 (예: 맑음, 비, 눈, 안개 등)- Weather information of the location where the vehicle is driving (eg sunny, rain, snow, fog, etc.)
- 차량이 주행 중인 도로의 종류 (예: 도심, 고속도로, 시골, 어린이 보호 구역 등)- the type of road the vehicle is traveling on (e.g. city center, highway, countryside, children's area, etc.)
- 차량이 주행 중인 도로의 정체 정보 (예: 원활, 정체 등)- Traffic congestion information on the road on which the vehicle is traveling (eg smooth, congested, etc.)
- 차량의 시야 밝기 정보 (예: 낮, 저녁, 밤 등)- Vehicle visibility information (eg day, evening, night, etc.)
- 태양의 방향 및 고도 정보 (예: 동쪽, 동남쪽, 북서쪽 등)- Direction and altitude information of the sun (eg east, southeast, northwest, etc.)
- 차량이 주행 중인 위치의 법규 정보 (예: 서울, 부산, LA (Los Angeles), NY (New York) 등)- Legal information of the location where the vehicle is driving (eg, Seoul, Busan, LA (Los Angeles), NY (New York), etc.)
본 발명에 적용 가능한 일 예로, 주행 환경 정보는 차량 외부의 이미지 정보를 별도의 딥러닝 알고리즘에 적용하여 실시간으로 또는 일정 주기마다 결정/추론될 수 있다. As an example applicable to the present invention, driving environment information may be determined/inferred in real time or at regular intervals by applying image information outside the vehicle to a separate deep learning algorithm.
본 발명에 적용 가능한 다른 예로, 상기 주행 환경 정보는 상기 딥러닝 알고리즘 뿐만 아니라 외부 신호 정보 (예: GPS, 인터넷 정보 등) 등을 종합하여 실시간으로 또는 일정 주기마다 결정/추론될 수 있다. As another example applicable to the present invention, the driving environment information may be determined/inferred in real time or at regular intervals by synthesizing external signal information (eg, GPS, Internet information, etc.) as well as the deep learning algorithm.
보다 구체적으로, 본 발명에 따른 딥러닝 알고리즘 설정 장치는 차량 외부의 (영상) 이미지 정보에 딥러닝 알고리즘을 적용한 결과 값과 외부에서 수신된 외부 신호 정보를 종합하여 다음과 같이 주행 환경 정보를 결정할 수 있다. More specifically, the device for setting a deep learning algorithm according to the present invention can determine driving environment information as follows by synthesizing the result of applying the deep learning algorithm to (video) image information outside the vehicle and external signal information received from the outside. have.
- 차량 외부의 이미지 정보를 이용하여 제1 주행 환경 정보를 추론함. 이를 위해, 딥러닝 알고리즘 설정 장치는 상기 차량 외부의 이미지 정보에 딥러닝 알고리즘을 적용하여 상기 제1 주행 환경 정보를 추론할 수 있다. - The first driving environment information is inferred using image information outside the vehicle. To this end, the deep learning algorithm setting apparatus may infer the first driving environment information by applying the deep learning algorithm to the image information outside the vehicle.
- 외부 신호 정보를 이용하여 제2 주행 환경 정보를 획득함. 일 예로, 딥러닝 알고리즘 설정 장치는 상기 외부 신호 정보로부터 상술한 각 세부 정보들을 획득할 수 있다.- Acquire the second driving environment information by using the external signal information. As an example, the apparatus for setting a deep learning algorithm may obtain each of the detailed information described above from the external signal information.
- 제1 주행 환경 정보 및 제2 주행 환경 정보를 모두 이용하여 차량의 주행 환경 정보를 결정함. 다만, 상기 제1 주행 환경 정보의 제1 세부 정보와 상기 제2 주행 환경 정보의 제2 세부 정보(이때, 제2 세부 정보는 제1 세부 정보에 대응함)가 상이할 경우, 상기 제1 세부 정보와 관련된 확률 값 및 대응하는 문턱 값의 비교 결과에 기반하여 상기 제1 세부 정보 또는 상기 제2 세부 정보를 상기 주행 환경 정보의 세부 정보로 결정함- The driving environment information of the vehicle is determined by using both the first driving environment information and the second driving environment information. However, when the first detailed information of the first driving environment information and the second detailed information of the second driving environment information (in this case, the second detailed information corresponds to the first detailed information) are different, the first detailed information Determining the first detailed information or the second detailed information as detailed information of the driving environment information based on a comparison result of a probability value related to and a corresponding threshold value
보다 구체적으로, 본 발명에 따른 딥러닝 알고리즘 설정 장치는 제1 주행 환경 정보와 제2 주행 환경 정보를 비교하여 최종적으로 차량 주행 환경 정보를 결정할 수 있다. 일 예로, 제1 주행 환경 정보의 제1 세부 정보와 상기 제2 주행 환경 정보의 제2 세부 정보 (이때, 상기 제2 세부 정보는 제1 세부 정보에 대응함)가 동일한 경우, 상기 딥러닝 알고리즘 설정 장치는 상기 동일한 세부 정보를 차량 주행 환경 정보의 세부 정보로 결정할 수 있다. 다만, 제1 주행 환경 정보의 제1 세부 정보와 상기 제2 주행 환경 정보의 제2 세부 정보가 상이한 경우, 상기 딥러닝 알고리즘 설정 장치는 상기 제1 세부 정보와 관련된 확률 값을 대응하는 문턱 값과 비교한 결과에 따라 상기 제1 세부 정보 또는 상기 제2 세부 정보를 상기 주행 환경 정보의 세부 정보로 결정할 수 있다. More specifically, the apparatus for setting a deep learning algorithm according to the present invention may compare the first driving environment information with the second driving environment information to finally determine the vehicle driving environment information. For example, when the first detailed information of the first driving environment information and the second detailed information of the second driving environment information (in this case, the second detailed information corresponds to the first detailed information) are the same, the deep learning algorithm setting The device may determine the same detailed information as detailed information of vehicle driving environment information. However, when the first detailed information of the first driving environment information and the second detailed information of the second driving environment information are different, the deep learning algorithm setting apparatus compares the probability value related to the first detailed information to a corresponding threshold value and According to the comparison result, the first detailed information or the second detailed information may be determined as detailed information of the driving environment information.
이때, 본 발명에 적용 가능한 문턱 값은 대응하는 세부 정보의 종류에 따라 상이하게 설정될 수 있다. 일 예로, 상기 문턱 값은 차량 주변의 날씨 정보, 상기 차량이 주행 중인 도로 종류, 상기 차량이 주행 중인 지역 정보 등에 따라 상이하게 설정될 수 있다. 보다 구체적인 예로, 상기 날씨 정보는 차량의 외부 이미지 (예: 차량 외부에 설치된 카메라로부터 획득한 영상 이미지 등)에 기초하여 추론된 제1 주행 환경 정보가 외부 신호 정보에 기초한 제2 주행 환경 정보보다 정확할 가능성이 상대적으로 높을 수 있다. 이에 따라, 상기 날씨 정보를 위한 문턱 값은 상대적으로 낮게 설정될 수 있다. 반면, 상기 지역 정보는 외부 신호 정보 (예: GPS, 지도, 인터넷 정보 등)에 기초한 제2 주행 환경 정보가 차량의 외부 이미지 (예: 차량 외부에 설치된 카메라로부터 획득한 영상 이미지 등)에 기초하여 추론된 제1 주행 환경 정보보다 정확할 가능성이 상대적으로 높을 수 있다. 이에 따라, 상기 지역 정보를 위한 문턱 값은 (상기 날씨 정보를 위한 문턱 값 대비) 상대적으로 높게 설정될 수 있다.In this case, the threshold value applicable to the present invention may be set differently according to the type of the corresponding detailed information. For example, the threshold value may be set differently according to weather information around the vehicle, a type of road on which the vehicle is traveling, information about a region on which the vehicle is traveling, and the like. As a more specific example, as for the weather information, the first driving environment information inferred based on an external image of the vehicle (eg, a video image obtained from a camera installed outside the vehicle, etc.) may be more accurate than the second driving environment information based on external signal information. The probability may be relatively high. Accordingly, the threshold value for the weather information may be set to be relatively low. On the other hand, in the area information, the second driving environment information based on external signal information (eg, GPS, map, Internet information, etc.) is based on an external image of the vehicle (eg, a video image acquired from a camera installed outside the vehicle). It may be more likely to be more accurate than the inferred first driving environment information. Accordingly, the threshold value for the local information may be set relatively high (compared to the threshold value for the weather information).
상술한 다양한 실시예와 같이, 본 발명에 따른 차량의 주행 환경 정보는 상술한 입력 정보를 이용한 딥러닝 알고리즘에 기초하여 결정될 수 있다. 다시 말해, 상기 주행 환경 정보는 상기 입력 정보가 적용된 딥러닝 알고리즘을 통해 획득될 수 있다. 이와 같은 과정을 통해, 결정된 주행 환경 정보는 다음 중 적어도 하나 이상을 포함할 수 있다.As in the various embodiments described above, the driving environment information of the vehicle according to the present invention may be determined based on a deep learning algorithm using the above-described input information. In other words, the driving environment information may be obtained through a deep learning algorithm to which the input information is applied. Through this process, the determined driving environment information may include at least one or more of the following.
- 차량이 주행 중인 위치의 날씨 정보 (예: 맑음, 비, 눈, 안개 등)- Weather information of the location where the vehicle is driving (eg sunny, rain, snow, fog, etc.)
- 차량이 주행 중인 도로의 종류 (예: 도심, 고속도로, 시골, 어린이 보호 구역 등)- the type of road the vehicle is traveling on (e.g. city center, highway, countryside, children's area, etc.)
- 차량이 주행 중인 도로의 정체 정보 (예: 원활, 정체 등)- Traffic congestion information on the road on which the vehicle is traveling (eg smooth, congested, etc.)
- 차량의 시야 밝기 정보 (예: 낮, 저녁, 밤 등)- Vehicle visibility information (eg day, evening, night, etc.)
- 태양의 방향 및 고도 정보 (예: 동쪽, 동남쪽, 북서쪽 등)- Direction and altitude information of the sun (eg east, southeast, northwest, etc.)
- 차량이 주행 중인 위치의 법규 정보 (예: 서울, 부산, LA (Los Angeles), NY (New York) 등)- Legal information of the location where the vehicle is driving (eg, Seoul, Busan, LA (Los Angeles), NY (New York), etc.)
추가적으로, 본 발명의 또 다른 실시예에 따르면, 딥러닝 알고리즘 설정 장치는 외부 신호 정보만을 이용하여 차량의 주행 환경 정보를 결정할 수도 있다. 다시 말해, 앞에서 상술한 실시예와 달리, 상기 딥러닝 알고리즘 설정 장치는 상기 외부 신호 정보를 포함하되 상기 차량 외부의 이미지 정보를 제외한 입력 정보를 이용하여 상기 차량의 주행 환경 정보를 결정할 수 있다. 이 경우, 상기 딥러닝 알고리즘 설정 장치는 상기 외부 신호 정보에 포함된 세부 정보들을 그대로 상기 주행 환경 정보로 적용하거나, 상기 세부 정보들을 이용하여 상기 주행 환경 정보의 각 세부 정보들을 별도로 결정(예: 외부 신호 정보에 포함된 2개 이상의 세부 정보들을 종합하여 주행 환경 정보의 특정 세부 정보를 결정함)하여 상기 주행 환경 정보를 결정할 수 있다. Additionally, according to another embodiment of the present invention, the apparatus for setting a deep learning algorithm may determine driving environment information of the vehicle using only external signal information. In other words, unlike the above-described embodiment, the deep learning algorithm setting apparatus may determine the driving environment information of the vehicle by using input information including the external signal information but excluding image information outside the vehicle. In this case, the deep learning algorithm setting device applies the detailed information included in the external signal information as it is as the driving environment information, or separately determines each detailed information of the driving environment information using the detailed information (eg, external The driving environment information may be determined by synthesizing two or more detailed information included in the signal information to determine specific detailed information of the driving environment information).
S220 단계에서, 딥러닝 알고리즘 설정 장치는 S210 단계에서 결정된 주행 환경 정보에 대응하는 딥러닝 모델 및 상기 딥러닝 모델의 딥러닝 파라미터 세트를 결정할 수 있다.In step S220, the deep learning algorithm setting apparatus may determine a deep learning model corresponding to the driving environment information determined in step S210 and a deep learning parameter set of the deep learning model.
도 3은 본 발명에 적용 가능한 딥러닝 모델 및 딥러닝 파라미터 세트를 간단히 나타낸 도면이다.3 is a diagram briefly showing a deep learning model and a deep learning parameter set applicable to the present invention.
도 3에 도시된 바와 같이, 본 발명에 따른 딥러닝 알고리즘 설정 방법은 하나 이상의 딥러닝 모델 및 각 딥러닝 모델 별로 설정된 하나 이상의 딥러닝 파라미터 세트에 기초하여 구현될 수 있다. 바람직하게는, 본 발명에 따른 딥러닝 알고리즘 설정 방법은, 복수 개의 딥러닝 모델 및 각 딥러닝 모델 별로 설정된 하나 이상의 딥러닝 파라미터 세트를 이용하여 구현될 수 있다.3, the deep learning algorithm setting method according to the present invention may be implemented based on one or more deep learning models and one or more deep learning parameter sets set for each deep learning model. Preferably, the deep learning algorithm setting method according to the present invention may be implemented using a plurality of deep learning models and one or more deep learning parameter sets set for each deep learning model.
본 발명에 따르면, S220 단계에서 딥러닝 알고리즘 설정 장치는 S210 단계를 통해 결정된 주행 환경 정보에 따라 적절한 딥러닝 모델 및 딥러닝 파라미터 세트를 결정할 수 있다. 일 예로, 상기 딥러닝 알고리즘 설정 장치는 결정된 주행 환경 정보에 따라 해당 환경에서 최적의 성능을 낼 수 있는 특정 딥러닝 모델 및 특정 딥러닝 파라미터 세트를 결정할 수 있다.According to the present invention, in step S220, the apparatus for setting a deep learning algorithm may determine an appropriate deep learning model and a set of deep learning parameters according to the driving environment information determined in step S210. For example, the apparatus for setting a deep learning algorithm may determine a specific deep learning model and a specific deep learning parameter set capable of providing optimal performance in a corresponding environment according to the determined driving environment information.
보다 구체적으로, 딥러닝 알고리즘 설정 장치는 주행 환경 정보에 포함된 시야 밝기 정보 (또는, 시간 (time) 정보, 예: 밤/낮), 날씨 정보 (예: 맑음, 비, 눈, 안개 등), 도로 정보 (예: 도심, 고속도로, 시골, 어린이 보호 구역 등) 등을 고려하여 해당 환경에서 최적의 성능을 낼 수 있는 딥러닝 모델 및 딥러닝 파라미터 세트를 결정할 수 있다.More specifically, the device for setting a deep learning algorithm includes information on the brightness of vision (or time information, eg, night/day), weather information (eg, sunny, rain, snow, fog, etc.) included in the driving environment information, By considering road information (e.g., city center, highway, countryside, child protection area, etc.), it is possible to determine a deep learning model and a set of deep learning parameters that can perform optimally in a given environment.
또는, 딥러닝 알고리즘 설정 장치는 S210 단계를 통해 결정된 주행 환경 정보 중 제1 정보 세트에 기초하여 최적의 딥러닝 모델을 결정/선택하고, 상기 S210 단계를 통해 결정된 주행 환경 정보 중 상기 제1 정보 세트를 포함하는 제2 정보 세트에 기초하여 상기 결정된 딥러닝 모델을 위한 최적의 딥러닝 파라미터 세트를 결정/선택할 수 있다. 본 발명에 있어, 상기 제1 정보 세트 및 상기 제2 정보 세트는, 각각 앞서 상술한 주행 환경 정보의 일부 또는 모든 정보를 포함할 수 있다. Alternatively, the deep learning algorithm setting apparatus determines/selects an optimal deep learning model based on the first information set among the driving environment information determined in step S210, and the first set of information among the driving environment information determined in step S210 It is possible to determine/select an optimal deep learning parameter set for the determined deep learning model based on a second information set including In the present invention, each of the first information set and the second information set may include some or all of the above-described driving environment information.
일 예로, 본 발명에 따른 딥러닝 알고리즘 설정 장치는 결정된 주행 환경 정보를 이용하여 다음의 케이스별로 상이한 딥러닝 모델 및 딥러닝 파라미터 세트를 결정/선택할 수 있다.For example, the apparatus for setting a deep learning algorithm according to the present invention may determine/select a different deep learning model and a set of deep learning parameters for each of the following cases by using the determined driving environment information.
- 야간의 고속도로 주행 케이스: 제1 딥러닝 모델 (예: EfficientDet D2 모델) 및 상기 제1 딥러닝 모델을 위한 복수의 딥러닝 파라미터 세트들 중 제1 딥러닝 파라미터 세트 (예: 야간 고속도로 주행에 최적화되어 학습된 파라미터 세트)- Night highway driving case: a first deep learning model (eg, EfficientDet D2 model) and a first deep learning parameter set among a plurality of deep learning parameter sets for the first deep learning model (eg, optimized for night highway driving) a set of learned parameters)
- 주간의 고속도로 주행 케이스: 제1 딥러닝 모델 (예: EfficientDet D2 모델) 및 상기 제1 딥러닝 모델을 위한 복수의 딥러닝 파라미터 세트들 중 제2 딥러닝 파라미터 세트 (예: 주간 고속도로 주행에 최적화되어 학습된 파라미터 세트)- Interstate driving case: a first deep learning model (eg, EfficientDet D2 model) and a second deep learning parameter set among a plurality of deep learning parameter sets for the first deep learning model (eg, optimized for interstate driving a set of learned parameters)
- 야간의 도심 주행 케이스: 제2 딥러닝 모델 (예: EfficientDet D3 모델) 및 상기 제2 딥러닝 모델을 위한 복수의 딥러닝 파라미터 세트들 중 제3 딥러닝 파라미터 세트 (예: 야간 도심 주행에 최적화되어 학습된 파라미터 세트)- Night city driving case: a second deep learning model (eg, EfficientDet D3 model) and a third deep learning parameter set among a plurality of deep learning parameter sets for the second deep learning model (eg, optimized for night city driving) a set of learned parameters)
- 주간의 도심 주행 케이스: 제2 딥러닝 모델 (예: EfficientDet D3 모델) 및 상기 제2 딥러닝 모델을 위한 복수의 딥러닝 파라미터 세트들 중 제4 딥러닝 파라미터 세트 (예: 주간 도심 주행에 최적화되어 학습된 파라미터 세트)- Daytime city driving case: a second deep learning model (eg, EfficientDet D3 model) and a fourth deep learning parameter set among a plurality of deep learning parameter sets for the second deep learning model (eg, optimized for daytime city driving) a set of learned parameters)
위 예시에서, EfficientDet 모델은 모델 사이즈를 최소화하고 성능을 최대화하는 효율성에 초점을 맞춘 객체 검출 (object detection) 모델을 포함할 수 있다. 이와 같이, 고속도로 주행의 경우 고속 주행을 위해 보다 빠른 반응 속도가 요구되는 바, 이를 구현할 수 있는 제1 딥러닝 모델 (예: EfficientDet D2 모델)이 활용될 수 있다. 반면, 도심 주행의 경우 차량의 주행 속도는 상대적으로 느리지만 도로가 복잡하고 보행자가 많아 훨씬 더 많은 객체를 높은 정확도로 검출해 내야 하는 바, 이를 위해 도심 주행에 더 큰 모델인 제3 딥러닝 모델 (예: EfficientDet D3 모델)이 활용될 수 있다.In the above example, the EfficientDet model may include an object detection model focused on efficiency to minimize model size and maximize performance. As such, in the case of highway driving, a faster reaction speed is required for high-speed driving, and a first deep learning model (eg, EfficientDet D2 model) that can implement this may be utilized. On the other hand, in the case of city driving, the driving speed of the vehicle is relatively slow, but the road is complicated and there are many pedestrians, so much more objects need to be detected with high accuracy. To this end, the third deep learning model, a larger model for city driving (eg EfficientDet D3 model) can be utilized.
S230 단계에서, 딥러닝 알고리즘 설정 장치는 S220 단계에서 결정된 딥러닝 모델에 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 차량의 자율 주행을 위한 딥러닝 알고리즘으로 설정할 수 있다. 이에 따라, 상기 딥러닝 알고리즘 설정 장치는 S220 단계에서 결정된 딥러닝 모델에 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 차량의 자율 주행을 위한 딥러닝 알고리즘으로 적용할 수 있다. 이를 통해, 상기 딥러닝 알고리즘 설정 장치는 주변 환경 정보에 따라 적응적으로 상기 자율 주행을 위한 딥러닝 알고리즘을 선택/적용할 수 있다.In step S230, the deep learning algorithm setting device may set the deep learning algorithm to which the deep learning parameter set determined in the deep learning model determined in step S220 is applied as a deep learning algorithm for autonomous driving of the vehicle. Accordingly, the deep learning algorithm setting apparatus may apply the deep learning algorithm to which the deep learning parameter set determined in the deep learning model determined in step S220 is applied as a deep learning algorithm for autonomous driving of the vehicle. Through this, the deep learning algorithm setting apparatus may select/apply the deep learning algorithm for autonomous driving adaptively according to surrounding environment information.
본 발명에 있어, 주행 환경 정보 결정 단계는 일정 주기마다 수행되거나 실시간으로 수행될 수 있다. 그리고, 상기 주행 환경 정보 결정 단계를 통해 결정된 차량의 주행 환경 정보가 직전에 결정된 상기 차량의 주행 환경 정보와 상이한 경우, 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계 및 딥러닝 알고리즘 설정 단계가 수행될 수 있다. In the present invention, the step of determining the driving environment information may be performed at regular intervals or in real time. And, when the driving environment information of the vehicle determined through the driving environment information determination step is different from the driving environment information of the vehicle determined immediately before, the deep learning model and deep learning parameter set determining step and the deep learning algorithm setting step may be performed have.
다시 말해, 본 발명에 따른 딥러닝 알고리즘 설정 장치는 일정 주기마다 또는 실시간으로 상술한 주행 환경 정보 결정 단계를 수행할 수 있다. 이때, 상기 딥러닝 알고리즘 설정 장치는 상기 주행 환경 정보 결정 단계를 통해 결정된 차량의 차량의 주행 환경 정보와 직전에 결정된 상기 차량의 주행 환경 정보를 비교할 수 있다. 이어, 상기 주행 환경 정보 결정 단계를 통해 결정된 차량의 차량의 주행 환경 정보와 직전에 결정된 상기 차량의 주행 환경 정보가 상이한 경우, 상기 딥러닝 알고리즘 설정 장치는 추가적으로 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계 및 딥러닝 알고리즘 설정 단계를 수행할 수 있다.In other words, the apparatus for setting a deep learning algorithm according to the present invention may perform the above-described driving environment information determination step at regular intervals or in real time. In this case, the apparatus for setting the deep learning algorithm may compare the driving environment information of the vehicle determined through the driving environment information determination step with the driving environment information of the vehicle determined just before. Then, when the driving environment information of the vehicle determined through the driving environment information determination step is different from the driving environment information of the vehicle determined immediately before, the deep learning algorithm setting apparatus additionally determines a deep learning model and a deep learning parameter set and deep learning algorithm setting steps.
이러한 동작을 통해, 본 발명에 따른 딥러닝 알고리즘 설정 장치는 불필요한 연산 동작을 최소화함으로써 보다 효율적이고 신속하게 주행 환경 정보에 따라 자율 주행을 위한 딥러닝 알고리즘을 설정할 수 있다.Through these operations, the apparatus for setting a deep learning algorithm according to the present invention can set a deep learning algorithm for autonomous driving according to driving environment information more efficiently and quickly by minimizing unnecessary computational operations.
도 4는 본 발명에 따른 딥러닝 알고리즘 설정 장치 및 주변 장치를 간단히 나타낸 도면이다.4 is a diagram briefly showing a device for setting a deep learning algorithm and a peripheral device according to the present invention.
본 발명의 실시예에 따라, 자율 주행을 위한 딥러닝 알고리즘 설정 장치 (400)는 자율 주행 차량의 자율 주행 제어 시스템에 포함되거나 또는 상기 자율 주행 제어 시스템과 별도의 장치로 구현될 수 있다. 다른 실시예로써, 상기 딥러닝 알고리즘 설정 장치 (400)는 상기 자율 주행 제어 시스템을 포함할 수도 있다. 다시 말해, 실시예에 따라 상기 딥러닝 알고리즘 설정 장치 (400)는 자율 주행 차량 시스템의 일부 장치로 구현될 수도 있고, 상기 자율 주행 차량 시스템을 포함하는 전체 시스템 장치로 구현될 수도 있다.According to an embodiment of the present invention, the apparatus 400 for setting a deep learning algorithm for autonomous driving may be included in an autonomous driving control system of an autonomous driving vehicle or implemented as a device separate from the autonomous driving control system. As another embodiment, the deep learning algorithm setting apparatus 400 may include the autonomous driving control system. In other words, according to an embodiment, the deep learning algorithm setting apparatus 400 may be implemented as a part of the autonomous driving vehicle system or as a whole system device including the autonomous driving vehicle system.
이와 같은 딥러닝 알고리즘 설정 장치 (400)는, 도 4에 도시된 바와 같이, 주행 환경 정보 결정부 (410), 딥러닝 모델 및 딥러닝 파라미터 세트 결정부 (420), 및 딥러닝 알고리즘 설정부 (430)를 포함할 수 있다. As shown in FIG. 4, such a deep learning algorithm setting device 400 includes a driving environment information determining unit 410, a deep learning model and deep learning parameter set determining unit 420, and a deep learning algorithm setting unit ( 430) may be included.
주행 환경 정보 결정부 (410)는 카메라 장치 (10), 또는 외부 정보 수신 장치 (20)로부터 획득된 입력 정보들을 이용하여 앞서 상술한 주행 환경 정보 결정 단계와 같이 주행 환경 정보를 결정할 수 있다.The driving environment information determining unit 410 may determine the driving environment information by using the input information obtained from the camera device 10 or the external information receiving device 20 as in the above-described driving environment information determining step.
딥러닝 모델 및 딥러닝 파라미터 세트 결정부 (420)는 상기 주행 환경 정보 결정부 (410)에 의해 결정된 주행 환경 정보를 이용하여 앞서 상술한 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계와 같이 딥러닝 모델 및 딥러닝 파라미터 세트를 결정/선택할 수 있다. 이때, 하나 이상의 딥러닝 모델에 대한 정보 및 각 딥러닝 모델 별 하나 이상의 딥러닝 파라미터 세트에 대한 정보는 별도의 저장 장치 (예: 데이터 베이스 등)에 저장될 수 있다. 이때, 상기 저장 장치는 실시예에 따라 본 발명에 따른 딥러닝 알고리즘 설정 장치 (400)에 포함되거나 상기 딥러닝 알고리즘 설정 장치 (400)외 외부에 위치할 수도 있다.The deep learning model and deep learning parameter set determining unit 420 uses the driving environment information determined by the driving environment information determining unit 410 to make a deep learning model like the above-described deep learning model and deep learning parameter set determining step. and a set of deep learning parameters may be determined/selected. In this case, information on one or more deep learning models and information on one or more deep learning parameter sets for each deep learning model may be stored in a separate storage device (eg, a database, etc.). In this case, the storage device may be included in the deep learning algorithm setting apparatus 400 according to the present invention or located outside the deep learning algorithm setting apparatus 400 according to an embodiment.
딥러닝 알고리즘 설정부 (430)는 앞서 상술한 딥러닝 알고리즘 설정 단계와 같이 결정된 딥러닝 모델 및 딥러닝 파라미터 세트를 자율 주행을 위한 딥러닝 알고리즘으로 설정할 수 있다.The deep learning algorithm setting unit 430 may set the deep learning model and the deep learning parameter set determined as in the above-described deep learning algorithm setting step as a deep learning algorithm for autonomous driving.
본 발명에 적용 가능한 일 예로, 딥러닝 알고리즘 설정 장치 (400)는 차량에 설치된 카메라 장치 (10), 외부 정보 수신 장치 (20) 등과 연결되어 상기 카메라 장치 (10) 및 외부 정보 수신 장치 (20)로부터 관련 정보를 획득할 수 있다. 본 발명에 적용 가능한 다른 예로, 상기 딥러닝 알고리즘 설정 장치 (400)는 상기 카메라 장치 (10) 및 외부 정보 수신 장치 (20)를 포함하여 상기 카메라 장치 (10) 및 외부 정보 수신 장치 (20)를 통해 획득한 관련 정보를 활용할 수도 있다.As an example applicable to the present invention, the deep learning algorithm setting device 400 is connected to the camera device 10 installed in the vehicle, the external information receiving device 20, etc., and the camera device 10 and the external information receiving device 20 Relevant information can be obtained from As another example applicable to the present invention, the deep learning algorithm setting device 400 includes the camera device 10 and the external information receiving device 20, including the camera device 10 and the external information receiving device 20 You can also use the relevant information obtained through
또한, 본 발명에 적용 가능한 일 예로, 딥러닝 알고리즘 설정 장치 (400)는 차량 시스템 내 자율 주행을 제어하는 자율 주행 제어 장치에 연결되어 상기 자율 주행 제어 장치가 이용하는 딥러닝 알고리즘을 설정/선택하여 상기 자율 주행 제어 장치로 제공할 수 있다. 이를 위해, 상기 딥러닝 알고리즘 설정 장치 (400)는 상기 자율 주행 제어 시스템으로 결정된 딥러닝 모델 및 딥러닝 파라미터 세트에 대한 정보를 제공함으로써 상기 결정된 딥러닝 모델 및 딥러닝 파라미터 세트를 자율 주행을 위한 딥러닝 알고리즘으로 설정할 수 있다. 다른 예로, 상기 딥러닝 알고리즘 설정 장치 (400)가 상기 자율 주행 제어 시스템을 포함하는 경우, 상기 딥러닝 알고리즘 설정 장치(400)는 상기 자율 주행 제어 시스템으로 하여금 상기 결정된 딥러닝 모델 및 딥러닝 파라미터 세트를 자율 주행을 위한 딥러닝 알고리즘으로 설정하도록 제어할 수도 있다.In addition, as an example applicable to the present invention, the deep learning algorithm setting device 400 is connected to an autonomous driving control device that controls autonomous driving in a vehicle system, and sets/selects a deep learning algorithm used by the autonomous driving control device to set/select the It can be provided as an autonomous driving control device. To this end, the deep learning algorithm setting device 400 provides information about the deep learning model and the deep learning parameter set determined by the autonomous driving control system, and thus uses the determined deep learning model and the deep learning parameter set for autonomous driving. It can be set as a learning algorithm. As another example, when the deep learning algorithm setting apparatus 400 includes the autonomous driving control system, the deep learning algorithm setting apparatus 400 causes the autonomous driving control system to determine the deep learning model and the deep learning parameter set can also be controlled to be set as a deep learning algorithm for autonomous driving.
또한, 본 발명에 따른 딥러닝 알고리즘 설정 장치 (400)는 앞서 상술한 다양한 딥러닝 알고리즘 설정 방법에 따라 동작할 수 있다. In addition, the deep learning algorithm setting apparatus 400 according to the present invention may operate according to the various deep learning algorithm setting methods described above.
추가적으로, 본 발명에 따른 컴퓨터 프로그램은, 컴퓨터와 결합하여, 앞서 상술한 다양한 자율 주행을 위한 딥러닝 알고리즘 설정 방법을 실행시키기 위하여 컴퓨터 판독가능 기록매체에 저장될 수 있다.Additionally, the computer program according to the present invention may be stored in a computer-readable recording medium in combination with a computer to execute the deep learning algorithm setting method for various autonomous driving described above.
전술한 프로그램은, 컴퓨터가 프로그램을 읽어 들여 프로그램으로 구현된 상기 방법들을 실행시키기 위하여, 상기 컴퓨터의 프로세서(CPU)가 상기 컴퓨터의 장치 인터페이스를 통해 읽힐 수 있는 C, C++, JAVA, 기계어 등의 컴퓨터 언어로 코드화된 코드(Code)를 포함할 수 있다. 이러한 코드는 상기 방법들을 실행하는 필요한 기능들을 정의한 함수 등과 관련된 기능적인 코드(Functional Code)를 포함할 수 있고, 상기 기능들을 상기 컴퓨터의 프로세서가 소정의 절차대로 실행시키는데 필요한 실행 절차 관련 제어 코드를 포함할 수 있다. 또한, 이러한 코드는 상기 기능들을 상기 컴퓨터의 프로세서가 실행시키는데 필요한 추가 정보나 미디어가 상기 컴퓨터의 내부 또는 외부 메모리의 어느 위치(주소 번지)에서 참조되어야 하는지에 대한 메모리 참조관련 코드를 더 포함할 수 있다. 또한, 상기 컴퓨터의 프로세서가 상기 기능들을 실행시키기 위하여 원격(Remote)에 있는 어떠한 다른 컴퓨터나 서버 등과 통신이 필요한 경우, 코드는 상기 컴퓨터의 통신 모듈을 이용하여 원격에 있는 어떠한 다른 컴퓨터나 서버 등과 어떻게 통신해야 하는지, 통신 시 어떠한 정보나 미디어를 송수신해야 하는지 등에 대한 통신 관련 코드를 더 포함할 수 있다.The above-described program is a computer such as C, C++, JAVA, machine language, etc. that the processor (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as the program. It may include code (Code) coded in the language. Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, this code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.
본 발명의 실시예와 관련하여 설명된 방법 또는 알고리즘의 단계들은 하드웨어로 직접 구현되거나, 하드웨어에 의해 실행되는 소프트웨어 모듈로 구현되거나, 또는 이들의 결합에 의해 구현될 수 있다. 소프트웨어 모듈은 RAM(Random Access Memory), ROM(Read Only Memory), EPROM(Erasable Programmable ROM), EEPROM(Electrically Erasable Programmable ROM), 플래시 메모리(Flash Memory), 하드 디스크, 착탈형 디스크, CD-ROM, 또는 본 발명이 속하는 기술 분야에서 잘 알려진 임의의 형태의 컴퓨터 판독가능 기록매체에 상주할 수도 있다.The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.
이상, 첨부된 도면을 참조로 하여 본 발명의 실시예를 설명하였지만, 본 발명이 속하는 기술분야의 통상의 기술자는 본 발명이 그 기술적 사상이나 필수적인 특징을 변경하지 않고서 다른 구체적인 형태로 실시될 수 있다는 것을 이해할 수 있을 것이다. 그러므로, 이상에서 기술한 실시예들은 모든 면에서 예시적인 것이며, 제한적이 아닌 것으로 이해해야만 한다.As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.
Claims (10)
- 차량 외부의 이미지 정보를 포함하는 입력 정보에 기초하여, 차량의 주행 환경 정보를 결정하는 주행 환경 정보 결정 단계;a driving environment information determination step of determining driving environment information of the vehicle based on input information including image information outside the vehicle;상기 결정된 주행 환경 정보에 대응하는 딥러닝 모델 및 상기 딥러닝 모델의 딥러닝 파라미터 세트를 결정하는 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계; 및a deep learning model and a deep learning parameter set determining step of determining a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and상기 결정된 딥러닝 모델에 상기 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 상기 차량의 자율 주행을 위한 딥러닝 알고리즘으로 설정하는 딥러닝 알고리즘 설정 단계를 포함하는 것을 특징으로 하는, A deep learning algorithm setting step of setting a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle,자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 1항에 있어서,The method of claim 1,상기 입력 정보는,The input information isGPS (Global Positioning System) 신호, 상기 차량이 주행 중인 도로와 관련된 방송 신호, 상기 차량이 주행 중인 도로와 관련된 전용 신호 중 적어도 하나 이상을 포함하는 외부 신호 정보를 더 포함하는 것을 특징으로 하는,GPS (Global Positioning System) signal, characterized in that it further comprises external signal information including at least one of a broadcast signal related to the road on which the vehicle is traveling, and a dedicated signal related to the road on which the vehicle is traveling,자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 2항에 있어서,3. The method of claim 2,상기 주행 환경 정보 결정 단계는,The driving environment information determination step includes:상기 차량 외부의 이미지 정보를 이용하여 제1 주행 환경 정보를 추론하는 단계;inferring first driving environment information using image information outside the vehicle;상기 외부 신호 정보를 이용하여 제2 주행 환경 정보를 획득하는 단계; 및obtaining second driving environment information by using the external signal information; and상기 제1 주행 환경 정보 및 상기 제2 주행 환경 정보를 모두 이용하여 상기 차량의 주행 환경 정보를 결정하되, 상기 제1 주행 환경 정보의 제1 세부 정보와 상기 제2 주행 환경 정보의 제2 세부 정보가 상이할 경우, 상기 제1 세부 정보와 관련된 확률 값 및 대응하는 문턱 값의 비교 결과에 기반하여 상기 제1 세부 정보 또는 상기 제2 세부 정보를 상기 주행 환경 정보의 세부 정보로 결정하는 단계를 포함하고,The driving environment information of the vehicle is determined using both the first driving environment information and the second driving environment information, and the first detailed information of the first driving environment information and the second detailed information of the second driving environment information if different, determining the first detailed information or the second detailed information as detailed information of the driving environment information based on a comparison result of a probability value related to the first detailed information and a corresponding threshold value do,상기 문턱 값은 대응하는 세부 정보의 종류에 따라 상이하게 설정되는 것을 특징으로 하는, The threshold value is characterized in that it is set differently according to the type of corresponding detailed information,자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 1항에 있어서,The method of claim 1,상기 차량의 주행 환경 정보는 상기 입력 정보를 이용한 딥러닝 알고리즘에 기초하여 결정되고,The driving environment information of the vehicle is determined based on a deep learning algorithm using the input information,상기 결정된 주행 환경 정보는,The determined driving environment information is상기 차량이 주행 중인 위치의 날씨 정보;weather information of a location in which the vehicle is driving;상기 차량이 주행 중인 도로의 종류;the type of road on which the vehicle is traveling;상기 차량이 주행 중인 도로의 정체 정보;congestion information of the road on which the vehicle is traveling;상기 차량의 시야 밝기 정보;information on the brightness of the field of view of the vehicle;태양의 방향 및 고도 정보; 또는Sun direction and altitude information; or상기 차량이 주행 중인 위치의 법규 정보, 중 적어도 하나 이상을 포함하는 것을 특징으로 하는,Law information of a location in which the vehicle is driving, characterized in that it includes at least one or more of자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 4항에 있어서,5. The method of claim 4,상기 결정된 딥러닝 모델은 상기 주행 환경 정보 중 제1 정보 세트에 기초하여 결정되고,The determined deep learning model is determined based on a first set of information among the driving environment information,상기 결정된 딥러닝 파라미터 세트는 상기 주행 환경 정보 중 상기 제1 정보 세트를 포함한 제2 정보 세트에 기초하여 결정되는 것을 특징으로 하는, The determined deep learning parameter set is characterized in that it is determined based on a second information set including the first information set among the driving environment information,자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 5항에 있어서,6. The method of claim 5,상기 제1 정보 세트는 상기 차량이 주행 중인 도로의 종류를 포함하는 것을 특징으로 하는, characterized in that the first set of information includes the type of road on which the vehicle is traveling,자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 제 1항에 있어서,The method of claim 1,상기 주행 환경 정보 결정 단계는 일정 주기마다 수행되거나 실시간으로 수행되고,The driving environment information determination step is performed at regular intervals or in real time,상기 주행 환경 정보 결정 단계를 통해 결정된 상기 차량의 주행 환경 정보가 직전에 결정된 상기 차량의 주행 환경 정보와 상이한 경우, 상기 딥러닝 모델 및 딥러닝 파라미터 세트 결정 단계 및 상기 딥러닝 알고리즘 설정 단계가 수행되는 것을 특징으로 하는, When the driving environment information of the vehicle determined through the driving environment information determining step is different from the driving environment information of the vehicle determined immediately before, the deep learning model and deep learning parameter set determining step and the deep learning algorithm setting step are performed characterized in that자율 주행을 위한 딥러닝 알고리즘 설정 방법.How to set up a deep learning algorithm for autonomous driving.
- 차량 외부의 이미지 정보를 포함하는 입력 정보에 기초하여, 차량의 주행 환경 정보를 결정하는 주행 환경 정보 결정부;a driving environment information determining unit configured to determine driving environment information of the vehicle based on input information including image information outside the vehicle;상기 결정된 주행 환경 정보에 대응하는 딥러닝 모델 및 상기 딥러닝 모델의 딥러닝 파라미터 세트를 결정하는 딥러닝 모델 및 딥러닝 파라미터 세트 결정부; 및a deep learning model and a deep learning parameter set determiner configured to determine a deep learning model corresponding to the determined driving environment information and a deep learning parameter set of the deep learning model; and상기 결정된 딥러닝 모델에 상기 결정된 딥러닝 파라미터 세트가 적용되는 딥러닝 알고리즘을 상기 차량의 자율 주행을 위한 딥러닝 알고리즘으로 설정하는 딥러닝 알고리즘 설정부를 포함하는 것을 특징으로 하는, A deep learning algorithm setting unit configured to set a deep learning algorithm to which the determined deep learning parameter set is applied to the determined deep learning model as a deep learning algorithm for autonomous driving of the vehicle,자율 주행을 위한 딥러닝 알고리즘 설정 장치.Deep learning algorithm setting device for autonomous driving.
- 제 8항에 있어서,9. The method of claim 8,상기 주행 환경 정보 결정부는 상기 입력 정보를 이용한 딥러닝 알고리즘에 기초하여 상기 주행 환경 정보를 결정하고,The driving environment information determining unit determines the driving environment information based on a deep learning algorithm using the input information,상기 결정된 주행 환경 정보는,The determined driving environment information is상기 차량이 주행 중인 위치의 날씨 정보;weather information of a location in which the vehicle is driving;상기 차량이 주행 중인 도로의 종류;the type of road on which the vehicle is traveling;상기 차량이 주행 중인 도로의 정체 정보;congestion information of the road on which the vehicle is traveling;상기 차량의 시야 밝기 정보;information on the brightness of the field of view of the vehicle;태양의 방향 및 고도 정보; 또는Sun direction and altitude information; or상기 차량이 주행 중인 위치의 법규 정보, 중 적어도 하나 이상을 포함하고,Including at least one or more of the legal information of the location in which the vehicle is driving,상기 결정된 딥러닝 모델은 상기 주행 환경 정보 중 일부 정보에 기초하여 결정되고,The determined deep learning model is determined based on some information of the driving environment information,상기 결정된 딥러닝 파라미터 세트는 모든 상기 주행 환경 정보에 기초하여 결정되는 것을 특징으로 하는, The determined deep learning parameter set is characterized in that it is determined based on all the driving environment information,자율 주행을 위한 딥러닝 알고리즘 설정 장치.Deep learning algorithm setting device for autonomous driving.
- 컴퓨터와 결합하여, 제1 항의 자율 주행을 위한 딥러닝 알고리즘 설정 방법을 실행시키기 위하여 컴퓨터 판독가능 기록매체에 저장된 컴퓨터 프로그램.In combination with a computer, a computer program stored in a computer-readable recording medium to execute the method of setting a deep learning algorithm for autonomous driving of claim 1.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/336,535 US20230331250A1 (en) | 2020-12-22 | 2023-06-16 | Method and apparatus for configuring deep learning algorithm for autonomous driving |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR10-2020-0180500 | 2020-12-22 | ||
KR1020200180500A KR102260246B1 (en) | 2020-12-22 | 2020-12-22 | Mehod and Device of configuring deep learning algorithm for autonomous driving |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/336,535 Continuation US20230331250A1 (en) | 2020-12-22 | 2023-06-16 | Method and apparatus for configuring deep learning algorithm for autonomous driving |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022139009A1 true WO2022139009A1 (en) | 2022-06-30 |
Family
ID=76391488
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2020/018864 WO2022139009A1 (en) | 2020-12-22 | 2020-12-22 | Method and apparatus for configuring deep learning algorithm for autonomous driving |
Country Status (3)
Country | Link |
---|---|
US (1) | US20230331250A1 (en) |
KR (1) | KR102260246B1 (en) |
WO (1) | WO2022139009A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115885310A (en) | 2021-07-30 | 2023-03-31 | 蒂普爱可斯有限公司 | Control method of image signal processor and control apparatus for performing the same |
KR102587693B1 (en) * | 2021-10-08 | 2023-10-12 | 주식회사 디알엠인사이드 | Image identification apparatus for protecting copyright of image and operation method thereof |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20150071419A (en) * | 2013-12-18 | 2015-06-26 | 국방과학연구소 | Method and Apparatus for Path Planning of Unmanned Ground Vehicle in Dynamic Environment |
KR20180109190A (en) * | 2017-03-27 | 2018-10-08 | 현대자동차주식회사 | Apparatus for controlling autonomous vehicle based on deep learning, system having the same and method thereof |
KR20190016332A (en) * | 2017-08-08 | 2019-02-18 | 주식회사 만도 | Deep learning based self-driving car, deep learning based self-driving control device, and deep learning based self-driving control method |
EP3477616A1 (en) * | 2017-10-27 | 2019-05-01 | Sigra Technologies GmbH | Method for controlling a vehicle using a machine learning system |
KR20200095381A (en) * | 2019-01-31 | 2020-08-10 | 주식회사 스트라드비젼 | Method and device for providing personalized and calibrated adaptive deep learning model for the user of an autonomous vehicle |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102481487B1 (en) | 2018-02-27 | 2022-12-27 | 삼성전자주식회사 | Autonomous driving apparatus and method thereof |
-
2020
- 2020-12-22 WO PCT/KR2020/018864 patent/WO2022139009A1/en active Application Filing
- 2020-12-22 KR KR1020200180500A patent/KR102260246B1/en active IP Right Grant
-
2023
- 2023-06-16 US US18/336,535 patent/US20230331250A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20150071419A (en) * | 2013-12-18 | 2015-06-26 | 국방과학연구소 | Method and Apparatus for Path Planning of Unmanned Ground Vehicle in Dynamic Environment |
KR20180109190A (en) * | 2017-03-27 | 2018-10-08 | 현대자동차주식회사 | Apparatus for controlling autonomous vehicle based on deep learning, system having the same and method thereof |
KR20190016332A (en) * | 2017-08-08 | 2019-02-18 | 주식회사 만도 | Deep learning based self-driving car, deep learning based self-driving control device, and deep learning based self-driving control method |
EP3477616A1 (en) * | 2017-10-27 | 2019-05-01 | Sigra Technologies GmbH | Method for controlling a vehicle using a machine learning system |
KR20200095381A (en) * | 2019-01-31 | 2020-08-10 | 주식회사 스트라드비젼 | Method and device for providing personalized and calibrated adaptive deep learning model for the user of an autonomous vehicle |
Also Published As
Publication number | Publication date |
---|---|
KR102260246B1 (en) | 2021-06-04 |
US20230331250A1 (en) | 2023-10-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3859708A1 (en) | Traffic light image processing method and device, and roadside device | |
WO2019098449A1 (en) | Apparatus related to metric-learning-based data classification and method thereof | |
WO2021221254A1 (en) | Method for performing continual learning on classifier in client capable of classifying images by using continual learning server and continual learning server using the same | |
WO2022139009A1 (en) | Method and apparatus for configuring deep learning algorithm for autonomous driving | |
CN112396093B (en) | Driving scene classification method, device and equipment and readable storage medium | |
KR102299564B1 (en) | Method and system for ai learning using de-identified image data | |
WO2019098418A1 (en) | Neural network training method and device | |
WO2021225296A1 (en) | Method for explainable active learning, to be used for object detector, by using deep encoder and active learning device using the same | |
WO2021225360A1 (en) | Method for performing on-device learning of machine learning network on autonomous vehicle by using multi-stage learning with adaptive hyper-parameter sets and device using the same | |
WO2021215740A1 (en) | Method and device for on-vehicle active learning to be used for training perception network of autonomous vehicle | |
WO2021235682A1 (en) | Method and device for performing behavior prediction by using explainable self-focused attention | |
CN109353345A (en) | Control method for vehicle, device, equipment, medium and vehicle | |
CN112180903A (en) | Vehicle state real-time detection system based on edge calculation | |
WO2020032506A1 (en) | Vision detection system and vision detection method using same | |
CN111680547A (en) | Method and device for identifying traffic countdown board, electronic equipment and storage medium | |
CN114241792B (en) | Traffic flow detection method and system | |
CN111860539A (en) | License plate color recognition method, device and medium | |
WO2019059460A1 (en) | Image processing apparatus and method | |
WO2022014831A1 (en) | Object detection method and device | |
CN115439815A (en) | Driving condition identification method, device, equipment, medium and vehicle | |
CN113221982A (en) | Vehicle identification method, vehicle identification model creation method and related components | |
CN106097751A (en) | Vehicle travel control method and device | |
WO2024150848A1 (en) | Method and apparatus for automatically converting two-dimensional drawing into three-dimensional drawing by using artificial intelligence learning model | |
CN117372996B (en) | Traffic signal lamp image detection method based on improved yolov model | |
WO2020231188A1 (en) | Classification result verifying method and classification result learning method which use verification neural network, and computing device for performing methods |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20967080 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 181023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20967080 Country of ref document: EP Kind code of ref document: A1 |