JP7317165B2 - Obstacle detection system for work vehicles - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle detection system for work vehicles used in riding work vehicles such as tractors and riding lawn mowers, and unmanned work vehicles such as unmanned lawn mowers.

In the work vehicle, a plurality of obstacle detection sensors that detect obstacles existing around the work vehicle and obtain the relative positions of the obstacles, a plurality of cameras that obtain images of the surroundings of the work vehicle, a plurality of A bird's-eye view image acquisition unit that acquires a bird's-eye view image of the surroundings of the work vehicle based on a plurality of images acquired by the camera, and a warning area that sets a warning area where it is necessary to warn the driver of the presence of obstacles. and a warning unit that warns the driver of the existence of the obstacle when the relative position of the obstacle is within the warning area on the bird's-eye view image, and is configured to monitor the surroundings of the work vehicle. Some are equipped with a perimeter monitoring device (see, for example, Patent Document 1).

International Publication No. 2012/169361

In the perimeter monitoring device described in Patent Document 1 mentioned above, a radar device with low object discrimination accuracy is employed as an obstacle detection sensor. Therefore, if there is tall grass growing in the warning area, or if dust or dust particles are floating in the warning area, the tall grass or floating objects will become an obstacle. The detection sensor may erroneously detect it as an obstacle. If such an erroneous detection occurs, the warning unit warns the driver that there is an obstacle within the warning area even though there is no obstacle that would impede the running of the work vehicle. become.

In view of this situation, a main object of the present invention is to construct an obstacle detection system for a work vehicle that can accurately detect obstacles existing around the work vehicle.

An obstacle detection system for a work vehicle according to one aspect of the present invention includes an image processing device that performs obstacle determination processing for determining an obstacle based on an image from an imaging device that captures an image of the surroundings of the work vehicle. The image processing device changes a processing target cycle per unit time for the imaging device according to at least one of a traveling direction and a vehicle speed of the work vehicle.

Diagram showing schematic configuration of automatic driving system for work vehicle Top view of the tractor showing the imaging range of each camera Side view of the tractor showing the measurement range of each lidar sensor and sonar Top view of the tractor showing the measurement range of each lidar sensor and sonar A plan view showing an example of a target route for automatic driving Block diagram showing a schematic configuration of an automatic driving system for work vehicles Block diagram showing a schematic configuration of an obstacle detection system for work vehicles A diagram showing an obstacle detection range and non-detection range in the range image of the front lidar sensor. FIG. 11 is a diagram showing an obstacle detection range and an obstacle non-detection range in the working device lowered state in the distance image of the rear rider sensor; FIG. 11 is a diagram showing an obstacle detection range and an obstacle non-detection range in a rear lidar sensor range image in the working device raised state; 4 is a flow chart showing a processing procedure of an image processing apparatus in image processing; A plan view showing the positional relationship between the mounting position of each camera, the origin of the vehicle body coordinates, and the distance calculation reference point. A diagram showing an example of the arrangement of camera images on a liquid crystal monitor or display device A diagram showing the processing target cycle per unit time for each camera in the time division method Flowchart showing the control operation of the obstacle detection device in the first specific control Flowchart showing the control operation of the obstacle detection device in the third specific control A diagram showing an example of arrangement of camera images on a liquid crystal monitor or a display device in another embodiment.

An embodiment in which an obstacle detection system for a work vehicle according to the present invention is applied to a tractor, which is an example of a work vehicle, will be described below as an example of a form for carrying out the present invention, based on the drawings.
In addition, the obstacle detection system for work vehicles according to the present invention is applicable to riding work vehicles other than tractors, such as riding lawn mowers, riding rice planters, combine harvesters, trucks, snow plows, wheel loaders, and unmanned grass cutting machines. It can be applied to unmanned working vehicles such as machines.

As shown in FIGS. 1 to 4, a tractor 1 exemplified in this embodiment has a rotary tillage device 3, which is an example of a working device, connected via a three-point link mechanism 2 to its rear portion so as to be able to move up and down and roll. It is As a result, the tractor 1 is configured for rotary tillage. The tractor 1 can automatically travel in a farm field A shown in FIG.
Various working devices such as a plow, a disk harrow, a cultivator, a subsoiler, a seeding device, a spreading device, and a mowing device can be connected to the rear portion of the tractor 1 instead of the rotary tillage device 3.

As shown in FIG. 6, the automatic traveling system includes an automatic traveling unit 4 mounted on a tractor 1, and a mobile communication terminal 5, which is an example of a wireless communication device set to be able to communicate wirelessly with the automatic traveling unit 4. It is included. The mobile communication terminal 5 is provided with a multi-touch display device (for example, a liquid crystal panel) 50 that enables various information display and input operations related to automatic driving.
As the mobile communication terminal 5, a tablet personal computer, a smart phone, or the like can be adopted. For wireless communication, a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark) or short-range wireless communication such as Bluetooth (registered trademark) can be employed.

As shown in FIGS. 1 to 3 and 6, the tractor 1 includes left and right drivable and steerable front wheels 10, left and right drivable rear wheels 11, a cabin 13 forming a ride-on driving section 12, a common rail An electronically controlled diesel engine (hereinafter referred to as the engine) 14 having a system, a bonnet 15 covering the engine 14, etc., a transmission unit 16 for shifting the power from the engine 14, and a full hydraulic power steering for the left and right front wheels 10. A steering unit 17, a brake unit 18 for braking the left and right rear wheels 11, an electro-hydraulic control type working clutch unit 19 for intermittently transmitting power to the rotary tillage device 3, and an electro-hydraulic control type elevation for driving the rotary tillage device 3 up and down. A drive unit 20, an electro-hydraulically controlled rolling unit 21 for driving the rotary tillage device 3 in the roll direction, and vehicle state detection including various sensors and switches for detecting various setting states and operating states of each part in the tractor 1. A device 22 and an in-vehicle control unit 23 having various control units are provided.
The engine 14 may be an electronically controlled gasoline engine having an electronic governor. Also, the power steering unit 17 may be of an electric type having an electric motor for steering.

As shown in FIGS. 1 and 3, the operation unit 12 includes a steering wheel 25 for manual steering, a seat 26 for passengers, and a multi-touch liquid crystal display that enables various information displays and input operations. A monitor 27 is provided. Although not shown, the operation unit 12 includes operation levers such as an accelerator lever and a shift lever, and operation pedals such as an accelerator pedal and a clutch pedal.

Although not shown, the transmission unit 16 includes an electronically controlled continuously variable transmission that changes the speed of the power from the engine 14, and an electronic controller that switches the power after the speed change by the continuously variable transmission between forward and reverse. It includes a hydraulically controlled forward/reverse switching device. The continuously variable transmission includes the I-HMT (Integrated Hydro-static Mechanical Transmission), which is an example of a hydromechanical continuously variable transmission that has higher transmission efficiency than a hydrostatic transmission (HST). Adopted. The forward/reverse switching device includes a hydraulic clutch for connecting and disconnecting forward drive power, a hydraulic clutch for connecting and disconnecting reverse drive power, and an electromagnetic valve for controlling oil flow to them.
Instead of the I-HMT, the continuously variable transmission may be a hydraulic mechanical transmission (HMT), a hydrostatic continuously variable transmission, or a belt-type continuously variable transmission. , etc., may be employed. Further, instead of a continuously variable transmission, the transmission unit 16 includes an electrohydraulically controlled stepped transmission having a plurality of hydraulic clutches for transmission and a plurality of electromagnetic valves for controlling oil flow to them. It may be

Although not shown, the brake unit 18 includes left and right brakes for individually braking the left and right rear wheels 11, and the left and right brakes are operated in conjunction with the stepping operation of the left and right brake pedals provided in the driving unit 12. A foot brake system, a parking brake system that operates the left and right brakes in conjunction with the operation of the parking lever provided in the driving unit 12, and a brake on the inside of the turn in conjunction with the steering of the left and right front wheels 10 at a set angle or more. It includes a turning brake system to operate.

The vehicle state detection device 22 is a general term for various sensors, switches, etc. provided in each part of the tractor 1 . As shown in FIG. 7, the vehicle state detection device 22 includes a vehicle speed sensor 22A that detects the vehicle speed of the tractor 1, a reverser sensor 22B that detects the operating position of a reverser lever for switching forward and backward, and a steering angle of the front wheels 10. A steering angle sensor 22C for detecting is included. Although not shown, the vehicle state detection device 22 includes a rotation sensor for detecting the output rotation speed of the engine 14, an accelerator sensor for detecting the operating position of an accelerator lever, and a speed change sensor for detecting the operating position of a shift lever. sensors, etc.

As shown in FIGS. 6 and 7, the in-vehicle control unit 23 includes an engine control section 23A for controlling the engine 14, a vehicle speed control section 23B for controlling the vehicle speed of the tractor 1 and switching between forward and backward travel, and a steering control section. A steering control unit 23C, a work device control unit 23D that controls work devices such as the rotary tillage device 3, a display control unit 23E that controls display and notification on the liquid crystal monitor 27, and an automatic travel control unit that performs control related to automatic travel. 23F, and a non-volatile in-vehicle storage unit 23G that stores a target route P (see FIG. 5) for automatic travel generated according to the travel areas divided in the field. Each control section 23A to 23F is constructed by an electronic control unit in which a microcontroller or the like is integrated, various control programs, or the like. The controllers 23A to 23F are interconnected via CAN (Controller Area Network) so as to be able to communicate with each other.
For mutual communication between the control units 23A to 23F, a communication standard other than CAN or a next-generation communication standard such as in-vehicle Ethernet or CAN-FD (CAN with Flexible Data rate) may be adopted.

The engine control unit 23A executes engine speed maintenance control for maintaining the engine speed at a speed corresponding to the operating position of the accelerator lever based on the detection information from the accelerator sensor and the detection information from the rotation sensor. do.

The vehicle speed control unit 23B controls the continuously variable transmission so that the vehicle speed of the tractor 1 is changed to a speed corresponding to the operation position of the shift lever based on the detection information from the shift sensor and the detection information from the vehicle speed sensor 22A. Vehicle speed control for controlling the operation and forward/reverse switching control for switching the transmission state of the forward/reverse switching device based on the information detected from the reverser sensor 22B are executed. The vehicle speed control includes deceleration stop processing for stopping the tractor 1 from traveling by controlling the speed reduction of the continuously variable transmission to the zero speed state when the shift lever is operated to the zero speed position.

The work device control unit 23D includes a work clutch control that controls the operation of the work clutch unit 19 based on the operation of the PTO switch, etc. Elevation control for controlling the operation, rolling control for controlling the operation of the rolling unit 21 based on the set value of the roll angle setting dial, etc. are executed. A PTO switch, a lift switch, a height setting dial, and a roll angle setting dial are included in the vehicle state detection device 22 .

As shown in FIG. 6, the tractor 1 is equipped with a positioning unit 30 for measuring the current position and current azimuth of the tractor 1 . The positioning unit 30 includes satellite navigation devices 31 and 3 that measure the current position and current direction of the tractor 1 using a GNSS (Global Navigation Satellite System), which is an example of a navigation satellite system (NSS). It has an inertial measurement unit (IMU) 32 which has an axial gyroscope, three-directional acceleration sensors, etc., and measures the attitude and orientation of the tractor 1, and the like. Positioning methods using GNSS include DGNSS (Differential GNSS: relative positioning system) and RTK-GNSS (Real Time Kinematic GNSS: interferometric positioning system). In this embodiment, RTK-GNSS suitable for positioning of mobile units is adopted. Therefore, as shown in FIG. 1, reference stations 6 that enable positioning by RTK-GNSS are installed at known positions around the field.

As shown in FIGS. 1 and 6, the tractor 1 and the reference station 6 are provided with GNSS antennas 33 and 60 for receiving radio waves transmitted from the positioning satellite 7 (see FIG. 1). Communication modules 34, 61, etc. are provided to enable wireless communication of each information including positioning information between. As a result, the satellite navigation device 31 of the positioning unit 30 receives the positioning information obtained by the GNSS antenna 33 on the tractor side receiving the radio waves from the positioning satellite 7, and the GNSS antenna 60 on the base station side receives the radio waves from the positioning satellite 7. Based on the positioning information obtained by receiving , the current position and current azimuth of the tractor 1 can be measured with high accuracy. Further, the positioning unit 30 has a satellite navigation device 31 and an inertial measurement device 32, so that the current position, current azimuth, and attitude angle (yaw angle, roll angle, pitch angle) of the tractor 1 can be measured with high precision. can be done.

In this tractor 1, the inertial measurement device 32, the GNSS antenna 33, and the communication module 34 of the positioning unit 30 are included in the antenna unit 35 shown in FIG. The antenna unit 35 is arranged at the left-right central portion of the upper portion on the front side of the cabin 13 . And the attachment position of the GNSS antenna 33 in the tractor 1 becomes a positioning object position when measuring the present position of the tractor 1 using GNSS.

As shown in FIG. 6, the mobile communication terminal 5 includes an electronic control unit integrated with a microcontroller, a terminal control unit 51 having various control programs, and a positioning control unit 51 with a communication module 34 on the tractor side. A communication module 52 and the like are provided to enable wireless communication of each information including information. The terminal control unit 51 includes a display control unit 51A that controls display and notification on the display device 50, a target route generation unit 51B that generates a target route P for automatic driving, and a target route generation unit 51B that generates A non-volatile terminal storage unit 51C for storing the target route P and the like are included. The terminal storage unit 51C stores various types of information used to generate the target route P, such as vehicle body information such as the turning radius and working width of the tractor 1, and field information obtained from the positioning information described above. there is The field information includes a plurality of shape identification points in the field A obtained using GNSS when the tractor 1 is driven along the outer periphery of the field A in order to identify the shape and size of the field A. Four corner points Ap1 to Ap4 (see FIG. 5) that serve as (shape identification coordinates), and rectangular shape identification that connects the corner points Ap1 to Ap4 to identify the shape and size of the field A. line AL (see FIG. 5), and so on.

The target route generator 51B generates the target route P based on the turning radius and working width of the tractor 1 included in the vehicle body information, the shape and size of the farm field A included in the farm field information, and the like.
For example, as shown in FIG. 5, a start point p1 and an end point p2 of automatic traveling are set in a rectangular farm field A, and the work traveling direction of the tractor 1 is set along the short side of the farm field A. If so, the target path generation unit 51B first divides the farm field A into a margin area adjacent to the outer periphery of the farm field A based on the four corner points Ap1 to Ap4 and the rectangular shape identification line AL. A1 and a running area A2 located inside the margin area A1.
Next, based on the turning radius and working width of the tractor 1, the target route generating unit 51B is arranged in parallel in the traveling area A2 at regular intervals corresponding to the working width in the direction along the long side of the field A. A plurality of parallel routes P1 are generated, and a plurality of turning routes P2 are generated that are arranged at the outer edge of each long side of the travel area A2 and connect the plurality of parallel routes P1 in the order of travel.
Then, the traveling area A2 is divided into a pair of non-working areas A2a set at the outer edges of each long side of the traveling area A2, and a working area A2b set between the pair of non-working areas A2a. , each parallel path P1 is divided into a non-work path P1a included in a pair of non-work areas A2a and a work path P1b included in a pair of work areas A2b.
As a result, the target route generator 51B can generate a target route P suitable for automatically driving the tractor 1 in the farm field A shown in FIG.

In the farm field A shown in FIG. 5, the margin area A1 prevents the rotary tillage device 3 from coming into contact with other objects such as ridges adjacent to the farm field A when the tractor 1 automatically travels along the outer periphery of the travel area A2. This area is secured between the outer edge of the field A and the running area A2 in order to prevent this. Each non-work area A2a is a ridge turning area for the tractor 1 to turn from the current work path P1b to the next work path P1b on the ridge of the field A.

In the target path P shown in FIG. 5, each non-working path P1a and each turning path P2 are paths along which the tractor 1 automatically travels without performing tillage work. It is a route that automatically travels while performing A start point p3 of each work path P1b is a work start point at which the tractor 1 starts tillage work, and an end point p4 of each work path P1b is a work stop point at which the tractor 1 stops tillage work. Each non-working path P1a includes a work stop point p4 before the tractor 1 turns on the turning path P2 and a work start point p3 after the tractor 1 turns on the turning path P2. Alignment path for alignment in the direction of travel. Of the connection points p5 and p6 between each parallel path P1 and each turning path P2, the connection point p5 on the end side of each parallel path P1 is the turning start point of the tractor 1, and is the connection point on the beginning side of each parallel path P1. A point p6 is the turning end point of the tractor 1 .

Note that the target route P shown in FIG. 5 is merely an example, and the target route generation unit 51B generates vehicle body information that differs depending on the model of the tractor 1 and the type of work, and the shape of the farm field A that differs depending on the farm field A. Various target paths P suitable for them can be generated based on field information such as size and size.

The target route P is stored in the terminal storage unit 51</b>C in a state of being associated with vehicle information, field information, etc., and can be displayed on the display device 50 of the mobile communication terminal 5 . The target path P includes the target vehicle speed of the tractor 1 on each parallel path P1, the target vehicle speed of the tractor 1 on each turning path P2b, the front wheel steering angle on each parallel path P1, and the front wheel steering angle on each turning path P2b. include.

The terminal control unit 51 transmits the field information, the target route P, and the like stored in the terminal storage section 51C to the in-vehicle control unit 23 in response to the transmission request command from the in-vehicle control unit 23 . The in-vehicle control unit 23 stores the received field information, the target route P, and the like in the in-vehicle storage section 23G. Regarding the transmission of the target route P, for example, the terminal control unit 51 may transmit all of the target route P from the terminal storage unit 51C to the vehicle-mounted control unit 23 at once before the tractor 1 starts automatic traveling. can be Further, the terminal control unit 51 divides the target route P into a plurality of divided route information for each predetermined distance, and every time the travel distance of the tractor 1 reaches the predetermined distance from the stage before the tractor 1 starts automatic travel Alternatively, a predetermined number of divided route information corresponding to the running order of the tractor 1 may be sequentially transmitted from the terminal storage section 51C to the vehicle-mounted control unit 23 .

In the in-vehicle control unit 23, detection information from various sensors, switches, and the like included in the vehicle state detection device 22 is input to the automatic driving control section 23F via the vehicle speed control section 23B, the steering control section 23C, and the like. ing. As a result, the automatic travel control unit 23F can monitor various setting states of the tractor 1, operating states of the respective units, and the like.

The automatic traveling control unit 23F is operated in a state in which the traveling mode of the tractor 1 has been switched to the automatic traveling mode by manual operation for satisfying various automatic traveling start conditions by a user such as a passenger or an administrator. When the display device 50 of the communication terminal 5 is operated to instruct the start of automatic traveling, the tractor 1 automatically travels according to the target route P while acquiring the current position and current direction of the tractor 1 with the positioning unit 30. Start automatic driving control.

During the execution of the automatic driving control, the automatic driving control unit 23F, for example, when the display device 50 of the mobile communication terminal 5 is operated by the user to issue an instruction to end the automatic driving, or when the driving unit 12 is on board. When the user operates a manual operation tool such as the steering wheel 25 or an accelerator pedal, the automatic travel control is ended and the travel mode is switched from the automatic travel mode to the manual travel mode. When the automatic travel control is to be restarted after the automatic travel control has ended in this way, the user first gets into the driving unit 12 and switches the travel mode of the tractor 1 from the automatic travel mode to the manual travel mode. Next, after performing manual operation for satisfying various automatic traveling start conditions, the traveling mode of the tractor 1 is switched from the manual traveling mode to the automatic traveling mode. In this state, by operating the display device 50 of the mobile communication terminal 5 to issue a command to start automatic driving, the automatic driving control can be restarted.

The automatic driving control by the automatic driving control unit 23F includes automatic driving control processing for transmitting an automatic driving control command for the engine 14 to the engine control unit 23A, automatic driving control for switching the vehicle speed and forward/backward movement of the tractor 1. Automatic vehicle speed control processing for transmitting a command to the vehicle speed control unit 23B, automatic steering control processing for transmitting a steering control command for automatic travel to the steering control unit 23C, and automatic travel for work devices such as the rotary tillage device 3 automatic control processing for work, which transmits a control command for work to the work device control unit 23D, and the like.

In the automatic engine control process, the automatic driving control unit 23F issues an engine rotation speed change command to the engine control unit 23A to change the engine rotation speed based on the set rotation speed included in the target route P. Send. The engine control unit 23A executes engine speed change control for automatically changing the engine speed in accordance with various control commands relating to the engine 14 transmitted from the automatic travel control unit 23F.

In the vehicle speed automatic control process, the automatic driving control unit 23F provides a gear shift operation command for instructing a gear shift operation of the continuously variable transmission based on the target vehicle speed included in the target path P, and Based on the traveling direction of the tractor 1, a forward/rearward switching command for instructing the forward/rearward switching operation of the forward/rearward switching device is transmitted to the vehicle speed control unit 23B. The vehicle speed control unit 23B automatically controls the operation of the continuously variable transmission according to various control commands regarding the continuously variable transmission, the forward/reverse switching device, and the like, which are transmitted from the automatic driving control unit 23F. , automatic forward/reverse switching control that automatically controls the operation of the forward/reverse switching device, and the like. The vehicle speed control includes, for example, an automatic deceleration stop process in which, when the target vehicle speed included in the target route P is zero speed, the continuously variable transmission is controlled to decelerate to the zero speed state to stop the tractor 1 from traveling. It is included.

In the steering automatic control process, the automatic driving control unit 23F transmits a steering command for steering the left and right front wheels 10 based on the front wheel steering angle included in the target route P, etc. to the steering control unit 23C. . The steering control unit 23C controls the operation of the power steering unit 17 to steer the left and right front wheels 10 according to the steering command sent from the automatic driving control unit 23F, and the left and right front wheels 10 are set. When the vehicle is steered beyond the angle, automatic brake turn control is executed to operate the brake unit 18 to operate the brake on the inner side of the turn.

In the automatic work control process, the automatic travel control unit 23F issues a work start command that instructs the rotary tiller 3 to switch to the work state based on the work start point p3 included in the target route P, and the target route P. Based on the work stop point p4 included in P, a work stop command for instructing switching to the non-working state of the rotary tillage device 3 and the like are transmitted to the working device control unit 23D. The work device control section 23D controls the operation of the work clutch unit 19 and the lifting drive unit 20 in accordance with various control commands regarding the rotary tillage device 3 transmitted from the automatic traveling control section 23F, thereby operating the rotary tillage device 3. Automatic work start control for lowering to the working height and automatic work stop control for stopping the rotary tillage device 3 and raising it to the non-working height, etc. are executed.

That is, the automatic traveling unit 4 described above includes a power steering unit 17, a brake unit 18, a work clutch unit 19, an elevation drive unit 20, a rolling unit 21, a vehicle state detection device 22, an in-vehicle control unit 23, a positioning unit 30, and , a communication module 34, and the like. By appropriately operating these, the tractor 1 can be automatically traveled accurately along the target route P, and the rotary tillage device 3 can properly perform tillage.

As shown in FIGS. 6-7, the tractor 1 is equipped with an obstacle detection system 80 that monitors the surroundings of the tractor 1 and detects obstacles that are present therearound. Obstacles detected by the obstacle detection system 80 include persons such as workers working in the field A, other work vehicles, and existing utility poles and trees in the field A.

As shown in FIGS. 1 to 4 and FIGS. 6 to 7, the obstacle detection system 80 includes four cameras (examples of imaging devices) 81 to 84 for imaging the surroundings of the tractor 1, and measurement sensors existing around the tractor 1. An active sensor unit 85 that measures the distance to an object, an image processor 86 that processes images from each of the cameras 81-84, and integration of information from the image processor 86 and measurement information from the active sensor unit 85. and an obstacle detection device 87 for processing and detecting obstacles. The image processing device 86 and the obstacle detection device 87 are constructed by an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. The active sensor unit 85, the image processing device 86, and the obstacle detection device 87 are connected to the in-vehicle control unit 23 via CAN so as to be able to communicate with each other.

As shown in FIGS. 1 to 3 and 7, the obstacle detection system 80 includes four cameras 81 to 84, a front camera 81 whose imaging range is set to a first imaging range Ri1 in front of the cabin 13, a cabin A rear camera 82 whose imaging range is set to a second imaging range Ri2 behind 13, a right camera 83 whose imaging range is set to a third imaging range Ri3 on the right from the cabin 13, and a left camera 83 from the cabin 13 It has a left camera 84 whose imaging range is set to the fourth imaging range Ri4.

The front camera 81 and the rear camera 82 are arranged on the left-right center line of the tractor 1 . The front camera 81 is arranged at the left-right central portion of the upper portion of the front end side of the cabin 13 in a front-down posture that looks down on the front side of the tractor 1 from obliquely above. As a result, the front camera 81 has a predetermined range on the front side of the vehicle body with the center line of the tractor 1 as the axis of symmetry set as the first imaging range Ri1. The rear camera 82 is arranged in a left-right center portion of the upper portion on the rear end side of the cabin 13 so as to look down on the rear side of the tractor 1 obliquely from above. As a result, the rear camera 82 sets a predetermined range on the rear side of the vehicle body with the center line of the tractor 1 as the axis of symmetry as the second imaging range Ri2. The right camera 83 is arranged at the upper front-rear central position on the right end side of the cabin 13 in a right-down posture that looks down on the right side of the tractor 1 from obliquely above. As a result, the predetermined range on the right side of the vehicle body of the right camera 83 is set as the third imaging range Ri3. The left camera 84 is arranged at the upper front-rear central position on the left end side of the cabin 13 in a left-down attitude that looks down on the left side of the tractor 1 from obliquely above. As a result, the predetermined range on the left side of the vehicle body of the left camera 84 is set as the fourth imaging range Ri4.

As shown in FIGS. 1, 3 to 4, and 7, the active sensor unit 85 includes a front rider sensor (an example of an active sensor) 85A whose measurement range is set to a first measurement range Rm1 in front of the cabin 13; A rear rider sensor (an example of an active sensor) 85B in which the second measurement range Rm2 behind 13 is set as the measurement range, and a third measurement range Rm3 on the right from the cabin 13 and a fourth measurement on the left from the cabin 13. It has a sonar (an example of an active sensor) 85C in which the range Rm4 is set to the measurement range. Each lidar sensor 85A, 85B uses a laser beam (for example, a pulsed near-infrared laser beam), which is an example of measurement light, to perform measurement in the first measurement range Rm1 or the second measurement range Rm2. , 85Ba, and rider control units 85Ab and 85Bb for generating distance images based on the measurement information from the measurement units 85Aa and 85Ba. The sonar 85C has a right ultrasonic sensor 85Ca, a left ultrasonic sensor 85Cb and a single sonar control 85Cc. Each of the rider control units 85Ab and 85Bb and the sonar control unit 85Cc is constructed by an electronic control unit in which microcontrollers and the like are integrated, various control programs, and the like. The rider control units 85Ab and 85Bb and the sonar control unit 85Cc are connected to the obstacle detection device 87 via CAN so as to be able to communicate with each other.

In each lidar sensor 85A, 85B, each measurement unit 85Aa, 85Ba measures the distance to the range-finding point based on the round-trip time for the irradiated laser light to reach and return to the range-finding point. ) method, the distance from each of the measuring units 85Aa and 85Ba to each distance measuring point (an example of an object to be measured) in the first measuring range Rm1 or the second measuring range Rm2 is measured. Each of the measurement units 85Aa and 85Ba scans the laser light vertically and horizontally over the entire first measurement range Rm1 or second measurement range Rm2 at high speed, and sequentially measures the distance to the distance measurement point for each scanning angle (coordinate). By doing so, three-dimensional measurement is performed in the first measurement range Rm1 or the second measurement range Rm2. Each of the measuring units 85Aa and 85Ba measures the intensity of the reflected light (hereinafter referred to as reflected intensity ) are measured sequentially. Each measurement unit 85Aa, 85Ba repeatedly measures in real time the distance to each ranging point in the first measurement range Rm1 or the second measurement range Rm2, each reflection intensity, and the like. Each rider control unit 85Ab, 85Bb generates a distance image from measurement information such as the distance to each ranging point measured by each measuring unit 85Aa, 85Ba and the scanning angle (coordinates) for each ranging point, and also generates an obstacle. is extracted, and the measurement information about the extracted range-finding point group is transmitted to the obstacle detection device 87 as the measurement information about the obstacle candidate.

Each rider control section 85Ab, 85Bb determines whether or not the distance value of each ranging point measured by each measuring section 85Aa, 85Ba satisfies the invalid condition, and treats the distance value that satisfies the invalid condition as an invalid value. It is transmitted to the object detection device 87 .
Specifically, each of the rider control units 85Ab and 85Bb utilizes the feature of contamination on the sensor surface of each of the rider sensors 85A and 85B, which is present at a close distance from each of the rider sensors 85A and 85B, to have the feature. Disable the distance value of the AF point. This prevents the obstacle detection device 87 from using the distance value of the range-finding point related to dirt on the sensor surface as information about the obstacle.
Each lidar control unit 85Ab, 85Bb utilizes the characteristics of floating objects such as dust and fog that are present at a short distance from the respective lidar sensors 85A, 85B but have very low reflection intensity, so that the distance measuring points having the characteristics are detected. The distance value of is an invalid value. This prevents the obstacle detection device 87 from using the distance value of the distance measuring point for the floating object as information about the obstacle.

As shown in FIGS. 1 and 3-4, the front lidar sensor 85A and the rear lidar sensor 85B are arranged on the left-right center line of the tractor 1, like the front camera 81 and the rear camera 82. As shown in FIG. The front rider sensor 85A is arranged in a left-right center portion of the upper portion of the front end side of the cabin 13 in a front-down posture that looks down on the front side of the tractor 1 from an obliquely upper side. As a result, the front rider sensor 85A sets a predetermined range on the front side of the vehicle body with the center line of the tractor 1 as the axis of symmetry as the first measurement range Rm1 by the measuring section 85Aa. The rear rider sensor 85B is arranged in a left-right central position on the upper portion of the rear end side of the cabin 13 in a rearwardly lowered posture in which the rearward side of the tractor 1 is viewed obliquely from above. As a result, the rear rider sensor 85B sets a predetermined range on the rear side of the vehicle body with the center line of the tractor 1 as the axis of symmetry as the second measurement range Rm2 by the measuring section 85Ba.

The front rider sensor 85A and the rear rider sensor 85B are interlocked with the forward travel of the tractor 1 when the forward/reverse switching device of the transmission unit 16 is switched to the forward transmission state, and the front rider sensor 85A is activated. , the rear rider sensor 85B is deactivated. Further, when the tractor 1 travels in reverse when the forward/rearward travel switching device of the transmission unit 16 is switched to the reverse transmission state, the front rider sensor 85A stops operating and the rear rider sensor 85B operates in conjunction with the switching. become.

As shown in FIGS. 1, 3 to 4, and 7, in the sonar 85C, the sonar control unit 85Cc controls the third measurement range Rm3 or the fourth The presence or absence of the measurement object in the measurement range Rm4 is determined. The sonar control unit 85Cc uses a TOF (Time Of Flight) method for measuring the distance to the range-finding point based on the round-trip time for the transmitted ultrasonic waves to reach and return to the range-finding point. The distance from 85Cb to the object to be measured is measured, and the measured distance to the object to be measured and the direction of the object to be measured are transmitted to the obstacle detection device 87 as measurement information regarding the obstacle candidate.

As shown in FIGS. 1 to 3, the right ultrasonic sensor 85Ca is attached to the right step 24 located between the right front wheel 10 and the right rear wheel 11 so as to face outward to the right of the vehicle body. As a result, the right ultrasonic sensor 85Ca has a predetermined range on the right outer side of the vehicle body set as the third measurement range Rm3. The left ultrasonic sensor 85Cb is attached to the left step 24 arranged between the left front wheel 10 and the left rear wheel 11 so as to face left outward of the vehicle body. As a result, the left ultrasonic sensor 85Cb has a predetermined range on the left outer side of the vehicle body set as the fourth measurement range Rm4.

As shown in FIGS. 4 and 8 to 10, the rider control units 85Ab and 85Bb perform cutting processing and masking processing on the measurement ranges Rm1 and Rm2 of the measurement units 85Aa and 85Ba based on vehicle body information. , a first detection range Rd1 and a second detection range Rd2 are set for detecting the obstacle candidates described above. In the cutting process, each of the rider control units 85Ab and 85Bb acquires the maximum lateral width of the vehicle body including the rotary tillage device 3 (in this embodiment, the lateral width of the rotary tillage device 3) through communication with the vehicle-mounted control unit 23, By adding a predetermined safety band to the maximum lateral width of the vehicle body, the detection target width Wd of the obstacle candidate is set. Then, in the first measurement range Rm1 and the second measurement range Rm2, the left and right ranges deviating from the detection target width Wd are set as the first non-detection range Rnd1 by the cutting process and excluded from the detection ranges Rd1 and Rd2. In the masking process, each of the rider control units 85Ab and 85Bb controls the range in which the front end side of the tractor 1 enters the first measurement range Rm1 and the range in which the rear end side of the rotary tillage device 3 enters the second measurement range Rm2. plus a predetermined safety band is set as the second non-detection range Rnd2 by the masking process, and excluded from the detection ranges Rd1 and Rd2. As a result, the obstacle candidate detection target ranges of the rider sensors 85A and 85B are limited to the first detection range Rd1 and the second detection range Rd2. Due to this limitation, the lidar sensors 85A and 85B detect an obstacle candidate that is out of the detection target width Wd and has no danger of colliding with the tractor 1, which increases the detection load and increases the detection load. This avoids the risk of erroneously detecting the front end side of the tractor 1 and the rear end side of the tractor 1, such as the rotary tillage device 3, which is in the second measurement range Rm2, as an obstacle candidate.
The second non-detection range Rnd2 shown in FIG. 8 is an example of a non-detection range suitable for the front side of the vehicle body where the left and right front wheels 10 and the bonnet 15 are present. A second non-detection range Rnd2 shown in FIG. 9 is an example of a non-detection range suitable for a working state in which the rotary tillage device 3 is lowered to the working height on the rear side of the vehicle body. A second non-detection range Rnd2 shown in FIG. 10 is an example of a non-detection range suitable for a non-working state in which the rotary tillage device 3 is raised to the retraction height on the rear side of the vehicle body. The second non-detection range Rnd2 on the rear side of the vehicle body switches appropriately in conjunction with the elevation of the rotary tillage device 3 .

Information about the first detection range Rd1, the second detection range Rd2, the first non-detection range Rnd1, and the second non-detection range Rnd2 is included in the distance image described above, and the distance image and the obstacle detection apparatus 87 has been sent.

As shown in FIG. 4, the detection ranges Rd1 and Rd2 of the rider sensors 85A and 85B are detected by the rider sensors 85A and 85B based on the collision determination process in which the collision prediction time is a set time (for example, 3 seconds). a deceleration control range Rdc set from the determination reference position to the deceleration start position; and a measurement limit position of the rider sensors 85A and 85B from the deceleration start position. and a notification control range Rnc that is set to a range up to . The determination reference position of the first detection range Rd1 is set at a position a certain distance L (for example, 2000 mm) away from the front end or rear end of the vehicle body including the rotary tillage device 3 in the longitudinal direction of the vehicle body.

The image processing device 86 performs image processing on the images sequentially transmitted from each of the cameras 81-84.
The image processing device 86 is subjected to learning processing for recognizing persons such as workers working in the field A, other working vehicles, and existing utility poles and trees in the field A as obstacles. ing.

The processing procedure of the image processing device 86 in image processing will be described below based on the flowchart shown in FIG.
The image processing device 86 first synthesizes the images from all the cameras 81 to 84 with the images sequentially transmitted from the cameras 81 to 84 to generate an all-around image of the tractor 1 (for example, a surround view). Surrounding image generation processing is performed (step #1), and image transmission for transmitting the generated omnidirectional image and images from the cameras 81 to 84 to the display control unit 23E on the tractor side and the display control unit 51A on the mobile communication terminal side Processing is performed (step #2).
As a result, the omnidirectional image generated by the omnidirectional image generator 86A, the image in the traveling direction of the tractor 1, and the like can be displayed on the liquid crystal monitor 27 of the tractor 1, the display device 50 of the mobile communication terminal 5, and the like. This display enables the user to visually recognize the surrounding conditions of the tractor 1 and the conditions in the running direction.
Next, based on the images sequentially transmitted from the cameras 81 to 84, the image processing device 86 determines whether there is an obstacle that hinders the traveling of the tractor 1 in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84. An obstacle determination process is performed to determine whether or not an obstacle exists (step #3). If an obstacle exists, coordinate calculation processing is performed to obtain the coordinates of the obstacle on the image in which the obstacle exists (step #4). , mounting angle, etc., a coordinate conversion process is performed to convert the coordinates to the coordinates based on the origin of the vehicle body coordinates (step #5). Then, a distance calculation process is performed to obtain the distance from the distance calculation reference point to the obstacle by straight line distance between the converted coordinates and the preset distance calculation reference point (step #6), and the converted coordinates are obtained. Detection information transmission processing is performed to transmit the detected distance to the obstacle to the obstacle detection device 87 as detection information on the obstacle (step #7). After that, return to step #1. On the other hand, if no obstacle exists, a non-detection transmission process is performed to transmit to the obstacle detection device 87 that the obstacle has not been detected (step #8), and then the process returns to step #1.
In this way, when an obstacle exists in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84, the image processing device 86 transmits obstacle detection information to the obstacle detection device 87. By receiving the obstacle detection information, the obstacle detection device 87 can detect the presence of an obstacle in the imaging ranges Ri1 to Ri4 of any one of the cameras 81 to 84, and detect the presence of the obstacle. position and distance to obstacles. Further, when there is no obstacle in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84, the image processing device 86 transmits non-detection of the obstacle to the obstacle detection device 87. The detection device 87 can detect that there is no obstacle in any of the imaging ranges Ri1-Ri4 of the cameras 81-84.

The vehicle body coordinate origin in the coordinate conversion process and the distance calculation reference point in the distance calculation process are set according to the mounting positions of the cameras 81-84. Specifically, as shown in FIG. 12, a vehicle body coordinate origin O1 and a distance calculation reference point Rp1 are set for the front camera 81 according to its mounting position. For the rear camera 82, a vehicle body coordinate origin O2 and a distance calculation reference point Rp2 are set according to its mounting position. For the right camera 83, a vehicle body coordinate origin O3 and a distance calculation reference point Rp3 are set according to its mounting position. For the left camera 84, a vehicle body coordinate origin O4 and a distance calculation reference point Rp4 are set according to its mounting position.
As a result, for example, when an obstacle exists in the first imaging range Ri1 of the front camera 81, the image processing device 86 obtains the coordinates of the obstacle on the image of the front camera 81 where the obstacle exists (coordinates calculation process), the obtained coordinates of the obstacle are converted into coordinates (x, y) based on the vehicle body coordinate origin O1 shown in FIG. process), the linear distance between the converted coordinates (x, y) and the distance calculation reference point Rp1 is obtained as the distance La from the distance calculation reference point Rp1 to the obstacle O (distance calculation process).
The relationship between the vehicle body coordinate origins O1 to O4, the distance calculation reference points Rp1 to Rp4, and the mounting positions of the cameras 81 to 84 can be changed in various settings.

Incidentally, in the omnidirectional image generation process described above, for example, the image processing device 86 displays the front image display area 90A of the omnidirectional image display area 90 such as the liquid crystal monitor 27 and the display device 50 shown in FIG. A processed image obtained by cutting out a predetermined area from the image of the camera 81 is arranged, a processed image obtained by cutting out a predetermined area from the image of the rear camera 82 is arranged in the rear image display area 90B, and a right camera is displayed in the right image display area 90C. By arranging a processed image obtained by cutting out a predetermined area from the image of 83, and arranging the processed image obtained by cutting out a predetermined area from the image of the left camera 84 in the left image display area 90D, generating an all-surrounding image. can be considered.

In the obstacle discrimination process described above, the image processing device 86 discriminates the existence or non-existence of obstacles at an ultra-high speed of several tens of frames (for example, 30 frames) per second for the images sequentially transmitted from the cameras 81 to 84. . The image processing device 86 performs obstacle discrimination processing for each of the cameras 81 to 84 in a time-division manner. The image processing device 86 changes the processing target period per unit time for each of the cameras 81 to 84 in the time division method according to the running direction and vehicle speed of the tractor 1 .
Specifically, if the traveling direction of the tractor 1 is the straight ahead direction and the vehicle speed is a standard speed (for example, 10 km/h), the image processing device 86 sets the processing target cycle per unit time to, for example, ( As shown in a), after four frames of image processing are continuously performed on the front image from the front camera 81, the right image from the right camera 83 or the left image from the left camera 84 is processed. The processing target cycle is changed to that for standard forward straight movement, which is set so that the image processing to be processed is performed for one frame.
If the traveling direction of the tractor 1 is the backward straight direction and the vehicle speed is the standard speed, the image processing device 86 calculates the processing target cycle per unit time from the rear camera 82 as shown in FIG. 14B, for example. After continuously performing image processing for four frames on the rearward image of the camera 83, image processing on the right image from the right camera 83 or the left image from the left camera 84 is performed for one frame. Change to the processing target cycle for standard backward straight running set as follows.
If the traveling direction of the tractor 1 is the forward turning right direction and the vehicle speed is the standard speed, the image processing device 86 determines the processing target cycle per unit time as shown in FIG. After performing image processing on the front image from 81 continuously for three frames, image processing on the right image from the right camera 83 is performed continuously for two frames, or A process for standard forward and right turn that is set to perform image processing for the right image from the camera 83 and image processing for the left image from the left camera 84 for each frame. Change to the target period.
If the traveling direction of the tractor 1 is the forward left turning direction and the vehicle speed is the standard speed, the image processing device 86 calculates the processing target cycle per unit time as shown in FIG. After performing image processing for three consecutive frames on the front image from the left camera 84, image processing for two consecutive frames on the left image from the left camera 84 is performed, or the right camera A processing target period for standard forward left turn, which is set so that image processing for the right image from 83 and image processing for the left image from the left camera 84 are performed one frame at a time. change to
If the traveling direction of the tractor 1 is the forward straight direction and the vehicle speed is decelerated from the standard speed, the image processing device 86 sets the processing target cycle per unit time to, for example, forward as shown in (e) of FIG. After image processing for three frames of the front image from the camera 81 is continuously performed, image processing for the right image from the right camera 83 or the left image from the left camera 84 is performed once. Change to the processing target cycle for low-speed forward straight movement, which is set to be performed for each frame.
If the traveling direction of the tractor 1 is the straight forward direction and the vehicle speed is increased from the standard speed, the image processing device 86 determines the processing target cycle per unit time as shown in FIG. After performing image processing for five consecutive frames on the front image from the front camera 81, image processing on the right image from the right camera 83 or the left image from the left camera 84 is performed. The cycle is changed to that for high-speed forward straight movement, which is set to be performed for one frame.

As described above, the image processing device 86 performs the obstacle discrimination processing in a time-sharing manner, so that the obstacle discrimination processing for the images from the cameras 81 to 84 with a large processing load can be performed by the single image processing device 86. It can be done quickly without delay.
Further, the image processing device 86 switches between a state in which either one of the front image and the rear image is processed and a state in which it is not processed according to the switching of the forward and backward movement of the tractor 1, thereby eliminating unnecessary image processing. This avoids a decrease in processing speed due to
Then, the image processing device 86 shortens the processing target period per unit time for each of the cameras 81 to 84 having an imaging range in the traveling direction of the tractor 1 according to the traveling direction of the tractor 1, thereby adjusting the traveling direction of the tractor 1. By slowing down the processing target period per unit time for each camera 81 to 84 that is not included in the imaging range, obstacle discrimination processing for images from each camera 81 to 84 can be sequentially and quickly performed by a single image processing device 86. In addition, the presence or absence of an obstacle in the traveling direction of the tractor 1 can be determined intensively, and collision avoidance with respect to the obstacle can be easily performed.
Furthermore, as the vehicle speed of the tractor 1 increases, the image processing device 86 shortens the processing target period per unit time for each of the cameras 81 to 84 whose imaging range is the traveling direction of the tractor 1, and By delaying the processing target period per unit time for each camera 81 to 84 that does not have an imaging range of , the obstacle discrimination processing for the images from each camera 81 to 84 is sequentially and smoothly performed by a single image processing device 86. As the vehicle speed of the tractor 1 increases, the presence or absence of obstacles in the direction of travel of the tractor 1 can be determined more intensively, making it easier to avoid collisions with obstacles.

It should be noted that the processing target period per unit time for each of the cameras 81 to 84 described above is merely an example, and can be changed in various ways according to the type of work and the condition of the field.
Further, as shown in FIG. 7, the image processing device 86 acquires the vehicle speed of the tractor 1 detected by the vehicle speed sensor 22A via the vehicle speed control section 23B. The image processing device 86 determines the traveling direction of the tractor 1 based on the operating position of the reverser lever obtained via the vehicle speed control section 23B and the steering angle of the front wheels 10 obtained via the steering control section 23C. do.

The obstacle detection device 87 detects the presence of an obstacle in the first imaging range Ri1 of the front camera 81 based on the obstacle detection information from the image processing device 86 when the tractor 1 is traveling forward. When the obstacle is detected, the first identification control is executed to identify the position of the obstacle in the first imaging range Ri1 and the distance to the obstacle.

The control operation of the obstacle detection device 87 in the first specific control will be described below based on the flowchart shown in FIG.
The obstacle detection device 87 determines whether the obstacle is within the first detection range Rd1 of the front rider sensor 85A based on the coordinates of the obstacle and the distance to the obstacle included in the detection information about the obstacle from the image processing device 86. A first position determination process is performed to determine whether or not the position is located at (step #11). If the obstacle is positioned within the first detection range Rd1, a second position determination process is performed to determine whether or not the obstacle is positioned within the notification control range Rnc of the first detection range Rd1 (step #12). ). If the obstacle is located within the notification control range Rnc, a first distance transmission process is performed to transmit the distance to the obstacle obtained from the image processing device 86 to the vehicle-mounted control unit 23 (step #13). Return to step #11. When the obstacle is not located in the notification control range Rnc, the obstacle is located in the deceleration control range Rdc or the stop control range Rsc of the first detection range Rd1. The coordinates (x, y) of the obstacle O and the distance La to the obstacle O shown in FIG. Whether the position of the obstacle O and the position of the obstacle candidate Oc match based on the coordinates (scanning angle) of the obstacle candidate Oc and the distance Lb to the obstacle candidate Oc shown in FIG. Matching determination processing for determining whether or not is performed (step #14). If they match, a second distance transmission process is performed to transmit the distance to the obstacle candidate obtained from the front lidar sensor 85A with high ranging accuracy to the in-vehicle control unit 23 as the distance to the obstacle (step # 15), and then return to step #11. If they do not match, it is determined that the measurement information about the obstacle candidate from the front rider sensor 85A is not the measurement information about the obstacle, and the distance to the obstacle obtained from the image processing device 86 is sent to the in-vehicle control unit 23. A third distance transmission process for transmission is performed (step #16), and then the process returns to step #11. If the obstacle is not located in the first detection range Rd1 in the first position determination process of step #11, it is determined whether or not the presence of the obstacle is detected in the first imaging range Ri1 of the front camera 81. Object detection determination processing is performed (step #17). If the presence of the obstacle is detected, the process returns to step #11 to continue the first specific control. If the presence of the obstacle is not detected, the first specific control is ended.

The obstacle detection device 87 detects the presence of an obstacle in the second imaging range Ri2 of the rear camera 82 based on the obstacle detection information from the image processing device 86 when the tractor 1 is traveling backward. When the obstacle is detected, the second identification control is executed to identify the position of the obstacle in the second imaging range Ri2 and the distance to the obstacle.
In the second specific control, the contents of the control are the same except that the first detection range Rd1 of the front rider sensor 85A in the first specific control is changed to the second detection range Rd2 of the rear rider sensor 85B. , the description of the second specific control is omitted.

The obstacle detection device 87 detects the presence of an obstacle in the third imaging range Ri3 of the right camera 83 based on the obstacle detection information from the image processing device 86 regardless of whether the tractor 1 is traveling forward or backward. When the presence of the obstacle is detected, the third identification control is executed to identify the position of the obstacle in the third imaging range Ri3 and the distance to the obstacle.

Hereinafter, based on the flowchart shown in FIG. 16, the control operation of the obstacle detection device 87 in the third specific control will be described.
Based on the coordinates of the obstacle and the distance to the obstacle included in the detection information about the obstacle from the image processing device 86, the obstacle detection device 87 detects the position of the obstacle in the third measurement range Rm3 of the sonar 85C. A third position determination process is performed to determine whether or not the vehicle is moving (step #21). If the obstacle is located in the third measurement range Rm3, the coordinates of the obstacle and the distance to the obstacle included in the detection information about the obstacle from the image processing device 86, and the obstacle from the sonar 85C. Based on the distance to the obstacle candidate and the direction of the obstacle candidate included in the measurement information about the candidate, matching determination processing is performed to determine whether or not the position of the obstacle matches the position of the obstacle candidate. (step #22). If they match, a fourth distance transmission process is performed to transmit the distance to the obstacle candidate obtained from the sonar 85C with high ranging accuracy to the in-vehicle control unit 23 as the distance to the obstacle (step #23). , and then returns to step #21. If they do not match, it is determined that the measurement information about the obstacle candidate from the front rider sensor 85A is not the measurement information about the obstacle, and the distance to the obstacle obtained from the image processing device 86 is sent to the in-vehicle control unit 23. A fifth distance transmission process for transmission is performed (step #24), and then the process returns to step #21. If the obstacle is not located in the third measurement range Rm3 in the third position determination process of step #21, it is determined whether or not the presence of the obstacle is detected in the third imaging range Ri3 of the right camera 83. Obstacle detection determination processing is performed (step #25). If the presence of the obstacle is detected, the process returns to step #21 to continue the third specific control. If the presence of the obstacle is not detected, the third specific control is terminated.

The obstacle detection device 87 detects the presence of an obstacle in the fourth imaging range Ri4 of the left camera 84 based on the obstacle detection information from the image processing device 86 regardless of whether the tractor 1 is traveling forward or backward. When the presence of the obstacle is detected, fourth identification control is performed to identify the position of the obstacle in the fourth imaging range Ri4 and the distance to the obstacle.
Note that in the fourth specific control, only the third measurement range Rm3 of the sonar 85C is changed to the fourth measurement range Rm4 in the above-described third specific control, and the contents of the control are the same. are omitted.

When the obstacle detection device 87 detects that the presence of an obstacle is not detected in the imaging ranges Ri1 to Ri4 of the cameras 81 to 84 based on the obstacle detection information from the image processing device 86, Measurement information invalidation processing for invalidating the measurement information from the active sensor unit 85 is performed.

As described above, the obstacle detection device 87 operates only when the position of the obstacle determined by the image processing device 86 with high object determination accuracy matches the position of the obstacle candidate determined by the active sensor unit 85. Since the distance to the obstacle candidate obtained from the active sensor unit 85 with high ranging accuracy is adopted as the distance to the obstacle, the distance to the obstacle candidate when the active sensor unit 85 erroneously discriminates it as an obstacle candidate. It is possible to avoid the possibility that the distance is adopted as the distance to the obstacle. As a result, the obstacle detection device 87 can acquire detection information regarding obstacles with high object discrimination accuracy and range finding accuracy.
In addition, since the obstacle detection device 87 acquires detection information regarding obstacles from the image processing device 86 that performs obstacle discrimination processing based on highly accurate camera images, each measurement range Rm1 to Rm4 of the active sensor unit 85 is obtained. , for example, when there is tall grass or the like, it is possible to avoid the risk of erroneously detecting the tall grass or the like as an obstacle that hinders the running of the tractor 1. In the detection ranges Rd1 and Rd2 of the lidar sensors 85A and 85B, the distance to all the distance measurement points in the detection ranges Rd1 and Rd2 measured by the active sensor unit 85 may be affected by dust, dust, etc. When the value becomes an invalid value, or when an abnormality such as contamination of the sensor surface occurs in the active sensor unit 85, based on the detection information regarding the obstacle from the image processing device 86, the existence of the obstacle or the obstacle is detected. It can detect the distance to an object.

As shown in FIGS. 6 and 7, the in-vehicle control unit 23 includes a collision avoidance control section 23H that executes collision avoidance control for avoiding collisions with obstacles based on detection information from the obstacle detection device 87. is The collision avoidance control section 23H is constructed by an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. The collision avoidance control section 23H communicates with the other control sections 23A to 23F of the in-vehicle control unit 23, the active sensor unit 85, the image processing device 86, and the obstacle detection device 87 via CAN (Controller Area Network). connected as possible.

The collision avoidance control by the collision avoidance control section 23H will be described below.
When the distance to the obstacle is acquired by the first distance transmission process from the obstacle detection device 87, the collision avoidance control unit 23H determines the notification control range of the first detection range Rd1 or the second detection range Rd2 shown in FIG. Since the obstacle is located in Rnc, the display control section 23E of the in-vehicle control unit 23 and the display control section of the terminal control unit 51 execute the first notification control to notify the existence of the obstacle in the notification control range Rnc. Command 51A. Accordingly, the first notification control is executed in each of the display control units 23E and 51A, and the presence of an obstacle in the notification control range Rnc of the first detection range Rd1 or the second detection range Rd2 for the tractor 1 is notified to the operation unit. Users such as 12 passengers and managers outside the vehicle can be notified.
When the distance to the obstacle is acquired by the second distance transmission processing or the third distance transmission processing from the obstacle detection device 87, the collision avoidance control section 23H uses the first detection range Rd1 or the second detection range Rd1 shown in FIG. Since the obstacle is positioned in the deceleration control range Rdc or the stop control range Rsc of the range Rd2, it is determined whether the position of the obstacle is in the deceleration control range Rdc or the stop control range Rsc based on the obtained distance to the obstacle. judge.
When the collision avoidance control unit 23H determines that an obstacle is positioned within the deceleration control range Rdc, the collision avoidance control unit 23H causes each display control unit 23E to execute second notification control to notify the existence of the obstacle within the deceleration control range Rdc. , 51A, and the vehicle speed controller 23B to execute automatic deceleration control. As a result, the display control units 23E and 51A execute the second notification control to inform the user that an obstacle exists in the deceleration control range Rdc of the first detection range Rd1 or the second detection range Rd2 for the tractor 1. be able to. Further, automatic deceleration control is executed in the vehicle speed control section 23B, and the vehicle speed of the tractor 1 decreases as the relative distance to the obstacle becomes shorter.
When the collision avoidance control unit 81b determines that an obstacle is positioned in the stop control range Rsc of the first detection range Rd1 or the second detection range Rd2, it notifies the presence of the obstacle in the stop control range Rsc. A third notification start command process is performed to command the display control units 23E and 51A to execute the third notification control, and the vehicle speed control unit 23B is commanded to execute the automatic stop control. As a result, the display control units 23E and 51A execute the third notification control to notify the user that an obstacle exists in the stop control range Rsc of the first detection range Rd1 or the second detection range Rd2 for the tractor 1. be able to. Further, automatic stop control is executed in the vehicle speed control section 23B, and the tractor 1 stops when an obstacle exists in the stop control range Rsc. As a result, collision of the tractor 1 against obstacles can be avoided.
When the distance to the obstacle is acquired by the fourth distance transmission process or the fifth distance transmission process from the obstacle detection device 87, the collision avoidance control unit 23H uses the third measurement range Rm3 or the fourth measurement range Rm3 shown in FIG. Since the obstacle is located in the range Rm4, the display control units 23E and 51A are instructed to execute the fourth notification control to notify the presence of the obstacle in the third measurement range Rm3 or the fourth measurement range Rm4. At the same time, it commands the vehicle speed control section 23B to execute the automatic stop control. As a result, the display control units 23E and 51A execute the fourth notification control to notify the user that an obstacle exists in the third measurement range Rm3 or the fourth measurement range Rm4 with respect to the tractor 1. FIG. Further, automatic stop control is executed in the vehicle speed control section 23B, and the tractor 1 stops when an obstacle exists in the third measurement range Rm3 or the fourth measurement range Rm4. As a result, collision of the tractor 1 against obstacles can be avoided.

As described above, an obstacle is detected in the deceleration control range Rdc or the stop control range Rsc in the detection ranges Rd1 and Rd2 of the rider sensors 85A and 85B, or in the third measurement range Rm3 or the fourth measurement range Rm4 of the sonar 85C. If so, collision avoidance with an obstacle is performed by automatic deceleration control and automatic stop control of the vehicle speed control section 23B based on the collision avoidance control of the collision avoidance control section 23H. Therefore, based on the detection information about the obstacle from the image processing device 86, the obstacle detection device 87 detects that the obstacle is within the aforementioned deceleration control range Rdc, the stop control range Rsc, the third measurement range Rm3, or the fourth measurement range. When it is detected that it is located in the range Rm4, the matching determination process described above is performed, and the position of the obstacle determined by the image processing device 86 matches the position of the obstacle candidate determined by the active sensor unit 85. In this case, the distance to the obstacle candidate measured by the active sensor unit 85 with high ranging accuracy is set as the distance to the obstacle, and this distance is transmitted to the collision avoidance control section 23H. As a result, based on the distance to the obstacle measured by the active sensor unit 85 with high ranging accuracy, the obstacle is detected by the automatic deceleration control and automatic stop control of the vehicle speed control unit 23B based on the collision avoidance control of the collision avoidance control unit 23H. Collision avoidance with As a result, collision avoidance with obstacles can be performed with high accuracy.
Further, when an obstacle is positioned within the notification control range Rnc of the detection ranges Rd1 and Rd2 of the rider sensors 85A and 85B, the automatic deceleration control of the vehicle speed control section 23B based on the collision avoidance control of the collision avoidance control section 23H and the Since collision avoidance with an obstacle is not performed by automatic stop control, the obstacle detection device 87 does not perform the match determination process described above, and the detection information about the obstacle from the image processing device 86 includes The distance to the obstacle is transmitted to the collision avoidance control section 23H. This avoids an increase in the processing load due to the consistency determination process.

[Another embodiment]
Another embodiment of the present invention will be described.
It should be noted that the configuration of each of the embodiments described below is not limited to being applied alone, and can be applied in combination with the configurations of other embodiments.

(1) The configuration of the work vehicle can be modified in various ways.
For example, the work vehicle may have a semi-crawler specification that includes left and right crawlers instead of the left and right rear wheels 11 .
For example, the work vehicle may be configured with full crawler specifications having left and right crawlers instead of the left and right front wheels 10 and the left and right rear wheels 11 .
For example, the work vehicle may be configured with a rear wheel steering specification in which the left and right rear wheels 11 function as steering wheels.
For example, the work vehicle may be configured to be electrically powered with an electric motor instead of the engine 14 .
For example, the work vehicle may have a hybrid specification that includes the engine 14 and an electric motor for traveling.

(2) Radar sensors may be employed as the active sensors 85A to 85C instead of the lidar sensors 85A and 85B and the sonar 85C. Also, lidar sensors may be employed for all of the active sensors 85A to 85C.

(3) The active sensors 85A-85C may be configured to measure the distance to the obstacle when the image processing device 86 detects the obstacle in their measurement ranges Rm1-Rm4.
According to this configuration, the active sensors 85A to 85C do not always need to perform measurements in their measurement ranges Rm1 to Rm4. Since the distance to the obstacle is measured only in this case, the accuracy of distance measurement to the obstacle can be improved while reducing the load required for distance measurement.

(4) The active sensors 85A to 85C may be configured to constantly perform measurements in their measurement ranges Rm1 to Rm4 and constantly transmit the measured values to the obstacle detection device 87.
According to this configuration, the obstacle detection device 87 can always receive the measurement values from the active sensors 85A-85C, so that if any of the imaging devices 81-84 detects an obstacle, , the position of the obstacle and the distance to the obstacle based on the measured values from the active sensors 85A to 85C can be specified at a stage with little time lag from the detection of the obstacle.

(5) A stereo camera or the like may be adopted as the imaging devices 81 to 84 .
In addition to the front camera 81, rear camera 82, right camera 83, and left camera 84 described above, a front camera having a different angle of view from the front camera 81 may be provided as the imaging devices 81-84.
In this case, for example, as shown in FIG. 17, a front image display area 90A in which an image of the front camera 81 is arranged and an image of the rear camera 82 are displayed in a full-circumference image display area 90 such as the liquid crystal monitor 27 or the display device 50. In addition to the rear image display area 90B arranged, the right image display area 90C where the image of the right camera 83 is arranged, and the left image display area 90D where the image of the left camera 84 is arranged, A second forward image display area 90E in which the image of the front camera is arranged may be provided.

<Additional remarks of the invention>
A first characteristic configuration of the present invention is an obstacle detection system for a work vehicle,
a plurality of imaging devices that capture images of the surroundings of the work vehicle; and an image processing device that performs an obstacle discrimination process for discriminating obstacles based on the images from the plurality of imaging devices in a time-sharing manner,
The image processing device changes a processing target period per unit time for the plurality of imaging devices in a time-division system according to the traveling direction and vehicle speed of the work vehicle.

According to this configuration, the image processing device performs obstacle discrimination processing based on the captured image with high object discrimination accuracy. Therefore, it is possible to avoid the risk of erroneously detecting tall grass or the like as an obstacle that hinders the running of the work vehicle. In addition, even if dust, dust, or the like floats up around the working vehicle, it is possible to avoid the risk of erroneously detecting the floating matter as an obstacle that hinders the traveling of the working vehicle.
Then, the image processing apparatus performs the obstacle discrimination processing in a time-division manner, so that the obstacle discrimination processing for the images from the imaging devices with a large processing load can be sequentially and quickly performed by a single image processing device.
Further, for example, the image processing device shortens the processing target cycle per unit time for each imaging device having an imaging range in the traveling direction of the work vehicle according to the traveling direction of the work vehicle, and images the traveling direction of the work vehicle. By slowing down the processing target period per unit time for each image pickup device that is not within the range, the obstacle discrimination processing for the images from each image pickup device can be sequentially and quickly performed by a single image processing device, and the work vehicle can be operated. It is possible to focus on determining whether or not an object existing in the running direction is an obstacle.
Furthermore, for example, as the vehicle speed of the work vehicle increases, the image processing device shortens the processing target period per unit time for each imaging device whose imaging range is the travel direction of the work vehicle, and also adjusts the travel direction of the work vehicle. By slowing down the processing target period per unit time for each imaging device that is not included in the imaging range, the obstacle discrimination processing for the images from each imaging device can be sequentially and smoothly performed by a single image processing device, and the working vehicle can be operated. As the vehicle speed increases, it is possible to quickly and intensively determine whether or not an object existing in the traveling direction of the work vehicle is an obstacle.
As a result, it was possible to construct an obstacle detection system for a work vehicle that can accurately detect obstacles existing around the work vehicle while simplifying the configuration.

The second characteristic configuration of the present invention is
a plurality of active sensors for measuring a distance to a measurement target existing around the work vehicle;
The obstacle detecting device detects the position of the obstacle and the distance to the obstacle based on the determination information from the image processing device and the measurement information from the active sensor.

According to this configuration, based on the determination information from the image processing device that performs the obstacle determination process based on the camera image with high object determination accuracy and the measurement information from the active sensor unit with high ranging accuracy, the obstacle is detected. Since the detection device detects the position of the obstacle and the distance to the obstacle, tall grass that exists in the direction of travel of the work vehicle and floating objects such as dust and dust that have risen around the work vehicle can be detected. It is possible to accurately detect the position of the obstacle and the distance from the work vehicle to the obstacle while avoiding the possibility of erroneous detection as an obstacle.

The third characteristic configuration of the present invention is
The image processing device obtains the coordinates of the obstacle on the image and the distance to the obstacle based on the image from the imaging device,
The active sensor is a lidar sensor that sequentially measures the distance for each coordinate of a plurality of ranging points that exist as the measurement object around the work vehicle and extracts ranging point groups of obstacle candidates,
The obstacle detection device detects a match when the coordinates of the obstacle and the distance to the obstacle from the image processing device match the distance for each coordinate of the range-finding point group from the lidar sensor. The distance of the range-finding point group is adopted as the distance to the obstacle, and if they do not match, the distance from the image processing device to the obstacle is adopted.

According to this configuration, the obstacle detection device includes the coordinates of the obstacle and the distance to the obstacle determined by the image processing device with high object determination accuracy, and the distance measurement point group as the obstacle candidate extracted by the lidar sensor. The distance to the obstacle candidate obtained from the lidar sensor with high ranging accuracy is adopted as the distance to the obstacle only when the distance for each coordinate matches the distance to the obstacle candidate. Therefore, it is possible to avoid the possibility that the distance to the obstacle candidate at the time is adopted as the distance to the obstacle.
As a result, the obstacle detection device can acquire detection information related to obstacles with high object discrimination accuracy and distance measurement accuracy.
In addition, the obstacle detection device may be affected by dust, dust, etc. floating around the lidar sensor as floating matter, which may cause the measurement accuracy of the lidar sensor to drop, or may cause an abnormality such as soiling of the sensor surface of the lidar sensor. In this case, the distance to the obstacle can be obtained from the image processing device.

The fourth characteristic configuration of the present invention is
The active sensor measures the distance to the obstacle when the obstacle is detected by the image processing device.

According to this configuration, when an obstacle is detected by the image processing device, the active sensor measures the distance to the obstacle. accuracy can be improved.

1 work vehicle 81 imaging device (front camera)
82 imaging device (front camera)
83 Imaging device (right camera)
84 imaging device (left camera)
86 Image processing device 85A Active sensor (front lidar sensor)
85B active sensor (rear rider sensor)
85C Active Sensor (Sonar)
87 Obstacle detector

Claims (4)

An image processing device that performs an obstacle discrimination process for discriminating obstacles based on images from a plurality of imaging devices that capture images of the surroundings of the work vehicle in a time-sharing manner ,
The image processing device is an obstacle detection system for a work vehicle that changes a processing target period per unit time for the plurality of imaging devices in a time-division system according to at least one of a traveling direction and a vehicle speed of the work vehicle. . Based on the measurement information from a plurality of active sensors that measure the distance to the measurement object existing around the work vehicle and the determination information from the image processing device, the position of the obstacle and the distance to the obstacle 2. The obstacle detection system for a work vehicle according to claim 1, further comprising an obstacle detection device for detecting distance. The image processing device obtains the coordinates of the obstacle on the image and the distance to the obstacle based on the images from the plurality of imaging devices,
The active sensor is a lidar sensor that sequentially measures the distance for each coordinate of a plurality of ranging points that exist as the measurement object around the work vehicle and extracts ranging point groups of obstacle candidates,
The obstacle detection device detects a match when the coordinates of the obstacle and the distance to the obstacle from the image processing device match the distance for each coordinate of the range-finding point group from the lidar sensor. 3. The obstacle for a work vehicle according to claim 2, wherein the distance of said range-finding point group is adopted as the distance to said obstacle, and if they do not match, the distance from said image processing device to said obstacle is adopted. detection system. 4. The obstacle detection system for a work vehicle according to claim 2, wherein the active sensor measures the distance to the obstacle when the image processing device detects the obstacle.