JP2024025398A - Display control device, head-up display device, and display control method - Google Patents

Abstract

To suppress deterioration of a virtual reality of an image (virtual object) caused by a change in attitude of a vehicle.SOLUTION: A HUD device carries out a first image adjustment processing that dynamically changes a position of a virtual image according to the change in the attitude of the vehicle with reference to a first reference position Po10, for suppressing a relative positional deviation between the virtual image and a foreground caused by the change in the attitude of the vehicle. When it is determined that the change in the attitude of the vehicle will be large, a second reference position Po20 is set in which a virtual position of the virtual image is off-set above (or below) the first reference position. A second image adjustment processing that dynamically changes the position of the virtual image according to the change in the attitude of the vehicle with reference to the second reference position, for suppressing the relative positional deviation between the virtual image and the foreground caused by the change in the attitude of the vehicle.SELECTED DRAWING: Figure 13

Description

The present disclosure relates to a display control device, a head-up display device, and a head-up display device that are used in a moving object such as a vehicle and superimpose an image on the foreground of the moving object (actual view in the forward direction of the moving object as seen from a vehicle occupant). Related to display control methods, etc.

A Head Up Display (HUD) device displays an image (virtual object) superimposed on the scenery in front of the vehicle, creating an augmented reality (augmented reality) that adds and emphasizes information to the real scene or real objects existing in the real scene. By expressing AR (Augmented Reality) and accurately providing desired information while suppressing the line of sight movement of the user driving the vehicle, it can contribute to safe and comfortable vehicle operation.

The head-up display device described in Patent Document 1 displays an image (virtual object) at a display reference position, and shifts the display position of the image (virtual object) from the display reference position based on the magnitude of the attitude change of the vehicle. By correcting so that there is no deviation, a shift in the positional relationship of the image (virtual object) with respect to the real scene or a real object existing in the real scene is suppressed. As a result, even if the attitude of the vehicle changes, the relative positional relationship between the image and the actual scene is maintained, so that the image can be further harmonized with the actual scene.

The area of the actual scene where the images were originally visually recognized as overlapping before the attitude change of the vehicle is set as the superimposed actual scene area. In a head-up display device, if the virtual image display area shifts to the extent that it does not overlap the superimposed real view area due to a large attitude change of the vehicle, when the image position is adjusted according to the large attitude change, a part of the adjusted image Or the entire image no longer fits within the virtual image display area (cannot be displayed). If part or all of the image is cut off due to the posture change (hereinafter, this phenomenon will also be referred to as "cut off"), the virtual reality of the virtual image will be impaired, and it is assumed that the viewer will feel uncomfortable.

As shown in FIG. 6, the head-up display device of Patent Document 1 restricts the shift of display content upward from the display reference position by Lu and downward from the display reference position in response to changes in the posture of the vehicle. This prevents the display content from being cut off.

International Publication 2020/208883

If an attempt is made to widen the upwardly shiftable area Lu, the display content will be displayed relatively lower in the display area. Furthermore, if it is attempted to widen the downwardly shiftable area Ld, the display content will be displayed relatively above the display area. In other words, the layout of the content displayed within the display area is approximately the same as when you want to provide a wide area where the upward shift can be made or when you want to provide a wide area where the downward shift can be made in response to changes in the vehicle's attitude. It is assumed that this will reduce the degree of freedom in layout of the content.

Conversely, if emphasis is placed on the layout of the content (virtual image), the area that can be shifted upwards or downwards may become narrower due to changes in the vehicle's attitude unintentionally. This makes it difficult to adjust the desired position of the image in response to changes in the vehicle's posture, and it is assumed that the impression that the virtual image is in harmony with the real scene will fade (the sense of virtual reality will deteriorate).

A summary of certain embodiments disclosed herein is provided below. It is to be understood that these aspects are presented solely to provide the reader with an overview of these particular embodiments and do not limit the scope of this disclosure. Indeed, the present disclosure may encompass combinations of the embodiments described below and various aspects not described below.

The summary of the present disclosure relates to reducing discomfort in viewing virtual images. More specifically, it also relates to suppressing a decrease in the virtual reality of an image (virtual object) due to changes in the posture of the vehicle.

Therefore, the display control device, head-up display device, display control method, etc. described in this specification employ the following means to solve the above problems. In this embodiment, in order to suppress relative positional deviation between the virtual image and the foreground due to changes in the attitude of the vehicle, the position of the virtual image is dynamically changed based on the first reference position according to changes in the attitude of the vehicle. If the first image adjustment process is executed and it is determined that the attitude variation of the vehicle becomes large, a second reference position is set by offsetting the reference position of the virtual image upward (or downward) from the first reference position. , in order to suppress a relative positional shift between the virtual image and the foreground due to the attitude change of the vehicle, the position of the virtual image is dynamically adjusted based on the second reference position in accordance with the attitude change of the vehicle. The gist thereof is to execute the second image adjustment process to change the image to the image.

Therefore, the display control device in the first embodiment described in this specification is mounted on a vehicle, has a display section that displays an image on a display surface, and directs the light of the image toward the projected section. A display control device that controls a head-up display device that displays a virtual image superimposed in a virtual image display area that overlaps with the foreground of the vehicle,
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors;
The processor includes:
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first image adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. An image adjustment process including a second image adjustment process of dynamically changing the position of the virtual image according to the attitude change information as a reference is executed.

The reference position is a position where a virtual image is displayed within the virtual image display area when the attitude of the vehicle is a predetermined reference attitude. The reference posture is a posture assumed when the vehicle normally travels, and for example, the pitching angle with respect to the road surface is zero [degree]. A virtual image that has a second reference position that is offset upward from the reference position does not display a virtual image even if the virtual image is corrected downward to compensate for the virtual image display area moving upward due to changes in vehicle attitude (backward tilting). It becomes easier to fit the virtual image within the area and display it (the virtual image is less likely to be cut off). Conversely, in a virtual image having a second reference position that is offset downward from the reference position, even if the virtual image is corrected upward in response to the virtual image display area moving downward due to changes in the attitude of the vehicle (forward tilting). , it becomes easier to fit and display the virtual image within the virtual image display area (the virtual image is less likely to be cut off).

According to the first embodiment, when the attitude variation of the vehicle is relatively small (or when the attitude variation is predicted to be relatively small) and when the attitude variation of the vehicle is relatively large (or when the attitude variation is relatively large). ), it is possible to change the reference position that serves as a reference for correcting the position of the virtual image in accordance with the posture change. Therefore, when the attitude change of the vehicle is relatively large, by offsetting the reference position upward, it is possible to prevent the virtual image from being cut off even when the position of the virtual image is corrected for the attitude change (backward tilting) of the vehicle. Furthermore, by offsetting the reference position downward, it is possible to make it difficult for the virtual image to be cut off even when the position of the virtual image is corrected for changes in the posture of the vehicle (forward tilting). Note that when the posture change is relatively small, the position of the virtual image can be corrected for the relatively small posture change without offsetting the reference position. In other words, when the posture change is relatively small, the position of the virtual image can be corrected for the posture change while the virtual image (content) is placed in the virtual image display area with a high degree of freedom; when the posture change is relatively large, It is assumed that by offsetting the reference position, it is less likely that the upper or lower side of the virtual image will be cut off.

Further, in a display control device of a second embodiment that may be subordinate to the first embodiment, the processor may, in the second image adjustment process,
The second reference position is set lower than the first reference position when viewed from an observer. According to this, by offsetting the reference position downward when the forward tilt attitude change is relatively large, it is possible to prevent the upper part of the virtual image from being cut off even when the position of the virtual image is corrected for the forward tilt of the vehicle. The advantage of being able to do so is also expected.

Further, in a display control device according to a third embodiment that is subordinate to the second embodiment, the processor may include:
In the first image adjustment process, dynamically changing the position of the virtual image upward and downward in accordance with the attitude change information with respect to the reference position,
In the second image adjustment process, the position of the virtual image is dynamically changed upward in accordance with the attitude change information, but not downward, based on the reference position. By offsetting the reference position downward, it is possible to prevent the upper part of the virtual image from being cut off even when the position of the virtual image is corrected for the forward tilt of the vehicle, but when the position of the virtual image is corrected for the backward tilt of the vehicle. It is assumed that the lower part of the virtual image is more likely to be cut off. In the third embodiment, in the second image adjustment process, the position of the virtual image is dynamically adjusted upward according to attitude change information, but not changed downward, thereby correcting the position of the virtual image with respect to the forward tilt of the vehicle. It is also envisioned that there are advantages in that the upper part of the virtual image is less likely to be cut off when the vehicle tilts backward, and the lower part of the virtual image is prevented from being cut off when the vehicle leans backward.

Further, in a display control device according to a fourth embodiment that may be subordinate to the first to third embodiments, the processor:
The larger the attitude change of the vehicle that is detected or predicted based on the attitude change information or the attitude change prediction information,
The amount of offset of the second reference position from the first reference position is set to be large. By offsetting the reference position, it is possible to suppress display defects caused by correcting the position of the virtual image in response to posture changes in one of forward and backward tilts, but display defects due to virtual image position correction in response to the other posture changes in forward and backward tilts. On the contrary, it is expected that this will occur more easily. According to the present embodiment, by setting the offset amount of the reference position according to the size of the posture change that may occur, display omission due to virtual image position correction for either forward or backward posture change can be suppressed. , it is also assumed that there is an advantage that it is possible to suppress display defects due to positional correction of the virtual image with respect to the other attitude change.

Further, in the display control device of the fifth embodiment that may be subordinate to the first to fourth embodiments, the virtual image is located at a position farther from the vertical center of the virtual image display area than the first virtual image and the first virtual image. including at least a second virtual image displayed in
When the processor determines that the attitude change of the vehicle will increase based on the attitude change information or the attitude change prediction information,
performing the second image adjustment process on the first virtual image;
The second virtual image is subjected to the first image adjustment process. According to the present embodiment, an advantage is also envisaged that even when the position of the virtual image is corrected for a relatively large attitude change of the vehicle leaning forward or leaning backward, it is possible to make it difficult for the virtual image to be cut off.

Further, in the display control device of the fifth embodiment that may be subordinate to the second embodiment, the virtual image includes at least a first virtual image and a second virtual image displayed below the first virtual image,
When the processor determines that the attitude change of the vehicle will increase based on the attitude change information or the attitude change prediction information,
performing the second image adjustment process on the first virtual image;
performing the first image adjustment process on the second virtual image;
In the second image adjustment process, the second reference position is set lower than the first reference position when viewed from the observer. According to the present embodiment, even when the position of the virtual image is corrected for a relatively large change in attitude of a vehicle leaning forward, an advantage is also envisaged in that the upper part of the virtual image is less likely to be cut off.

Further, in the display control device of the sixth embodiment that may be subordinate to the first to fifth embodiments, when the processor detects that the amplitude of the attitude fluctuation of the vehicle has become equal to or less than a predetermined value, the When returning the second reference position to the first reference position, the reference position is gradually changed. According to this, it is also assumed that the sense of discomfort that may be given to the observer due to the instantaneous return of the virtual image from the second reference position to the first reference position is reduced.

Further, in a display control device according to a sixth embodiment that may be subordinate to the first to fifth embodiments, a display unit that is mounted on a vehicle and displays an image on a display surface;
a relay optical system that directs display light from the display section to a projected section;
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors, the virtual image being superimposed and viewed in a virtual image display area overlapping with the foreground of the vehicle. A head-up display device,
The processor includes:
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first position adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. performing a position adjustment process including a second position adjustment process of dynamically changing the position of the virtual image according to the attitude change information as a reference;
A head-up display device characterized by:

Further, a display control device according to a sixth embodiment that may be subordinate to the first to fifth embodiments includes a display unit that is mounted on a vehicle and displays an image on a display surface, and directs the light of the image to a projected unit. A display control method for controlling a head-up display device that displays a virtual image superimposed in a virtual image display area that overlaps with the foreground of the vehicle by directing the head-up display device,
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first position adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. a second position adjustment process of dynamically changing the position of the virtual image according to the attitude change information as a reference;
A display control method characterized by:

FIG. 1 is a diagram showing an example in which the vehicle display system of this embodiment is applied to a vehicle. FIG. 2 is a diagram showing the configuration of a head-up display device. FIG. 3 is a block diagram of a vehicle display system of some embodiments. FIG. 4 is a diagram illustrating a model space in which content and virtual viewpoints are arranged, according to some embodiments. FIG. 5 is a diagram illustrating an example of a foreground visually recognized by an observer and an image (virtual image) displayed superimposed on the foreground while the vehicle is running. FIG. 6 is a diagram showing the virtual image before the position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the viewer faces forward. shows. FIG. 7A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the foreground that are visible when the viewer faces forward. A virtual image is shown. FIG. 7B is a diagram showing the virtual image after the second image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visible when the observer looks forward. A virtual image is shown. FIG. 8A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the viewer faces forward. A virtual image is shown. FIG. 8B is a diagram showing the virtual image after the second image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the observer faces forward. A virtual image is shown. FIG. 9 is a diagram for explaining the limit CT of the position adjustment amount. FIG. 10 is a diagram for explaining a plurality of position adjustment ranges set for each of a plurality of contents. FIG. 11 is a diagram illustrating image adjustment processing for large posture changes in a comparative example. FIG. 12 is a diagram illustrating the image adjustment process in the case where it is not determined that the change in the posture of the vehicle will become large in this embodiment. FIG. 13 is a diagram illustrating image adjustment processing in the present embodiment when it is determined that the change in the posture of the vehicle becomes large. FIG. 14 is a diagram illustrating image adjustment processing when it is determined that the change in the posture of the vehicle will increase in this embodiment. FIG. 15 is a diagram illustrating image adjustment processing in the present embodiment when it is determined that the vehicle attitude variation becomes large. FIG. 16 is a diagram illustrating a flow of display control in some embodiments.

1-10 and 12-16, a description of the configuration and operation of an exemplary vehicle display system is provided below. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Of course, changes (including deletion of components) can be made to the embodiments described below. Further, in the following description, description of known technical matters will be omitted as appropriate in order to facilitate understanding of the present invention.

Please refer to FIG. FIG. 1 is a diagram showing an example of the configuration of a virtual image display system for a vehicle. In FIG. 1, the left and right direction (in other words, the width direction of the vehicle 1) of the vehicle (an example of a moving object) 1 is the X axis (the positive direction of the X axis is the left direction when facing the front of the vehicle 1). ), and the vertical direction (in other words, the height direction of the vehicle 1) along a line segment perpendicular to the left-right direction and perpendicular to the ground or a surface corresponding to the ground (road surface 6 in this case) is the Y-axis (Y-axis The positive direction is the upward direction), and the front-rear direction along the line segment orthogonal to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight direction of the vehicle 1). This point also applies to other drawings.

As shown in the figure, a vehicle display system 10 provided in a vehicle (mobile object) 1 displays the positions and line of sight of the left eye 700L and right eye 700R of an observer (typically a driver seated in the driver's seat of the vehicle 1). An external sensor 411 consisting of an eye position detection unit (line of sight detection unit) 409 for detecting pupils (or faces), a camera (for example, a stereo camera) that captures an image of the front of the vehicle 1 (in a broad sense, the surroundings), and the vehicle. 1, a head-up display device (hereinafter also referred to as a HUD device) 20, and a display control device 30 that controls the HUD device 20. Note that the eye position detection unit (line of sight detection unit) 409 and the vehicle external sensor 411 may be omitted.

FIG. 2 is a diagram showing one aspect of the configuration of a head-up display device (HUD device) 20. As shown in FIG. The HUD device 20 is installed, for example, in a dashboard (numeral 5 in FIG. 1). This HUD device 20 houses an image display device (display section) 40, a relay optical system 80, and the image display device 40 and the relay optical system 80, and directs display light K from the image display device 40 from the inside to the outside. The casing 22 has a light exit window 21 through which light can be emitted. The HUD device 20 expresses virtual content FU by displaying the virtual image V0 in the air.

The image display device (display unit) 40 is here a parallax type 3D display device. This stereoscopic display device (parallax type 3D display device) 40 includes a display device 50 which is an autostereoscopic display device using a multi-view image display method that can control depth expression by visually recognizing a left viewpoint image and a right viewpoint image. , and a light source unit 60 that functions as a backlight. Note that the image display device (display unit) 40 is not limited to a stereoscopic image display device that displays 3D images, but may also display 2D images. That is, the virtual image V0 may be displayed in 3D or 2D.

The display control device 30, which will be described later, transmits the left eye display light K10 of the left viewpoint image V1 to the left eye 700L of the observer and the right eye 700R by executing, for example, image rendering processing (graphic processing), display drive processing, etc. Control the mode of content FU displayed by the HUD device 20 (perceived by the observer) by directing the right-eye display light K20 of the right-view image V2 and adjusting the left-view image V1 and the right-view image V2. Can be done. Note that the display control device 30, which will be described later, controls the display (display device 50) so as to reproduce a light field that (approximately) reproduces the light rays output in various directions from points existing in a certain space. You can.

The relay optical system 80 includes curved mirrors (concave mirrors, etc.) 81 and 82 that reflect light from the image display device 40 and project image display lights K10 and K20 onto the windshield (projection target member) 2. However, it may further include other optical members (which may include a refractive optical member such as a lens, a diffractive optical member such as a hologram, a reflective optical member, or a combination thereof).

In FIG. 1, the image display device 40 of the HUD device 20 displays images with parallax (parallax images) for each of the left and right eyes. As shown in FIG. 1, each parallax image is displayed as V1 and V2 formed on a virtual image display area (virtual image plane) VS. The focus of each eye of the observer (person) is adjusted to match the position of the virtual image display area VS. Note that the position of the virtual image display area VS is referred to as an "adjustment position (or imaging position)" and may also be a predetermined reference position (for example, the center 205 of the eyebox 200 of the HUD device 20, the observer's viewpoint position, or , a specific position of the vehicle 1, etc.) to the virtual image display area VS is referred to as an adjustment distance (imaging distance).

Further, the virtual image display area VS is a virtual (apparent) surface set in the real space in front of the occupant (a viewer such as a driver), corresponding to the display surface 50a of the display 50. Further, the virtual image display area VS includes, for example, an elevational surface VS1 perpendicular to the road surface 6, an inclined surface VS2 inclined with respect to the road surface 6, a road surface superimposed surface VS3 superimposed on the road surface 6, and a side closer to the occupant (viewer). There are surfaces (not shown) that are elevations (including pseudo-elevations) and whose far side is an inclined surface. In display using surfaces other than the vertical surface VS1, the display distance of the virtual image varies depending on the display position on the virtual image display area, and therefore depth can be expressed.

FIG. 3 is a block diagram of a virtual image display system for a vehicle, according to some embodiments. The display control device 30 includes one or more I/O interfaces 31, one or more processors 33, one or more display control processing circuits 35, and one or more memories 37. FIG. 3 is only one embodiment, and the illustrated components may be combined into fewer components or there may be additional components. For example, display control processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33.

As shown, processor 33 and display control processing circuit 35 are operably coupled to memory 37 . More specifically, the processor 33 and the display control processing circuit 35 execute programs stored in the memory 37 to generate and/or transmit image data, for example, in the vehicle display system 10 (image data). The display device 40) can be controlled. Processor 33 and/or display control processing circuitry 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), and at least one field programmable gate array (FPGA). ), or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operably coupled to I/O interface 31 . The I/O interface 31 communicates with, for example, a vehicle ECU 401 (described later) and/or other electronic devices (numerals 403 to 419 described later) provided in the vehicle according to the CAN (Controller Area Network) standard. communication). Note that the communication standard adopted by the I/O interface 31 is not limited to CAN, and includes, for example, CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), and MOST (Media Oriented Systems Transport). Wired communication interfaces such as: MOST (registered trademark), UART, or USB, or local communication interfaces such as personal area networks (PAN), e.g. Bluetooth networks, 802.11x Wi-Fi networks, etc. It includes an in-vehicle communication (internal communication) interface that is a short-range wireless communication interface within several tens of meters such as an area network (LAN). Further, the I/O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, It may also include an external communication interface such as a wide area communication network (for example, the Internet communication network) according to cellular communication standards such as 5G.

As shown in the figure, the processor 33 is interoperably connected to the I/O interface 31, so that it can communicate information with various other electronic devices connected to the vehicle display system 10 (I/O interface 31). It becomes possible to give and receive. The I/O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, a vehicle position detection section 405, an operation detection section 407, an eye position detection section 409, an external sensor 411, a brightness detection section 413, and an attitude detection section 415. , a mobile information terminal 417, an external communication device 419, and the like are operably connected. Note that the I/O interface 31 may include a function to process (convert, calculate, analyze) information received from other electronic devices connected to the vehicle display system 10.

Image display device 40 is operably coupled to processor 33 and display control processing circuit 35 . Accordingly, the image displayed by light modulation element 51 may be based on image data received from processor 33 and/or display control processing circuit 35. The processor 33 and the display control processing circuit 35 control the image displayed by the light modulation element 51 based on information obtained from the I/O interface 31.

The software components stored in memory 37 include a drawing module 510 and an image adjustment module 520 (position adjustment module 522, depression angle adjustment module 524, size adjustment module 526).

The drawing module 510 forms an image M based on information (navigation information, vehicle information, etc.) acquired by the display control device 30, and temporarily stores the formed image M in a buffer (not shown). The image M displayed on the display 50 is visually recognized by the observer 700 as a virtual image V0. At this time, the virtual image V0 represents the content FU.

The drawing module 510 calculates each vertex data of the content FU to be drawn in each drawing frame. In this case, a model space for each content FU is constructed. Then, data for each vertex to be drawn is calculated on the "model coordinate system (local coordinate system)" for each virtual object. The drawing module 510 converts the content FU drawn in the model coordinate system into a two-dimensional image by projecting it onto a predetermined projection plane (virtual image display area VS to be described later) based on the virtual viewpoint VP, and converts the content FU into a two-dimensional image. Let be image M. Note that the drawing module 510 may arrange each content FU arranged in the "model coordinate system (local coordinate system)" in the space of the "world coordinate system." That is, the vertex data of each content FU to be drawn calculated on the "model coordinate system" may be placed on the "world coordinate system." Note that some or all of the content FU does not need to be placed on the "world coordinate system."

FIG. 4 is a diagram illustrating a model space in which content FU and virtual viewpoints VP are arranged. In FIG. 4, the coordinate system of the virtual viewpoint VP is such that the depth direction is the Z1-axis direction, the left-right direction is the X1-axis direction (corresponding to the width direction X of the vehicle 1), and the vertical direction is the Y1-axis direction (corresponding to the width direction (corresponds to the vertical direction Y). The observer 700 visually recognizes the virtual image V0 formed (imaged) in the virtual image display area VS via the projection target section 2, and perceives that the content FU is located at a predetermined target position MP.

For example, when the content FU1 is an arrow that guides a course, the arrow of the virtual image V10 is arranged so that the content FU1 is viewed from the virtual viewpoint VP0 as if it were placed at a predetermined target position MP1 in the real scene. It is displayed in the virtual image display area VS. That is, when an image (virtual image V10 here) obtained by projectively transforming the content FU into the virtual image display area VS is displayed based on the virtual viewpoint VP, the image is projected from the same position as the virtual viewpoint VP (for example, the center 205 of the eyebox 200). When viewed by the observer 700, as shown in FIG. 5, the content FU1 can be perceived as being placed at a predetermined target position MP1 as seen from the virtual viewpoint VP.

Normally, the virtual plane 100 on which the target position MP1 on which the content FU1 is arranged (set) is arranged is made to match the height of the surface of the foreground (road surface 6) (that is, the set height is set to 0 m). ). However, this is an example and is not limited. The depression angle β is the angle (looking down angle) between the horizontal direction (Z1-X1 plane) and the content FU (target position MP) when viewed from a predetermined virtual viewpoint VP.

Further, in another example, the target position MP may be set at a position higher than the height of the surface of the foreground (road surface 6) (that is, the set height may be set to 0.5 m or 1 m). For example, the second content FU2 indicating the speed of the vehicle 1 may be placed at the target position MP2 closer to the virtual viewpoint VP than the first content FU1. With this arrangement, parallax can be created between the first content FU1 and the second content FU2 by movement of the virtual viewpoint VP (here, movement according to the attitude change of the vehicle 1). The positions of the virtual image V10 (first content FU1) and virtual image V20 (second content FU2) displayed in the virtual image display area VS are adjusted as the virtual viewpoint VP moves (here, movement according to the attitude change of the vehicle 1). However, the amount of position adjustment of the virtual image V10 (first content FU1) is set to be larger than the amount of position adjustment of the virtual image V20 (second content FU2). In another example, the target position MP may be set at a position lower than the height of the surface of the foreground (road surface 6) (that is, the set height may be set to -1 m or -2 m).

The image adjustment module 520 (position adjustment module 522, depression angle adjustment module 524, size adjustment module 526) in FIG. (position adjustment process), process to adjust the depression angle (depression angle adjustment process), and process to adjust the size (size adjustment process).

FIG. 6 is a diagram showing the virtual image before the position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the viewer faces forward. shows. In FIG. 6, it is assumed that the vehicle attitude AT10 is the reference attitude AT11 (pitching angle α0). Content FU is placed at target position MP (first area 110) on virtual plane 100. The content FU has a predetermined size and is arranged so as to overlap the first area 110 of the virtual plane 100. In FIG. 6, the size of the content FU is the first length L10 in the depth direction of the first area 110. The arbitrary virtual plane 100 is a virtual plane on which the content FU is arranged, and is set, for example, parallel to the front, rear, left, and right directions of the vehicle 1 (it may generally coincide with the road surface 6). The drawing module 510 displays an image M, which is the source of the virtual image V21, on the display device 50 so as to display an image obtained by projectively transforming the content FU into the virtual image display area VS1 (here, the virtual image V21) based on the virtual viewpoint VP1. Display. Here, the distance from the virtual viewpoint VP1 to the first region 110 along the virtual plane 100 is set as D0, and the distance from the virtual plane 100 to the virtual viewpoint VP1 is set as h0. The reference attitude AT11 is the vehicle attitude when the pitching angle α0 is the reference value of the vehicle and is stored in the memory 37 in advance, and is typically a state where the pitching angle of the vehicle is zero (parallel to the road surface 6). It is. Note that the reference attitude AT11 (base pitching angle α0) may be varied. Specifically, when the pitching angle α of the vehicle does not change much for a predetermined period of time or more, the stable pitching angle α at that time is set (updated) as the reference pitching angle α0 (reference attitude AT11), and is stored in the memory 37. You may memorize it.

FIG. 7A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the foreground that are visible when the viewer faces forward. A virtual image is shown. In FIG. 7A, it is assumed that the vehicle attitude AT10 is an AT12 in which the vehicle 1 is tilted forward by a relatively smaller pitching angle α12 than the vehicle attitude AT11 before the first image adjustment process. Here, the term "forward tilt" refers to the fact that the front of the vehicle 1 is lowered (in other words, the rear of the vehicle 1 is raised) based on the vehicle attitude AT11 shown in FIG. 6. When the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. ) by B12. When the first image adjustment process is not performed, the virtual image V21 shown in FIG. 6 is shifted downward to position G2 in the right diagram of FIG. It is shifted by the shift amount B12. That is, in the content FU seen from the observer's viewpoint, the virtual image V21 is shifted from the desired target position MP1. The processor 33 in some embodiments moves the virtual image position G2 upward (in the Y-axis positive direction) so as to suppress (cancel) the shift amount B12 due to posture fluctuation by executing the first image adjustment process. The first position adjustment amount C12 (C10) is corrected (the virtual image V22 whose position has been adjusted is displayed). Preferably, the first position adjustment amount C12 (C10) is made equal to the image shift amount B12 (B10) due to the attitude change (C10=B10), so that the virtual image V20 after the attitude change is adjusted to the first area 110 ( The target position MP1) can be maintained. According to this, the image shift amount B10 due to the posture change is canceled out by the first position adjustment amount C10, and is not recognized by the viewer. However, the first position adjustment amount C10 only needs to be able to reduce the image shift amount B10 due to posture variation, and may be smaller than the image shift amount B10. According to this, it is possible to suppress a positional shift of the virtual image due to a change in the vehicle attitude.

FIG. 7B is a diagram showing the virtual image after the second image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visible when the observer looks forward. A virtual image is shown. In FIG. 7B, it is assumed that the vehicle attitude AT10 is an AT13 in which the vehicle 1 is tilted forward by a relatively large pitching angle α13 (>α12) than the vehicle attitude AT11 before the second image adjustment process. When the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. 7B, the virtual image display area VS shifts downward (in the Y-axis negative direction) by B13 relative to the real scene (road surface) 6 due to attitude change. If the position adjustment is not performed, the virtual image V21 shown in FIG. 6 will be shifted by the shift amount B13 to the position G3 in the right diagram of FIG. 7B due to the shift amount B13 due to the attitude change of the virtual image display area VS. In order to offset the downward shift amount B13 due to this attitude change, it is sufficient to move the virtual image V20 upward by the shift amount B13, but if the virtual image V20 is moved upward by the shift amount B13, the virtual image display area I will be leaving VS13.

In some embodiments, when the attitude change of the vehicle 1 is large (an example of a predetermined condition described later), the processor 33 executes the second image adjustment process to reduce the downward adjustment of the virtual image display area VS due to the attitude change. With respect to the shift amount B13, the virtual image position G3 is corrected upward (in the Y-axis positive direction) by a second position adjustment amount C23 (C20) which is smaller than the first position adjustment amount C13 (C10) in the first image adjustment process. (Displaying the virtual image V23 whose position has been adjusted). By making the second position adjustment amount C23 (C20) smaller than the image shift amount B13 (B10) due to posture change (C20<B10), the second position adjustment amount C23 (C20) is set to be smaller than the image shift amount B13 (B10) due to posture change. The virtual image V23 can be maintained within the virtual image display area VS13, although the virtual image V23 is largely shifted to the second area 120 (a position different from the target position MP1) closer to the target position MP1). According to this, the image shift B13 (B10) due to the posture change is not canceled out by the second position adjustment amount C23 (C20), and therefore is recognized by the observer.

The content FU is arranged with a predetermined angular relationship with respect to the road surface 6. Specifically, for example, the content FU is arranged so as to be visually recognized parallel to the road surface 6. However, when the pitching angle α of the vehicle 1 changes, the angular relationship between the content FU and the road surface 6 changes. Specifically, when the virtual image V20 is displayed parallel to the road surface 6, the parallel relationship between the virtual image V20 and the road surface 6 is shifted by the pitching angle α due to the pitching angle α. The deviation in the angular relationship between the virtual image V20 and the road surface 6 due to the attitude change (pitching) of the vehicle 1 can be corrected by adjusting the angle β of the virtual image V20 with the left-right direction as an axis (the downward angle (depression angle) of the virtual image V20). Can be corrected.

In at least the second image adjustment process, the processor 33 in this embodiment adjusts the depression angle β of the virtual image V20 (content FU) with respect to the attitude change of the vehicle 1. The processor 33 dynamically increases the depression angle β in accordance with the increase in the pitching angle α in the forward tilting direction. Conversely, the processor 33 dynamically decreases the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

In some embodiments, the processor 33 adjusts the depression angle β of the virtual image V20 (content FU) with respect to the attitude change of the vehicle 1 in the first image adjustment process in addition to the second image adjustment process. good. Similar to the second image adjustment process, the processor 33 dynamically increases the depression angle β in accordance with the increase in the pitching angle α in the forward tilting direction. Conversely, the processor 33 dynamically decreases the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

In some embodiments, the processor 33 may adjust the depression angle β by the same angle as the pitching angle change amount α in the first image adjustment process. Specifically, when the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. 7A, the amount of change α in the pitching angle is α12. In the virtual viewpoint VP1 shown in FIG. 6, the angle of the virtual viewpoint VP2 with respect to the virtual plane 100 is changed by α12 based on the amount of change α12 of the forward pitching angle shown in FIG. 7A. As a result, the depression angle β12 looking down on the content FU located at the target position MP1 with respect to the virtual viewpoint VP2 becomes larger than the depression angle β11 by α12. Further, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a coefficient.

Furthermore, in the second image adjustment process, the processor 33 in some embodiments may adjust the depression angle β by the same angle as the pitching angle change amount α. Specifically, when the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. 7B, the amount of change α in the pitching angle is α13. The virtual viewpoint VP1 shown in FIG. 6 moves to the position of the virtual viewpoint VP3 according to the amount of change α13 in the forward pitching angle shown in FIG. 7B, and the angle of the virtual viewpoint VP3 with respect to the virtual plane 100 changes by α13. In accordance with this, the processor 33 may increase the depression angle β of the content FU expressed by the virtual image V23 by the amount of change α13 in the forward pitching angle. Further, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a coefficient.

Preferably, in some embodiments, the processor 33 uses the adjustment amount of the depression angle β (depression angle adjustment amount E20) for the pitching angle change amount α in the second image adjustment process as the pitching angle change in the first image adjustment process. The adjustment amount of the depression angle β with respect to the amount α (the depression angle adjustment amount E10) may be made larger. The virtual image V23 that has been subjected to the second image adjustment process is visually recognized to be more shifted from the target position MP1 than the virtual image V22 that has been subjected to the first image adjustment process. Specifically, when tilting forward, the virtual image V23 subjected to the second image adjustment process is located closer to the observer's near side with respect to the target position MP1 (110) than the virtual image V22 subjected to the first image adjustment process. It is shifted to a position 120 overlapping the actual scene (road surface 6) and is visually recognized. When the content FU is placed near the viewer along the virtual plane 100 parallel to the road surface 6, the angle of depression β looking down on the content FU with respect to the virtual viewpoint VP becomes large. Conversely, when the content FU is placed far from the viewer, the angle of depression β when looking down on the content FU with respect to the virtual viewpoint VP becomes small. Therefore, in some more preferred embodiments, the processor 33 sets the amount of adjustment of the depression angle β of the content FU in the second image adjustment process (depression angle adjustment amount) to the amount of depression angle adjustment due to a change in the pitching angle α of the vehicle 1; The depression angle adjustment amount may be a combination of the depression angle adjustment amount due to the shift of the content FU in the perspective direction of the virtual plane 100 (as an example, set to approximately match the road surface 6). The depression angle β13 in FIG. 7B is the depression angle adjustment amount due to the content FU shifting the depression angle β11 in FIG. and the depression angle adjustment amount due to a change in the pitching angle α of the vehicle 1 are corrected by the depression angle adjustment amount.

FIG. 8A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the viewer faces forward. A virtual image is shown. In FIG. 8A, it is assumed that the vehicle attitude AT10 is an AT14 in which the vehicle 1 is tilted backward by a relatively smaller pitching angle α14 than the vehicle attitude AT11 before the first image adjustment process. Here, the term "backward tilt" refers to the fact that the front of the vehicle 1 is raised (in other words, the rear of the vehicle 1 is lowered) based on the vehicle attitude AT11 shown in FIG. 6 . When the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8A, the virtual image display area VS shifts upward (in the Y-axis positive direction) by B14 relative to the real scene (road surface) 6 due to attitude change. When the first image adjustment process is not performed, the virtual image V21 shown in FIG. 6 is shifted upward to position G4 in the right diagram of FIG. It is shifted by the shift amount B14. That is, the virtual image V21 shifts from the target position MP1 where the content FU is desired to be placed. By executing the first image adjustment process, the processor 33 in some embodiments moves the virtual image position G4 downward (in the Y-axis negative direction) so as to suppress (offset) the shift amount B14 due to posture variation. The first position adjustment amount C14 (C10) is corrected (the virtual image V24 whose position has been adjusted is displayed). Preferably, the first position adjustment amount C14 (C10) is made equal to the image shift amount B14 (B10) due to the attitude change (C10=B10), thereby reducing the virtual image V21 that would be displayed at the position G4 after the attitude change. , can be maintained in the first region 110 (target position MP1). According to this, the image shift B14 (B10) due to the attitude change is canceled out by the first position adjustment amount C14 (C10) and is not recognized by the observer. However, the first position adjustment amount C10 only needs to be able to reduce the image shift amount B10 due to posture variation, and may be smaller than the image shift amount B10. According to this, it is possible to suppress positional deviations due to changes in vehicle posture.

In some embodiments, the processor 33 may adjust the depression angle β by the same angle as the amount of change α in the pitching angle of the vehicle 1 in the backward tilting direction in the first image adjustment process. Specifically, when the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8A, the amount of change α in the pitching angle is α14. In the virtual viewpoint VP1 shown in FIG. 6, the angle of the virtual viewpoint VP4 with respect to the virtual plane 100 changes by α14 due to the amount of change α14 in the backward pitching angle shown in FIG. 8A. As a result, the depression angle β14 looking down on the content FU located at the target position MP1 with reference to the virtual viewpoint VP4 becomes smaller than the depression angle β11 by α14. Further, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a coefficient.

FIG. 8B is a diagram showing the virtual image after the second image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the observer faces forward. A virtual image is shown. In FIG. 8B, it is assumed that the vehicle attitude AT10 is an AT15 in which the vehicle 1 is tilted backward by a relatively large pitching angle α15 (α14>) than the vehicle attitude AT11 before the second image adjustment process. When the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8B, the virtual image display area VS shifts upward (in the Y-axis positive direction) by B15 relative to the real scene (road surface) 6 due to attitude change. If the position adjustment is not performed, the virtual image V21 shown in FIG. 6 will be shifted by the shift amount B15 to the position G5 in the right diagram of FIG. 8B due to the shift amount B15 due to the attitude change of the virtual image display area VS. In order to offset the upward shift amount B15 due to this attitude change, the virtual image that would be displayed at position G5 should be moved downward by the shift amount B15, but the virtual image that would be displayed at position G5 should be moved downward. If it is moved by the shift amount B15, it will move out of the virtual image display area VS15.

In some embodiments, when the posture change is large (an example of a predetermined condition described later), the processor 33 executes the second image adjustment process to increase the upward shift amount B15 of the virtual image display area VS due to the posture change. , the virtual image position G5 is corrected downward (in the Y-axis positive direction) by a second position adjustment amount C25 (C20) smaller than the first position adjustment amount C15 (C10) in the first image adjustment process (position adjustment display the virtual image V25). By making the second position adjustment amount C25 (C20) smaller than the image shift amount B15 (B10) due to posture change (C20<B10), the second position adjustment amount C25 (C20) is set to be smaller than the image shift amount B15 (B10) due to posture change, so that the second position adjustment amount The virtual image V25 can be maintained within the virtual image display area VS15, although the virtual image V25 is largely shifted to the fifth area 150 (a position different from the target position MP1) on the far side than the target position MP1). According to this, the image shift amount B10 due to the posture change is not canceled out by the second position adjustment amount C20, and therefore is recognized by the observer. The processor 33 in this embodiment dynamically reduces the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

In some preferred embodiments, the processor 33 converts the amount of adjustment of the depression angle β (depression angle adjustment amount E20) to the amount of change α of the pitching angle in the second image adjustment process into the amount of change in the pitching angle in the first image adjustment process. The adjustment amount of the depression angle β with respect to α (the depression angle adjustment amount E10) may be made larger. The virtual image V25 that has been subjected to the second image adjustment process is visually recognized to be more shifted from the target position MP1 than the virtual image V23 that has been subjected to the first image adjustment process. Specifically, in the backward tilt, the virtual image V25 subjected to the second image adjustment process is a real view that is further away from the observer with respect to the target position MP1 (110) than the virtual image V24 subjected to the first image adjustment process. It will be visually recognized as being shifted to a position 150 overlapping the (road surface 6). When the content FU is placed away from the viewer along the virtual plane 100 parallel to the road surface 6, the depression angle β of the content FU with respect to the virtual viewpoint VP becomes small. Therefore, the depression angle β15 in FIG. 8B is the depression angle resulting from shifting the depression angle β11 in FIG. The adjustment amount and the depression angle adjustment amount due to a change in the pitching angle α of the vehicle 1 may be corrected by the depression angle adjustment amount.

The processor 33 (image adjustment module 520) calculates the virtual image V0 (the source of the virtual image V0) from information indicating the display position of the image M before executing the position adjustment process (information indicating the reference position Po) included in the drawing data. Set the limit CT of the position adjustment amount of image M). Specifically, for example, the image adjustment module 520 sets a position adjustment range VT, and sets a limit CT of the position adjustment amount C based on the position adjustment range VT and the reference position Po.

FIG. 9 is a diagram for explaining the limit CT of the position adjustment amount. If the lower end of the position adjustment range VT is set close to the reference position Po of the virtual image V10, the limit CTd of the downward position adjustment amount of the virtual image V10 is set short. On the other hand, if the upper end of the position adjustment range VT is set far away from the reference position Po of the virtual image V10, the limit CTu of the upward position adjustment amount of the virtual image V10 is set longer. The processor 33 (image adjustment module 520) may set the position adjustment range VT individually for each of the plurality of virtual images V0 displayed in the virtual image display area VS, or may set it commonly for all the virtual images V0. The position adjustment range VT may be the virtual image display area VS (the entire virtual image display area VS may be set as the position adjustment range VT).

The image adjustment module 520 has table data (not shown) that associates the attitude change (pitch angle α) of the vehicle 1 with the position adjustment amount C of the virtual image V10, and based on the table data, the image adjustment module 520 adjusts the position of the virtual image V10. The limit (attitude threshold) of the backward pitching angle αTu that is assumed when the position adjustment amount C reaches the downward position adjustment amount limit CTd, and the position adjustment amount C of the virtual image V10 is the upward position adjustment amount A limit (posture threshold value) αTd of the pitching angle of forward tilt that is assumed to reach the limit CTu may be set.

When the virtual image V10 (virtual image V20) is shifted downward within the position adjustment range VT, the virtual image V10 (virtual image V20) undergoes attitude fluctuation until it reaches the limit CTda (CTdb in the case of V20) of the downward position adjustment amount. (first position adjustment process), but for larger posture changes, the position is adjusted at the downward position adjustment amount limit CTda (CTdb in case of V20) (lower peripheral edge VTd). (second position adjustment process). Conversely, when the virtual image V10 (virtual image V20) is shifted upward within the position adjustment range VT, the virtual image V10 (virtual image V20) is shifted until it reaches the limit CTua (CTub in case of V20) of the upward position adjustment amount. It shifts dynamically in response to posture changes (first position adjustment process), but for larger posture changes, the position adjusted at the upper position adjustment amount limit CTa (CTub in case of V20) (upper periphery) (second position adjustment process).

FIG. 10 is a diagram for explaining a plurality of position adjustment ranges VT set for each of a plurality of contents. The first virtual image V10 is based on a reference position Poa within the first position adjustment range VTa, and has a limit CTua of the amount of upward position adjustment up to the upper peripheral edge Vtua, and an amount of downward position adjustment up to the lower peripheral edge Vtda. A limit CTda is set. The second virtual image V20 is based on the reference position Pob within the second position adjustment range VTb, and has a limit CTub of the upward position adjustment amount up to the upper peripheral edge portion Vtub, and a downward position adjustment amount up to the lower peripheral edge portion Vtdb. A limit CTdb is set. If either the first position adjustment amount C10a of the first content FU1 or the first position adjustment amount C10b of the second content FU2 reaches the position adjustment amount limit CTu (or CTd), the content that has reached the limit In addition, the other content may also be switched from the first position adjustment process to the second position adjustment process at the same time.

FIG. 11 is a diagram illustrating image adjustment processing for large posture changes in a comparative example. From time t91 to t92, the pitching angle α is a forward tilt smaller than the preset posture threshold 902. The position adjustment amount 910 adjusts the virtual image display area VS in accordance with the posture change (forward tilt of the pitching angle 901) so as to strongly suppress (preferably cancel) the downward shift of the virtual image caused by the posture change. The first position adjustment amount 911 is set to dynamically correct the position of the virtual image in the upper direction. Thereby, the displacement of the virtual image is preferably canceled out (the position of the virtual image is maintained at the target position 960).

From time t92 to t93, the pitching angle 901 is a forward tilt greater than the preset posture threshold 902 (the first position adjustment amount 911 is greater than the preset upper limit 951 of the position adjustment amount). The position adjustment amount 910 is smaller than the first position adjustment amount 911 that strongly suppresses (preferably cancels out) the downward shift of the virtual image caused by posture fluctuations, and is The second position adjustment amount 912 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. In this case, due to attitude change (forward tilting of pitching angle 901), the virtual image shifts downward with respect to the target position 960 (in other words, due to forward tilting, the virtual image moves to a position where it overlaps with the foreground (road surface) on the side near the viewer). shift).

From time t93 to t94, the pitching angle 901 is a forward tilt smaller than the preset posture threshold 902 (the first position adjustment amount 911 is smaller than the preset upper limit 951 of the position adjustment amount). From time t94 to t95, the pitching angle 901 is a backward tilt that is smaller than a preset backward tilt posture threshold 903 (the first position adjustment amount 911 is smaller than the preset downward position adjustment limit 952). small). The position adjustment amount 910 is adjusted to adjust the posture change (forward tilt of the pitching angle 901) so as to strongly suppress (preferably cancel) the downward (upward in t94 to t95) shift of the virtual image caused by the posture change. (backward tilt from t94 to t95)) is set to a first position adjustment amount 911 that dynamically corrects the position of the virtual image in the virtual image display area VS upward (downward from t94 to t95). Thereby, the displacement of the virtual image is preferably canceled out (the position of the virtual image is maintained at the target position 960).

From time t95 to t96, the pitching angle 901 is a backward tilt that is greater than a preset backward tilt attitude threshold 903 (the first position adjustment amount 911 is greater than the preset downward position adjustment limit 952). big). The position adjustment amount 910 is smaller than the first position adjustment amount 911 that strongly suppresses (preferably cancels) the upward shift of the virtual image caused by the attitude change, and is smaller than the first position adjustment amount 911 that strongly suppresses (preferably cancels) the upward shift of the virtual image caused by the attitude change, and is The second position adjustment amount 912 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. In this case, due to the attitude change (backward tilting of pitching angle 901), the virtual image shifts upward with respect to the target position 960 (in other words, due to backward tilting, the virtual image moves to a position where it overlaps with the foreground (road surface) on the far side of the viewer). shift). The image processing from time t96 to t99 is the same as the image processing from time t91 to t92 and from t93 to t95, and a description thereof will be omitted.

In the comparative example, the reference position 931 is set at a position relatively close to the vertical center of the position adjustment range VT. Therefore, the position adjustment of the virtual image for forward tilt is allowed at a medium level from the reference position 931 to the upward limit 951, and the virtual image position adjustment for backward tilt is also allowed at a medium level from the reference position 931 to the downward limit 952. Permissible.

In this embodiment, when it is determined that the vehicle attitude change will increase, by offsetting the reference position upward, the virtual image is less likely to be cut off even when the position of the virtual image is corrected for the vehicle attitude change (backward tilt). can do. Furthermore, by offsetting the reference position downward, it is possible to make it difficult for the virtual image to be cut off even when the position of the virtual image is corrected for changes in the posture of the vehicle (forward tilting). For example, if a relatively small posture change that is unlikely to result in cut-off is predicted, the virtual image position adjustment for forward tilting is allowed at a moderate level from the reference position 931 to the upward limit 951, and the virtual image position adjustment for backward tilting is also allowed. , a medium range from the reference position 931 to the downward limit 952 is allowed. If a relatively large posture change that is likely to cause parting is predicted, for example, the position adjustment of the virtual image with respect to forward tilting is permitted to a large extent from the reference position 931 to the upper limit 951, and the position of the virtual image with respect to backward tilting is allowed. Adjustment is also permitted to a small extent from the reference position 931 to the downward limit 952.

FIG. 12 is a diagram illustrating the image adjustment process in the case where it is not determined that the change in the posture of the vehicle will become large in this embodiment. In the present embodiment, the processor 33 (image adjustment module 520) changes the reference position Po to an initial first reference position Po10 (corresponding to the first target position MP10) when it is not determined that the attitude change of the vehicle will be large. The position of the virtual image is adjusted based on the attitude change of the vehicle so that the positional deviation of the virtual image from the first target position MP10 is suppressed.

From time t11 to t12, the pitching angle α is a forward tilt smaller than the preset posture threshold αTd (the first position adjustment amount C10 is smaller than the preset upper limit CTu of the position adjustment amount). The position adjustment amount C adjusts the virtual image display area VS in accordance with the posture change (forward tilting of the pitching angle α) so as to strongly suppress (preferably cancel) the downward shift of the virtual image caused by the posture change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the upper direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the first target position MP10).

From time t12 to t13, the pitching angle α is a backward tilt smaller than the preset attitude threshold αTu (the first position adjustment amount C10 is smaller than the preset limit CTd of the downward position adjustment amount). The position adjustment amount C adjusts the virtual image display area VS in accordance with the attitude change (backward tilting of the pitching angle α) so as to strongly suppress (preferably cancel out) the upward shift of the virtual image caused by the attitude change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the lower part. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the first target position MP10). The image processing from time t13 to t14 is the same as the image processing from time t11 to t13, and a description thereof will be omitted.

FIG. 13 is a diagram illustrating image adjustment processing in the present embodiment when it is determined that the change in the posture of the vehicle becomes large. In the present embodiment, the processor 33 (image adjustment module 520) moves the reference position Po upward or downward from the initial first reference position Po10 (corresponding to the target position MP10) when it is determined that the attitude change of the vehicle becomes large. The setting is changed to the second reference position Po20 (corresponding to the second target position MP20) which is offset to Adjust the position of the virtual image so that it is suppressed.

From time t21 to t22, the pitching angle α is a forward tilt smaller than the preset posture threshold αTd (the first position adjustment amount C10 is smaller than the preset upper limit CTu of the position adjustment amount). The position adjustment amount C adjusts the virtual image display area VS in accordance with the posture change (forward tilting of the pitching angle α) so as to strongly suppress (preferably cancel) the downward shift of the virtual image caused by the posture change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the upper direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the first target position MP10).

From time t22 to t23, the pitching angle α is a backward tilt that is smaller than the preset backward tilt attitude threshold αTu (the first position adjustment amount C10 is smaller than the preset downward position adjustment limit CTd). small). The position adjustment amount C adjusts the virtual image display area VS in accordance with the attitude change (backward tilting of the pitching angle α) so as to strongly suppress (preferably cancel out) the upward shift of the virtual image caused by the attitude change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the downward direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the second target position MP20).

From time t23 to t24, the pitching angle α is a backward tilt that is larger than a preset backward tilt attitude threshold αTu (the first position adjustment amount C10 is smaller than the preset downward position adjustment limit CTd). big). The position adjustment amount C is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels out) the upward shift of the virtual image caused by the attitude change, and is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels) the upward shift of the virtual image caused by the attitude change, and is The second position adjustment amount C20 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. In this case, the virtual image shifts upward with respect to the second target position MP20 due to the attitude change (backward tilting of the pitching angle α) (in other words, the virtual image overlaps with the foreground (road surface) on the far side of the viewer due to the backward tilting). position). The image processing from time t24 to t25 is the same as the image processing from time t21 to t23, and a description thereof will be omitted.

According to this embodiment, when the attitude change of the vehicle is relatively small (or when the attitude change is predicted to be relatively small) and when the attitude change of the vehicle is relatively large (or when the attitude change is relatively large), (if predicted), it is possible to change the reference position that serves as a reference for correcting the position of the virtual image in accordance with the posture change. Therefore, when the attitude change of the vehicle is relatively large, by offsetting the reference position upward, it is possible to prevent the virtual image from being cut off even when the position of the virtual image is corrected for the attitude change (backward tilting) of the vehicle. Furthermore, by offsetting the reference position downward, it is possible to make it difficult for the virtual image to be cut off even when the position of the virtual image is corrected for changes in the posture of the vehicle (forward tilting). Note that when the posture change is relatively small, the position of the virtual image can be corrected for the relatively small posture change without offsetting the reference position. In other words, when the posture change is relatively small, the position of the virtual image can be corrected for the posture change while the virtual image (content) is placed in the virtual image display area with a high degree of freedom; when the posture change is relatively large, By offsetting the reference position, it is assumed that there is an advantage that the upper or lower part of the virtual image is less likely to be cut off.

In some embodiments, in the first image adjustment process, the processor 33 (image adjustment module 520) adjusts the position of the virtual image upward based on the reference position Po in accordance with the pitching angle α (an example of attitude change information). In the second image adjustment process, as shown in FIG. 14, the position of the virtual image is dynamically changed upward in accordance with the pitching angle α (an example of attitude change information) with reference to the reference position Po. (t31 to t32, t33 to t34 in FIG. 14), and do not change downward (t32 to t33, t34 to t35 in FIG. 14). In the present embodiment, in the second image adjustment process, the position of the virtual image is dynamically adjusted upward according to attitude change information, but not changed downward, thereby correcting the position of the virtual image with respect to the forward tilt of the vehicle. It is also envisaged that there are advantages in that it is possible to make it difficult for the upper side of the virtual image to be cut off, and to prevent the lower side of the virtual image from being cut off when the vehicle leans backward.

FIG. 15 is a diagram illustrating image adjustment processing when it is determined that the change in the posture of the vehicle will increase in this embodiment. In this embodiment, when a posture change (pitching angle α) larger than a predetermined threshold is detected, the processor 33 (image adjustment module 520) determines that the posture change of the vehicle becomes large, and changes the reference position Po to the initial position. The setting is changed from the first reference position Po10 (corresponding to target position MP10) to a second reference position Po20 (corresponding to second target position MP20) offset upward or downward (in the example of FIG. 15, offset downward), and the vehicle Based on the attitude change, the position of the virtual image is adjusted so that the positional deviation of the virtual image from the target position MP20 is suppressed.

From time t41 to t42, the pitching angle α is a forward tilt smaller than the preset posture threshold αTs (the first position adjustment amount C10 is smaller than the preset upward position adjustment amount threshold CTs). The position adjustment amount C adjusts the virtual image display area VS in accordance with the posture change (forward tilting of the pitching angle α) so as to strongly suppress (preferably cancel) the downward shift of the virtual image caused by the posture change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the upper direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the first target position MP10).

At time t42, the pitching angle α is a forward tilt that reaches a preset posture threshold αTs (the first position adjustment amount C10 reaches a preset upward position adjustment amount threshold CTs). When a posture change (pitching angle α) larger than a predetermined threshold αTs is detected, the processor 33 (image adjustment module 520) determines that the posture change of the vehicle becomes large, and changes the reference position Po to the initial first reference position. The setting is changed from Po10 (corresponding to target position MP10) to a second reference position Po20 (corresponding to second target position MP20) offset upward or downward (in the example of FIG. 15, offset downward) to compensate for changes in vehicle attitude. Based on this, the position of the virtual image is adjusted so that the positional deviation of the virtual image from the target position MP20 is suppressed.

From time t42 to t43, the pitching angle α is a backward tilt that is larger than the preset backward tilt attitude threshold αTu, but the first position adjustment amount C10 based on the second reference position Po20 is It is smaller than the limit CTd of the downward position adjustment amount. The position adjustment amount C adjusts the virtual image display area VS in accordance with the attitude change (backward tilting of the pitching angle α) so as to strongly suppress (preferably cancel out) the upward shift of the virtual image caused by the attitude change. The first position adjustment amount C10 is set to dynamically correct the position of the virtual image in the downward direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the second target position MP20).

From time t44 to t45, the pitching angle α is a backward tilt that is larger than the preset backward tilt attitude threshold αTu (the first position adjustment amount C10 based on the second reference position Po20 is also direction position adjustment amount limit CTd). The position adjustment amount C is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels out) the upward shift of the virtual image caused by the attitude change, and is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels) the upward shift of the virtual image caused by the attitude change, and is The second position adjustment amount C20 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. In this case, the virtual image shifts upward with respect to the second target position MP20 due to the attitude change (backward tilting of the pitching angle α) (in other words, the virtual image overlaps with the foreground (road surface) on the far side of the viewer due to the backward tilting). position).

The image adjustment process shown in FIG. 16 is performed by the processor 33 executing the image adjustment module 520 stored in the memory 37. In step S110, the image adjustment module 520 acquires the drawing data generated by the drawing module 510. In step S120, the image adjustment module 520 extracts the virtual image V20 (the original of the virtual image V20) from the information indicating the display position of the image M before executing the image adjustment process (information indicating the reference position Po) included in the acquired drawing data. Set the limit CT of the position adjustment amount of image M). Specifically, for example, the image adjustment module 520 sets a position adjustment range VT, and sets a limit CT of the position adjustment amount C based on the position adjustment range VT and the reference position Po.

Next, in step S130, the image adjustment module 520 acquires information indicating the attitude change of the vehicle 1 (attitude change information) from the attitude detection unit 415. Posture detection unit 415 includes one or more sensors such as a gyro sensor, an acceleration sensor, and a height sensor. The attitude detection unit 415 calculates the vehicle attitude such as pitching angle and roll angle, the frequency of change in the vehicle attitude, etc. as attitude change information from the sensor values such as the angular velocity, acceleration, and height of the moving object, and displays the information in the display control device. It may be output to 30. That is, the attitude change information may include a frequency of change in the vehicle attitude (vibration frequency), etc. in addition to the vehicle attitude (pitch angle, roll angle, etc.). Part or all of the function for the posture detection unit 415 to calculate the posture change information may be provided in the display control device 30.

In step S140, the image adjustment module 520 calculates a first position adjustment amount C10 used in a first image adjustment process S151, which will be described later. First, the image adjustment module 520 calculates the amount of attitude variation (angular deviation amount) of the vehicle 1 based on the attitude variation information acquired from the attitude detection unit 415. For example, the image adjustment module 520 calculates the angle (pitch angle) α of the vehicle 1 around the pitch axis by performing an integral calculation on the angular velocity detected by the attitude detection unit 415. Thereby, the amount of deviation (angle) of the vehicle 1 in the rotational direction about the Y axis (the pitch axis) shown in FIG. 1 can be calculated. Note that in this embodiment, the pitching angle is calculated, but the yaw angle or the roll angle may also be calculated. For example, all angles around the X, Y, and Z axes may be calculated. However, part or all of the function of calculating the amount of posture variation (angular deviation amount) in the image adjustment module 520 is provided by a device other than the display control device 30 that can communicate with the display control device 30, and The device 30 may receive information indicating the amount of posture variation (angular deviation amount) of the vehicle 1 from the other device via the I/O interface 31. That is, some display control devices 30 may omit the function of calculating the posture variation amount (angular shift amount) in the image adjustment module 520.

In step S140, the image adjustment module 520 further calculates a first position adjustment amount C10 for correcting the display position of the virtual image V20, based on the attitude variation amount (angular deviation amount) of the vehicle 1. Specifically, the image adjustment module 520 converts the amount of deviation (pitching angle) into the number of pixels, and adjusts the amount of adjustment to restore the number of pixels corresponding to the deviation (image shift amount B10 due to posture fluctuation). Determine. Preferably, the image adjustment module 520 adjusts a position adjustment amount (first position adjustment amount C10) in the opposite direction equal to the image shift amount B10 due to the attitude change in order to restore the positional shift of the virtual image V20 due to the attitude change of the vehicle 1. calculate.

In step S150 of some embodiments, the image adjustment module 520 generates image data by performing image adjustments on the drawing data obtained in step S110. In S150, the image adjustment module 520 rearranges the pixels of the drawing data as pixels of the image data based on the position adjustment amount C, the depression angle adjustment amount E, and the size adjustment amount F. In S160, the image adjustment module 520 outputs the image data generated (adjusted) in S150 to the display device 50.

In step S150 of some embodiments, the image adjustment module 520 determines whether or not the following predetermined conditions are satisfied, and if the predetermined conditions are not satisfied, the image adjustment module 520 executes the first image adjustment process S151, and If the predetermined conditions are met, second image adjustment processing S152 is executed. However, the predetermined conditions for determining whether to execute the first image adjustment process S151 or the second image adjustment process S152 are not limited to the following. (1) The amount of posture variation acquired in step S130 is larger than the predetermined threshold αTs. (2) The speed of the amount of posture variation (speed of change in posture) acquired in step S130 is faster than a predetermined threshold. (3) The speed of the vehicle 1 is faster than a predetermined threshold.

When the predetermined condition is satisfied, the processor 33 (image adjustment module 520) sets the reference position Po to the initial first reference position Po10 (corresponding to the first target position MP10), and sets the reference position Po to the first reference position Po10 (corresponding to the first target position MP10) based on the attitude change of the vehicle. The position of the virtual image is adjusted so that the positional deviation of the virtual image from the first target position MP10 is suppressed (first image adjustment process S151).

In the first image adjustment process S151, if the pitching angle α is smaller than the preset posture threshold αT (the first position adjustment amount C10 is smaller than the preset limit CT of the position adjustment amount), the processor 33 performs the step The initial first reference position Po10 (corresponding to the first target position MP10) is determined based on the first position adjustment amount C10 (position adjustment amount C) that is dynamically set in step S140 in accordance with the attitude change acquired in S130. Using this as a reference, the position of the virtual image V20 (the image M that is the source of the virtual image V20) is adjusted (first position adjustment process). Preferably, the first position adjustment amount C10 is set so as to offset the positional shift of the image caused by the posture change.

On the other hand, in the first image adjustment process S151, when the pitching angle α is smaller than the preset attitude threshold αT (the first position adjustment amount C10 is smaller than the preset position adjustment amount limit CT) , for larger posture fluctuations, the display is fixed at the position (upper periphery VTu or lower periphery VTd) adjusted by the limit CT of the downward position adjustment amount (CT in case of V20) (second position). adjustment process).

If the predetermined condition is not satisfied, the processor 33 (image adjustment module 520) sets the reference position Po to a second reference position Po20 (which is offset upward or downward from the initial first reference position Po10 (corresponding to the target position MP10)). (corresponding to the second target position MP20), and adjust the position of the virtual image (corresponding to the second image Adjustment processing S152).

In the second image adjustment process S152, if the pitching angle α is smaller than the preset posture threshold αT (the first position adjustment amount C10 is smaller than the preset limit CT of the position adjustment amount), the processor 33 performs the step Based on the first position adjustment amount C10 (position adjustment amount C) dynamically set in step S140 in accordance with the attitude change acquired in S130, the second reference position Po20 (corresponding to the second target position MP20) is set as a reference. Then, the position of the virtual image V20 (the image M that is the source of the virtual image V20) is adjusted (first position adjustment process). Preferably, the first position adjustment amount C10 is set so as to offset the positional shift of the image caused by the posture change.

On the other hand, in the second image adjustment process S152, if the pitching angle α is smaller than the preset attitude threshold αT (the first position adjustment amount C10 is smaller than the preset limit CT of the position adjustment amount), , for larger posture fluctuations, the display is fixed at the position (upper periphery VTu or lower periphery VTd) adjusted by the limit CT of the downward position adjustment amount (CT in case of V20) (second position). adjustment process).

In the first position adjustment process in some embodiments, in addition to the position adjustment, the image adjustment module 520 adjusts the first depression angle adjustment amount E10 (depression angle adjustment amount E) in accordance with the attitude change acquired in step S130. may be dynamically changed, and the depression angle of the virtual image V20 (the image M that is the source of the virtual image V20) may be adjusted based on the first depression angle adjustment amount E10 (the depression angle adjustment amount E).

In the first position adjustment process in some embodiments, the first position adjustment amount C10 is not set to offset the image position shift caused by the attitude change (in other words, if the image position shift caused by the attitude change is canceled out), the first position adjustment amount C10 is In addition to the position adjustment, the image adjustment module 520 dynamically changes the size adjustment amount F in accordance with the posture change acquired in step S130, and based on this size adjustment amount F, the image adjustment module 520 adjusts the virtual image V20 ( The size of the image M) that is the source of the virtual image V20 may be adjusted.

In the second position adjustment process in some embodiments, if the first position adjustment amount C10 calculated in step S140 is larger than the threshold (limit) CT of the movement adjustment amount set in step S120, the image adjustment module 520 dynamically changes the second depression angle adjustment amount 210 (depression angle adjustment amount E) in accordance with the attitude change acquired in step S130, and based on this second depression angle adjustment amount E20 (depression angle adjustment amount E), the virtual image is The depression angle of V20 (the image M that is the source of the virtual image V20) is adjusted. The depression angle adjustment amount E20 in the second position adjustment process is a depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image that occurs due to posture fluctuations (changes in the pitching angle α). Preferably, the depression angle adjustment amount E20 in the second position adjustment process is a depression angle adjustment amount that corrects a depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude variation (change in pitching angle α). An angle of depression adjustment amount may be added to express a shift to the vicinity (or a shift to a far distance) due to a change in the angle α).

Furthermore, in the second position adjustment process in some embodiments, the image adjustment module 520 dynamically changes the second position adjustment amount C20 (position adjustment amount C) in accordance with the posture change acquired in step S130. You can. The image adjustment module 520 may include table data, arithmetic expressions, etc. for setting the second position adjustment amount C20 (position adjustment amount C) from the attitude change acquired in step S130. In the second position adjustment process in some embodiments, in addition to the depression angle adjustment, the image adjustment module 520 also adjusts the second position adjustment amount C20 (second position adjustment amount) that is dynamically set according to the attitude change acquired in step S130. The position of the virtual image V20 (the image M that is the source of the virtual image V20) may be adjusted based on the 1-position adjustment amount C10).

In the position adjustment process in the second position adjustment process, the image adjustment module 520 calculates a second position adjustment amount C20 that is dynamically changed based on the amount of posture variation of the vehicle 1. For example, the second position adjustment amount C20 is obtained by multiplying the first position adjustment amount C10, which is determined based on the amount of posture variation of the vehicle 1, by a coefficient smaller than 1. Further, the position adjustment module 522 of some embodiments may read out the second position adjustment amount C20 stored in the memory 37, which is not based on the amount of posture variation of the vehicle 1. In this embodiment, the adjustment amount in the pitch axis direction is calculated, but the adjustment amount in the yaw axis direction and the roll direction may also be calculated. Regarding the roll angle, an adjustment amount is determined so as to return the deviation amount of the roll angle to the original value while leaving the angle as it is. However, part or all of the function of calculating the second position adjustment amount C20 of the image adjustment module 520 is provided by a device other than the display control device 30 that can communicate with the display control device 30, and the display control device 30 is , a display parameter (second position adjustment amount C20) for adjusting the position of the virtual image may be input from the other device via the I/O interface 31.

When a predetermined condition is satisfied, the display control device 30 (processor 33) of the present embodiment executes the second image adjustment process (step S152) to adjust the virtual image according to the deviation of the virtual image due to the attitude change of the moving object. Adjust the depression angle β of V20. An example of calculating the depression angle β of the virtual image V20 according to the attitude change (angular deviation amount) of the vehicle 1 using an arithmetic expression is shown below. The position and angle of the image M, which is the source of the virtual image V20, are changed by changing the position and angle of the viewpoint VP according to changes in the posture of the vehicle 1, and by rendering according to the content (virtual object) FU seen from the virtual viewpoint VP. (depression angle β) and size may be changed.

The image adjustment module 520 (depression angle adjustment module 524) calculates the depression angle β of the virtual image V20 (content FU) according to the attitude change (angular deviation amount) of the vehicle 1.

First, as shown in FIGS. 6 to 8B, the angle (pitching angle) around the pitch axis of the vehicle 1 is set to α (a negative value when tilting forward and a positive value when tilting backward), and from the virtual viewpoint VP to the virtual plane. 100 (the height h when the angular deviation amount α is zero is h0), and the depth direction from the virtual viewpoint VP to the near end of the area on the virtual plane 100 where the virtual image V20 appears to overlap. The distance Z is defined as D (the distance D when the angular deviation amount α is zero is D0).

The image adjustment module 520 (depression angle adjustment module 524) may calculate the distance D according to the pitching angle α using Equation 1 below (but is not limited to this).

Further, the image adjustment module 520 (depression angle adjustment module 524) may calculate the depression angle β according to the distance D using Equation 2 below (but is not limited to this).

Note that the image adjustment module 520 (depression angle adjustment module 524) is not limited to the above calculation method as long as it can adjust the depression angle β of the virtual image V20 according to the attitude (angular deviation amount) of the vehicle 1. For example, the image adjustment module 520 (depression angle adjustment module 524) stores in the memory 37 in advance table data that associates the correction coefficient of the depression angle β of the virtual image V20 with the posture fluctuation (angular deviation amount) of the vehicle 1, The correction coefficient may be read out based on the input information (signal) indicating the attitude change (angular deviation amount) of the vehicle 1, and the depression angle β of the content FU1 (virtual image V20) may be adjusted.

Furthermore, the image adjustment module 520 (depression angle adjustment module 524) may calculate the depression angle adjustment amount Δβ based on the distance D to the content (virtual object) FU on the virtual plane 100 using the following formula 3 (this (but not limited to).

More preferably, the image adjustment module 520 (depression angle adjustment module 524) calculates the depression angle adjustment amount Δβ using the following formula 4, taking into account that the tilt angle θt of the virtual image display area VS is tilted by the angular deviation amount α. (but not limited to)

Further, the display control device 30 (processor 33) of some embodiments adjusts the size of the virtual image V20 according to a shift in the virtual image due to a change in the posture of the moving object. An example of calculating the size of the virtual image V20 according to the attitude change (angular deviation amount) of the vehicle 1 using an arithmetic expression is shown below. By changing the position and angle of VP according to changes in the posture of the vehicle 1, by rendering according to the content (virtual object) FU seen from the virtual viewpoint VP, the position of the image M that is the source of the virtual image V20, The angle (depression angle β) and size may be changed.

The image adjustment module 520 (size adjustment module 526) calculates the size of the virtual image V20 (content FU) according to the attitude change (angular deviation amount) of the vehicle 1. Here, the size of the virtual image V20 is assumed to be the vertical angle (angle of view) of the virtual image V20 viewed from the reference position. The angle of view is the angle between a line connecting the upper end of the virtual image V20 from a predetermined reference position (for example, the center 205 of the eyebox 200) and a line connecting the lower end of the virtual image V20 from the predetermined reference position, It corresponds to the size of the virtual image V20 in the vertical direction (Y-axis direction) as seen from the observer.

The image adjustment module 520 (size adjustment module 526) sets the size according to the distance D, where L is the length in the depth direction Z of the area on the virtual plane 100 where the virtual images V20 appear to overlap from the virtual viewpoint VP1. It may be calculated using Formula 5 (but is not limited to this).

Note that the image adjustment module 520 (size adjustment module 526) is not limited to the calculation method described above, as long as it can adjust the size of the virtual image V20 according to the attitude (angular deviation amount) of the vehicle 1. For example, the image adjustment module 520 (size adjustment module 526) stores table data in advance in the memory 37 in which the correction coefficient for the size of the virtual image V20 is associated with the attitude (angular deviation amount) of the vehicle 1, and inputs the The correction coefficient may be read out based on information (signal) indicating the attitude (angular deviation amount) of the vehicle 1, and the size of the virtual image V20 may be adjusted. However, a part or all of the functions of the image adjustment module 520 (size adjustment module 526) are provided in a separate device from the display control device 30 that can communicate with the display control device 30, and the display control device 30 is Display parameters for adjusting the size of the virtual image may be input from the device via the I/O interface 31. That is, some display control devices 30 may omit the image adjustment module 520 (size adjustment module 526).

The head-up display device 20 described in this specification includes one of the display control devices 30 in some embodiments, a light modulation element 51 that emits display light, and a light modulation element 51 that receives the display light from the light modulation element 51. A relay optical system 80 facing the projection section 2 is provided. In this case as well, advantages similar to those described above are envisaged.

The operations of the processing processes described above may be performed by executing one or more functional modules of an information processing device, such as a general-purpose processor or an application-specific chip. All these modules, combinations of these modules, and/or combinations with known hardware that can replace their functions fall within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. It is noted that the functional blocks described in FIG. 3 may optionally be combined or one functional block separated into two or more sub-blocks to implement the principles of the described embodiments. It will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

In step S150 of some embodiments, the image adjustment module 520 determines whether or not the following predetermined conditions are satisfied, and if the predetermined conditions are not satisfied, the image adjustment module 520 executes the first image adjustment process S151, and If the predetermined conditions are met, a second image adjustment process S152 may be executed. (4) There is a posture change in which the amount of posture change obtained in step S130 has a predetermined frequency. (5) There is a brake operation obtained from the vehicle ECU 401. (6) The amount of depression of the brake pedal is greater than a predetermined threshold. However, the predetermined conditions for determining whether to execute the first image adjustment process S151 or the second image adjustment process S152 are not limited to these.

1 : Vehicle 2 : Projected part 6 : Road surface 10 : Vehicle display system 20 : Head-up display device 21 : Light exit window 22 : Housing 30 : Display control device 31 : I/O interface 33 : Processor 35 : Display control Processing circuit 37: Memory 40: Image display device 50: Display device 50a: Display surface 51: Light modulation element 60: Light source unit 80: Relay optical system 100: Virtual plane 200: Eye box 401: Vehicle ECU
403: Road information database 405: Vehicle position detection unit 407: Operation detection unit 409: Eye position detection unit 411: External sensor 413: Brightness detection unit 415: Attitude detection unit 417: Mobile information terminal 419: External communication device 510: Drawing module 520 : Image adjustment module 522 : Position adjustment module 524 : Depression angle adjustment module 526 : Size adjustment module 700 : Observer AT10 : Vehicle posture B10 : Image shift amount C : Position adjustment amount C10 : First position adjustment amount C20 : First 2-position adjustment amount CT: Limit CTs: Threshold CTu: Limit D: Distance E: Depression angle adjustment amount F: Size adjustment amount FU: Content FU1: First content FU2: Second content Ld: Shiftable area Lu: Shiftable area MP : Target position MP1 : Target position MP20 : Second target position Po : Reference position Po10 : First reference position Po20 : Second reference position Poa : Reference position Pob : Reference position VP : Virtual viewpoint VTa : First position adjustment range VTb : Second position adjustment range VTd: Lower peripheral edge VTu: Upper peripheral edge α: Pitching angle αT: Attitude threshold β: Depression angle θt: Tilt angle

Claims (8)

A head-up device is mounted on a vehicle and has a display unit that displays an image on a display surface, and by directing the light of the image toward the projected area, a virtual image is superimposed and visible in a virtual image display area that overlaps with the foreground of the vehicle. A display control device that controls a display device,
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors;
The processor includes:
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first image adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. performing an image adjustment process including a second image adjustment process of dynamically changing the position of the virtual image according to the posture change information as a reference;
A display control device characterized by: In the second image adjustment process, the processor:
setting the second reference position lower than the first reference position when viewed from the observer;
The display control device according to claim 1, characterized in that: The processor includes:
In the first image adjustment process, dynamically changing the position of the virtual image upward and downward in accordance with the attitude change information with respect to the reference position,
In the second image adjustment process, based on the reference position, dynamically changing the position of the virtual image upward in accordance with the attitude change information, but not changing it downward;
The display control device according to claim 2, characterized in that: The processor includes:
The larger the attitude change of the vehicle that is detected or predicted based on the attitude change information or the attitude change prediction information,
setting a large offset amount of the second reference position from the first reference position;
The display control device according to claim 1, characterized in that: The virtual image includes at least a first virtual image and a second virtual image displayed below the first virtual image,
When the processor determines that the attitude change of the vehicle will increase based on the attitude change information or the attitude change prediction information,
performing the second image adjustment process on the first virtual image;
performing the first image adjustment process on the second virtual image;
The display control device according to claim 1, characterized in that: When the processor detects that the amplitude of the posture fluctuation of the vehicle has become equal to or less than a predetermined value, the processor gradually changes the reference position when returning the second reference position to the first reference position.
The display control device according to claim 1, characterized in that: a display unit installed in a vehicle and displaying an image on a display surface;
a relay optical system that directs display light from the display section to a projected section;
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors, the virtual image being superimposed and viewed in a virtual image display area overlapping with the foreground of the vehicle. A head-up display device,
The processor includes:
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first image adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. performing an image adjustment process including a second image adjustment process of dynamically changing the position of the virtual image according to the posture change information as a reference;
A head-up display device characterized by: A head-up device is mounted on a vehicle and has a display unit that displays an image on a display surface, and by directing the light of the image toward the projected area, a virtual image is superimposed and visible in a virtual image display area that overlaps with the foreground of the vehicle. A display control method for controlling a display device, the method comprising:
acquiring attitude change information indicating attitude change of the vehicle;
1) A predetermined first reference position is set, and in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle, the attitude change information is set based on the first reference position. a first image adjustment process that dynamically changes the position of the virtual image;
2) If it is determined that the attitude change of the vehicle will increase based on the attitude change information or attitude change prediction information that predicts that the attitude change of the vehicle will increase, the reference position of the virtual image is changed to the reference position of the virtual image. A second reference position offset at least upward or downward from the first reference position is set, and the second reference position is set in order to suppress a relative positional shift between the virtual image and the foreground due to a change in the attitude of the vehicle. a second image adjustment process of dynamically changing the position of the virtual image according to the posture change information as a reference;
A display control method characterized by:

Images