KR102457559B1 - Electronic device and method for correcting image based on object included image - Google Patents

Abstract

The present invention relates to an electronic device and method for correcting an image based on an object included in an image. According to various embodiments of the present disclosure, the electronic device includes a display and a processor, the processor comprising: displaying an image on the display, detecting at least one or more straight lines satisfying a specified length based at least on an object included in the image, determining at least one of the at least one or more straight lines as a reference line, and the reference line is It may be set to correct the image so as to be parallel to at least one of a horizontal axis and a vertical axis of the image.
Various other embodiments are possible other than the various embodiments disclosed herein.

Description

ELECTRONIC DEVICE AND METHOD FOR CORRECTING IMAGE BASED ON OBJECT INCLUDED IMAGE

Various embodiments of the present disclosure relate to an electronic device and method for correcting an image based on an object included in an image.

In general, an electronic device provides a camera function to generate and store an image or a moving picture by photographing a subject. A tilted image may exist among the images acquired through the camera function. In this case, the electronic device provides an editing function for correcting a tilted image. The user can correct the image with a desired angle (eg, composition) by adjusting the angle of the image through the editing function for correcting the tilted image.

The editing function to correct skewed images allows the detailed operation of rotating the angle of an image by 90 degrees or adjusting it to a fine angle (eg 1°), but manually adjusting the angle of the image after selecting it It can be inconvenient for users who have to do it.

The electronic device according to various embodiments of the present disclosure may provide an image corrected so that a reference line automatically determined based on a preset condition is parallel to at least one of a horizontal central axis and a vertical central axis of the image.

The electronic device according to various embodiments of the present disclosure may provide at least one or more straight lines detected based on the included object. The at least one or more straight lines are lines for adjusting the angle of the image, and may be selectable by the user. The electronic device may determine at least one straight line selected by the user from among the one or more straight lines as a reference line, and provide an image corrected so that the determined reference line is parallel to at least one of a horizontal central axis or a vertical central axis of the image.

According to various embodiments of the present disclosure, an electronic device includes a display and a processor, wherein the processor displays an image on the display and at least meets a specified length based at least on an object included in the image. It may be configured to detect one or more straight lines, determine at least one of the at least one or more straight lines as a reference line, and correct the image so that the reference line is parallel to at least one of a horizontal axis or a vertical axis of the image.

According to various embodiments of the present disclosure, a method of correcting an image based on an object included in an image of an electronic device satisfies a specified length based on at least an operation of displaying the image on a display and an object included in the image detecting at least one or more straight lines, determining at least one of the at least one or more straight lines as a reference line, and correcting the image so that the reference line is parallel to at least one of a horizontal axis or a vertical axis of the image can do.

According to various embodiments of the present invention, in a computer-readable recording medium recording a program for executing a method for correcting an image based on an object included in the image in a computer, the method includes: displaying, detecting at least one or more straight lines satisfying a specified length based at least on an object included in the image, determining at least one of the at least one or more straight lines as a reference line, and the reference line is the image and correcting the image to be parallel to at least one of a horizontal axis and a vertical axis.

The electronic device according to various embodiments of the present disclosure may provide an image corrected according to a reference line determined based on a preset condition. Accordingly, the user may be provided with an image suitable for a horizontal or vertical composition easily and accurately through a reference line determined based on a preset condition rather than manual adjustment.

According to various embodiments of the present disclosure, the electronic device may provide the user with one or more straight lines for adjusting the composition of the image based on the object included in the image. Accordingly, the user can easily edit the tilted image into an image having a desired composition.

1 is a block diagram of an electronic device for correcting an image based on an object included in an image in a network environment according to various embodiments of the present disclosure;
2 is a block diagram of a camera module according to various embodiments.
3 is a block diagram illustrating an electronic device for correcting an image based on an object included in an image according to various embodiments of the present disclosure;
4 is a flowchart illustrating a method of correcting an image based on a reference line determined by an electronic device according to various embodiments of the present disclosure;
5A and 5B are diagrams for explaining a method of correcting an image according to various embodiments.
6 is a diagram for describing a method of correcting an image based on a reference line determined by an electronic device according to various embodiments of the present disclosure;
7 is a flowchart illustrating a method of correcting an image based on a reference line selected by a user according to various embodiments of the present disclosure;
8 is a diagram for describing a method of correcting an image based on a reference line selected by a user according to various embodiments of the present disclosure;
9 is a flowchart illustrating a method of correcting an image based on a reference line selected from a preview image obtained from a camera module according to various embodiments of the present disclosure;
10 is a diagram for describing a method of correcting an image based on a reference line selected from a preview image obtained from a camera module according to various embodiments of the present disclosure;

1 is a block diagram of an electronic device 101 for correcting an image based on an object included in an image in the network environment 100 according to various embodiments of the present disclosure.

Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with the electronic device 102 through a first network 198 (eg, short-range wireless communication) or a second network 199 ( For example, it may communicate with the electronic device 104 or the server 108 through long-distance wireless communication. According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input device 150 , a sound output device 155 , a display device 160 , an audio module 170 , and a sensor module ( 176 , interface 177 , haptic module 179 , camera module 180 , power management module 188 , battery 189 , communication module 190 , subscriber identification module 196 , and antenna module 197 . ) may be included. In some embodiments, at least one of these components (eg, the display device 160 or the camera module 180 ) may be omitted or another component may be added to the electronic device 101 . In some embodiments, for example, as in the case of a sensor module 176 (eg, a fingerprint sensor, iris sensor, or illuminance sensor) embedded in a display device 160 (eg, a display), some components It can be integrated and implemented.

The processor 120, for example, runs software (eg, the program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing and operations. The processor 120 loads and processes commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) into the volatile memory 132 , and stores the result data in the non-volatile memory 134 . can be stored in According to an embodiment, the processor 120 is operated independently of the main processor 121 (eg, central processing unit or application processor), and additionally or alternatively, uses less power than the main processor 121, Alternatively, the auxiliary processor 123 (eg, a graphic processing unit, an image signal processor, a sensor hub processor, or a communication processor) specialized for a specified function may be included. Here, the auxiliary processor 123 may be operated separately from or embedded in the main processor 121 .

In this case, the auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, in a sleep) state. : While in the application execution) state, together with the main processor 121 , at least one of the components of the electronic device 101 (eg, the display device 160 , the sensor module 176 , or the communication module ( 190)) and related functions or states. According to one embodiment, the coprocessor 123 (eg, image signal processor or communication processor) is implemented as some component of another functionally related component (eg, camera module 180 or communication module 190). can be The memory 130 includes various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 , for example, software (eg, the program 140 ). ) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 is software stored in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input device 150 is a device for receiving commands or data to be used by a component (eg, the processor 120) of the electronic device 101 from the outside (eg, a user) of the electronic device 101, for example, For example, it may include a microphone, mouse, or keyboard.

The sound output device 155 is a device for outputting a sound signal to the outside of the electronic device 101, and includes, for example, a speaker used for general purposes such as multimedia playback or recording playback, and a receiver used exclusively for receiving calls. may include According to an embodiment, the receiver may be formed integrally with or separately from the speaker.

The display device 160 is a device for visually providing information to the user of the electronic device 101 , and may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment, the display device 160 may include a touch circuitry or a pressure sensor capable of measuring the intensity of the pressure applied to the touch.

The audio module 170 may interactively convert a sound and an electrical signal. According to an embodiment, the audio module 170 acquires a sound through the input device 150 , or an external electronic device (eg, a sound output device 155 ) connected to the electronic device 101 by wire or wirelessly. Sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 may generate an electrical signal or data value corresponding to an internal operating state (eg, power or temperature) of the electronic device 101 or an external environmental state. The sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, Alternatively, it may include an illuminance sensor.

The interface 177 may support a designated protocol capable of connecting to an external electronic device (eg, the electronic device 102 ) in a wired or wireless manner. According to an embodiment, the interface 177 may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 is a connector capable of physically connecting the electronic device 101 and an external electronic device (eg, the electronic device 102 ), for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector. (eg a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. The haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

The power management module 188 is a module for managing power supplied to the electronic device 101 , and may be configured as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 is a device for supplying power to at least one component of the electronic device 101 , and may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module 190 establishes a wired or wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108), and establishes the established communication channel. It can support performing communication through The communication module 190 may include one or more communication processors that support wired communication or wireless communication, which are operated independently of the processor 120 (eg, an application processor). According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : LAN (local area network) communication module, or power line communication module), and using the corresponding communication module among them, the first network 198 (eg, Bluetooth, WiFi direct, or infrared data association (IrDA)) communication network) or the second network 199 (eg, a cellular network, the Internet, or a telecommunication network such as a computer network (eg, LAN or WAN)). The above-described various types of communication modules 190 may be implemented as a single chip or as separate chips.

According to an embodiment, the wireless communication module 192 may use the user information stored in the subscriber identification module 196 to distinguish and authenticate the electronic device 101 in the communication network.

The antenna module 197 may include one or more antennas for externally transmitting or receiving a signal or power. According to an example, the communication module 190 (eg, the wireless communication module 192 ) may transmit a signal to or receive a signal from the external electronic device through an antenna suitable for a communication method.

Some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) to signal (eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the electronic devices 102 and 104 may be the same or a different type of the electronic device 101 . According to an embodiment, all or some of the operations performed by the electronic device 101 may be performed by one or a plurality of external electronic devices. According to an embodiment, when the electronic device 101 is to perform a function or service automatically or upon request, the electronic device 101 performs the function or service by itself instead of or in addition to it. At least some related functions may be requested from the external electronic device. Upon receiving the request, the external electronic device may execute the requested function or additional function, and may transmit the result to the electronic device 101 . The electronic device 101 may provide the requested function or service by processing the received result as it is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

2 is a block diagram 200 of a camera module 180, according to various embodiments.

Referring to FIG. 2 , the camera module 180 includes a lens assembly 210 , a flash 220 , an image sensor 230 , an image stabilizer 240 , a memory 250 (eg, a buffer memory), or an image signal processor. (260). The lens assembly 210 may collect light emitted from a subject, which is an image to be captured. The lens assembly 210 may include one or more lenses. According to an embodiment, the camera module 180 may include a plurality of lens assemblies 210 . In this case, the camera module 180 may be, for example, a dual camera, a 360 degree camera, or a spherical camera. The plurality of lens assemblies 210 may have the same lens properties (eg, angle of view, focal length, auto focus, f number, or optical zoom), or at least one lens assembly may have at least one lens assembly and at least one lens assembly with another lens assembly. may have different lens properties of The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens. The flash 220 may emit a light source used to enhance light emitted from the subject. The flash 220 may include one or more light emitting diodes (eg, a red-green-blue (RGB) LED, a white LED, an infrared LED, or an ultraviolet LED), or a xenon lamp.

The image sensor 230 may obtain an image corresponding to the subject by converting light transmitted from the subject through the lens assembly 210 into an electrical signal. According to an embodiment, the image sensor 230 is, for example, one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, the same It may include a plurality of image sensors having properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 230 may be implemented as, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The image stabilizer 240 responds to the movement of the camera module 180 or the electronic device 101 including the same, and at least partially compensates for a negative effect (eg, image shake) caused by the movement on a photographed image. At least one lens or image sensor 230 included in the assembly 210 may be moved or controlled in a specific direction (eg, read-out timing is adjusted, etc.). According to an embodiment, the image stabilizer 240 may be implemented as, for example, an optical image stabilizer, and a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 180 . ) can be used to detect the movement.

The memory 250 may temporarily store at least a portion of the image acquired through the image sensor 230 for a next image processing operation. For example, when image acquisition is delayed according to the shutter or when a plurality of images are acquired at high speed, the acquired original image (eg, high-resolution image) is stored in the memory 250 , and a corresponding copy thereof is stored in the memory 250 . An image (eg, a low-resolution image) may be previewed through the display device 160 . Thereafter, when a specified condition is satisfied (eg, a user input or a system command), at least a portion of the original image stored in the memory 250 may be obtained and processed by, for example, the image signal processor 260 . According to an embodiment, the memory 250 may be configured as at least a part of the memory 130 or as a separate memory operated independently of the memory 130 .

The image signal processor 260 performs image processing (eg, depth map generation, 3D modeling, panorama generation, feature point extraction, Image compositing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening) may be performed. Additionally or alternatively, an image signal processor ( The 260 may perform control (eg, exposure time control, readout timing control, etc.) on at least one (eg, image sensor 230) among components included in the camera module 180. Image The image processed by the signal processor 260 is stored back in the memory 250 for further processing, or an external component of the camera module 180 (eg, the memory 130 , the display device 160 , the electronic device 102 ). ), the electronic device 104 , or the server 108. According to an embodiment, the image signal processor 260 is configured as at least a part of the processor 120 or is independent of the processor 120 . In the case of a separate processor, the images processed by the image signal processor 260 are processed by the processor 120 as they are or after additional image processing is performed by the display device 160 . can be displayed through

According to an embodiment, the electronic device 101 may include two or more camera modules 180 each having different properties or functions. In this case, for example, at least one camera module 180 may be a wide-angle camera or a front camera, and at least one other camera module may be a telephoto camera or a rear camera.

3 is a block diagram illustrating an electronic device for correcting an image based on an object included in an image according to various embodiments of the present disclosure;

Referring to FIG. 3 , an electronic device 300 (eg, the electronic device 101 of FIG. 1 ) includes a communication circuit 310 (eg, the communication module 190 of FIG. 1 ), a memory 320 (eg, FIG. 1 ). 1 ), a touch screen display 330 (eg, the display device 160 of FIG. 1 ), a camera module 340 (eg, the camera module 180 of FIG. 1 ), and a processor 350 ). (eg, the processor 120 of FIG. 1 ).

According to various embodiments of the present disclosure, the communication circuit 310 (eg, the communication module 190 of FIG. 1 ) is connected to an external electronic device (eg, the electronic device 102 or the server 108 of FIG. 1 ). data can be sent and received. For example, the communication circuit 310 may be connected to a network through wireless communication or wired communication to communicate with an external electronic device.

According to various embodiments of the present disclosure, the memory 320 (eg, the memory 130 of FIG. 1 ) may be electrically connected to the processor 350 .

In an embodiment, the memory 320 may store a program for detecting at least one or more straight lines based at least on an object included in an image. The memory 320 may store a program for determining at least one of the detected at least one or more straight lines as a reference line. The memory 320 may store a program for processing, for example, horizontal correction or vertical correction of an image based on the determined reference line.

According to various embodiments of the present disclosure, the touch screen display 330 (eg, the display device 160 of FIG. 1 ) may be integrally configured including the display unit 331 and the touch panel 333 . The display unit 331 may display various screens according to the use of the electronic device 300 . In addition, the display unit 331 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or a microelectromechanical systems (MEMS) display, or an electronic paper display. may include. The display unit 331 may display, for example, various contents (eg, text, image, video, icon, or symbol) to the user. The touch panel 333 may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body.

In an embodiment, the touch screen display 330 may display an image under the control of the processor 350 . For example, the image may include at least one of an image stored in the electronic device 300 , an image received from an external electronic device through the communication circuit 310 , and a preview image acquired from the camera module 340 .

The touch screen display 330 may display at least one or more straight lines detected based on at least an object included in the image to be superimposed on the image under the control of the processor 350 .

In one embodiment, the touch screen display 330 is calibrated so that a reference line determined by the processor 350 or selected by the user under the control of the processor 350 is parallel to at least one of a horizontal axis or a vertical axis of the image. Images can be displayed.

In an embodiment, the touch screen display 330 may overlap an indicator indicating a tilted region based on the corrected image under the control of the processor 350 .

According to various embodiments of the present disclosure, the camera module 340 (eg, the camera module 180 of FIG. 1 ) transmits the collected image to the display unit 331 as a preview image to allow the user to view the camera module 340 . You can check the acquired image through The camera module 340 may generate image data by photographing an image collected at the point in time when the input requesting photographing is generated in response to an input for requesting photographing.

According to various embodiments of the present disclosure, the processor 350 (eg, the processor 120 of FIG. 1 ) controls the overall operation of the electronic device 300 and the signal flow between internal components of the electronic device 300 , and , data processing, and control power supply from a battery (eg, battery 189 in FIG. 1 ) to the components.

In an embodiment, the processor 350 may display an image on the touch screen display 330 and detect at least one or more straight lines satisfying a specified length based at least on an object included in the image. The processor 350 may determine at least one of the detected at least one or more straight lines as a reference line.

In an embodiment, the processor 350 may determine a long straight line among the detected at least one or more straight lines as a reference line.

In an embodiment, the processor 350 divides the detected at least one straight line into a horizontal straight line or a vertical straight line according to whether an angle of a straight line satisfying a specified length is included in a predetermined range with respect to the central axis of the image. can When the straight line satisfying the specified length includes a horizontal straight line and a vertical straight line, the processor 350 may determine the vertical straight line as a reference line.

In an embodiment, the processor 350 may determine the horizontal straight line or the vertical straight line as a reference line based on characteristics of the image. For example, the characteristic of the image may include at least one of whether the image is a portrait image, a landscape image, an indoor or outdoor image, an image photographed in a portrait mode or a landscape mode, and the size of the image.

In an embodiment, the processor 350 may correct the image so that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image.

In an embodiment, the processor 350 may display the detected at least one or more straight lines on the touch screen display 330 by superimposing it on the image. When an input for selecting at least one of the at least one or more straight lines displayed on the image is sensed, the processor 350 may determine the selected at least one straight line as a reference line. The processor 350 may correct the image so that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image.

In an embodiment, the image may be a preview image obtained from the camera module 340 . The processor 350 may detect at least one or more straight lines satisfying a specified length based at least on the object included in the preview image. The processor 350 may display the detected at least one or more straight lines to be superimposed on the preview image. When an input for selecting at least one of at least one or more straight lines displayed to be superimposed on the preview image is detected, the processor 350 may determine the selected at least one straight line as a reference line. The processor 350 may display the corrected image on the touch screen display 330 such that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image in response to the input for selecting the at least one. The processor 350 may overlap an indicator indicating a tilted region based on the corrected image. When an image capturing signal is sensed, the processor 350 may capture an image, and may correct and store the captured image so that the determined reference line is parallel to at least one of a horizontal axis or a vertical axis of the image.

4 is a flowchart illustrating a method of correcting an image based on a reference line determined by an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 4 , the processor (eg, the processor 350 of FIG. 3 ) may display an image on the display (eg, the touch screen display 330 of FIG. 3 ) in operation 401 . For example, the image includes at least one of an image stored in an electronic device (eg, the electronic device 300 of FIG. 3 ) and an image received from an external electronic device through a communication circuit (eg, the communication circuit 310 of FIG. 3 ). may include

In an embodiment, the image may be a preview image obtained from a camera module (eg, the camera module 340 of FIG. 3 ).

The operation of correcting the preview image obtained from the camera module will be described in detail with reference to FIGS. 9 and 10 to be described later.

In an embodiment, the processor may detect an input for performing a function of correcting an angle of an image while the image is displayed. In response to an input for performing a function of correcting the angle of the image, the processor may perform operation 403 to be described later. However, the present invention is not limited thereto.

In an embodiment, in operation 403 , the processor may detect at least one or more straight lines satisfying a specified length based at least on the object included in the image. For example, at least one or more straight lines satisfying the specified length may be straight lines having a predetermined length or more. For example, the predetermined length or more may mean a length of 2/3 or more based on the horizontal length (or vertical length) of the image. However, the present invention is not limited thereto.

A method of detecting at least one or more straight lines for an object included in the image is a well-known technique and is obvious to a person skilled in the art, so a detailed description thereof will be omitted.

In an embodiment, the processor may detect a straight line having a length greater than or equal to a predetermined length based on at least an object included in the image, for example, a horizontal length of an image (or a vertical length of the image).

In operation 405 , the processor may determine at least one of the detected at least one or more straight lines as a reference line.

In an embodiment, the processor may determine a long straight line among the detected at least one or more straight lines as a reference line.

In an embodiment, the processor may divide the detected straight line into a horizontal straight line or a vertical straight line according to whether an angle of a straight line satisfying a specified length among the detected at least one straight line is included in a predetermined range with respect to the central axis of the image. When the straight line satisfying the specified length includes a horizontal straight line and a vertical straight line, the processor may set to give priority to the vertical straight line over the horizontal straight line, and may determine the vertical straight line as a reference line.

In an embodiment, the processor is configured to, when the detected at least one or more straight lines include a horizontal straight line and a vertical straight line, a characteristic of the image (eg, whether it is a portrait image, a landscape image, an indoor or outdoor image, a vertical direction or a horizontal direction). The vertical straight line or the horizontal straight line may be determined as a reference line based on the photographed image and the size of the image).

For example, when the image is an indoor image, the processor may set to give priority to a horizontal straight line over a vertical straight line, and may determine a long horizontal straight line among the horizontal straight lines as a reference line. Alternatively, when the image is an outdoor image, the processor may set to give priority to a vertical straight line over a horizontal straight line, and may determine a long vertical straight line among the vertical straight lines as a reference line.

For example, in the case of an image photographed in a vertical direction, the processor may be set to give a priority for determining a vertical straight line as a reference line rather than a horizontal straight line. Alternatively, in the case of an image photographed in a horizontal direction, the processor may be set to give a priority for determining a horizontal straight line as a reference line rather than a vertical straight line.

In an embodiment, the processor may correct the image so that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image in operation 407 .

For example, when the determined reference line is a horizontal straight line, the image may be corrected to be parallel to a horizontal axis of the image. When the determined reference line is a vertical straight line, the image may be corrected to be parallel to a vertical axis of the image.

In this regard, it will be described in detail with reference to FIGS. 5A and 5B to be described later.

In an embodiment, the processor may display a reference line based on correcting the image to be parallel to at least one of a horizontal axis and a vertical axis of the image to be superimposed on the corrected image. Accordingly, the user can intuitively check by which reference line the image is parallel-corrected.

In an embodiment, the processor may display a reference line based on correcting the image to be parallel to at least one of a horizontal axis and a vertical axis of the image and at least one or more straight lines detected in operation 403 to be superimposed on the corrected image . The reference line displayed on the corrected image and at least one or more straight lines may be displayed to be distinguished from each other. In this case, when the composition (eg, angle) of the image corrected by the reference line determined by the electronic device is not the composition desired by the user, it may be an example of an operation scenario in which the user can select a desired straight line as the reference line. . In an embodiment, an operation scenario in which the user can select a desired straight line as a reference line may be performed in a similar manner to the embodiment of FIG. 7 to be described later.

In an embodiment, the processor may perform an image editing function of editing the corrected image. For example, the image editing function may include enlarging, reducing, adjusting a ratio, cropping an image, adjusting brightness, and applying an effect of the image.

In one embodiment, the processor stores the corrected image in a memory (eg, memory 320 in FIG. 3 ) or via a communication circuit (eg, communication circuit 310 in FIG. 3 ) to an external electronic device (eg, FIG. 1 ). of the electronic device 102 or the server 108).

In an embodiment, the image stored in the memory or transmitted to the external electronic device may be a tilted image based on a parallel-corrected image by a reference line.

5A and 5B are diagrams for explaining a method of correcting an image according to various embodiments.

5A and 5B are diagrams for explaining a method of correcting an image so that the reference line determined in operation 407 in FIG. 4 is parallel to at least one of a horizontal axis or a vertical axis of the image.

Referring to FIG. 5A , the processor (eg, the processor 350 of FIG. 3 ) may correct the image so that the determined reference line is parallel to the horizontal central axis 510 or the vertical central axis 520 that is the central axis of the image.

In one embodiment, when the angle of the reference line 530 with respect to the central axes 510 and 520 is included in a certain range, for example, -45° to 45° or 135° to 225°, the processor is configured to The image may be corrected to be parallel to the vertical central axis 520 of the image.

In one embodiment, when the angle of the reference line 530 with respect to the central axes 510 and 520 is not included in a predetermined range, for example, 45° to 45° and 135° to 225°, the processor determines that the reference line 530 is The image may be corrected to be parallel to the horizontal central axis 510 of the image.

The aforementioned predetermined range, for example, 45° to 45°, 135° to 225°, determines whether the reference line is corrected to be parallel to the horizontal central axis 510 of the image or to be corrected to be parallel to the vertical central axis 520 . It is intended to easily describe the determining operation, but is not limited thereto.

As described above, in FIG. 5A , it has been described that the reference line is corrected in parallel with respect to the central axis of the entire image, but the present invention is not limited thereto. For example, the processor may correct the reference line to be parallel to the central axis of one side of the entire image.

For example, referring to FIG. 5B , the processor may detect at least one or more straight lines satisfying a specified length based on at least an object included in the image 550 . The processor may determine a straight line 580 on one side of the image 550 among the detected at least one straight line as a reference line. The processor may correct the image so that the determined reference line 580 is parallel to a horizontal central axis 560 or a vertical central axis 570 that is a central axis of one side of the image 550 . For example, when the angle of the reference line 580 is within a predetermined range (eg, -45° to 45° or 135° to 225°), the processor determines that the reference line 580 is a vertical central axis of one side of the image 550 . The image may be corrected to be parallel to 570 . Alternatively, if the angle of the reference line 580 does not fall within a certain range (eg, 45° to 45° and 135° to 225°), the processor determines that the reference line 580 is a horizontal central axis of one side of the image 550 ( 560), the image may be corrected.

6 is a diagram for describing a method of correcting an image based on a reference line determined by an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 6 , the processor (eg, the processor 350 of FIG. 3 ) performs at least one or more straight lines (eg, the first straight line) based on at least the object included in the image 600 as shown in <610>. 611 , the second straight line 613 , the third straight line 615 , the fourth straight line 617 , and the fifth straight line 619 ) can be detected.

As shown in operation 403 of FIG. 4 , the at least one or more straight lines, for example, the first straight line 611 and the second straight line 613 may be a vertical straight line having a length of 2/3 or more of the vertical length of the image 600 . In addition, the third straight line 615 to the fifth straight line 619 may be a horizontal straight line having a length equal to or greater than 2/3 of the horizontal length of the image 600 .

In an embodiment, the processor may determine at least one of the one or more straight lines (eg, the first straight line 611 to the fifth straight line 619) as the reference line. For example, when it is set to select a long straight line among the detected at least one or more straight lines as a reference line, the processor uses a fifth straight line 619 among the first straight lines 611 to 619 as a reference line. can decide

In one embodiment, the processor determines that the fifth straight line 619 determined as the reference line is the horizontal axis of the image 600 (eg, the horizontal central axes 510 and 560 of FIGS. 5A and 5B ) as shown in <630>. The image 600 may be horizontally corrected to be parallel to .

In one embodiment, as shown in <610> of FIG. 6, at least one or more detected straight lines (eg, the first straight line 611 to the fifth straight line 619) are displayed to be superimposed on the image 600 However, this is for ease of explanation, and in the embodiments of FIGS. 4 and 6 , the operation of displaying the at least one straight line to be superimposed on the image may be omitted. For example, when the processor detects that the function of correcting the image angle is detected while the image 600 is displayed as shown in <610>, the processor corrects it as shown in <630> based on the reference line determined by the processor image can be displayed.

7 is a flowchart illustrating a method of correcting an image based on a reference line selected by a user according to various embodiments of the present disclosure;

Referring to FIG. 7 , the processor (eg, the processor 350 of FIG. 3 ) may display an image on the display (eg, the touch screen display 330 of FIG. 3 ) in operation 701 . For example, the image includes at least one of an image stored in an electronic device (eg, the electronic device 300 of FIG. 3 ) and an image received from an external electronic device through a communication circuit (eg, the communication circuit 310 of FIG. 3 ). may include

In an embodiment, the image may be a preview image obtained from a camera module (eg, the camera module 340 of FIG. 3 ).

In an embodiment, the processor may detect an input for performing a function of correcting an angle of an image while the image is displayed. In response to an input for performing a function of correcting the angle of the image, the processor may perform operation 703 to be described later.

In an embodiment, the processor may detect at least one or more straight lines satisfying a specified length based on at least an object included in the image in operation 703 . For example, at least one or more straight lines satisfying the specified length may be straight lines having a predetermined length or more. For example, the predetermined length or more may mean a length of 2/3 or more based on the horizontal length (or vertical length) of the image. However, the present invention is not limited thereto.

In an embodiment, the processor may display the detected at least one or more straight lines in operation 705 by superimposing them on the image.

In an embodiment, the one or more straight lines may be displayed to overlap the image by a preset number. For example, if the preset number is 5 and at least one or more straight lines satisfying the specified length are detected, for example, 8 or more, the processor superimposes only 5 straight lines with high priority among the 8 or more straight lines on the image can be displayed For example, the processor may display top 5 straight lines having a long length among the 8 or more detected straight lines.

In an embodiment, when an input for selecting at least one of the at least one or more straight lines displayed on the image is detected in operation 707 , the processor may determine the selected at least one straight line as a reference line in operation 709 .

In one embodiment, the processor determines that the reference line determined in operation 711 is a horizontal axis of the image (eg, the horizontal central axes 510 and 560 of FIGS. 5A and 5B ) or a vertical axis (eg, the vertical central axis 520 of FIGS. 5A and 5B ). , 570)), the image may be corrected to be parallel to at least one.

In one embodiment, since operation 711 of FIG. 7 is the same as operation 407 of FIG. 4 described above, a detailed description thereof is replaced with a description related to FIG. 4 .

In an embodiment, the processor may store the corrected image in a memory (eg, the memory 320 of FIG. 3 ) in operation 713 .

In an embodiment, the processor transmits the corrected image to an external electronic device (eg, the electronic device 102 or the server 108 of FIG. 1 ) through a communication circuit (eg, the communication circuit 310 of FIG. 3 ). can

In an embodiment, the image stored in the memory or transmitted to the external electronic device may be a tilted image based on a parallel-corrected image by a reference line.

8 is a diagram for describing a method of correcting an image based on a reference line selected by a user according to various embodiments of the present disclosure;

Referring to FIG. 8 , the processor (eg, the processor 350 of FIG. 3 ) performs at least one or more straight lines (eg, a first straight line) based on at least an object included in an image 800 as shown in <810>. 811 , the second straight line 813 , the third straight line 815 , the fourth straight line 817 , and the fifth straight line 819 ) can be detected. For example, the detected at least one or more straight lines (eg, the first straight line 811 to the fifth straight line 819 ) may be a straight line having a length of 2/3 or more of a horizontal length or a vertical length of the image 800 . .

In an embodiment, the processor may receive an input 821 for selecting one of the at least one or more straight lines (eg, the first straight line 811 to the fifth straight line 819 ). For example, when an input 821 for selecting a third straight line 815 from among the first straight line 811 to the fifth straight line 819 is received, the processor may determine the third straight line 815 as a reference line. .

In one embodiment, the processor determines that the third straight line 815 determined as the reference line is the horizontal axis of the image 800 (eg, the horizontal central axes 510 and 560 of FIGS. 5A and 5B ) as shown in <830>. The image 800 may be horizontally corrected to be parallel to .

9 is a flowchart illustrating a method of correcting an image based on a reference line selected from a preview image obtained from a camera module according to various embodiments of the present disclosure;

Referring to FIG. 9 , the processor (eg, the processor 350 of FIG. 3 ) displays a preview image obtained from the camera module (eg, the camera module 340 of FIG. 3 ) in operation 901 (eg, the touch of FIG. 3 ). may be displayed on the screen display 330).

In an embodiment, the processor may detect at least one or more straight lines satisfying a specified length based at least on the object included in the preview image in operation 903 .

In an embodiment, the processor may display the detected at least one or more straight lines in operation 905 to be superimposed on the preview image.

In an embodiment, even if the movement of the camera module is sensed in the detected at least one or more straight lines, a state in which the at least one or more reference lines are displayed to be superimposed on the preview image may be maintained.

In an embodiment, when an input for selecting at least one of the at least one or more straight lines displayed to be superimposed on the preview image is detected in operation 907 , the processor may determine the selected at least one straight line as a reference line in operation 909 .

In an embodiment, in response to the input of selecting at least one of operation 907, the processor may display the corrected image so that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image. The operation of displaying the corrected image may be an operation for providing the user to check the composition of the corrected image in advance before storing the corrected image according to the image capturing signal detected in operation 911 to be described later. have.

In an embodiment, the processor may further display an indicator (eg, a guide line) indicating an area for a tilt result on the corrected image. For example, the operation of displaying the indicator indicating the area for the tilt result may mean indicating the area of the image to be stored according to the image capturing signal detected in operation 911 to be described later. The indicator may be displayed to be superimposed on the corrected image in the form of a closed curve. In addition, the indicator may be distinguishedly displayed (eg, indicated by a dotted line) in the corrected image.

In one embodiment, when the image capturing signal is detected in operation 911, the processor captures an image, and the determined reference line is the horizontal axis of the image (eg, the horizontal central axes 510 and 560 of FIGS. 5A and 5B) or the vertical axis ( Example: The photographed image may be corrected and stored to be parallel to at least one of the vertical central axes 520 and 570 of FIGS. 5A and 5B .

In an embodiment, the processor transmits the corrected image to an external electronic device (eg, the electronic device 102 or the server 108 of FIG. 1 ) through a communication circuit (eg, the communication circuit 310 of FIG. 3 ). can

10 is a diagram for describing a method of correcting an image based on a reference line selected from a preview image obtained from a camera module according to various embodiments of the present disclosure;

Referring to FIG. 10 , a processor (eg, the processor 350 of FIG. 3 ) displays an image 1000 obtained from a camera module (eg, the camera module 340 of FIG. 3 ) (eg, the touch screen of FIG. 3 ). may be displayed on the display 340 . The processor may detect at least one or more straight lines based at least on the object included in the image 1000 .

For example, as shown in <1010>, the processor may receive an input for selecting at least one of the detected at least one or more straight lines (not shown), for example, the first straight line 1011 .

In an embodiment, the processor may determine the selected first straight line 1011 as a reference line, and display a vertically corrected preview image based on the first straight line 1011 determined as the reference line. In an embodiment, the vertically corrected preview image may be an image for providing a user to check the composition of the corrected image in advance before capturing the image.

In an embodiment, the processor may display a guide line 1021 (eg, a rectangular shape) indicating a tilt result on the vertically corrected preview image. For example, as the guide 1021 indicating the tilt result is displayed on the vertically corrected preview image, the user can check the area of the image to be stored in advance.

In an embodiment, the processor determines that when an image capturing signal is detected, as shown in <1030>, the first straight line 1011 determined as the reference line is the vertical axis of the image 1000 (eg, the vertical center of FIGS. 5A and 5B ). An image vertically corrected to be parallel to the axes 520 and 570 (eg, the image 1021 according to the tilt result) may be stored.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, at least one of a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, and a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

It should be understood that the various embodiments of this document and the terms used therein are not intended to limit the technology described in this document to the specific embodiments, and include various modifications, equivalents, and/or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like components. The singular expression may include the plural expression unless the context clearly dictates otherwise. In this document, expressions such as “A or B”, “at least one of A and/or B”, “A, B or C” or “at least one of A, B and/or C” refer to all of the items listed together. Possible combinations may be included. Expressions such as “first”, “second”, “first” or “second” can modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another. The components are not limited. When an (eg, first) component is referred to as being “connected (functionally or communicatively)” or “connected” to another (eg, second) component, that component is It may be directly connected to the component or may be connected through another component (eg, a third component).

As used herein, the term “module” includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of one or more functions. For example, the module may be configured as an application-specific integrated circuit (ASIC).

Various embodiments of the present document include instructions stored in a machine-readable storage media (eg, internal memory 136 or external memory 138) that can be read by a machine (eg, a computer). It may be implemented as software (eg, the program 140). The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device 101 ) according to the disclosed embodiments. When the instruction is executed by a processor (eg, the processor 120), the processor may directly or use other components under the control of the processor to perform a function corresponding to the instruction. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

According to an exemplary embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

Each of the components (eg, a module or a program) according to various embodiments may be composed of a singular or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be It may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component are executed sequentially, parallel, iteratively, or heuristically, or at least some operations are executed in a different order, are omitted, or other operations are added. can be

310: communication circuit
320: memory
330: touch screen display
340: camera module
350: processor

Claims (20)

In an electronic device,
display; and
A processor comprising:
displaying an image on the display and detecting at least one or more straight lines that satisfy a specified length based at least on an object included in the image,
Determining at least one of the at least one or more straight lines as a reference line,
correcting the image so that the reference line is parallel to at least one of a horizontal axis or a vertical axis of the image, and
When the detected at least one or more straight lines include a horizontal straight line and a vertical straight line, one of the horizontal straight line and the vertical straight line is determined as the reference line based on the characteristics of the image,
The characteristic of the image includes at least one of a scene of the image, a photographing environment of the image, and a photographing direction of the image. The method of claim 1, wherein the processor comprises:
An electronic device configured to display the one or more straight lines to be superimposed on the image using the display. 3. The method of claim 2,
The processor is
The electronic device is configured to determine the selected at least one straight line as the reference line based at least on the basis of a user input for selecting at least one straight line among the at least one or more straight lines displayed to be superimposed on the image. The method of claim 1,
Further comprising an image sensor,
The processor is
An electronic device configured to acquire the image using the image sensor and display it on the display. The method of claim 1,
The processor is
An electronic device configured to determine a long straight line among the one or more straight lines as the reference line. The method of claim 1,
The processor is
An electronic device configured to determine the vertical straight line as the reference line when the detected at least one or more straight lines include a horizontal straight line and a vertical straight line. The method of claim 1,
The scene of the image includes a person and a landscape,
The photographing environment of the image includes indoor and outdoor, and
The photographing direction of the image includes a vertical direction and a horizontal direction,
The processor is
If the image is an image including the person, determining the vertical straight line among the vertical straight line and the horizontal straight line as the reference line,
When the image is an image including the landscape, the horizontal straight line among the vertical straight line and the horizontal straight line is determined as the reference line,
When the image is an image taken indoors, the horizontal straight line among the vertical straight line and the horizontal straight line is determined as the reference line,
When the image is an image taken outdoors, the vertical straight line among the vertical straight line and the horizontal straight line is determined as the reference line,
If the image is the image taken in the vertical direction, determining the vertical straight line among the vertical straight line and the horizontal straight line as the reference line, and
an electronic device configured to determine the horizontal straight line among the vertical straight line and the horizontal straight line as the reference line when the image is the image taken in the horizontal direction. 5. The method of claim 4,
The processor is
When a signal for photographing the image obtained using the image sensor is detected, the image is photographed, and the photographed image is corrected and stored so that the determined reference line is parallel to at least one of a horizontal axis or a vertical axis of the image set electronics. The method of claim 1,
The processor is
An electronic device configured to overlap an indicator indicating a tilted area based on the corrected image. A method of correcting an image based on an object included in an image of an electronic device, the method comprising:
displaying an image on a display;
detecting at least one or more straight lines satisfying a specified length based on at least an object included in the image;
determining at least one of the at least one or more straight lines as a reference line; and
correcting the image so that the reference line is parallel to at least one of a horizontal axis and a vertical axis of the image,
The operation of determining the at least one as the reference line comprises:
When the detected at least one or more straight lines include a horizontal straight line and a vertical straight line, determining one of the horizontal straight line and the vertical straight line as the reference line based on the characteristics of the image,
The characteristic of the image includes at least one of a scene of the image, a shooting environment of the image, and a shooting direction of the image. ◈Claim 11 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
After detecting the at least one or more straight lines, the method further comprising: displaying the detected at least one or more straight lines to be superimposed on the image. ◈Claim 12 was abandoned when paying the registration fee.◈ 12. The method of claim 11,
Further comprising the operation of detecting a user input for selecting at least one straight line among the at least one or more straight lines displayed to be superimposed on the image,
The operation of determining the at least one as a reference line,
and determining at least one straight line selected by the user's input as the reference line. ◈Claim 13 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The operation of displaying the image on the display comprises:
An operation of displaying an image acquired using an image sensor on the display. ◈Claim 14 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The operation of determining the at least one as a reference line,
and determining, as the reference line, a long straight line among the at least one or more straight lines. ◈Claim 15 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The operation of determining the at least one as a reference line,
and determining the vertical straight line as the reference line when the detected at least one or more straight lines include a horizontal straight line and a vertical straight line. delete ◈Claim 17 was abandoned when paying the registration fee.◈ 14. The method of claim 13,
detecting a signal for capturing the image acquired using the image sensor;
photographing the image and correcting the photographed image so that the determined reference line is parallel to at least one of a horizontal axis and a vertical axis of the image; and
and storing the corrected image. ◈Claim 18 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The method further comprising the operation of displaying an indicator indicating a tilted area based on the corrected image to be overlapped. ◈Claim 19 was abandoned at the time of payment of the registration fee.◈ 11. The method of claim 10,
Further comprising the operation of providing an image editing function to edit the corrected image,
The image editing function includes at least one of enlarging, reducing, adjusting a ratio, cropping an image, adjusting brightness, and applying an effect of the image. ◈Claim 20 was abandoned when paying the registration fee.◈ A computer-readable recording medium recording a program for executing a method of correcting an image based on an object included in the image in a computer, the method comprising:
displaying an image on a display;
detecting at least one or more straight lines satisfying a specified length based on at least an object included in the image;
determining at least one of the at least one or more straight lines as a reference line; and
correcting the image so that the reference line is parallel to at least one of a horizontal axis and a vertical axis of the image,
The operation of determining the at least one as the reference line comprises:
When the detected at least one or more straight lines include a horizontal straight line and a vertical straight line, determining one of the horizontal straight line and the vertical straight line as the reference line based on the characteristics of the image,
The characteristic of the image includes at least one of a scene of the image, a photographing environment of the image, and a photographing direction of the image.

Images