KR102380133B1 - Method and apparatus for providing internet of vehicle application service in multi access edge computing - Google Patents

Abstract

An objective of the present invention is to reduce operation amounts by preventing redundant data in data received from a vehicle to provide an internet-of-vehicle (IoV) application service from being processed. An operating method of a first MEC device providing an internet-of-vehicle (IoV) application service comprises: a step of receiving metadata of images photographed by vehicle communication nodes from the vehicle communication nodes; a step of determining image data to be requested from at least some vehicle communication nodes among the vehicle communication nodes based on the received metadata of the images; a step of transmitting information on the images to be requested to the at least some vehicle communication nodes; a step of receiving the requested images from the at least some vehicle communication nodes; and a step of storing the images received from the at least some vehicle communication nodes in a database.

Description

METHOD AND APPARATUS FOR PROVIDING INTERNET OF VEHICLE APPLICATION SERVICE IN MULTI ACCESS EDGE COMPUTING

The following description relates to a method and apparatus for providing an Internet of vehicle (IoV) application service, and relates to a method and apparatus for providing an Internet of vehicle application service in a multi-access edge computing (MEC) environment will be.

Recently, through time- and data-driven computing operations (eg, image/video-based on-road object recognition, scene recognition), IoV (such as HD map generation, road construction detection, and 3D scene reconstruction) Internet of vehicle) applications are emerging recently. IoV applications require the collection and analysis of real-time traffic detection data provided by multiple sensors (ie cameras, lidar, radar, etc.) installed in the vehicle to provide a better user experience and safety for drivers and passengers. .

Therefore, high-performance computing resources are required to process large amounts of data in a short response time. Most vehicles will not be able to handle these computing tasks with limited resources within the vehicle. To solve this technical problem, multi-access edge computing (MEC), called a new paradigm, provides computing resources like a cloud (fog cloud) to mobile users in close proximity to a radio access network. was proposed for It has also been proposed to offload computing tasks generated in client vehicles to MEC devices in edge networks such as 5G communication base stations.

With multi-access edge computing, autonomous vehicles can handle time-sensitive computational tasks and data-intensive computational tasks for new Internet-of-vehicles (IoV) applications with computational offloading. However, since large amounts of data generated by autonomous vehicles located at the same location are usually duplicated, a serious problem may arise due to limited network bandwidth.

In addition, such a computationally intensive operation on the edge server side may place a serious load on the resource-limited MEC device, thereby reducing the performance efficiency of the application.

An object of the present disclosure is to provide a method and apparatus for providing a vehicle Internet application service that reduces the amount of computation by preventing processing of duplicate data among data received from a vehicle to provide the vehicle Internet application service.

The technical objects to be achieved in the present disclosure are not limited to the above, and other technical problems not mentioned are common knowledge in the technical field to which the technical configuration of the present disclosure is applied from the embodiments of the present disclosure to be described below. can be considered by those with

The present invention is characterized in that, among data received from a vehicle, the amount of calculation for providing a vehicle Internet application service is lowered by preventing overlapping data processing and distributing the data processing to edge computing nodes.

As an example of the present disclosure, a method of operating a first MEC device that provides an Internet of vehicle (IoV) application service includes: receiving metadata of an image captured by each of the vehicle communication nodes from vehicle communication nodes; ; determining image data to be requested from at least some of the vehicle communication nodes among the vehicle communication nodes based on the received metadata of the image; transmitting information about the requested images to the at least some of the vehicle communication nodes; receiving the requested images from the at least some vehicle communication nodes; and storing the images received from the at least some vehicle communication nodes in a database.

Here, the meta data of the image may include location information of a location where the image was captured, information about the time of capturing the image, and information about a depth of field (DOF).

Here, the determining of the image data to be requested from the at least some of the vehicle communication nodes may include: comparing an area of images previously stored in the database with an area indicated by the received metadata of the image; determining an overlapping area overlapping the pre-stored image area from among areas indicated by the metadata of the received image based on the comparison result; and excluding the overlapping area from the area indicated by the metadata of the received image, wherein the requested image data may be an image from which the overlapping area is excluded from the image.

Here, the method includes: calculating an overlap, which is a ratio of an overlapping area between each area of the meta data and the pre-stored images; and determining meta data having the highest degree of overlap among the meta data, wherein the requested image data includes, among images corresponding to the determined meta data having the highest degree of overlap, the overlapping area It may be an excluded image.

Here, the operating method includes calculating a validity parameter of each of the images stored in the database based on timestamp information of each of the images stored in the database; checking the validity of each of the images stored in the database based on the validity parameter; and deleting at least some images whose validity has not been confirmed from among the images stored in the database.

Here, the operating method includes: receiving parameters for dividing and allocating a task from a plurality of MEC devices other than the first MEC device; calculating a cost according to assignment of a subtask that is a part of the task based on the parameters; allocating subtasks to the plurality of MEC devices; receiving the subtask performance result data from each of the plurality of MEC devices; and performing the task based on data as a result of performing the subtask.

here. The task may be a task of stitching images stored in the database, and the subtask may be a task of stitching some images among images stored in the database.

Here, the parameters are a unit cost of a resource consumed due to transmission of data for allocating a subtask, a price per unit computing resource consumed due to the execution of the subtask, and data consumed due to transmission of data as a result of performing the subtask. It may include information about the unit cost of the resource. , how it works.

As an example of the present disclosure, a first MEC providing an Internet of vehicle (IoV) application service includes a transceiver; and a processor coupled to the transceiver, the processor configured to: receive meta data from vehicle communication nodes of an image captured by each of the vehicle communication nodes; determining image data to be requested from at least some of the vehicle communication nodes among the vehicle communication nodes based on the received metadata of the image; transmitting information about the images to be requested to the at least some of the vehicle communication nodes; receive the requested images from the at least some vehicular communication nodes; In addition, the images received from the at least some of the vehicle communication nodes may be stored in a database.

Here, the meta data of the image may include location information of a location where the image was captured, information about the time of capturing the image, and information about a depth of field (DOF).

Here, in determining the image data to be requested from the at least some vehicle communication nodes, the processor compares a region of images previously stored in the database with a region indicated by the received metadata of the image; determining an overlapping area overlapping the pre-stored areas of the images from among areas indicated by the metadata of the received images based on the comparison result; In addition, from the area indicated by the metadata of the received image, except for the overlapping area, the image data to be requested may be an image in which the overlapping area is excluded from the image.

Here, the processor calculates an overlapping degree, which is a ratio of an overlapping area between each area of the metadata and the pre-stored images; and determining metadata having the highest degree of overlap among the metadata, wherein the requested image data includes, among images corresponding to the determined metadata having the highest degree of overlap, the overlapping area It may be an excluded image.

Here, the processor calculates a validity parameter of each of the images stored in the database based on timestamp information of each of the images stored in the database; check the validity of each of the images stored in the database based on the validity parameter; In addition, from among the images stored in the database, at least some images whose validity has not been confirmed may be deleted.

Here, the processor receives parameters for dividing and allocating a task from a plurality of MEC devices other than the first MEC device; calculating a cost for allocating a subtask that is part of the task based on the parameters; allocating a subtask to the plurality of MEC devices; receive the subtask performance result data from each of the plurality of MEC devices; In addition, the task may be performed based on data as a result of performing the subtask.

Here, the task may be a task of stitching images stored in the database, and the subtask may be a task of stitching some images among images stored in the database.

Here, the parameters are a unit cost of a resource consumed due to transmission of data for allocating a subtask, a price per unit computing resource consumed due to the execution of the subtask, and data consumed due to transmission of data as a result of performing the subtask. It may include information about the unit cost of the resource.

The following effects may be obtained by the embodiments based on the present disclosure.

According to the present disclosure, the MEC device preferentially receives metadata of images from vehicle communication nodes and determines image data to be additionally requested based on the received metadata, thereby can be stitched Accordingly, the present disclosure may provide a vehicle Internet application service method that consumes a small amount of computation.

Effects that can be obtained in the embodiments of the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned are the technical fields to which the technical configuration of the present disclosure is applied from the description of the embodiments of the present disclosure below. It can be clearly derived and understood by those of ordinary skill in the art. That is, unintended effects of implementing the configuration described in the present disclosure may also be derived by those of ordinary skill in the art from the embodiments of the present disclosure.

The accompanying drawings below are provided to help understanding of the present disclosure, and together with the detailed description, may provide embodiments of the present disclosure. However, the technical features of the present disclosure are not limited to specific drawings, and features disclosed in each drawing may be combined with each other to constitute a new embodiment. Reference numerals in each drawing may refer to structural elements.
1 is a diagram illustrating a structure of a communication network applied in the present disclosure.
2 is a diagram illustrating a method of stitching an image based on metadata of an image applied in the present disclosure.
3 is a diagram illustrating a communication network applied in the present disclosure in a grid form.
4 is a diagram illustrating a distribution of images distributed in a space-time domain applied in the present disclosure.
5 is a diagram illustrating a result of classifying an overlapping region and an additional region in an image applicable to the present disclosure.
6 is a diagram illustrating a structure of a communication network including an MEC device for distributing computing tasks applied in the present disclosure.
7 is a diagram illustrating an operation of an MEC device for distributing computing tasks applied in the present disclosure.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

Recently, Internet of vehicle (IoV) applications that provide convenience to vehicle users through time- and data-oriented computing operations have been developed. In order to provide IoV applications, an IoV application service provider needs to collect and analyze real-time traffic data obtained from various sensors (eg, camera, lidar, radar, etc.) installed in a vehicle.

In order to process a large amount of data within a short response time, high-performance computing resources are required. Accordingly, a multi-access edge clouding (MEC) technology capable of providing computing resources like a cloud (fog cloud) to mobile users close to a radio access network has been proposed. In addition, the MEC technology has been proposed to provide a method for an edge network such as a 5G communication base station to offload a computing task generated by a client vehicle to lower-level servers.

Most studies in the field of MEC technology have been conducted to improve an optimization method for user task offloading and a method for allocating computing/communication resources to reduce service delay time or improve energy efficiency. However, a large amount of traffic data obtained from vehicle communication nodes in an MEC environment may often be missing or not processed.

For example, the same or similar large amount of image data and/or analysis results may be duplicated in the vicinity of a moving vehicle communication node. The overlapping images obtained from vehicle communication nodes in the same place may be merged into one image through image stitching and/or image merging. The merged image can provide more accurate and detailed surrounding environment information.

However, if each of the vehicle communication nodes transmits all images and/or images captured by the MEC device to the MEC device for computational operation, it is difficult for the MEC device to guarantee a low-latency service due to overlapping images. Therefore, communication nodes in the MEC environment should minimize waste of communication resources and computational resources by deleting redundant data.

MEC devices merge or stitch images and images received from a vehicle communication node to provide a more detailed environment around the vehicle exterior. MEC devices may consume very large computing costs by executing tasks that require high-performance computational resources, such as image stitching. In addition, when the computing cost of the MEC device is high, there is a risk of providing a delay and low quality service due to excessive consumption of computing operations.

To solve the above problems, the IoV application service provider may merge MEC devices located in a certain range of places and configure computing resources into a virtualization pool. And MEC devices can share the computing workload. However, when MEC devices located in adjacent ranges are merged, costs are incurred in both computing computational resources and communication resources. Here, the MEC devices are operated and maintained by different mobile operators (other service providers). Therefore, for economic reasons, MEC device operators may refuse to participate in the merge group without a certain remuneration.

Therefore, in order to solve the above problems, by first removing redundant data from vehicle communication nodes and optimizing MEC operation costs (ie communication and computing costs), computing resources for collaboration between merged MEC devices are allocated. a mechanism was developed.

An application provider that performs image stitching processing based on data duplication mitigation and collaboration of an image set collected from vehicle communication nodes must communicate effectively with communication nodes of a network and efficiently manage computing resources. Therefore, the framework of the mechanism proposed in the present disclosure is largely composed of two steps.

First, in order to provide a load reconfiguration service, the vehicle communication node transmits metadata (eg, location information, time, depth information, etc.) It is transmitted to the MEC device through the LTE/DSRC (dedicated short range communication) protocol. And the MEC device determines the optimal set of images from among all the images based on the metadata of the collected images. That is, the vehicle communication node does not transmit all images but preferentially transmits only image metadata, thereby reducing network bandwidth usage.

Second, a single MEC device may not be able to perform a high-performance operation capable of satisfying a latency condition required to provide an IoV application service due to a limitation of computing resources. Accordingly, the present disclosure is designed to provide a low-cost, high-performance computing service based on efficient MEC collaboration by distributing high-performance computational tasks. One MEC device may set a resource use (lease) cost in consideration of idle computing resources, work sizes, and transmission costs of adjacent MEC devices. The MEC device may select an edge server in an adjacent area to allocate a task based on the determined cost of using computing resources, and allocate or perform subtasks that are part of the task at an optimal service cost.

1 is a diagram illustrating a structure of a communication network applied in the present disclosure.

Referring to FIG. 1 , a communication network may include vehicle communication nodes 102-1 to 102-9, MEC devices 104-1 to 104-3, and a central controller.

The vehicle communication nodes 102-1 to 102-9 are nodes that are attached to a vehicle and perform a communication function. The vehicle communication nodes 102-1 to 102-9 may include an image sensor, and may periodically photograph an image around the vehicle communication node by using the image sensor. That is, the vehicle communication nodes 102-1 to 102-9 may acquire a set of captured images. Here, the image sensor may be an image sensor such as a camera, radar, or lidar.

The vehicle communication nodes 102-1 to 102-9 automatically generate meta data of an image whenever an image is captured, and store meta data of the generated image. Metadata of the image may indicate information such as a photographing location, photographing method, and photographing time of the image. For example, when the different vehicle communication nodes 102-1 to 102-9 each take a picture at the same time and at the same place, the different vehicle communication nodes 102-1 to 102-9 are photographed One image may be recognized as the same image. That is, the image acquired by each of the vehicle communication nodes 102-1 to 102-9 may include duplicated images.

The MEC devices 104-1 to 104-3 may collect images captured by the vehicle communication nodes 102-1 to 102-9, and may provide a road reconstruction service by stitching the collected images. The MEC devices 104 - 1 to 104 - 3 may obtain a partial view of road. The MEC devices 104 - 1 to 104 - 3 may be connected to a base station, and may receive images from vehicle communication nodes through the connected base station. The MEC devices 104 - 1 to 104 - 3 may provide a road reconstruction service for constructing an around view of the vehicle by utilizing image stitching technology. The MEC devices 104-1 to 104-3 may stitch images provided from the vehicle communication nodes 102-1 to 102-9 and reconstruct the road image.

The MEC devices 104 - 1 to 104 - 3 may be connected to a central controller. The central controller may be connected to the MEC devices 104-1 to 104-3 to control operations of the MEC devices 104-1 to 104-3. The central controller may receive image data for a road reconstruction service from the MEC devices, and collect the received image data. The central controller can secure a global view.

Each of the plurality of vehicle communication nodes 102-1 to 102-9 is a user equipment (UE), a terminal equipment (TE), an advanced mobile station (AMS), a high reliability-mobile station (HR-MS), a terminal (terminal) ), access terminal, mobile terminal, station, subscriber station, mobile station, portable subscriber station, node, device (device), may be referred to as an on board unit (OBU), and the like.

Each of the plurality of base stations connected to the MEC devices 104-1 to 104-3 is a NodeB (NB), an evolved NodeB (eNB), a gNB, an advanced base station (ABS), a high reliability-base station (HR-BS) ), BTS (base transceiver station), wireless base station (radio base station), wireless transceiver (radio transceiver), access point (access point), access node (node), RAS (radio access station), MMR-BS (mobile multihop) relay-base station), RS (relay station), ARS (advanced relay station), HR-RS (high reliability-relay station), HNB (home NodeB), HeNB (home eNodeB), RSU (road side unit), RRH (radio remote head), TP (transmission point), may be referred to as a transmission and reception point (TRP).

Each of the plurality of base stations connected to the MEC devices 104-1 to 104-3 is MIMO transmission, coordinated multipoint (CoMP) transmission, carrier aggregation (CA) transmission, transmission in an unlicensed band, Device to device communication (D2D) (or Proximity services), vehicle to everything (V2X), Internet of Things (IoT) communication, dual connectivity (DC), etc. can support

The vehicle communication nodes 102-1 to 102-9 may be connected to a base station connected to the MEC devices 104-1 to 104-3 through a vehicle to infrastructure (V2I) link. A base station connected to the MEC devices 104-1 to 104-3 may be connected to a base station connected to other MEC devices 104-1 to 104-3 through an X2 link. In addition, base stations connected to the MEC devices 104-1 to 104-3 may be connected to the central controller through a fiber link.

Each of the vehicle communication nodes 102-1 to 102-9 preferably secures a wide image of the surrounding environment in order to improve driving performance. Therefore, the vehicle communication nodes 102-1 to 102-9 photograph the road condition through the image sensor and transmit the photographed image to the MEC devices 104-1 to 104-3. However, since the capacity of the transmitted image data is large, a load may occur in the network due to the image transmission of the vehicle communication nodes 102-1 to 102-9. In particular, since the vehicle communication nodes 102-1 to 102-9 located in the same place capture the same image, the vehicle communication nodes 102-1 to 102-9 located in the same place transmit the image. In this case, unnecessary load may occur on the network.

In addition, since network and computing resources are limited, the MEC devices 104-1 to 104-3 may not receive all images transmitted by the vehicle communication nodes 102-1 to 102-9, or all Images may not be analyzed. Accordingly, the MEC devices 104-1 to 104-3 must efficiently use computing resources to find or collect the most useful images.

In order to provide an IoV application service such as a road reconfiguration service, the MEC devices 104 - 1 to 104 - 3 may consume a large amount of computing resources and perform tasks. Here, the MEC devices 104-1 to 104-3 may not support the response time and performance required by the IoV application due to limited computing resources.

According to an embodiment of the present invention, the vehicle communication node and the MEC device can efficiently use computing resources by first exchanging metadata of an image. Specifically, each of the vehicle communication nodes may transmit metadata of the image, which is information necessary for the MEC device to identify the duplicated image, to a base station connected to the MEC device. The MEC device collects metadata of each image captured by vehicles.

The MEC device may identify, analyze, and process duplicate data based on the metadata of the collected image. Since the MEC device can obtain an image of the entire connected road structure in advance, information on the road condition may be obtained in advance. Accordingly, the MEC device can quickly and efficiently determine whether a new image is needed, thereby reducing the load on the network.

The MEC device may request data excluding overlapping data from the vehicle communication node. The vehicle communication node may transmit image data requested from the MEC device to the MEC device. The MEC device may stitch image data received from the vehicle communication node to previously secured images.

In addition, in order to reduce the delay time due to the computing operation of the service, the MEC device may cooperatively control the computing service by transferring the excessive computing task to the MEC devices located in the neighboring area. Therefore, the MEC servers can cooperatively perform tasks by sharing each other's redundant tasks. Accordingly, the processing efficiency of all operations can be improved.

In particular, the MEC device performing the image stitching operation divides the image into a plurality of smaller subset units. In addition, the MEC device may distribute and allocate the image stitching task to MEC devices located in a neighboring area based on the current network state information. Here, the network state information may include information about the number of connectable MEC devices and/or idle resource information of each connectable MEC device located in a nearby area. In order to induce participation of MEC devices located in nearby areas in a cooperative computing structure, the MEC server must pay for the use of computing resources.

As a result of image stitching through cooperation with external MEC devices, the MEC device reconstructs images captured in the vehicle to compose an image of the environment around the vehicle. The MEC device, which has succeeded in constructing an image of the surrounding environment, may transmit the image of the surrounding environment to the vehicle communication node to improve the driving performance of the vehicle. In addition, the MEC device may continuously update the surrounding environment image.

An operation method for providing an IoV application service of each of the vehicle communication node and the MEC device included in the communication network of FIG. 1 may be as described below.

2 is a diagram illustrating a method of stitching an image based on metadata of an image applied in the present disclosure.

Referring to FIG. 2 , communication nodes constituting a communication system may exchange image metadata and image data, and may perform an operation for providing an IoV application based on image data obtained as a result of the exchange. Here, the devices of the communication system may support inter-terminal communication such as D2D communication and/or vehicle-to-vehicle communication such as V2X.

In step S201, the vehicle communication node may take an image of the area around the vehicle communication node through the provided image sensor. The vehicle communication node may generate metadata of an image captured through the image sensor, and may store metadata of the generated image. Here, the meta data of the image may include information about location information of a place where the image was photographed, photographing time, depth, and the like.

In step S203, the vehicle communication node may transmit metadata of the captured image to the MEC device. The vehicle communication node may transmit metadata including information about the location information of the location where the image was taken, the photographing time, and information about the depth of the image to the MEC device, and information on the identifier of the vehicle communication node that transmits the metadata together can be transmitted

In step S205 , the MEC device may detect an overlapping portion between the reconstructed region and the previously secured image based on the metadata of the image acquired from the vehicle communication node. A specific example of detecting an overlapping portion based on metadata and information on metadata of an image acquired by the MEC device may be described below.

3 is a diagram illustrating a communication network applied in the present disclosure in a grid form.

Referring to FIG. 3 , the communication network may be represented in a grid form on a two-dimensional plane. A grid of a communication network may be divided by roads, and each road may be divided into smaller load segment units (s _j to s _j+3 ). Vehicle communication nodes (vehicles) can be represented as moving on a road in a two-dimensional plane. In addition, the base station may be located on a grid of a two-dimensional plane.

The vehicle communication node may take an image within the coverage of the image sensor. For example, if the coverage of the image sensor is an area of d _{1 -} longitudinal length and d _s transverse length, then the vehicle communication node has a grid and a load contained in the area of d _{1 -} longitudinal length and d _s transverse length. You can take an image of a segment. In addition, the vehicle communication node may generate metadata of the image including information about location information of a place where the image was captured, a shooting time, depth, and the like.

A distribution of images captured by vehicle communication nodes on a grid of a communication network and a representation method of the distributed images may be as described below.

4 is a diagram illustrating a distribution of images distributed in a space-time domain applied in the present disclosure.

Referring to FIG. 4 , each of the vehicle communication nodes of the communication network may capture an image of the grid of the communication network. Here, the horizontal axis of FIG. 4 indicates the road segment (Ω={s _j, s _j+1 , s _j+2 , s _{j+3, ...} }) of the road, and the vertical axis indicates time.

Specifically, each of the vehicle communication nodes may periodically take an image of the road. When the vehicle communication node drives the road e _k at time t, the captured image is expressed as l(e _k ,t)={l(s _j ,t)|Ω∋s _j } including the road segments can be The vehicle communication node may photograph continuous road segments for a predetermined time period, and images continuously photographed by the vehicle communication node may be expressed as l(e _k , t _a , t _b ). Here, t _a may indicate a starting point of a time interval for photographing consecutive segments of a vehicle communication node, and t _b may indicate an end point of the time interval. The depth of image (DOF) of the image l(e _k ,t) may be expressed as d(s _j ,t).

The vehicle communication node may generate metadata of the image including information on location information, photographing time, depth, etc. of a place where the image is photographed. That is, the metadata of the image may include respective parameters of L(e _k ,t _a , t _b ) and respective parameters of d(s _j ,t).

The image taken by the vehicle communication node may include an image of the set of road segments. A validity parameter θ may be determined based on a time stamp of each of the images captured by the vehicle communication node. The validity parameter may define the freshness of the image. Also, the validity parameter may be used to determine whether different images are identical.

If the difference in time stamps of different images taken in the same area is less than θ, the two images are regarded as the same image. Accordingly, the photographed images l(s _j ,t), l(s _j+1 ,t), l(s _j+2 ,t)) have a depth of d(s _j , t)+d(s _{j +1} ,t)+d(s _j+2 , t) and may be expressed as one image l(e _k ,t)) in the form of a rectangle having a time stamp of θ.

The MEC device may distinguish an overlapping area and an additional area from among the images captured by the vehicle communication node based on the metadata of the image. A specific method of distinguishing the overlapping area and the additional area among the images captured by the vehicle communication node may be as described below.

5 is a diagram illustrating a result of classifying an overlapping region and an additional region in an image applicable to the present disclosure.

Referring to FIG. 5 , the MEC device may store image data from vehicle communication nodes in advance in a database.

The MEC device may receive the metadata of the image S from the vehicle communication node, and may configure an area of the image S based on the metadata of the image S. The MEC device may distinguish a redundancy area of the image S with existing image data (eg, X or Y) and an additional area that does not overlap with the existing image data. For example, when the existing image data previously secured by the MEC device is X, the image S includes only the additional area. On the other hand, when the existing image data previously secured by the MEC device is Y, the image S includes an additional area and an overlapping area.

The MEC device may calculate the degree of overlap of the images captured by the vehicle communication nodes based on each of the metadata of the image S received from the vehicle communication nodes. That is, the MEC device includes the entire area (F(X∪S) or F(Y∪S) including the image S captured by the vehicle communication node and the existing image data (eg, X or Y). ) and the difference value (F(X∪S)-F(X) or F(Y∪S)-F(Y)) of the image (S) taken by the vehicle communication node. It is possible to calculate the degree of overlap of the image (S). In addition, the MEC device may calculate the ratio of the additional area of the image S captured by the vehicle age communication node based on the overlapping calculation diagram. According to the example of FIG. 5 , the ratio of the additional area of S to X is greater than the ratio of the additional area of S to Y.

Referring back to FIG. 2 , in step S207, the MEC device may request an additional image from the vehicle communication node. That is, the MEC device may request the vehicle communication node for an image of the additional area determined based on metadata of the image received from the vehicle communication nodes. The MEC device may transmit an additional image request message including metadata for the additional region of the image to the vehicle communication node.

The MEC device may preferentially request an image having the lowest degree of overlap from among the images captured by the vehicle communication nodes from the vehicle communication node. In addition, the MEC device may sequentially request images having a low degree of overlap among the images captured by the vehicle communication nodes from the vehicle communication node.

In step S209 , the vehicle communication node may transmit an additional image to the MEC device. For example, the vehicle communication node may edit a part of the captured image based on metadata included in the additional image request message received from the MEC device. In addition, the vehicle communication node may transmit an additional image obtained as a result of editing the image to the MEC device.

In step S211, the MEC device may stitch the previously acquired image and the acquired image. That is, the MEC device may reconstruct the image of the road included in the grid by stitching the requested image obtained from the vehicle communication node to the previously secured image.

The MEC device may reconstruct the images of the road based on the validity parameter of the images every time slot. For example, when a predetermined time has elapsed since the specific image was acquired, the MEC device may delete the specific image. In addition, the MEC device may periodically update the image corresponding to the deleted part. That is, the MEC device may periodically update the road image for providing the IoV application service by performing the operations of steps S203 to S211.

6 is a diagram illustrating a structure of a communication network including an MEC device for distributing computing tasks applied in the present disclosure.

Referring to FIG. 6 , a communication network may include base stations connected to each of MEC devices 604 - 1 , 606 - 1 , and 606 - 2 . That is, the communication network may include a plurality of MEC devices (604-1, 606-1, 606-2). At least one MEC device 604-1 of the plurality of MEC devices 604-1, 606-1, and 606-2 includes images (img) taken from the vehicle communication nodes 602-1 and 602-2. ₁ to img ₅ ) may be received. The MEC device 604-1 may generate a road reconstruction map by stitching the received images, and may provide a road reconstruction service to the vehicle communication nodes 602-1 and 602-2.

Among the plurality of MEC devices, the MEC device 604-1, which has received images from the vehicle communication nodes 602-1 and 602-2, is located around the It transmits (broadcasts) a work request to the MEC devices (606-1, 606-2). Finally, the MEC device may allocate a computing task to the surrounding MEC devices 606 - 1 and 606 - 2 at a minimum cost according to the determined cost. For example, the MEC device may allocate stitching operations of img ₁ to img ₃ to the MEC device 606 - 1 and may allocate stitching operations of img ₄ to img ₅ to the MEC device 606 - 2 .

The surrounding MEC devices 606 - 1 and 606 - 2 have available resources. Upon receiving the message transmitted from the requesting MEC device 604-1, the surrounding MEC devices 606-1 and 606-2 consume idle computing resources to provide a computing service, and according to the cost determined for outsourcing, Calculate the cost of computing services. If the requesting MEC device 604-1 selects a nearby MEC server, the selected MEC devices 606-1 and 606-2 perform image stitching and compensate the requested MEC server 604-1 (amount used) will deliver the results together.

The operation of MEC devices that distribute computing tasks such as image stitching may be as described below.

7 is a diagram illustrating an operation of an MEC device for distributing computing tasks applied in the present disclosure.

Referring to FIG. 7 , the operation may be performed by each of a plurality of MEC devices included in the communication network of FIG. 6 . The first MEC device may divide the task performed to provide the IoV application service into subtask units. In addition, the first MEC device may divide and allocate the divided subtasks to a plurality of MEC devices. Here, the task may be an image stitching task for merging images obtained from vehicle communication nodes, and the subtask may be a task of stitching some images among images obtained from vehicle communication nodes.

In step S701 , the first MEC device may exchange parameters for calculating a cost due to subtask allocation with the second MEC device and the third MEC device. For example, the second MEC device and the third MEC device may transmit a parameter for calculating the cost to the first MEC device. When the first MEC device secures parameters for calculating the cost of the second MEC device and the third MEC device in advance, step S701 may be omitted. Here, the parameter for calculating the cost may include information about the computing performance of each of the MEC devices, the effective computing resource, the cost of the effective computing resource, and the like.

In step S703 , the first MEC device may calculate a cost incurred by distributing subtasks. The first MEC device may calculate a cost for outsourcing the subtask to adjacent MECs according to the number and how many idle resources it can provide to the assigned subtask. The first MEC device may calculate the cost based on Equation (1).

In Equation 1, p _m,b,T indicates a unit cost of resources consumed to transmit data for a subtask through the X2 link. p _m,b,C indicates the price of computing resources consumed in units of subtasks. And, p _{m, b, R} represents a price per data size received as a result of performing the subtask. ts _m is the working data size, DR _m is the required computing resource, and θ _m is the scale factor.

Here, the price unit for the task transmitted through the X2 link is expressed as a constant. MEC devices may set different prices per computing resources consumed for computing computing services. For example, an MEC device with more idle computing resources may offer a higher computational rate (computational throughput) and thus may be priced higher.

Therefore, p _m,b,C can be expressed as in Equation (2).

In Equation 2, K _b may indicate the operation rate (or operation throughput) of the adjacent MEC device b. and η _b indicates the adjustment coefficient. According to Equation 2, Equation 1 is modified to Equation 3.

And, as a result of the outsourcing of the subtask, the consumption time may be expressed as Equation (4).

In Equation 4, t _m,b,T indicates a transmission time for sending data for executing subtasks from the requested MEC device to the adjacent MEC b, and t _m,b,C is the It indicates an operation time, and t _{m, b, R} may indicate a time for receiving execution result data of subtasks. And, r indicates the transmission rate of the X2 link between the MEC servers in the same cooperative space. Given B means the number of MEC devices close to the cooperating group, the requested MEC device cost should be paid for outsourced computing work according to the cost calculated according to Equation 5.

In Equation 5, yb indicates the number of subtasks distributed to the adjacent MEC b, and p _m,b indicates a value calculated by Equation 3

In step S705 , the first MEC device may allocate subtasks to a plurality of MEC devices. The first MEC device transmits images acquired from the vehicle communication nodes to the second MEC device and the third MEC device, and requests to perform a subtask, thereby performing a stitching operation of the images to the second MEC device and the third MEC device. can be dispersed.

In step S707-1, the second MEC device may perform the subtask allocated from the first MEC device, and in operation S707-2, the third MEC device may perform the subtask allocated from the first MEC device . For example, in step S707-1, the second MEC device may stitch images received from the first server, and in step S707-2, the third MEC device may stitch images received from the first MEC device. can

Meanwhile, the total of execution times of the image stitching operation performed by the MEC devices is the sum of the required times of each of the outsourcing tasks satisfying the delay time T _m required for the image stitching operation m. An MEC that accepts the image stitching assignment request but does not meet the required latency may cancel the processing of the assigned subtask and pay a penalty fee for violating the service level agreement (SLA).

In step S709 , the first MEC device may receive data as a result of performing a subtask by the second MEC device and data as a result of performing a subtask by the third MEC device. For example, the first MEC device may receive the image stitched by each of the second MEC device and the third MEC device.

In step S711 , the first MEC device may perform a task based on data received from a plurality of MEC devices. For example, the first MEC device may generate the road reconstruction map by stitching the image stitched by the second MEC device and the image stitched by the third MEC device.

The invention of the present disclosure can be applied in a distributed cloud environment such as an MEC environment, multi-cloud, and edge cloud, and is sensitive to service delay time and required to provide services in autonomous driving and artificial intelligence fields that require high-performance computing resources It can be used in various ways because it can satisfy the delay time condition.

According to the present disclosure, it is possible to improve resource use efficiency of the MEC network, and by improving quality of service (QoS), revenues of mobile network operators and MEC service providers can be increased.

Example methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, other steps may be included in addition to the illustrated steps, other steps may be excluded from some steps, or additional other steps may be included except some steps.

Various embodiments of the present disclosure do not list all possible combinations, but are intended to describe representative aspects of the present disclosure, and matters described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure includes software or machine-executable instructions (eg, operating system, application, firmware, program, etc.) that cause an operation according to the method of various embodiments to be executed on a device or computer, and such software or and non-transitory computer-readable media in which instructions and the like are stored and executable on a device or computer.

Claims (16)

A method of operating a first MEC device that provides an Internet of vehicle (IoV) application service, comprising:
receiving meta data of an image captured by each of the vehicle communication nodes from vehicle communication nodes;
determining image data to be requested from at least some of the vehicle communication nodes among the vehicle communication nodes based on the received metadata of the image;
transmitting information about the requested images to the at least some of the vehicle communication nodes;
receiving the requested images from the at least some vehicle communication nodes; and
storing the images received from the at least some vehicle communication nodes in a database,
The step of determining the image data to be requested from the at least some of the vehicle communication nodes,
comparing an area of images previously stored in the database with an area indicated by the received metadata of the image;
determining an overlapping area overlapping the pre-stored image area from among areas indicated by the metadata of the received image based on the comparison result; and
and excluding the overlapping area from among the areas indicated by the metadata of the received image. The method according to claim 1,
The metadata of the image is
Including location information of a location in which the image was captured, information about a time taken for the image, and information about a depth of field (DOF), the operating method. The method according to claim 1,
The image data to be requested is,
The method of any one of the preceding claims, wherein the overlapping area is excluded from the image. 4. The method according to claim 3,
calculating an overlap, which is a ratio of an overlapping area between each area of the meta data and the pre-stored images; and
determining meta data having the highest degree of overlap among the meta data;
The image data to be requested is,
and an image from which the overlapping region is excluded from among the images corresponding to the determined metadata having the highest degree of overlap. The method according to claim 1,
calculating a validity parameter of each of the images stored in the database based on timestamp information of each of the images stored in the database;
checking the validity of each of the images stored in the database based on the validity parameter; and
and deleting at least some images whose validity has not been confirmed from among the images stored in the database. The method according to claim 1,
receiving parameters for dividing and allocating a task from a plurality of MEC devices other than the first MEC device;
calculating a cost according to assignment of a subtask that is a part of the task based on the parameters;
allocating subtasks to the plurality of MEC devices;
receiving the subtask performance result data from each of the plurality of MEC devices; and
and performing the task based on data as a result of performing the subtask. 7. The method of claim 6,
The task is
It is an operation of stitching images stored in the database,
The subtask is
An operation of stitching some images from among the images stored in the database. 7. The method of claim 6,
The parameters are
Regarding the unit cost of resources consumed due to data transmission for allocating the subtask, the price per unit computing resource consumed due to the execution of the subtask, and the unit cost of resources consumed due to the transmission of data as a result of performing the subtask A method comprising information. In the first MEC providing an Internet of vehicle (IoV) application service,
transceiver; and
a processor coupled to the transceiver, the processor comprising:
receive meta data of an image captured by each of the vehicle communication nodes from vehicle communication nodes;
determining image data to be requested from at least some of the vehicle communication nodes among the vehicle communication nodes based on the received metadata of the image;
transmitting information about the images to be requested to the at least some of the vehicle communication nodes;
receive the requested images from the at least some vehicular communication nodes; And
storing the images received from the at least some vehicle communication nodes in a database,
the processor is
In determining the image data to be requested from the at least some of the vehicle communication nodes,
comparing an area of images previously stored in the database with an area indicated by the received metadata of the image;
determining an overlapping area overlapping the pre-stored areas of the images from among areas indicated by the metadata of the received images based on the comparison result; And
The first MEC device, wherein the overlapping area is excluded from the area indicated by the metadata of the received image. 10. The method of claim 9,
The metadata of the image is
The first MEC device, including information about the location information of the photographed place of the image, photographed time information, and depth of field (DOF). 10. The method of claim 9,
The image data to be requested is,
The first MEC device, which is an image in which the overlapping region is excluded from among the images. 12. The method of claim 11,
The processor is
calculating an overlapping degree, which is a ratio of an overlapping area between each area of the meta data and the pre-stored images; And
determining meta data having the highest degree of overlap among the meta data;
The image data to be requested is,
The first MEC device, wherein the overlapping region is excluded from the image corresponding to the determined metadata having the highest degree of overlap. 10. The method of claim 9,
The processor is
calculating a validity parameter of each of the images stored in the database based on timestamp information of each of the images stored in the database;
check the validity of each of the images stored in the database based on the validity parameter; And
Among the images stored in the database, the first MEC device for deleting at least some images that are not validated. 10. The method of claim 9,
The processor is
receive parameters for dividing and allocating a task from a plurality of MEC devices other than the first MEC device;
calculating a cost for allocating a subtask that is part of the task based on the parameters;
allocating a subtask to the plurality of MEC devices;
receive the subtask performance result data from each of the plurality of MEC devices; And
A first MEC device for performing the task based on the subtask performance result data. 15. The method of claim 14,
The task is
It is an operation of stitching images stored in the database,
The subtask is
The first MEC device, which is a task of stitching some images among the images stored in the database. 15. The method of claim 14,
The parameters are
Regarding the unit cost of resources consumed due to data transmission for allocating the subtask, the price per unit computing resource consumed due to the execution of the subtask, and the unit cost of resources consumed due to the transmission of data as a result of performing the subtask A first MEC device comprising information.

Images