JP7317901B2 - Information processing device, information processing method and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

A technique is disclosed for sending teacher data to a server device for causing the server device to perform machine learning.

Japanese Patent No. 6880290

Machine learning is generally performed by a server device, as in the conventional technology described above. However, if the data contains information that can identify individual behavior, etc., it may not be preferable to send the data to an external server device such as a company. In addition, since there are various restrictions on the transmission and reception of data containing information that can identify an individual's behavior, etc., it may not be easy to freely transmit and receive data. Therefore, when performing machine learning, it is preferable to be able to determine whether or not to send data according to data and circumstances.

The present application has been made in view of the above, and an object of the present application is to determine whether or not to send data according to data and circumstances.

An information processing apparatus according to the present application includes an acquisition unit that acquires data including information that can identify an individual's behavior, such as position information and sensor data, and a context that serves as a criterion for determining whether to send the data to the outside. a determination unit for determining whether or not to send the data to the server side; a transmission unit for sending the data to the server side when it is determined to send the data to the server; A receiving unit that receives the result of machine learning with a global model using data, and a learning unit that performs machine learning with a local model using the data when it is determined not to send the data to the server side. It is characterized by

According to one aspect of the embodiment, it is possible to determine whether or not to send the data depending on the data and the situation.

FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to the embodiment; FIG. 3 is a diagram illustrating a configuration example of a terminal device according to the embodiment; FIG. 4 is a diagram illustrating a configuration example of an information providing apparatus according to the embodiment; FIG. 5 is a diagram showing an example of a user information database. FIG. 6 is a diagram showing an example of the history information database. FIG. 7 is a diagram showing an example of a recommendation information database. FIG. 8 is a flow chart showing a processing procedure according to the embodiment. FIG. 9 is a diagram illustrating an example of a hardware configuration;

Embodiments for implementing an information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail below with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. Also, in the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[1. Outline of information processing method]
First, an outline of an information processing method performed by an information processing apparatus according to an embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. Note that in FIG. 1, a case in which it is determined whether or not to send data according to the data and the situation will be described as an example.

As shown in FIG. 1 , the information processing system 1 includes a terminal device 10 and an information providing device 100 . The terminal device 10 and the information providing device 100 are communicably connected to each other by wire or wirelessly via a network N (see FIG. 2). In this embodiment, the terminal device 10 cooperates with the information providing device 100 .

The terminal device 10 is a smart device such as a smartphone or tablet used by a user U (user), and communicates with an arbitrary server device via a wireless communication network such as 4G (Generation) or LTE (Long Term Evolution). It is a portable terminal device capable of performing The terminal device 10 has a screen such as a liquid crystal display and has a touch panel function, and the user U can perform a tap operation, a slide operation, a scroll operation, or the like with a finger, a stylus, or the like, and display data such as contents. Accepts various operations for . An operation performed on an area where content is displayed on the screen may be an operation on the content. In addition, the terminal device 10 may be an information processing device such as a desktop PC (Personal Computer) or a notebook PC as well as a smart device.

The information providing device 100 cooperates with the terminal device 10 of each user U, and provides the terminal device 10 of each user U with API (Application Programming Interface) services for various applications (hereinafter referred to as apps) and various It is an information processing device that provides data, and is realized by a server device, a cloud system, or the like.

Also, the information providing device 100 may be an information processing device that provides some web service online to the terminal device 10 of each user U. FIG. For example, the information providing apparatus 100 provides Web services such as Internet connection, search service, SNS (Social Networking Service), electronic commerce, electronic payment, online game, online banking, online trading, accommodation/ticket reservation, video/music distribution, Services such as news, maps, route search, route guidance, route information, operation information, and weather forecast may be provided. In practice, the information providing apparatus 100 may cooperate with various servers that provide web services as described above, mediate web services, or take charge of web service processing.

Note that the information providing device 100 can acquire user information about the user U. FIG. For example, the information providing device 100 acquires information about attributes of the user U, such as the user's U gender, age, and area of residence. The information providing apparatus 100 stores and manages identification information indicating the user U (user ID, etc.) and information related to the attributes of the user U. FIG.

Further, the information providing apparatus 100 acquires various types of history information (log data) indicating actions of the user U from the terminal apparatus 10 of the user U or from various servers based on the user ID or the like. For example, the information providing device 100 acquires a location history, which is a history of the user U's location and date and time, from the terminal device 10 . The information providing apparatus 100 also acquires a search history, which is a history of search queries input by the user U, from a search server (search engine). Further, the information providing apparatus 100 acquires a viewing history, which is a history of content viewed by the user U, from the content server. The information providing apparatus 100 also acquires a purchase history (payment history), which is a history of product purchases and payment processing of the user U, from an electronic commerce server or a payment processing server. Further, the information providing apparatus 100 may acquire the exhibition history and sales history, which are the history of the user U's exhibition in the marketplace, from the electronic commerce server or the settlement server. In addition, the information providing apparatus 100 acquires the posting history, which is the posting history of the user U, from a posting server that provides a word-of-mouth posting service or an SNS server.

[1-1. Processing Flow of Information Processing Method]
As shown in FIG. 1, the terminal device 10 of the user U acquires data measured or recorded by the functions of the terminal device 10 (step S1). For example, the terminal device 10 acquires, as data, location information of the terminal device 10, sensor data, attribute information of the user U, history information (operation history, payment history, etc.) and the like. The terminal device 10 acquires detailed position information with a first precision (for example, 10-meter precision, 10-meter mesh) using a positioning method such as GPS (Global Positioning System) or RTK (Real Time Kinematic). In addition, the terminal device 10 may acquire sensor data measured by each sensor mounted on or connected to the terminal.

In this embodiment, when the terminal device 10 of the user U acquires data such as position information and sensor data, based on a predetermined rule, the local model (local model) of the terminal side (terminal device 10) , and a global model (overall model) on the server side (information providing apparatus 100) to perform machine learning (step S2). That is, it determines which of the local model and the global model is to be updated. The local model and the global model have different accuracies. For example, the terminal device 10 of the user U determines which of the server-side global model and the terminal-side local model to learn based on the acquired data and context.

Subsequently, when the terminal device 10 of the user U determines that machine learning is to be performed in the global model on the server side (information providing device 100), based on a predetermined rule, the terminal device 10 to be transmitted to the server side (information providing device 100). It is determined whether or not the data should be abstracted and sent (step S3). For example, the terminal device 10 of the user U determines whether to send the data to be sent to the server side (the information providing device 100) in the raw data state or abstracted from the acquired data and context.

<context>
In this embodiment, the terminal device 10 of the user U determines from the context whether to learn on-device (terminal side) or online (server side). That is, the terminal device 10 of the user U determines whether or not to send the acquired data based on the context. The terminal device 10 of the user U also determines whether to send the data to be sent to the server side (the information providing device 100) in a raw data state or in an abstracted state based on the context. Examples of contexts that serve as criteria for determination include the following situations.

(1) Data privacy condition (surrounding congestion condition)
The terminal device 10 of the user U may transmit data such as location information without abstraction when the user U is in a crowded area such as a train station or downtown. That is, the terminal device 10 does not abstract data when there are many unspecified people in the same place and a large number of the same data (same location information, etc.) are collected, or when it is difficult to identify an individual. You can send In this case, the terminal device 10 of the user U determines to learn the data online (on the server side).

In addition, when the user U's terminal device 10 is in a situation where there are no people around the user U, such as at home or in a quiet place, for example, the position information is stored with 1-kilometer accuracy (for example, 1-km mesh unit) or It is abstracted and sent to a regional unit (area unit). That is, the terminal device 10 may abstract and send data when there is no other person in the same place and it is easy to identify the individual. In this case, the terminal device 10 of the user U determines to learn the data on-device (terminal side). Also, the terminal device 10 determines to learn the abstracted data (abstracted data) online (on the server side). In practice, the terminal device 10 may not send data to the server side regardless of whether or not abstraction is implemented.

(2) Remaining battery level (balance with battery consumption)
In many terminal devices, when machine learning is performed on-device, the battery (battery, etc.) remaining amount is consumed rapidly. Therefore, when the remaining battery (remaining battery power) of the terminal device 10 is low (for example, when the remaining battery power is less than a predetermined threshold), the terminal device 10 of the user U can be online (on the server side). Decide to learn. Conversely, when the terminal device 10 has a large remaining battery (remaining battery power) (for example, when the remaining battery power is equal to or greater than a predetermined threshold), the terminal device 10 performs machine learning on the device (terminal side). decide to do so.

(3) Data Acquisition Permission and Tacit Approval The user U's terminal device 10 implicitly acquires (for example, acquires in the background) data such as the location information of the user U so that the user U does not know. It is determined whether to learn on-device (terminal side) or online (server side) depending on whether it is explicitly acquired (for example, acquired by a navigation application) so that user U can understand it. do.

For example, when the user U's data is implicitly acquired, the terminal device 10 of the user U abstracts the data and sends the data to the server side. In this case, the terminal device 10 determines to learn the abstracted data (abstracted data) online (on the server side). Also, the terminal device 10 of the user U determines to learn the raw data on-device (terminal side). In this case, prioritize on-device machine learning.

Further, when the user U's data is explicitly acquired, the terminal device 10 of the user U sends the raw data to the server side. In this case, the terminal device 10 of the user U determines to learn the raw data online (on the server side). At this time, the terminal device 10 of the user U may determine to learn the raw data on-device (terminal side) as well, or may determine not to learn the raw data on-device (terminal side). In other words, prioritize online machine learning.

Note that the terminal device 10 of the user U may match or synchronize the local model and the global model with respect to learning timing. For example, when the terminal device 10 of the user U has not updated the global model on the server side when the data is acquired, updating of the local model on the terminal side may be restricted or prohibited. Alternatively, the terminal device 10 of the user U may not update the model itself when the context on the terminal side does not change.

<Online (server side) machine learning>
Subsequently, when the terminal device 10 determines to learn online (on the server side), the terminal device 10 determines whether the data to be transmitted is to be abstracted and transmitted, in raw data state or abstracted data. converted data to the information providing apparatus 100 (step S4). At this time, when the terminal device 10 determines that the data to be transmitted is to be transmitted in the raw data state, the terminal device 10 transmits the acquired data in the raw data state to the information providing device 100 . Further, when the terminal device 10 determines that the data to be transmitted is to be abstracted and transmitted, the terminal device 10 transmits abstracted data obtained by abstracting the acquired data to the information providing device 100 .

For example, the terminal device 10 uses a second precision (for example, 1-kilometer precision, 1-kilometer precision, 1 km mesh) is transmitted to the information providing apparatus 100 . The coarse position information with the second accuracy may be information (region name, address, etc.) in units of regions (areas) such as prefectures and municipalities, or information in units of regional meshes (regional mesh code etc.). Further, the terminal device 10 may perform fraction processing (rounding processing) when generating rough position information with the second precision from detailed position information with the first precision.

Moreover, it is possible within the scope of common technical knowledge to encrypt and transmit transmission data (raw data or abstract data). Alternatively, tunneling may be used to establish a closed virtual direct line between the terminal side and the server side. For example, a VPN (Virtual Private Network) may be constructed between the terminal side and the server side.

At this time, the terminal device 10 may transmit the transmission data in the form of vector data obtained by converting (vectoring) the transmission data into a vector. Vector data is irreversible information, and the original data cannot be restored from the vector values of the vector data. For example, the terminal device 10 may perform so-called embedding to generate an embedding vector based on transmission data.

Subsequently, the information providing device 100 receives transmission data (raw data or abstract data) from the terminal device 10, performs machine learning based on the received data, and constructs a global model (step S5). For example, the information providing device 100 and the terminal device 10 perform grouping and patterning of received data by machine learning. Machine learning is also performed using the received data and the current and future positions and behaviors of the user U as learning data. Alternatively, the information providing device 100 performs machine learning using the received data and the provided information to be provided to the user U as learning data. The provided information includes recommendation information such as advertisements and coupons for nearby stores.

Subsequently, the information providing device 100 inputs the received data into the global model, makes an inference, and obtains an inference result (step S6). For example, the information providing apparatus 100 estimates the current or future position and behavior of the user U, or estimates recommended information (provided information) to be provided to the user U, based on the received data.

Subsequently, the information providing device 100 provides information based on the inference result to the terminal device 10 of the user U (step S7). In this embodiment, the information providing device 100 provides recommendation information to the terminal device 10 of the user U based on the inference result. For example, the information providing device 100 provides information on nearby stores, distributes coupons, and the like.

At this time, the information providing apparatus 100 may use a global model when determining this provided information. For example, the information providing apparatus 100 provides a plurality of pieces of recommended information that are estimated to have a viewing accuracy that exceeds a predetermined threshold based on the global model and the received data.

Note that the recommendation information is only an example. Actually, it may be information corresponding to or related to the position information, or various information based on inference from the position information. For example, it may be information related to user behavior or the like based on inference from position information.

<Machine learning on-device (terminal side)>
On the other hand, when the terminal device 10 of the user U determines to learn on-device (terminal side), it performs machine learning based on the raw data and builds a local model (step S8). That is, the terminal device 10 performs on-device machine learning. For example, the terminal device 10 performs grouping and patterning of raw data by machine learning. In addition, the terminal device 10 performs machine learning using the raw data and the current and future positions and behaviors of the user U as learning data. At this time, the terminal device 10 may perform machine learning by combining raw data and sensor data collected by a sensor mounted (or connected) to the terminal device 10 .

Subsequently, the terminal device 10 of the user U inputs the raw data into the local model, makes an inference, and obtains an inference result (step S9). For example, the terminal device 10 estimates the current or future position and behavior of the user U based on the raw data. At this time, the terminal device 10 may make an inference by combining the position information and sensor data collected by a sensor mounted (or connected) to the terminal device 10 as raw data.

For example, the terminal device 10 combines position information and sensor data as raw data, and even when the position information is the same (when the user is in a high-rise building, a large facility, etc.), the light sensor data is bright. If the illuminance is high, it may be assumed that the user is in an "entrance or aisle", and if the light sensor data is dark (low illuminance), it may be assumed that the user is in a "restaurant". Alternatively, if the air pressure sensor data is high (high air pressure), it is assumed to be on a "low floor", and if the air pressure sensor data is low (low air pressure), it is assumed to be on a "high floor". can be inferred.

In addition to the sensor data, the terminal device 10 simply detects whether the terminal device 10 is being touched (contact/operation), which application is being used (application activation), and whether it is being charged (battery level). You may learn and infer by combining information, such as. These pieces of information can be easily obtained, and are appropriate information for achieving high accuracy of the user's location information. That is, the terminal device 10 may perform learning and inference by combining the position information of the terminal device 10 and the state information/operation information of the terminal device 10 . Of course, the terminal device 10 may perform learning and inference by combining this information and sensor data.

Subsequently, the terminal device 10 of the user U determines information (presentation information) to be presented to the user U based on the inference result using the raw data (step S10). For example, the terminal device 10 of the user U narrows down a plurality of pieces of recommended information (provided information) received from the information providing device 100 based on the inference result using the raw data, and presents information (presented information ). That is, the terminal device 10 controls information (presentation information) to be presented to the user U based on a plurality of pieces of recommended information received from the information providing device 100 and raw data.

At this time, the terminal device 10 may use the local model when determining information (presentation information) to be presented to the user U. FIG. For example, the terminal device 10 displays only the recommended information of which the viewing accuracy exceeds a predetermined threshold for the local model and the raw data, among a plurality of pieces of recommended information provided from the information providing device 100 .

<Re-learning online (on the server side)>
In addition, the terminal device 10 receives a plurality of pieces of provided information (recommended information) in which the server-side global model is selected with 100-meter accuracy (for example, 100-m mesh units), and the terminal-side local model for these pieces of provided information. Scoring is performed, and one of a plurality of pieces of provided information is output as presentation information.

Then, if none of the provided information is output as presentation information as a result of scoring with the terminal-side local model, the terminal device 10 transmits data for re-learning with the server-side global model, Tell it to retrain on the server-side global model. As a result, the information providing apparatus 100 on the server side re-learns the global model. The data sent to the server side may be raw data or abstracted data (abstracted data).

On the other hand, when the terminal device 10 on the terminal side displays any of the plurality of pieces of provided information selected in the global model on the server side, the displayed provided information (presented information) is If the user does not click/tap (the percentage of clicks/tap is low), the local model is re-learned using the data collected in the device itself.

Here, in the above explanation, the learning target is determined based on the presence or absence and ratio of clicks/tappings on the provided information (presented information) displayed from among the multiple provided information selected by the global model, but other than the above Various patterns are also assumed.

For example, the terminal device 10 of the user U determines to update the global model when a predetermined period of time has passed, when a predetermined number of pieces of data have been collected, or when there is a purchase history of a predetermined amount or more. do.

In addition, the terminal device 10 of the user U changes when the current position of the user U changes, or when the user U moves more than a predetermined threshold within a predetermined period (during travel, business trip, commuting to work, etc.). determines to update the local model.

<Subject of Judgment>
In the above embodiment, the terminal side or the server side may determine whether or not to send the data to the server side. For example, the following procedure may be used for determination. The data sent to the server side may be raw data or abstracted data (abstracted data).

(Determination on the terminal side)
When acquiring data, the terminal device 10 of the user U judges, based on various criteria, whether to send the data to the server side or to process the data only on the terminal side without sending it to the server side. For example, as described above, the criterion is the remaining battery level, the expected battery consumption of the task to be performed, the degree of privacy of the data, and other contexts. The degree of privacy of data refers to the state of data acquisition, such as whether the data was explicitly acquired in a crowded public facility, or whether it was implicitly/secretly acquired in a situation where no one was around. show.

(Decision on the server side)
When acquiring data, the terminal device 10 of the user U inquires of the information providing device 100 (on the server side) whether to send the data to the server side. At this time, the terminal device 10 may transmit only one item (or several items) of data to the server side as a trial and inquire whether subsequent (future) transmissions are desired.

In response to an inquiry from the terminal device 10, the information providing device 100 (server side) determines whether or not to send the data to the server side. For example, as a criterion for determination, the information providing apparatus 100 (server side) determines if the data is rare, if the data is valuable in the area, or if data has recently been sent from the user or the area. If the data has not been received, the terminal device 10 determines to send the data to the server side. Then, the information providing device 100 (server side) replies to the terminal device 10 of the user U as to whether or not the user desires to transmit data.

The terminal device 10 of the user U receives the response from the information providing device 100 (server side), and the information providing device 100 (server side) confirms whether or not there is a desire to transmit data. Then, when the information providing apparatus 100 (on the server side) desires to transmit data, the terminal apparatus 10 considers the context and transmits the data if there is no problem.

As described above, in this embodiment, when a user's terminal device acquires data such as location information and sensor data, the global model on the server side and the local model on the terminal side are generated according to the content and context of the data. Decide whether to learn with the model or not. At this time, which model is to be updated may be set on a rule basis according to the type of user context that has changed.

[2. Configuration example of information processing system]
Next, the configuration of the information processing system 1 including the information providing device 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a terminal device 10 and an information providing device 100. As shown in FIG. These various devices are communicatively connected via a network N by wire or wirelessly. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Also, the number of devices included in the information processing system 1 shown in FIG. 2 is not limited to the illustrated one. For example, in FIG. 2, only one terminal device 10 is shown for simplification of illustration, but this is only an example and is not limited, and two or more devices may be provided.

The terminal device 10 is an information processing device used by the user U. FIG. For example, the terminal device 10 includes smart devices such as smartphones and tablet terminals, feature phones, PCs (Personal Computers), PDAs (Personal Digital Assistants), game machines and AV equipment with communication functions, car navigation systems, smart watches, These include wearable devices such as head-mounted displays, smart glasses, smart speakers, and cameras. However, in practice, it is not limited to these examples.

In addition, the terminal device 10 is compatible with wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation: fifth generation mobile communication system), Bluetooth (registered trademark), wireless LAN (Local It is possible to communicate with the information providing apparatus 100 by connecting to the network N via short-range wireless communication such as Area Network).

The information providing device 100 is, for example, a PC, a server device, a mainframe, a workstation, or the like. Note that the information providing apparatus 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be described using FIG. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. As shown in FIG. 3, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40. Prepare.

(Communication unit 11)
The communication unit 11 is connected to the network N (see FIG. 2) by wire or wirelessly, and transmits and receives information to and from the information providing apparatus 100 via the network N. FIG. For example, the communication unit 11 is implemented by a NIC (Network Interface Card), an antenna, or the like.

(Display unit 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic EL display (Organic Electro-Luminescent Display). Also, the display unit 12 is a touch panel display, but is not limited to this.

(Input unit 13)
The input unit 13 is an input device that receives various operations from the user U. For example, the input unit 13 has buttons and the like for inputting characters, numbers, and the like. The input unit 13 may be an input/output port (I/O port), a USB (Universal Serial Bus) port, or the like. Moreover, when the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13 . Also, the input unit 13 may be a microphone or the like that receives voice input from the user U. FIG. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives signals (radio waves) transmitted from GPS (Global Positioning System) satellites, and based on the received signals, position information (for example, latitude and longitude). That is, the positioning unit 14 positions the position of the terminal device 10 . GPS is merely an example of GNSS (Global Navigation Satellite System).

Also, the positioning unit 14 can measure the position by various methods other than GPS. For example, the positioning unit 14 may measure the position using various communication functions of the terminal device 10 as described below as auxiliary positioning means for position correction and the like.

(Wi-Fi positioning)
For example, the positioning unit 14 measures the position of the terminal device 10 using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 performs Wi-Fi communication or the like and measures the position of the terminal device 10 by measuring the distance to a nearby base station or access point.

(beacon positioning)
The positioning unit 14 may also use the Bluetooth (registered trademark) function of the terminal device 10 to measure the position. For example, the positioning unit 14 positions the position of the terminal device 10 by connecting with a beacon transmitter connected by the Bluetooth (registered trademark) function.

(geomagnetic positioning)
Further, the positioning unit 14 positions the position of the terminal device 10 based on the geomagnetism pattern of the structure measured in advance and the geomagnetic sensor provided in the terminal device 10 .

(RFID positioning)
Further, for example, if the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used at station ticket gates, shops, etc., or has a function of reading an RFID tag In this case, the location used is recorded together with the information that the payment was made by the terminal device 10 . The positioning unit 14 may measure the position of the terminal device 10 by acquiring such information. Also, the position may be measured by an optical sensor provided in the terminal device 10, an infrared sensor, or the like.

The positioning unit 14 may measure the position of the terminal device 10 using one or a combination of the positioning means described above, if necessary.

(Sensor unit 20)
The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10 . The connection may be wired connection or wireless connection. For example, the sensors may be detection devices other than the terminal device 10, such as wearable devices and wireless devices. In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, an atmospheric pressure sensor 23, an air temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( camera) 28.

The sensors 21 to 28 described above are only examples and are not limited. That is, the sensor unit 20 may be configured to include a part of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of the sensors 21 to 28. .

The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects physical movements of the terminal device 10 such as movement direction, speed, and acceleration of the terminal device 10 . The gyro sensor 22 detects physical movements of the terminal device 10 such as inclination in three axial directions based on the angular velocity of the terminal device 10 and the like. The atmospheric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

Since the terminal device 10 includes the above-described acceleration sensor 21, gyro sensor 22, barometric pressure sensor 23, etc., techniques such as pedestrian dead-reckoning (PDR: Pedestrian Dead-Reckoning) using these sensors 21 to 23, etc. , the position of the terminal device 10 can be determined. This makes it possible to acquire indoor position information that is difficult to acquire with a positioning system such as GPS.

For example, a pedometer using the acceleration sensor 21 can calculate the number of steps, walking speed, and distance walked. Further, by using the gyro sensor 22, it is possible to know the traveling direction, the direction of the line of sight, and the inclination of the body of the user U. Also, from the atmospheric pressure detected by the atmospheric pressure sensor 23, the altitude at which the terminal device 10 of the user U is present and the number of floors can be known.

The temperature sensor 24 detects the temperature around the terminal device 10, for example. The sound sensor 25 detects sounds around the terminal device 10, for example. The optical sensor 26 detects the illuminance around the terminal device 10 . The magnetic sensor 27 detects, for example, geomagnetism around the terminal device 10 . The image sensor 28 captures an image around the terminal device 10 .

The atmospheric pressure sensor 23, the temperature sensor 24, the sound sensor 25, the optical sensor 26, and the image sensor 28 described above detect the atmospheric pressure, temperature, sound, and illuminance, respectively, or capture an image of the surroundings to detect the terminal device 10. It is possible to detect the surrounding environment and situations. In addition, it is possible to improve the accuracy of the location information of the terminal device 10 based on the surrounding environment and situation of the terminal device 10 .

(control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input/output port, and various circuits. Also, the control unit 30 may be configured by hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a transmission unit 31 , a reception unit 32 , a processing unit 33 , an acquisition unit 34 , a determination unit 35 , a conversion unit 36 and a learning unit 37 .

(Sending unit 31)
The transmission unit 31 receives, for example, various information input by the user U using the input unit 13, various information detected by the sensors 21 to 28 mounted on or connected to the terminal device 10, and information measured by the positioning unit 14. The position information of the terminal device 10 and the like can be transmitted to the information providing device 100 via the communication unit 11 .

(Receiver 32)
The receiving unit 32 can receive various information provided by the information providing apparatus 100 and requests for various information from the information providing apparatus 100 via the communication unit 11 .

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10 including the display unit 12 and the like. For example, the processing unit 33 can output various types of information transmitted by the transmitting unit 31 and various types of information received by the receiving unit 32 from the information providing apparatus 100 to the display unit 12 for display. Note that the processing unit 33 may include an acquisition unit 34, a determination unit 35, a conversion unit 36, and a learning unit 37 described below.

(Acquisition unit 34)
The acquisition unit 34 acquires data via the positioning unit 14 , the sensor unit 20 or the input unit 13 . Specifically, the acquisition unit 34 acquires the position information of the terminal device 10, sensor data, attribute information of the user U, history information (operation history, payment history, etc.) and the like as data. For example, the acquisition unit 34 acquires position information with a first precision (for example, 10-meter precision, 10-meter mesh) using a positioning method such as GPS (Global Positioning System) or RTK (Real Time Kinematic). The acquisition unit 34 also acquires sensor data measured by each sensor mounted on or connected to the terminal.

(Determination unit 35)
The determination unit 35 determines whether or not to send data to the information providing apparatus 100 (server side) according to the data and context. At this time, when the transmitting unit 31 determines to transmit the data to the information providing apparatus 100 (server side), it transmits the data to the information providing apparatus 100 (server side) via the communication unit 11 . The receiving unit 32 receives the result of machine learning in the global model using data from the information providing apparatus 100 (server side) via the communication unit 11 .

Further, when determining that the data is to be sent to the information providing apparatus 100 (server side), the determination unit 35 determines whether to abstract the data. At this time, when the transmission unit 31 determines to abstract the data, the transmission unit 31 transmits the abstracted data to the information providing apparatus 100 (server side) via the communication unit 11 . The receiving unit 32 receives the result of machine learning in the global model using the abstracted data from the information providing apparatus 100 (server side) via the communication unit 11 . Further, when the transmission unit 31 determines not to abstract the data, the transmission unit 31 transmits the raw data to the information providing apparatus 100 (server side) via the communication unit 11 . The receiving unit 32 receives the machine learning result of the global model using the raw data from the information providing apparatus 100 (server side) via the communication unit 11 .

Further, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) according to the degree of privacy of the data. For example, the determination unit 35 does not abstract the data when there are an unspecified number of people in the same place and a large number of the same data (same location information, etc.) are collected, or when it is not easy to identify an individual. , the data is sent to the information providing apparatus 100 (server side). In this case, the determination unit 35 determines to learn the data online (on the server side). In addition, when there is no other person in the same place and it is easy to identify the individual, the determination unit 35 may abstract the data and send it to the information providing device 100 (server side), Data may be determined to be learned on-device (terminal side).

Further, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) according to the remaining battery level of the terminal device that acquired the data. For example, when the remaining battery (remaining battery power) of the determining unit 35 is low (for example, when the remaining battery power is less than a predetermined threshold), the determination unit 35 performs machine learning online (on the server side). to judge. Conversely, when the remaining battery (remaining battery power) of the determining unit 35 is large (for example, when the remaining battery power is equal to or greater than a predetermined threshold), the determination unit 35 performs machine learning on the device (terminal side). decide to do so.

Further, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) depending on whether the acquisition of the data is explicit or implicit. Specifically, the determination unit 35 determines whether the acquisition of data such as the location information of the user U is implicitly acquired (for example, acquired in the background) so that the user U does not know, or whether the user U As you can see from the above, it is determined whether to learn on-device (terminal side) or online (server side) depending on whether it is explicitly acquired (for example, acquired by a navigation app).

For example, when the acquisition of user U data is implicit, the determination unit 35 abstracts the data and sends it to the server side. In this case, the determination unit 35 determines to learn the abstracted data (abstracted data) online (on the server side). Also, the determination unit 35 determines to learn the data on-device (terminal side). In this case, prioritize on-device machine learning.

Further, when the acquisition of the data of the user U is explicitly performed, the determination unit 35 sends the acquired data to the server side in the state of raw data. In this case, the determination unit 35 determines to learn the raw data online (on the server side). At this time, the determination unit 35 may determine that the raw data should be learned on the on-device (terminal side), or may be determined not to be learned on the on-device (terminal side). In other words, prioritize online machine learning.

Further, the determination unit 35 determines not to update the local model based on the data until the global model is updated based on the data. That is, the determination unit 35 may match or synchronize the learning timing between the local model and the global model. For example, if the server-side global model is not updated when the data is acquired, the determination unit 35 may restrict or prohibit updating of the terminal-side local model. Alternatively, the determination unit 35 may not update the model itself when the context on the terminal side does not change.

(Converter 36)
The conversion unit 36 abstracts data and converts it into abstracted data. At this time, when the determination unit 35 determines that the data should be abstracted, the conversion unit 36 abstracts the data and converts it into abstracted data.

Further, when the acquisition unit 34 acquires the position information as data and the determination unit 35 determines to abstract the data, the conversion unit 36 converts the position information from the position information indicating a specific point to the wide-area position information. . For example, the conversion unit 36 converts detailed position information with a first precision (eg, 10-meter precision, 10-m mesh), which is raw data, to a second precision (eg, 1-kilometer precision) that is lower than the first precision. , 1 km mesh). The coarse position information with the second accuracy may be information (region name, address, etc.) in units of regions (areas) such as prefectures and municipalities, or information in units of regional meshes (regional mesh code etc.). Further, the conversion unit 36 may perform fraction processing (rounding processing) when converting detailed position information with the first precision into coarse position information with the second precision.

(Learning part 37)
The learning unit 37 performs machine learning with a local model using raw data. That is, the learning unit 37 performs on-device machine learning. For example, when the learning unit 37 determines not to send data to the information providing apparatus 100 (server side), the learning unit 37 performs machine learning with a local model using raw data.

Also, the learning unit 37 determines information (presentation information) to be presented to the user U based on the inference result of the local model using the raw data. For example, the learning unit 37 determines recommended information such as advertisements and coupons for stores near the current position of the user U as information to be presented to the user U (presentation information). At this time, the transmission unit 31 may request recommendation information such as advertisements and coupons for stores near the current location of the user U via the communication unit 11 . Further, when the learning unit 37 receives a plurality of pieces of provided information (recommendation information, etc.) from the information providing device 100 (server side), the learning unit 37 learns the information providing device based on the inference result of the local model using raw data. A plurality of pieces of provided information (recommended information, etc.) received from 100 are narrowed down, and information (presented information) to be presented to the user U is determined.

(storage unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory device such as RAM (Random Access Memory) or flash memory, or a storage device such as HDD (Hard Disk Drive), SSD (Solid State Drive), or optical disk. be. Various programs, various data, and the like are stored in the storage unit 40 .

Note that the storage unit 40 may store various types of information depending on the purpose. For example, the storage unit 40 stores user U's demographic (demographic attribute), psychographic (psychological attribute), geographic (geographical attribute), behavioral (behavioral attribute), and other attributes. Information may be stored. For example, the storage unit 40 stores name, family structure, hometown (local), occupation, position, income, qualification, residence type (detached house, condominium, etc.), car availability, school/commuting time, school/commuting route, Commuter pass sections (stations, lines, etc.), frequently used stations (other than the nearest station to your home or place of work), lessons (places, time zones, etc.), hobbies, interests, lifestyle, etc. good. The storage unit 40 also stores a location history that is a history of the location and date and time of the user U, a search history that is a history of search queries input by the user U, a browsing history that is a history of content browsed by the user U, Purchase history (payment history), which is the history of user U's product purchases and payment processing; exhibition history and sales history, which is the history of user U's listings on the marketplace; posting history, which is the history of user U's postings etc. may be stored. In addition, the storage unit 40 may store a usage history of various applications installed in the terminal device 10 by the user U, a payment history of payment (electronic payment) using the terminal device 10 of the user U, and the like. . These pieces of information can be used as learning data along with location information and sensor data.

[4. Configuration example of information providing device]
Next, the configuration of the information providing apparatus 100 according to the embodiment will be described using FIG. FIG. 4 is a diagram showing a configuration example of the information providing device 100 according to the embodiment. As shown in FIG. 4, the information providing device 100 has a communication section 110, a storage section 120, and a control section .

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Also, the communication unit 110 is connected to the network N (see FIG. 2) by wire or wirelessly.

(storage unit 120)
The storage unit 120 is implemented by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or flash memory, or a storage device such as an HDD, SSD, or optical disk. As shown in FIG. 4 , the storage unit 120 has a user information database 121 , history information database 122 and recommendation information database 123 .

(User information database 121)
The user information database 121 stores user information about the user U. FIG. For example, the user information database 121 stores various information such as user U attributes. FIG. 5 is a diagram showing an example of the user information database 121. As shown in FIG. In the example shown in FIG. 5, the user information database 121 has items such as "user ID (Identifier)", "age", "gender", "home", "place of work", and "interest".

“User ID” indicates identification information for identifying the user U. The “user ID” may be the user U's contact information (telephone number, e-mail address, etc.), or may be identification information for identifying the user U's terminal device 10 .

"Age" indicates the age of the user U identified by the user ID. Note that the "age" may be information indicating a specific age of the user U (for example, 35 years old) or information indicating the age of the user U (for example, 30's). . Alternatively, the "age" may be information indicating the date of birth of the user U, or information indicating the generation of the user U (for example, born in the 80's). "Gender" indicates the gender of the user U identified by the user ID.

"Home" indicates location information of the home of the user U identified by the user ID. In the example shown in FIG. 5, "home" is represented by an abstract code such as "LC11", but may be latitude/longitude information or the like. Also, for example, "home" may be an area name or an address.

"Place of work" indicates location information of the place of work (school in the case of a student) of the user U identified by the user ID. In the example shown in FIG. 5, the "place of work" is illustrated as an abstract code such as "LC12", but may be latitude/longitude information or the like. Also, for example, the "place of work" may be an area name or an address.

"Interest" indicates the interest of the user U identified by the user ID. That is, "interest" indicates an object in which the user U identified by the user ID is highly interested. For example, the "interest" may be a search query (keyword) that the user U has entered into a search engine and searched for. In the example shown in FIG. 5, one "interest" is shown for each user U, but a plurality of "interests" may be shown.

For example, in the example shown in FIG. 5, the age of the user U identified by the user ID "U1" is "twenties" and the gender is "male". Also, for example, the user U identified by the user ID "U1" indicates that the home is "LC11". Also, for example, the user U identified by the user ID "U1" indicates that the place of work is "LC12". Also, for example, the user U identified by the user ID "U1" indicates that he is interested in "sports".

Here, in the example shown in FIG. 5, abstract values such as “U1”, “LC11” and “LC12” are used, but “U1”, “LC11” and “LC12” are concrete values. It is assumed that information such as character strings and numerical values is stored. Hereinafter, abstract values may also be illustrated in diagrams relating to other information.

The user information database 121 is not limited to the above, and may store various types of information depending on the purpose. For example, the user information database 121 may store various information about the terminal device 10 of the user U. FIG. In addition, the user information database 121 stores user U's demographics (demographic attributes), psychographics (psychological attributes), geographics (geographical attributes), behavioral attributes (behavioral attributes), etc. Information about attributes may be stored. For example, the user information database 121 includes name, family structure, hometown (local), occupation, position, income, qualification, residence type (detached house, condominium, etc.), presence or absence of car, commuting time, commuting time, commuting time. Information such as routes, commuter pass sections (stations, lines, etc.), frequently used stations (other than the nearest station to your home or place of work), lessons (places, time zones, etc.), hobbies, interests, lifestyle, etc. may

(History information database 122)
The history information database 122 stores various types of information related to history information (log data) indicating user U's actions. FIG. 6 is a diagram showing an example of the history information database 122. As shown in FIG. In the example shown in FIG. 6, the history information database 122 has items such as "user ID", "location history", "search history", "browsing history", "purchase history", and "posting history".

“User ID” indicates identification information for identifying the user U. "Position history" indicates a position history that is a history of the user's U position and movement. Also, "search history" indicates a search history that is a history of search queries input by the user U. FIG. "Browsing history" indicates a browsing history that is a history of contents browsed by the user U. FIG. "Purchase history" indicates the purchase history of the user U's purchases. In addition, “posting history” indicates a posting history that is a history of posts by the user U. FIG. In addition, the “posting history” may include questions about user U's property.

For example, in the example shown in FIG. 6, the user U identified by the user ID "U1" moves along the "location history #1", searches along the "search history #1", It indicates that the content was browsed according to the "Purchase history #1", the predetermined product etc. was purchased at the predetermined store etc. according to the "Purchase history #1", and the content was posted according to the "Posting history".

Here, in the example shown in FIG. 6, abstract history such as "U1", "location history #1", "search history #1", "browsing history #1", "purchase history #1", and "posting history #1" "U1", "location history #1", "search history #1", "browsing history #1", "purchase history #1" and "posting history #1" , information such as specific character strings and numerical values are stored.

Note that the history information database 122 may store various types of information, not limited to the above, depending on the purpose. For example, the history information database 122 may store the user U's usage history of a predetermined service. In addition, the history information database 122 may store the user U's store visit history, facility visit history, and the like. In addition, the history information database 122 may store a history of payment (electronic payment) using the terminal device 10 of the user U, and the like.

(Recommendation information database 123)
The recommendation information database 123 stores various types of information related to information provided to the user U (recommendation information). FIG. 7 is a diagram showing an example of the recommendation information database 123. As shown in FIG. In the example shown in FIG. 7, the recommendation information database 123 has items such as "area", "region", "facility", and "provided information".

(Recommendation information database 123)
The recommendation information database 123 stores various types of information related to information provided to the user U (recommendation information). FIG. 7 is a diagram showing an example of the recommendation information database 123. As shown in FIG. In the example shown in FIG. 7, the recommendation information database 123 has items such as "area", "region", "facility", and "provided information".

"Region" indicates a region corresponding to rough location information in the second precision, which is abstract data. The “region” may be a region (area) unit such as a prefecture or a municipality, or may be a regional mesh unit (regional mesh code, etc.). In addition, "area" indicates an area (district, etc.) included in the region. For example, the area indicates "Akasaka" and the area indicates "Akasakamitsuke". Further, the "facility" is a facility such as a store corresponding to the detailed location information with the first precision, which is the raw data, and indicates a facility such as a store existing within the area. For example, "facility" indicates "restaurant A" located in area "Akasakamitsuke". Further, "provided information" indicates recommendation information and various types of information related to facilities such as shops. For example, the provided information may be an advertisement, a coupon, or the like regarding a facility such as a store.

For example, in the example shown in FIG. 7, when the rough location information with the second accuracy, which is the abstract data, indicates the area "Akasaka", the information providing apparatus 100 determines that the facility "Restaurant A to the terminal device 10 of the user U. Further, when the detailed position information with the first accuracy, which is raw data, indicates the facility "restaurant A" located in the area "Akasakamitsuke", the information providing apparatus 100 provides the provided information "restaurant A" regarding the facility "restaurant A". This indicates that the terminal device 10 of the user U is provided with candidates for provision information related to each store, etc., including "coupons".

For example, the information providing device 100 provides the terminal device 10 of the user U with candidates for provision information related to stores, etc. located in the area "Akasaka" based on rough position information with second accuracy indicating the area "Akasaka." . At this time, if the information providing apparatus 100 can estimate that the user U is in the area "Akasaka-mitsuke" from the rough position information with the second accuracy indicating the area "Akasaka", the information providing apparatus 100 The terminal device 10 of the user U is provided with candidates for information to be provided regarding the store or the like located there.

It should be noted that the recommendation information database 123 may store various types of information, not limited to the above, depending on the purpose. For example, the recommendation information database 123 may store information related to conditions for providing recommendation information (for example, attributes or action history of the user U who is the target of provision) in addition to the region. In addition, the recommendation information database 123 may store information regarding the time period (for example, 17:00 to 20:00) during which the recommendation information is provided.

(control unit 130)
Returning to FIG. 4, the description is continued. The control unit 130 is a controller, and for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like controls the information providing apparatus 100. Various programs (corresponding to an example of an information processing program) stored in the internal storage device are executed by using a storage area such as a RAM as a work area. In the example shown in FIG. 4 , the control unit 130 has a collection unit 131 , an estimation unit 132 and a provision unit 133 .

(Collection unit 131)
The collection unit 131 collects raw data or abstract data from the terminal device 10 of each user U via the communication unit 110 .

Also, the collection unit 131 may acquire user information about each user U via the communication unit 110 . For example, the collection unit 131 acquires identification information (such as a user ID) indicating the user U, location information of the user U, attribute information of the user U, and the like from the terminal device 10 of the user U. FIG. Further, the collection unit 131 may acquire identification information indicating the user U, attribute information of the user U, and the like when the user U is registered as a user. The collection unit 131 then registers the user information in the user information database 121 of the storage unit 120 .

Also, the collection unit 131 may acquire various types of history information (log data) indicating behavior of each user U via the communication unit 110 . For example, the collection unit 131 acquires various types of history information indicating actions of the user U from the terminal device 10 of the user U or from various servers based on the user ID or the like. The collection unit 131 then registers various types of history information in the history information database 122 of the storage unit 120 .

(Estimation unit 132)
The estimator 132 performs learning and inference based on raw data or abstracted data. For example, the estimation unit 132 performs machine learning based on raw data or abstracted data to construct a global model. For example, the estimating unit 132 performs machine learning using raw data or abstract data and rough current and future positions and behaviors of the user U as learning data. Alternatively, the estimation unit 132 performs machine learning using raw data or abstract data and recommendation information (recommendation) to be provided to the user U as learning data.

Also, the estimating unit 132 inputs raw data or abstracted data to the global model, performs inference, and obtains an inference result. For example, the estimation unit 132 estimates a rough current or future position or behavior of the user U, or estimates recommended information to be provided to the user U, based on raw data or abstracted data.

(Providing unit 133)
The providing unit 133 provides, via the communication unit 110, provision information based on an inference result corresponding to raw data or abstract data to the terminal device 10 of the user U who is the transmission source of the data. For example, the providing unit 133 provides recommendation information to the terminal device 10 of the user U based on the inference result of the global model.

[5. Processing procedure]
Next, a processing procedure by the terminal device 10 according to the embodiment will be described with reference to FIG. FIG. 8 is a flow chart showing a processing procedure according to the embodiment. Note that the processing procedure described below is repeatedly executed by the control unit 30 of the terminal device 10 .

As shown in FIG. 8, the acquisition unit 34 of the terminal device 10 acquires data via the positioning unit 14, the sensor unit 20, or the input unit 13 (step S101). For example, the acquisition unit 34 acquires position information of the terminal device 10, sensor data, attribute information of the user U, history information (operation history, payment history, etc.) and the like as data.

Subsequently, the determination unit 35 of the terminal device 10 determines whether or not to send the data to the information providing device 100 (server side) according to the data and context (step S102).

At this time, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) according to the degree of privacy of the data. For example, the determination unit 35 does not abstract the data when there are an unspecified number of people in the same place and a large number of the same data (same location information, etc.) are collected, or when it is not easy to identify an individual. , the data is sent to the information providing apparatus 100 (server side). In this case, the determination unit 35 determines to learn the data online (on the server side). In addition, when there is no other person in the same place and it is easy to identify the individual, the determination unit 35 may abstract the data and send it to the information providing device 100 (server side), Data may be determined to be learned on-device (terminal side).

Further, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) according to the remaining battery level of the terminal device that acquired the data. For example, when the remaining battery (remaining battery power) of the determining unit 35 is low (for example, when the remaining battery power is less than a predetermined threshold), the determination unit 35 performs machine learning online (on the server side). to judge. Conversely, when the remaining battery (remaining battery power) of the determining unit 35 is large (for example, when the remaining battery power is equal to or greater than a predetermined threshold), the determination unit 35 performs machine learning on the device (terminal side). decide to do so.

Further, the determination unit 35 determines whether or not to send the data to the information providing apparatus 100 (server side) depending on whether the acquisition of the data is explicit or implicit. Specifically, the determination unit 35 determines whether the acquisition of data such as the location information of the user U is implicitly acquired (for example, acquired in the background) so that the user U does not know, or whether the user U As you can see from the above, it is determined whether to learn on-device (terminal side) or online (server side) depending on whether it is explicitly acquired (for example, acquired by a navigation app).

Subsequently, when the determination unit 35 of the terminal device 10 determines to send the data to the information providing device 100 (server side) (step S102: Yes), it determines whether or not to abstract the data (step S103). .

Subsequently, when the transmission unit 31 of the terminal device 10 determines that the data is to be abstracted (step S103: Yes), the transmission unit 31 transmits the abstracted data to the information providing device 100 (server side) via the communication unit 11. ) (step S104). At this time, the conversion unit 36 abstracts the data and converts it into abstracted data.

Alternatively, when the transmission unit 31 of the terminal device 10 determines not to abstract the data (step S103: No), the data is sent to the information providing device 100 (server side) in raw data via the communication unit 11. Send (step S105).

Subsequently, when the collection unit 131 of the information providing device 100 (on the server side) collects raw data or abstract data from the terminal device 10 of each user U via the communication unit 110, It is determined to perform learning and inference with the global model using the modified data (step S106). That is, the transmission unit 31 of the terminal device 10 requests learning and inference in the global model by transmitting data to the information providing device 100 (server side).

At this time, the collection unit 131 of the information providing device 100 (on the server side) collects raw data or abstract data from the terminal device 10 of each user U via the communication unit 110 . The estimator 132 performs learning and inference on a global model using raw data or abstracted data. The provision unit 133 provides, via the communication unit 110, provision information (recommendation information or the like) based on the inference result of the global model to the terminal device 10 of the user U who is the transmission source of the data.

Subsequently, the receiving unit 32 of the terminal device 10 receives the provided information based on the inference result of the global model from the information providing device 100 (server side) via the communication unit 11 (step S107).

On the other hand, when the determination unit 35 determines not to send data to the information providing device 100 (server side) (step S102: Yes), the learning unit 37 of the terminal device 10 performs machine learning with a local model using raw data. (Step S108).

Subsequently, the learning unit 37 of the terminal device 10 makes an inference with the local model using the raw data and obtains an inference result (step S109).

Subsequently, the learning unit 37 of the terminal device 10 determines information (presentation information) to be presented to the user U based on the inference result of the local model using the raw data (step S110). For example, the learning unit 37 of the terminal device 10 determines recommended information such as advertisements and coupons for stores near the current location of the user U as information to be presented to the user U (presentation information). At this time, the transmission unit 31 may request recommendation information such as advertisements and coupons for stores near the current location of the user U via the communication unit 11 . Further, when the learning unit 37 receives a plurality of pieces of provided information (recommendation information, etc.) from the information providing device 100 (server side), the learning unit 37 learns the information providing device based on the inference result of the local model using raw data. A plurality of pieces of provided information (recommended information, etc.) received from 100 are narrowed down, and information (presented information) to be presented to the user U is determined.

[6. Modification]
The terminal device 10 and the information providing device 100 described above may be implemented in various different forms other than the above embodiments. So, below, the modification of embodiment is demonstrated.

In the above embodiment, part or all of the processing executed by the information providing device 100 may actually be executed by the terminal device 10 . For example, the processing may be completed stand-alone (by the terminal device 10 alone). In this case, it is assumed that the terminal device 10 has the functions of the information providing device 100 in the above embodiment. In addition, in the above-described embodiment, the terminal device 10 cooperates with the information providing device 100, so from the user U's point of view, it appears that the terminal device 10 is executing the processing of the information providing device 100 as well. That is, from another point of view, it can be said that the terminal device 10 includes the information providing device 100 .

In the above embodiment, the information providing apparatus 100 (server side) employs federated learning, and the terminal device 10 (terminal side) of each user U uses a shared model (core model). model) can be provided. Then, based on the shared model, the terminal device 10 creates a pattern by performing on-device machine learning using the raw data (and sensor data) and the result information as learning data, and uses the pattern to create the raw data (and Inference may be performed using sensor data) as input data.

Further, in the above-described embodiment, the terminal device 10 may be a smart phone or the like that realizes advanced camera functions by on-device AI (Artificial Intelligence). Also, the terminal device 10 may be a robot, self-driving car, drone, or the like equipped with an on-device AI.

[7. effect〕
As described above, the information processing device (terminal device 10) according to the present application includes an acquisition unit 34 that acquires data, and a determination unit 35 that determines whether to send data to the server side according to data and context. and, if it is determined to send the data to the server side, a transmission unit 31 that transmits the data to the server side, a reception unit 32 that receives the result of machine learning in the global model using the data from the server side, and the data and a learning unit 37 that performs machine learning with a local model using the data when it is determined not to send the data to the server side.

The information processing apparatus according to the present application further includes a conversion unit 36 that abstracts data and converts it into abstracted data.

Further, when determining to send the data to the server side, the determining unit 35 determines whether to abstract the data. When determining to abstract the data, the conversion unit 36 abstracts the data and converts it into abstracted data. The transmission unit 31 transmits the abstracted data to the server side. The receiving unit 32 receives the result of machine learning in the global model using the abstracted data from the server side.

The acquisition unit 34 also acquires position information as data. If the conversion unit 36 determines to abstract the data, the conversion unit 36 converts the position information indicating a specific location into wide-area position information.

Further, the determination unit 35 determines whether or not to send the data to the server side according to the degree of privacy of the data.

Further, the determination unit 35 determines whether or not to send the data to the server side according to the remaining battery level of the terminal device 10 that acquired the data.

Further, the determination unit 35 determines whether or not to send data to the server side depending on whether the acquisition of data is explicit or implicit.

Further, the determination unit 35 determines not to update the local model based on the data until the global model is updated based on the data.

The information processing apparatus according to the present application can determine whether or not to send data according to the data and the situation by one or a combination of the above-described processes.

[8. Hardware configuration]
Also, the terminal device 10 and the information providing device 100 according to the above-described embodiments are implemented by a computer 1000 configured as shown in FIG. 9, for example. The information providing apparatus 100 will be described below as an example. FIG. 9 is a diagram illustrating an example of a hardware configuration; The computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are buses. It has a form connected by 1090.

Arithmetic device 1030 operates based on programs stored in primary storage device 1040 and secondary storage device 1050, programs read from input device 1020, and the like, and executes various types of processing. The arithmetic unit 1030 is implemented by, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used for various calculations by the arithmetic device 1030 . The secondary storage device 1050 is a storage device in which data used for various calculations by the arithmetic device 1030 and various databases are registered. State Drive), flash memory, or the like. The secondary storage device 1050 may be an internal storage or an external storage. Also, the secondary storage device 1050 may be a removable storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) memory card. Also, the secondary storage device 1050 may be a cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a display, a projector, and a printer. (Digital Visual Interface), HDMI (registered trademark) (High Definition Multimedia Interface), and other standardized connectors. Also, the input I/F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, keypad, buttons, scanner, etc., and is realized by, for example, USB.

Also, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

Also, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

Note that the input device 1020 includes, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), PDs (Phase change rewritable discs), magneto-optical recording media such as MOs (Magneto-Optical discs), and tapes. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

Network I/F 1080 receives data from other devices via network N and sends the data to arithmetic device 1030, and also transmits data generated by arithmetic device 1030 via network N to other devices.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output I/F 1060 and input I/F 1070 . For example, arithmetic device 1030 loads a program from input device 1020 or secondary storage device 1050 onto primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 100 , the arithmetic device 1030 of the computer 1000 implements the functions of the control unit 130 by executing a program loaded on the primary storage device 1040 . Further, arithmetic device 1030 of computer 1000 may load a program acquired from another device via network I/F 1080 onto primary storage device 1040 and execute the loaded program. Further, the arithmetic unit 1030 of the computer 1000 may cooperate with another device via the network I/F 1080, and call functions, data, etc. of the program from another program of the other device for use.

[9. others〕
Although the embodiments of the present application have been described above, the present invention is not limited by the contents of these embodiments. In addition, the components described above include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the components described above can be combined as appropriate. Furthermore, various omissions, replacements, or modifications of components can be made without departing from the gist of the above-described embodiments.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, the information providing apparatus 100 described above may be implemented by a plurality of server computers, and depending on the function, may be implemented by calling an external platform or the like using an API (Application Programming Interface), network computing, or the like. can be changed flexibly.

Further, the above-described embodiments and modifications can be appropriately combined within a range that does not contradict the processing contents.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the acquisition unit can be read as acquisition means or an acquisition circuit.

1 information processing system 10 terminal device 11 communication unit 14 positioning unit 20 sensor unit 30 control unit 31 transmission unit 32 reception unit 33 processing unit 34 acquisition unit 35 determination unit 36 conversion unit 37 learning unit 100 information providing device 110 communication unit 120 storage unit 121 user information database 122 history information database 123 recommendation information database 130 control unit 131 collection unit 132 estimation unit 133 provision unit

Claims (10)

an acquisition unit that acquires data containing information that can identify individual behavior, such as location information and sensor data ;
a determination unit that determines whether or not to send the data to a server side according to a context serving as a criterion for determining whether or not to send the data to the outside ;
a transmitting unit configured to transmit the data to the server when it is determined that the data is to be transmitted to the server;
a receiving unit that receives results of machine learning in a global model using the data from the server side;
and a learning unit that performs machine learning with a local model using the data when it is determined not to send the data to the server side. a conversion unit that abstracts the data and converts it into abstracted data;
The information processing apparatus according to claim 1, further comprising: The determining unit, when determining to send the data to the server side, determines whether to abstract the data,
When determining that the data should be abstracted, the conversion unit abstracts the data and converts it into abstracted data;
The transmission unit transmits the abstracted data to the server side,
The information processing apparatus according to claim 2, wherein the receiving unit receives a result of machine learning in a global model using the abstracted data from the server side. The acquisition unit acquires position information as the data,
4. The information processing apparatus according to claim 2, wherein, when determining to abstract the data, the conversion unit converts the position information from position information indicating a specific point to wide-area position information. . The information processing according to any one of claims 1 to 4, wherein the determination unit determines whether or not to send the data to the server side according to the degree of privacy of the data. Device. 6. The method according to any one of claims 1 to 5, wherein the determination unit determines whether or not to send the data to the server side according to the remaining battery level of the terminal device that acquired the data. The information processing device described. 7. The method according to any one of claims 1 to 6, wherein the determining unit determines whether to send the data to the server side according to whether the acquisition of the data is explicit or implicit. The information processing device described. The determination unit determines not to update the local model based on the data until the global model is updated based on the data. The information processing device according to . An information processing method executed by an information processing device,
an acquisition step of acquiring data containing information that can identify an individual's behavior, such as location information and sensor data ;
a determination step of determining whether or not to send the data to a server side according to a context serving as a criterion for determining whether or not to send the data to the outside ;
a transmitting step of transmitting the data to the server side when it is determined that the data is to be transmitted to the server side;
a receiving step of receiving results of machine learning on a global model using the data from the server side;
and a learning step of performing machine learning with a local model using the data when it is determined not to send the data to the server side. an acquisition procedure for acquiring data containing information that can identify individual behavior, such as location information and sensor data ;
a determination procedure for determining whether or not to send the data to a server side according to a context serving as a criterion for determining whether or not to send the data to the outside ;
a transmission procedure for transmitting the data to the server side when it is determined that the data is to be transmitted to the server side;
a receiving procedure for receiving results of machine learning on the global model using the data from the server side;
an information processing program for causing a computer to execute a learning procedure for performing machine learning with a local model using the data when it is determined not to send the data to the server side.

Images