JP6067120B2 - Integration of robotic systems with one or more mobile computing devices - Google Patents

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is priority to US Provisional Application No. 61 / 693,687, filed Aug. 27, 2012 ("Integration of a Robotic System with or More Mobile Computing Devices") All rights are claimed and incorporated herein by reference.

  The present invention relates to a robot system that can be integrated with a mobile computing device.

  One of the problems in building a robot system that is mobile, autonomous and / or otherwise aware of and operating with respect to the environment is the cost. Manufacturing this system at a cost that can support sales to the consumer market has traditionally been an obstacle in the development of robots for consumer applications. Despite the huge market opportunities to give products some intelligence or functional capabilities based on artificial intelligence, these additions are made from a business perspective due to the cost structure of these products (especially consumer products) It has become difficult.

  This cost is due to the use of dedicated hardware and firmware (eg, control systems and autonomous components) for this type of robotic system. Because the consumer product market (especially consumer products related to entertainment products) is cost competitive, it has not been possible to profitably produce and sell products for such robotic applications.

  According to various embodiments of the present invention, a mobile computing device (eg, a smartphone and / or tablet) is utilized as a platform for robot system support and control. Since such devices are rapidly spreading to consumers, it is feasible and realistic for consumers to use a sufficient number of such devices with robotic systems.

  By using a mobile computing device (eg, a smartphone) to support at least a portion of the operation of the robot system, the technology of the present invention enables a significant reduction in hardware requirements in the robot system. In addition, such devices primarily have a rich user interface, advanced computing capabilities and built-in wireless connectivity, making them ideal for use as a robust control system for robotic systems as described herein. It becomes the target.

  According to various embodiments of the invention, a robotic system is integrated with one or more mobile computing devices. Examples of such mobile computing devices include one or more smartphones, tablet computers, laptop computers, gaming consoles, kiosks, and the like. In other embodiments, the techniques of the present invention may be performed using any other suitable computing device (eg, desktop computer, web-based computing architecture, etc.). Can be networked with each other and / or with physical components (eg, agents under the control of the user), such as Bluetooth, WiFi and / or other wireless networking technologies. Can be used.

  In various embodiments described herein, a representation of a physical configuration under user control of individual components (or agents) of a system in physical space is replicated in virtual space. Some real-time equivalence can be maintained between the physical space and the virtual space so that a virtual environment that mirrors the physical environment can be executed. In this way, the present invention supports a system in which events occurring in one environment can directly affect the sequence of events occurring in the other environment. As a result, the elements in the virtual space are truly independent and unified on the peer footing together with the elements in the physical space.

  The integration of virtual and physical has been considered the existence of augmented reality, and to date, the application of augmented reality has yielded numerous examples of one-way effects (physical effects on the virtual and vice versa) However, it remains inadequate for the construction of a robust interactive system. In at least one embodiment of the present invention, a truly symbiotic system is implemented where the virtual environment affects the physical environment in a consistent manner and vice versa.

  In at least one embodiment, the system of the present invention is implemented for entertainment applications (eg, presentation of video games in physical space). Games and entertainment are advantageous applications of the present invention in that they provide interactivity between multiple users. In at least one embodiment, the system of the present invention provides an environment in which a user controls one or more agents of the system while controlling one or more other agents by artificial intelligence.

  The accompanying drawings illustrate several embodiments of the present invention and, together with the description, serve to explain the principles of the invention according to the embodiments. Those skilled in the art will recognize that the specific embodiments shown in the drawings are illustrative only and are not intended to limit the scope of the present invention.

1 is a block diagram illustrating an architecture for carrying out the present invention, according to one embodiment. FIG. FIG. 4 illustrates an exemplary integration of events and functions in physical and virtual space, according to one embodiment. FIG. 2 is a block diagram illustrating aspects of the role of a host device (base station) according to at least one embodiment, showing various components and structures that implement the relationship between the user controlling the vehicle and the vehicle itself. In accordance with one embodiment, an example is shown in which a body covering a vehicle is used to reflect a characteristic difference in the corresponding representation of the vehicle in the virtual environment, thereby strengthening the relationship between the physical environment and the virtual environment. 6 illustrates an example of the use of a virtual accessory to execute digitally rendered elements in physical space, according to one embodiment. FIG. 6 illustrates an exemplary embodiment of the present invention for executing a game participation environment in which vehicles compete on a race track, according to one embodiment.

  For purposes of illustration, the present invention is described herein in the context of a system for executing a car racing game. In this system, a user-controlled agent is a physical vehicle or accessory associated with game participation and is competing on a physical raceway. For further details on such systems and their virtual and physical environment integration mechanisms, see related US patent application Ser. No. 13 / 707,512 filed Dec. 6, 2012 (“Distributed System of Autonomous Controlled Mobile”). Agents "(case number ANK001CONT)). This application is hereby incorporated by reference. However, those skilled in the art will recognize that the techniques described herein can be implemented in other contexts and environments and are not limited to vehicles on physical raceways. Thus, the term “vehicle” described herein applies to any mobile agent capable of being controlled and operated in the manner described herein and is described herein. It is also expressed in the virtual environment.

  Although the present invention is primarily described herein in the context of entertainment applications, those skilled in the art will recognize that the present invention can be practiced in many other contexts (eg, contexts not necessarily related to entertainment). Will.

System Architecture Referring now to FIG. 1, an architecture for carrying out the present invention according to one embodiment is illustrated. In the system 100 shown in FIG. 1, game participation is hosted by the host device 108 and may be performed on any suitable computing device. The computing device is mobile or stationary, for example, a smartphone, tablet, laptop computer, etc., and / or any combination thereof. In at least one embodiment, the host device 108 supports and executes various algorithms (performing game operations) included in the software. Herein, the host device 108 and associated software are collectively referred to as a base station or central controller.

  Any different variety of devices can function as the host device 108, examples of which are control software for smartphones, tablet computers, laptop computers, desktop computers, video game consoles and / or systems. Any other computing device capable of supporting In at least one embodiment, such devices can use any suitable operating system, including but not limited to iOS or MacOS (available from Apple Inc., Cuepartino, Calif.). Available), Android (available from Google, Inc., Mountain View, Calif.), Or Windows® (available from Microsoft Corporation, Redmond, Washington). In at least one embodiment, the host device 108 is an iPhone or iPad (available from Apple Inc., Cuepartino, Calif.) And executes a suitable software application (“app”). In at least one embodiment, the software that controls the host device 108 operates the agents 104A-104F in physical space and schedules game participation according to rules, actions controlled by the user and / or artificial intelligence, It can be provided via any suitable means (eg, downloadable application (app)) including appropriate functions and game participation structures for collaboration and execution. In at least one embodiment, the host The device 108 maintains the state of the agents 104A-104F and sends and receives commands to and from the agents 104A-104F The host device 108 may also include an appropriate user interface to facilitate user interaction with the system. .

  In at least one embodiment, agents 104A-104F are mobile agents (eg, vehicles), referred to herein by such names, but may be other objects or components.

  In at least one embodiment, host device 108 is a central node for all activity and control commands sent to agents 104A-104F and / or other components (eg, accessories 105, 106), although The command originates from an algorithm executing on the host device 108 or is routed through the host device 108, but is physically present or controlled by a user 109D-109K at a remote location Generated from 101D to 101K. In other embodiments, a more distributed architecture can be implemented, in which case host device 108 need not be a central node for all activity and control commands.

  1 shows a specific number of controllers 101D to 101K, agents 104B to 104H, accessories 105 and 106 (which can be regarded as a kind of agent), AI-controlled vehicles 104J (which can be regarded as a kind of agent). , And other components. Those skilled in the art will appreciate that the specific quantities of these components described in FIG. 1 and herein are exemplary only, and that the present invention may be omitted using any other quantity and / or where appropriate. It will be appreciated that some of the components to be implemented can be implemented.

  In the architecture of FIG. 1, the system 100 runs centrally, and the controllers 101D-101K and agents (eg, vehicles 104A-104F) and other components communicate with the host device 108. As shown, in at least one embodiment, multiple users 109 (or players) can control multiple agents in the form of vehicles 104A-104F, while other agents / vehicles 104J are controlled by artificial intelligence. Can be done.

  As shown in FIG. 1, any number of external devices may be connected to the host device 108 via any suitable communication protocol (eg, cellular / internet connection 107). The various external devices may or may not be the same as the host device 108. Some or all of the external devices function as a player controller. FIG. 1 shows a device that can be used as a player controller (for example, a game console 101B having an arbitrary number of controllers 101J and 101K (controlled by users 109J and 109K, respectively)) (controlled by a user 109D). ) Various examples of laptop computer 101D, stand-alone controller 101E (controlled by user 109E), and smartphones 101F, 101G, 101H (controlled by users 109F, 109G, 109H, respectively) are shown. In at least one embodiment, any or all of the controllers 101 can be iPhone or iPad (available from Apple Inc., Cuepartino, Calif.) And execute a suitable software application (“app”). The controllers 101J, 101K, and 101E may be any appropriate type (for example, a controller that is generally used with a console game device).

  In the embodiment shown in FIG. 1, the game is hosted on the host device 108. The host device 108 supports game participation in a physical environment in a physical environment (e.g., a race track) and in a virtual environment under the direction of software, and the status of the virtual environment is stored in memory on the host device 108 and / or other Keep in place.

  Referring now also to FIG. 6, an example embodiment of the present invention for executing a game participation environment is illustrated. In this game participation environment, according to one embodiment, the vehicle 104 (race car) competes on the runnable surface 601 (eg, race raceway). However, those of ordinary skill in the art will appreciate that such an embodiment is merely an example of the practice of the invention, for example, the system can be run in a completely different physical environment, using agents other than vehicles and / or different types. It will be appreciated that the raceway may be used or may not be used at all.

  As described in the related US Pat. No. 13 / 707,512 for “Distributed System of Autonomously Controlled Mobile Agents”, the runnable surface 601 is a physical model of one or more roads in at least one embodiment. Yes, and may include objects (eg, pauses, traffic lights 105, railroad crossings, etc.). The vehicle 104 is a mobile agent capable of independent movement. The vehicle 104 may be physically modeled following a car, raceway, ambulance, animal or any other desired form. In at least one embodiment, each vehicle includes one or more sensors 604. These sensors 604 can read information from the travelable surface 601 and a communication module (not shown). The communication module can send and receive commands and / or other information to the host device 108, eg, via wireless means.

  As shown in FIG. 6, the travelable surface 601 may include any number of segments 602, although such a segmented arrangement is optional. Such segments 602 can connect at designated attachment points and can be reconfigured to build any desired structure. This structure is called a travelable surface 601. The road segment 602 includes a continuous area that can be navigated by one or more vehicles 104 called a travelable section, and the ends are interconnected using a simple click-in mechanism that exists at each connection point. Each road segment 602 can also optionally send power to an adjacent road piece 602 and may optionally include a microcontroller for advanced functions (eg, traffic lights 603). The travelable surface 601 can also be provided as a single piece. The runnable surface 601 (and / or segment 602) can be foldable, collapsible, harvestable, or otherwise compact for storage.

  In at least one embodiment, the base station software running on the host device 108 runs a virtual version of a physical game. The game continuously maintains equivalence with events in the physical environment by updating information related to the position, direction, speed of the vehicle 104 and other aspects that characterize the game events. In at least one embodiment, the host device 108 ensures that at any point in time the game state is in the physical environment and the virtual environment is the same (or is substantially the same) or virtual The game state in the environment at least ensures that the representation of the physical state is at least a sufficient level of accuracy for game participation purposes.

  In at least one embodiment, the artificial intelligence software executes on the host device 108 and issues a command (via a wireless communication mechanism or other mechanism) to execute on the raceway 601. 104J is controlled. In other embodiments, software for controlling the vehicle 104J may be located elsewhere and / or executed on the vehicle 104J itself.

  In at least one embodiment, the host device 108 may simultaneously function as a control unit for a human user 109A that controls the vehicle 104 (in the illustrated example, the human user 109A uses the host device 108 to control the vehicle 104A. Control). Such functionality can be provided on the host device 108, and at the same time the host device 108 serves as a conduit and interpreter for control commands coming from other devices 101D-101K that control other vehicles 104B-104F. Also works. In another embodiment, the host device 108 does not function as a control unit for the human user 109 but operates as a dedicated central controller.

  In at least one embodiment, agents under user control (eg, vehicles 104B-104F) may not be consistent in form or function. For example, the user 109 may be given an opportunity to control objects or elements other than vehicles (eg, traffic lights, railway crossings, turrets, drawbridges, pedestrians, etc.).

  The player controllers 101D to 101K may communicate directly with the host device 108 or may communicate via an intermediate device. For example, in FIG. 1, the controllers 101J and 101K communicate with the host device 108 via the game console 101B. Similarly, any number of connection stages can be configured between the player controller and the host device (eg, one or more smartphones connected to the host device can be reconnected to the host device via a series of networked devices. Connecting).

  FIG. 1 shows an example in which vehicles 104B to 104F are controlled by human users 109B to 109F, respectively. Additional agents (referred to as accessories 105, 106) may also be controlled by a human user 109 or operate automatically (eg, under the direction of artificial intelligence software running on the host device 108 or elsewhere). can do. Each accessory 105, 106 is a physical or virtual item and can be powered or passive, and can be used to directly influence aspects of the game participation environment and / or other agents 104. Good. In this example, the accessory 105 is a physical traffic signal as shown in FIG. Other examples of physical accessories may be barriers, breakers, drawbridges and the like. Such a device may be communicatively connected to the host device 108 to control actions associated with game participation. In at least one embodiment, user 109 may affect game participation by changing the physical state of accessory 105.

The smart accessory 105 can also exist as a passive element in physical space. An example of a passive smart accessory is a sticker intended to indicate a potential danger on the runnable surface 601 located at any point before or during play. Such a sticker may include a coding scheme that allows for a unique identification of the vehicle 104 (for example, when the vehicle 104 passes the sticker). In at least one embodiment, a user 109 controlling one or more such passive smart accessories can selectively activate or deactivate the accessories individually or by group. Such accessories are passive in the physical environment, but can represent a range of different features (eg, the following) in the virtual environment.
An oil film that can stop steering and braking of a vehicle that passes through, a landmine, a nail, and a power source that increases the performance of the first vehicle 104 after activation and allows it to pass physically.

  Although the passive accessory 105 does not necessarily need to physically indicate a state change, the host device 108 may report a virtual state change and send such a change to the user 109 via the controller 101 in real time. The behavior of the vehicle 104 can also change in response to a change in the virtual state of the accessory 105.

  The accessory 106 is an example of a virtual accessory, does not include a physical component other than a computing device (for example, a smartphone or a tablet computer), and includes an appropriate output device (for example, a display screen). The virtual accessory 106 can be physically placed at a specific location within the physical gaming environment to render the appropriate accessory in both appearance and state. Referring now to FIG. 5, an example of a virtual accessory (tablet 106) is illustrated. This virtual accessory (tablet 106), according to one embodiment, performs digitally rendered elements (turret image 503) in physical space. The image 503 can be moved and animated to appear to interact with the physical vehicle 104 in the game participation area. For example, the image 50 can be shown as shooting a physical vehicle 104. The physical vehicle 104 can also appear to have been shot. These behaviors and interactions mirror the virtual environment in which the turret 501 is shooting the virtual representation 502 of the vehicle 104, as also shown in FIG. In various embodiments, the accessories 105, 106 need not rely on a human user when operating, but can operate under the control of artificial intelligence software running on the host device 108 and / or elsewhere.

  Those skilled in the art will appreciate that as the number of users 109 and the number of AI controlled opponents increase, the performance requirements for the host device 108 also increase. Depending on the number of agents 104 and the capabilities of the host device 108, game performance may be affected, for example, due to increased computational demands. In at least one embodiment of the invention, the system is executed in a distributed environment. In a distributed environment, for example, the host device 108 can distribute its logical portion to any number of connected devices and can support logical execution. These examples include smartphones, tablet computers, laptops, game consoles, etc., but any suitable device that can provide the support necessary to perform the logic assigned to it. It may be a thing. In at least one embodiment, for example, some of the processing tasks associated with operating system 100 can be distributed to one or more controllers 101D-101H.

  Distribution need not be local. That is, in at least one embodiment, logic can be distributed to, for example, one or more remotely located servers. The modular design for the structure of the host device 108 is useful for simple logic distribution, and the type of logical process that is offloaded from the host device 108 may not be one particular type of function or process. In at least one embodiment, the most billing of the host device's 108 resources is first assigned to somewhere, for example by prioritizing logical distribution according to operational and memory requirements.

  The wireless interface used for communication with the controllers 101D to 101H and / or communication between the controllers 101D to 101H may not be the same as that used when connecting the agents 104A to 104F under the control of the user 109. For example, the host device 108 can communicate with the controllers 101D to 101H via Wi-Fi, and the host device 108 can communicate with the agents 104A to 104F via Bluetooth. In such a case, the host device 108 can function as a bridge between a high power protocol (eg, Wi-Fi) and a low power protocol (eg, Bluetooth). The advantage of such an approach is that the output budget is limited when the vehicle 104 is controlled by the user 109 via the host device 108 or directly by the host device 108 (in the case of the vehicle 104J under AI control). Can be understood if there is.

  Another benefit gained through the use of Bluetooth (especially Bluetooth Low Energy (BTLE or BLE)) or similar capable wireless protocols allows agent 104 to communicate with BTLE / wireless devices that are also used. There is a point that can be used. In one embodiment, for example, a user 109 desiring to control a particular vehicle 104 or an active smart accessory 105 may use an intended controller 101 (eg, a smartphone with BTLE) of the desired vehicle 104. Can be placed nearby. Utilizing BTLE's ability to determine the relative distance or proximity to another BTLE enabled device, user 109 can place a device with two BTLEs within a certain distance threshold range. . In at least one embodiment, as a result, data exchange between a smartphone (eg, 101F) and the vehicle 104 can be facilitated, thereby providing the user 109 with an option to select the vehicle 104 for play. can do. This selection is then relayed to the host device 108 as an indication of the combination between the vehicle 104 and the user's 109 smartphone 101 and is ultimately designated as the control device for the vehicle 104.

  In various embodiments, BTLE data exchange can be used in other ways between vehicles 104 and / or similarly wireless-capable agents. For example, a user or observer can receive information about the status of agent 104's participation in the game, usage over the lifetime and / or achievement history, and / or to diagnose or customize the unit. it can.

  As described above, the controllers 101D-101H can be implemented using any suitable device. Again, lower function controllers 101J, 101K can be used (eg, wireless gamepad or joystick). If a gamepad or joystick 101J, 101K that does not have a wireless communication module that supports direct communication with the host device 108 is used, then a dongle that is plugged through the game console 101B or other intermediate or into an appropriate port on the host device 108 A connection to the host device 108 can be achieved through the use of (not shown). With such a dongle, a wireless connection to the controller 101 is obtained, and communication is sent through the port at the insertion destination. In an alternative embodiment of the dongle, there is a unit that performs a bridge between a wireless protocol compatible with the controller 101 and a wireless protocol compatible with the host device 108.

  In addition to sending user 109 commands to the vehicles 104B-104F through the host device 108, the controllers 101D-101H can also receive update information from the host device 108 that reflects the current state of the game. In at least one embodiment, some or all of the controllers 101D-101H are provided with one or more output devices (e.g., displays, speakers, haptic output mechanisms, etc.) to provide such received status information. Based on this, you can improve the playing experience. Such enhancements, for example, represent renderings, tactile outputs (eg, vibrations) and / or actions on a game raceway to provide increased realism or details that are not normally visible to the user 109. There is audio and / or audio that enhances such behavior.

  In at least one embodiment, visual, tactile, and / or audio information presented to the user 109 through some or all of the controllers 101D-101H may be unique for each device. Various agents 104 may be in different states for any given time (eg, for position, velocity, state, motion). In addition, if multiple users 109 control a single agent 104, the user roles or controls may be different. As such, the various queues and data presented to each user 109 can be individually tailored to the current state of the agent 104 and the specific role of the user 109. For example, in at least one embodiment, the racing game may present a rendering of the race course viewed from the vehicle 104 while displaying vehicle-specific data to the user 109. In an embodiment where more than one user 109 may share control of a single agent 104 (eg, a vehicle with weapons) that assumes the role of a driver while one user 109 operates a weapon, to the controller 101. It may be appropriate that the various forms of information transmitted differ in one or more ways for each user 109 associated with the agent 104.

  In various embodiments, since the users of controllers 101D-101H may be physically present, controllers 101D-101H communicate directly with host device 108 (via a wireless protocol (eg, Bluetooth)). Alternatively, users of the controllers 101D to 101H can be remotely located and connected via a host network (for example, the network 107). Controllers 101D-101H may rely on information about the game state returned from host device 108.

  In at least one embodiment, the tight coupling and equivalence maintenance between the state of the virtual play environment and the physical play environment allows the user 109 to control the vehicle 104 on the remotely located raceway 601. . Such an architecture allows users who are not remotely located to participate through dependence on the virtual representation of the game. In at least one embodiment, the remote user 109 can have the same level of control over the vehicle 104 as the local user 109 and can view events that are functionally identical to the local user 109. .

  In at least one embodiment, a bystander can observe game events from a remote location by using an external network connection. In the example of FIG. 1, a bystander or observer 110 observes a game event via a tablet 103 that communicates with a host device 108 via a cellular / Internet connection 107. Since actions occurring in the physical space are mirrored in real time in the virtual environment, the tablet 103 (or any other device used to display game participation activity) may have a game participation event in the physical space. A wealth of rendering and further information about the state of the user 109 and other details that the observer 110 may be interested in may be provided.

  In at least one embodiment, the remote user 109 (or even a local user) further provides an auxiliary device (not shown) for game display, separate from the controller 101 that the user is using to control the physical agent 104. Can be used. For example, when the screen of the remote user's controller 101 is small, by using such an auxiliary device as the second screen, it is advantageously possible to experience the game experience more deeply. For example, user specific information can be displayed on one display (eg, a display on the controller 101), and visualization of game participation can be provided on the auxiliary display. Using a separate device as a monitor reduces the possibility of limited display and is more familiar to console gamers who are accustomed to using a handheld gamepad to control the behavior while watching the behavior on another screen. Can provide a certain experience.

  In the examples described herein, we have described the dependency on rendering or other representation of game participation in virtual space, but playing game participation in physical space in real time on a replicated physical raceway Is completely possible. Since equivalence is maintained between virtual and physics, the remote user 109 can directly connect the physical raceway 601 that hosts game participation directly on the physical raceway that fully aligns through mirroring of the virtual model state. Play can be played. Using wireless communication between the host device 108 and a controller for replication of the physical raceway to move the vehicle on the duplicate raceway and behave in a manner that replicates (or at least approximates) the motion. Can do.

The host device 108 and / or other components can provide additional functionality. For example, one or more controllers 101 may provide supplemental information related to events occurring during the game and / or function as a direct platform for further game participation. Such a device may have any suitable component that provides the desired level of interactivity, for example:
A user interface that can respond to touch, button control, basic direction or orientation / acceleration of the device, and / or any other suitable user input display screens one or more audio components (eg, speakers) And microphone)

  Those skilled in the art will recognize that the above list is exemplary only and not exhaustive.

  Such a set of improvements is collectively referred to as non-actionable information. These examples include individual audio, video, and / or haptic output or any suitable combination. Specific types of output include, for example, sound effects, music, vibrations, images, animations, film clips, and the like. Any of these can be used alone or in any suitable combination to improve the realism or otherwise enrich the perceived aspects of the game experience. Alternatively, these enhancements may take the form of games and / or data forms that provide summary or supplemental information about the user 109 (eg, vehicle performance data, hourly racer rankings, competition standings and / or others). data from).

  In at least one embodiment, a highly interactive game controller 101 (eg, a smartphone and a tablet) can be used to enhance the gaming experience. Such improvements can be achieved by using the control and connection capabilities of these devices to generate play scenarios outside of the direct action occurring in the physical environment and / or (whether at the game or outside the game). ) Done by changing the mode of the game or the agent operating inside. In the context of a racing game, for example, after pulling the vehicle 104 during a pit stop on a physical race track, game participation switches to the controller 101 and the user 109 is a fully digital video player that is close to the traditional video game format. In context, as a member of the pit crew, the problem of the service vehicle 104 is faced. As another example, the vehicle 104 may drive to a specific point in physical space (eg, a building placed on a playset). At this particular point, game participation switches to a display on the game controller that the driver or team of the vehicle 104 continuously uses in the building in the digitally rendered environment. In the scenario described in such an example, game participation may switch seamlessly between a physical play environment and a digital environment. The digital environment may be supported by a highly interactive controller 101 and / or controller 101 contemplated by a separate display console (not shown) that communicates with the controller 101 and / or the host device 108. In such cases, if game participation parts or segments that occur in a truly digital space (eg, traditional video games) are integrated into the entire system, do these experience parts necessarily disconnect from the physical domain? Or does not mean do not affect. Considering an example where user 109 drives physical vehicle 104 to a location that provides access to the digital portion of play (eg, a pit stop), user 109 is no longer controlling physical vehicle 104 in an active state. However, the digital space may provide an opportunity to equip or repair the vehicle 104, thereby affecting the performance of the vehicle 104 after the user 109 resumes control. As another example, the user 109 similarly enters a truly digital environment that allows control of a physical accessory (eg, a drawbridge on a race course). In that case, the user 109 opens and closes the raceway section 602 to traffic in a physical environment form in a truly digital space. Similarly, such control may also be provided for virtual accessories 106 (eg, those hosted by Android-based or IOS-based devices that are played during a physical play environment). These devices interact with the physical agent 104 and / or affect its behavior.

  In at least one embodiment in which the present invention is implemented as a competitive racing game, the controller 101 may provide functions to manipulate the speed and steering of the vehicle 104. However, the game control can be performed more extensively by using the capability of a smartphone or a device having the same capability as the controller 101. For example, in a “death race” scenario, the vehicle 104 may be equipped with a weapon that can be pointed at the competing vehicle 104. In such an embodiment, the bi-directional controller 101 can be used to support functions beyond basic vehicle speed and steering. Additional functionality can be provided (eg, selecting a target vehicle 104 between active state representations of all vehicles 104 that are aimed at a weapon mounted on the vehicle 104 or are competing on the runnable surface 601). .

  As described above, in at least one embodiment, multiple users 109 can control a single agent 104 in a game. For example, in a racing game embodiment, three users 109 may control a single vehicle 104 on which one user 109 is steering and controlling speed, and another user 109 may be a gun pointing forward. Aiming and firing, the third user 109 operates a gun facing backward. In such a case, the controller 101 provides information that is individually adjusted according to the role played by each user 109. For example, a user 109 driving a vehicle 104 discovers display information and controls the same manner as a typical racing game, while a user 109 operating a gun is rendered in a physical play environment. You can see the view. In this view, the displayed field of view is from a position on the vehicle 104 and includes a raceway element and a virtual representation of the competing vehicle 104 in a position and trajectory relative to the actual position in the physical play environment.

  When the user 109 is operating a weapon in this example, the resulting control and interactive experience may make the smart device similar to a first person shooter game.

  The gun need not be represented (although it may be represented) by any physical structure or element on the physical vehicle 104 itself, instead of such physical structure (or such physical structure). (In addition to) may be expressed in a virtual environment. In at least one embodiment, when a gun is fired, such a rendering of the fire can be represented in a virtual environment, and optionally the physical vehicle 104 can provide a constant activation of such weapons. Visual and / or audio representations may be provided (eg, flashes, physical barrel movements, sounds, etc.). A firing target (eg, another vehicle 104) may respond to it as if it had been fired, eg, by rerouting, flipover, disabling, etc. Visual feedback can be provided in the virtual and / or physical environment, for example by emitting light from the target as if it were in a flame or otherwise disabled. Muzzle sparks and sound can be provided by mounting LEDs or other visual and / or audio components on the vehicle 104 to reproduce the sound of a machine gun. Such output can be synchronized with the firing of the corresponding gun in the virtual space.

  In at least one embodiment, the vehicle 104 may be designed to present (simulated) damage in physical space. For example, the LED may indicate that the vehicle 104 is damaged, or the change in the color of the LED array may indicate the current damage state of the vehicle 104. In at least one other embodiment, more complex techniques can be used to replicate or simulate damage caused to the vehicle 104 by weapon collision, impact, or other means. Examples of these are the dropping of parts from the vehicle 104 or mechanism. When such a fall is triggered, for example, kinetic energy generated when a collision or explosive force occurs on the vehicle 104 is applied. In at least one embodiment, the vehicle 104 exhibits simulated damage by changing the behavior to simulate behavior, such as punctured tires, damaged steering or engine components.

Bi-directional impacts between physical and virtual environments In at least one embodiment, the system of the present invention provides simultaneous equivalence in virtual and physical space by arbitrating events at a low level. Maintain in space, thereby providing a bi-directional effect between the physical environment and the virtual environment. Referring now to FIG. 2, the integration of such events and functions is illustrated in both physical and virtual space. In this example, when the racing game is executed, the mobile agent is a vehicle competing in the physical space, and these vehicles are similarly equipped with virtual weapons.

  The sequence of events shown in the drawings includes virtual events that affect physical events and vice versa. While the car is competing on the physical course, the base station maintains a virtual representation of the race state in real time so that the position, speed, acceleration, course and other metrics characteristics of the moving vehicle are in memory. Are continuously tracked for playback. This memory mirrors state changes in the physical world. In such a case, the virtual representation of the vehicle may have multiple functions or characteristics that do not exist on the physical automobile. Using an exemplary gun, user 109 can fire another vehicle in virtual space, but this gun is not in physical space. Since the virtual state and the physical state are closely connected, an event occurring on one side can affect the other state.

  These series of events shown in the example in FIG. 2 occur as follows. The vehicle 104K is indicated in the virtual environment 202 by a vehicle representation 204K. In the virtual environment 202, the vehicle representation 204K fires the vehicle representation 204L (representing the vehicle 104L in the physical environment 201) with a virtual gun. Since the vehicle 104K in the physical environment 201 does not have an actual gun that fires a projectile, the host device 108 operating the game may use a physical law based on the relative positions of the vehicle representations 204K and 204L and the direction of the gun. Can be determined to collide with the vehicle representation 204L at position 2 in the figure. The host device 108 also determines that the target vehicle representation 204L is displaced from the course in the virtual environment 202 due to virtual energy from the collision and the resulting collision.

  As described above, in at least one embodiment, the system maintains equivalence between the virtual environment and the physical environment. Therefore, in response to the above-mentioned weapon colliding with the vehicle representation 204L, the virtual displacement of the vehicle representation 204L in the physical environment 201 is reproduced by the control algorithm of the host device 108. Thus, the physical vehicle 104L is artificially propelled in a manner that mimics the displacement of the vehicle representation 204L in the virtual environment 202. In the example of FIG. 2, a physical vehicle 104L fired by a virtual weapon at position 2 is artificially deflected from the current course in physical space. As a result, it collides with another vehicle 104M at position 3 in the physical space.

  When a collision occurs between the physical vehicles 104L and 104M at the position 3, the vehicle 104M is physically displaced from the original course vector as shown in the position 4 by the actual energy transfer. When such a course change in the physical environment 201 is detected, the host device 108 causes the vehicle representations 204L and 204M to continue the vehicles 104L and 104M in the physical environment 201 by correspondingly adjusting the virtual environment 202. Be able to mirror automatically. Therefore, the vehicle representation 204M deviates from the course as well.

  In this way, a series of events in alternating states occur, and a result is generated in the physical environment 201 due to the occurrence in the virtual environment 202. As a result, due to the influence in the physical environment 201, a dynamic law or The order of events in the virtual environment 202 is affected. The above scenario illustrates the close nature of the physical environment 201 and the virtual environment 202 in the system of the present invention. Rather than simply connecting virtual components with physical components, the various embodiments of the present invention are truly symbiotic and bi-directional, so that events are affected due to events and changes in one state (environment). On the other hand, changes can occur.

  In at least one embodiment, the system of the present invention does not necessarily maintain an accurate balance of each other's state among the states, but can be set to maintain a preferred primary state. For example, in at least one embodiment, the system can be configured to mirror only the events in which the virtual environment 202 dominates the physical environment 201 and the physical environment 201 occurs in the virtual environment 202. In at least one embodiment, the opposite configuration is also feasible. Any suitable priority order priority scheme can be established between the physical environment 201 and the virtual environment 202.

  In at least one embodiment, the system of the present invention provides additional benefits through the use of a controller 101 and / or host device 108 that has the ability to connect to an external server network (not shown), thereby improving the user experience. Enable. In at least one embodiment, a user 109 controlling one or more agents 104 via the controller 101 downloads virtual goods used in the game and / or digital content (eg, on sound effects or agents) The order of the light patterns when the LED is activated can be downloaded. Any suitable virtual accessory or digital content can be made available via any suitable e-commerce mechanism (eg, apps and / or resource downloads). Such content can be made available for download or attachment via a remote server by other means that allow the user to personalize or enhance their experience. In at least one embodiment, such methods (e.g., AI controlled vehicle personality, new commentators, new scenarios for existing race courses, content schemes for rendering events occurring in virtual space, and Any of various aspects of the gaming experience can be altered or enhanced through (and / or vehicle customization and enhancement).

  In addition to communication with the player controller 101 and the agent 104, in at least one embodiment, the host device 108 coordinates the commands from the user 109 with the game's operational framework to physically handle events that occur in the virtual environment 202. Coordinate with events occurring in the physical environment 201 including the agent 104.

  Referring now to FIG. 3, a block diagram illustrating some aspects of the role of the host device 108 (base station) in accordance with at least one embodiment is illustrated. FIG. 3 shows the components and structures that can be executed between the vehicle 104 controlled by the user 109 (using the controller 101) and the vehicle 104 itself. FIG. 3 also shows several mechanisms. With these mechanisms, in at least one embodiment, the user 109 cooperates or with the assistance of artificial intelligence (AI) planners 305A and 305B that run with other users 109 or on the host device 108, or both, The vehicle 104 can be controlled. In the illustrated example, four users 109M to 109Q control four agents (three vehicles 104M, 104N, and 104P and one smart accessory 105) using the controllers 101M to 101Q. Two AI planners 305A, 105B are also provided. Between the controller 101 and the agents 104M, 104N, 104P, 105, a plurality of structures that operate within the framework (base station) of the host device 108 are provided. Arrows indicate the direction of information flow.

  For illustrative purposes and clarity, FIG. 3 does not necessarily show the complete flow of all structures or information, but only shows information related to vehicle control. For example, a state in which information moves from the controller 101 to the filter 301 in one direction is illustrated. Those skilled in the art will appreciate that the control mechanisms used in connection with the present invention may also include additional data movement not shown in FIG. 3 (eg, data sent from the host device 108 to various other components (eg, It will be recognized that data on race performance reports, sounds, images, animations and / or other uncontrolled information).

  In at least one embodiment, vehicle commands generated from the user 109 are sent to the filter 301 via the controller 101. Filter 301 may refine the instructions received from controller 101 depending on the current status of vehicle 104 under control of user 109 or other aspects of the game. For example, in at least one embodiment, the speed filter 301 can be implemented. Under normal circumstances, the vehicle 104 may have an allowable moving speed range. For example, if the vehicle 104 suffers (virtual) damage as part of the game participation, the speed filter 301 can reduce this range by eliminating the upper end of this range. Other types of filters 301 can also be provided, whereby each vehicle 104 may have a plurality of different filters 104 that affect different aspects of performance. The filter 104 can control or change such parameters (e.g., speed, lane change speed, and / or device usage (e.g., bullets in an onboard weapon)).

  In at least one embodiment, an effects system 302 is provided. The effects system 302 can be input directly to a specific agent 104M, 104N, 104P, 105. The effects system 302 operates as a global actor and has a wide ability to influence vehicle behavior. This is done in some of several ways. In various embodiments, the effects system 302 simulates possible outcomes of the interaction results of the vehicle 104 and external factors that can affect the performance of the one or more vehicles 104.

  In at least one embodiment, a game engine 303 is provided. The game engine 303 includes rule sets and scenarios, and mainly induces play. Game engine 303 is closely connected to effects system 302 and often triggers effects. Similarly, effects system 302 may notify game engine 303 about events that have partially initiated an action, and may be affected by the outcome of a series of events as determined by game engine 303. For clarity, the line connecting the user 109 to the effects system 302 is omitted in FIG. 3, but those skilled in the art will recognize the effects system 302 directly or indirectly by actions taken by the user 109. You will recognize that you can trigger automatically.

  An example of actions taken directly by the user 109 in a modified racing scenario, in at least one embodiment, the user 109 may widen a virtual oil slick behind the vehicle 104 (more specifically, the oil slick is physical Since it does not exist in the target space, it is actually spread behind the vehicle representation 204 in the virtual environment 202). When the following vehicle 104 passes through the virtual oil film, it loses control (possibly temporarily). This is performed, for example, by the effects system 302, which (temporarily) reduces or eliminates the ability of the user 109 to control steering or braking of the following vehicle 104.

  To give an example of an indirect trigger of effects system 302, in at least one embodiment, when a user 109 collides with his vehicle 104 or another user 109's vehicle 104, a game is played under typical car racing rules. The engine 303 indicates to the effect system 302 that a yellow flag condition has occurred, and the effect system 302 can set a speed limit in accordance with the yellow flag condition parameter. Such a restriction may remain in place, for example, after the raceway is removed, until the game engine 303 subsequently raises the yellow flag condition.

  In at least one embodiment as shown in FIG. 3, when command information is sent past the filter 301, the arbiter 304 effectively issues a direct command to the agent, regardless of the vehicle 104 or accessory 105. In at least one embodiment, the arbiter 304 directly optimizes communication to the agents 104M, 104N, 104P, 105 that are agents or destinations. In addition, in some cases, arbiter 304 may share part of the role of filter 301 to manage or modify user commands.

  For example, the arbiter 304 may have a function of reducing or eliminating redundant commands coming from the user 109. Arbiter 304 is also useful in situations as shown for vehicle 104N (two users 109N, 109P and AI planner 305A controlling a single vehicle 104N). This is an example of a situation where a plurality of users 109 are controlling the vehicle 104. As another example, as described above, the vehicle 104 may be equipped with weapons that can be aimed and fired, with one user 109 controlling the vehicle 104 and another user 109 operating his weapon. In this case, the arbiter 304 integrates and orders separate command sets suitable for performing operations in the agent 104 desired by the separate user 109.

  In another example, by combining the user 109 with the AI planner 305, the AI planner 305 becomes a driver and the user 109 operates the weapon of the vehicle 104 or vice versa. By relying on the ability of the AI planner 305 to assist the user 109 in controlling the vehicle 104, multiple benefits can be obtained even in situations where one user 109 has the ability to operate all systems on the vehicle 104. For example, it may rely on the planner 305 to steer or smart operate around slower opponents on the course, thereby allowing the user 109 to control the speed of the vehicle 104. By designing some of the controls for the planner 305, one advantage is the broader participation of beginners or younger competitors who do not have the motor skills necessary to compete with older or more skilled users 109 There is a point to become. In this way, the planner 305 can assist the user 109 in controlling the vehicle 104 and can provide such support in a number of different ways depending on the game rules and / or the user 109 preferences.

  In another embodiment, the example of the planner 305 that controls steering of the vehicle 104 can be reversed so that the planner 305 controls the speed while the user 109 holds steering control. In a more detailed scenario of a jointly controlled vehicle 104 (or other agent), the user 109 may be confident in steering and control speed on an open course, but pass or avoid obstacles on a crowded course. May not have itself. In that case, the system may be configured to prompt the planner 305 to mediate in situations where active avoidance is necessary or when passing obstacles.

  In at least one embodiment, one user 109 may be able to control multiple agents 104 (which may include the vehicle 104 and / or the accessory 105). For example, a user 109 is configured to control one agent 104 and follow other agents. In other embodiments, the user 109 command is replicated between multiple agents 104 and responds in a consistent manner (eg, responds with all directions or velocities simultaneously). In more complex scenarios, user 109 may provide higher level commands to a group of agents 104. In the case of a racing game, for example, the plurality of vehicles 104 under such control can respond to the high-level goal direction of the user 109 and collide with another user's vehicle 104 to be off the course. In response to such a command, the agent 104 positions itself around the target vehicle 104, partially or completely surrounding the target vehicle 104, and pushes the target vehicle 104 off the course. In these cases, there may be multiple sets of actions that are performed stepwise or in sequence by multiple agents 104 acting in concert.

  In at least one embodiment, whether control of the vehicle 104 is maintained by one user 109, a single AI planner 305 or multiple users 109, or is co-maintained between a combination of the user 109 and the planner 305 Regardless, the arbiter 304 manages commands for separate and potentially independent vehicles 104 and makes selections and / or sequencing in relays to the vehicles 104. A number of behavioral aspects can be imparted to the AI planner 305 that controls the vehicle 104. For example, in addition to schemes related to drive control and strategy, AI planner 305 may embody personality characteristics that affect general behavior. For example, the AI planner 305 may simulate an aggressive driver who tends to push other vehicles 104 out of the runnable surface 601 or a driver who tries to avoid the other users 109 and avoid the confusion and proceed the course. In at least one embodiment, such an AI scheme retains a property associated with or intended for a particular event occurring on the runnable surface 601, for example, a message tricking other drivers. It can start for other users 109.

  In at least one embodiment, the capability for assistance control is provided in the vehicle 104 itself. For example, even in the absence of assistance from the host device 108, in at least one embodiment, the vehicle 104 may have the ability to maintain the relative lateral position on the runnable surface 601 through curves and corners, and the user 109 No active steering from is required. Similarly, the vehicle 104 is capable of managing its speed throughout the course according to the speed defined as part of the raceway identification (eg, encoded in a readable manner on the raceway segment 602). May be included. Also, by the same localization means through the recognition of the information encoded on the segment 602, it is either out of the course or not present on the course because there is no such information in the information detection area The vehicle 104 is determined. In such a case, even in at least one embodiment, the user 109 can control the vehicle 104, but such control is performed when the vehicle 104 is localized on the travelable surface 601. May be limited to part of the ability In some embodiments, the response of the vehicle 104 to the control of the user 109 may be similar to that of a conventional remote control vehicle.

  In other embodiments, the host device 108 may perform game control in other ways away from actual game participation. For example, the host device 108 may control various features of the vehicle 104 that operate in a physical environment as defined in software. These features include, for example, performance characteristics and vehicle capabilities. In an ongoing example of a vehicle-based game, for example, dominating the physical aspects of the vehicle's acceleration profile, maximum speed, handling and other performance in software (not by physical limitations that exist in the real world) Can do. In at least one embodiment, the system considers how these definitions are affected by outcomes in game participation in both physical and virtual space.

Differences between agents in a virtual environment The ability of software algorithms to control important aspects of the capabilities and behavior of agents 104 provides an opportunity to obtain differences in capabilities and behavior between agents 104 with the same hardware. When looking at the advantages of hardware design consistency from a simple manufacturing and cost perspective, the benefits of the same building agent 104 may be strong from a business perspective. With the present invention, it is possible to make some distinction between individual agents 104 in a group by controlling the difference by software-based means (while maintaining hardware consistency), thereby reducing the manufacturing cost. This leads to savings, and has its own advantages.

  For example, in the context of an agent that is a vehicle 104 adapted to compete in a racing or competitive driving environment as described herein, it may be advantageous to manufacture the vehicle 104 with the same hardware components. As a result, a difference in the running, behavior and response modes of the vehicle 104 occurs. For example, one vehicle 104 may function as a high-performance sports car while another vehicle may be a large raceway with low acceleration and high inertia or a lightweight compact car with a small turning radius. Such differences may or may not be indicated visually by covers that fit on different bodies or chassis (but this is for aesthetic purposes only, in at least one embodiment) Does not physically affect the performance characteristics of the vehicle in any significant manner). In software, such attributes can be easily attributed to the vehicle 104, so that the vehicle 104 functions and responds in virtual space according to the attribute to which it belongs, while the basic capabilities of the physical agent are the same. A system can be run.

  By symbiosis by the fusion of the physical environment 201 and the virtual environment 202 as described above, the main dynamic laws of the intended (simulated) physical difference of the vehicle 104 can be placed in the virtual environment 202 almost or completely. it can. For example, when two vehicles having the same weight and capacity are provided in the physical environment, by assigning behaviors to these vehicles 104 through software, it is possible to reduce the mass, power, maneuverability, etc. in the physical world. Different characteristics can be simulated. In at least one embodiment, the system of the present invention uses the physical characteristics of the vehicle representation 204 and its interaction in the virtual environment 202 to control the movement and behavior of the vehicle 104 in the physical environment 201.

  For example, two vehicles 104 having the same or similar physical mobility profiles are provided so that weight, acceleration, maneuverability and similar metrics are the same or similar to each other in the physical world. The main difference between the vehicles 104 is that in the virtual environment 202, one expresses an armored tank (low speed and high weight) and the other expresses a sedan (high speed and light weight). In at least one embodiment, applying a certain visual distinction to the physical vehicle 104 enhances the characteristic differences of the corresponding virtual representation 204. For example, the difference in virtual characteristics can be reflected by different body markings on the chassis with different markings, designs and / or colors.

  Referring now to FIG. 4, an example is illustrated. In this example, according to one embodiment, the body cover on the vehicles 104R, 104S in the physical environment 201 is used to reflect the difference in characteristics of the corresponding vehicle representations 204R, 204S in the virtual environment 202, thereby making the physical Strengthen the relationship between environment 201 and virtual environment 202. The cover of the vehicle 104S is a cover that looks like a tank, and the cover of the vehicle 104R is a cover that looks like a sedan. In at least one embodiment, simulate and reflect the fact that a full size heavy equipment vehicle (eg, a tank) outperforms a full size car even though the physical vehicles 104R, 104S are actually similar in mass and size. As such, the vehicle representations 204R, 204S are configured to interact. Although both the vehicles 104R and 104S shown in FIG. 4 operate in the physical environment 201, the dynamics of dominant body movement at the time of collision are defined in the virtual environment 202 in order to maintain the consistency of the interaction. It has been done.

  In the example of FIG. 4, the vehicle 104R (representing a sedan) is moving at high speed on a path that collides with the vehicle 104S (representing a tank). As described above, the physical environment 201 and the virtual environment 202 are closely connected to each other. Since the mass properties of these two physical vehicles 104R, 104S are the same or very similar, if a collision occurs at position 2 due to Newton's mechanism, a result will occur at position 3, in which case (the tank is The vehicle 104S (representing) moves and rotates, and the vehicle 104R (representing a sedan) moves on the course in the same way as before the collision, regardless of the low speed. This is because energy transfer to the vehicle 104S is necessary in a collision. However, in at least one embodiment, the system maintains consistency with the characteristics attributed to the representations 204R, 204S of the two vehicles in the virtual environment 202 so that the two vehicles in the physical environment 201 are consistent. The result of the collision of 104R, 104S is artificially followed by the physics of the collision in the virtual environment 202. Specifically, because the mass of the vehicle representation 204S is much larger than the mass of the vehicle representation 204R, due to the result at position 4, the vehicle representation 204R (sedan) effectively bounces from the vehicle representation 204S (tank), It has little effect on the momentum and position of the vehicle representation 204S. As shown at position 5 in the physical environment 201, the physical vehicles 104R, 104S are artificially made to follow the trajectory of the corresponding vehicle representations 204R, 204S, thereby extending the simulation to physical space. The

  In essence, in a collision between two such vehicles 104, the vehicle representation 204 thus has a significantly different mass, so the speed and direction of the vehicle 104R changes significantly. This is because the inertia of the vehicle representation 204S dominates the interaction. When actual physical laws of collision between actual vehicles 104R and 104S occur, an equal collision occurs as a result. To displace the tank. Thus, in at least one embodiment, in order to maintain the difference attributable to the vehicle in virtual space, the result of the collision is determined by the physical laws governing the vehicle parameters defined in the software and the interaction in the virtual environment. Ruled.

  Thus, in at least one embodiment, the specific details and order of events in a collision are determined according to an algorithm that governs movement in the virtual environment 202. In at least one embodiment, consistency is maintained between the physical environment and the virtual environment. In one possible approach, by enhancing such consistency, an algorithm that determines the outcome of the collision in the virtual environment 202 is used to directly influence the control of the vehicle 104 in the physical environment 201, thereby The vehicle 104 is moved in the collision event in the same (or similar) manner as in the vehicle representation 204. In at least one embodiment, the system of the present invention provides an event in which a desired level of equivalence is occurring in the virtual environment 202 by enhancing the priority protocol between the virtual environment 202 and the physical environment 201. And events in the physical environment 201. In this way, the system of the present invention enables bi-directional effects of events as described above.

  Thus, in at least one embodiment, vehicles 104R, 104S in physical environment 201 respond in cooperation with their counterparts in virtual environment 202. As a result, the vehicles 104R and 104S behave and respond according to parameters defined in the virtual environment 202 and physical laws added to such parameters in the virtual environment 202.

  In this particular example, a collision occurs between the virtual environment 202 and the actual environment 201. Since the mass characteristics assigned to the vehicle representation 204 in the virtual environment 202 do not match the mass characteristics of the physical vehicle 104, the result of the collision is different in these two environments. Here, the operation protocol gives priority to the physical laws of collision as determined in the virtual environment 202. Although the physical vehicle 104 is configured to respond according to its physical laws, in other situations it may be desirable to prioritize the physical laws and behavior of the physical environment 201 and adjust the virtual environment 202 accordingly. .

  The example shown in FIG. 4 shows the application of virtual parameters to conventional Newtonian physics, but those skilled in the art will be in a collision or any other mode of motion that is usually governed by real-world physical laws. It will be appreciated that any set of rules can be defined to govern the movement of the virtual body. In other cases, for example, a frictionless surface in a portion of the virtual environment 202 causes the physical vehicle 104 to travel on a corresponding portion of the travelable surface 601 so that the inertial or physical travelable surface 601 is flat. In such a case, the steering ability or the stopping ability and the drift ability are lost in response to the inclination simulation in which the vehicle speed and the acceleration ability are lowered as if the vehicle is rising. Numerous types of virtual forces can be introduced arbitrarily and can affect the movement of the vehicle 104 in a manner that is different from real world forces that behave only according to real world physical laws. In this way, the system of the present invention can simulate and perform behaviors that do not follow other rules (not real world laws of physics).

Error correction and raceway information A situation where the virtual environment 202 and the physical environment 201 are intended to act as mirrors to each other (ie, a situation where the laws of physics and mechanics are consistent and always intended to produce the same result) ), An imbalance may occur between events in the virtual environment 202 and the physical environment 201 due to the occurrence of an error.

  For example, when modeling the movement of the vehicle 104 when it starts traveling around a circular course, aspects such as acceleration of the vehicle 104, chassis geometry and mass, course adjustment, friction between the wheel / runnable surface 601, etc. As a result of the difference between the modeling method and the reality, errors may result. Such errors can be exacerbated if no correction is made. Therefore, if there is no protocol that establishes and maintains equivalence between the event occurring in the physical environment 201 and the event in the virtual environment 202, there is a possibility that a deviation will immediately occur between the two.

  In at least one embodiment, the system may be configured to correct such errors based on a priority enforcement scheme. This priority enforcement method specifies whether an error should be corrected according to a physical environment or a virtual environment. For example, in at least one embodiment, the system of the present invention may establish a protocol that ensures equivalence between a physical environment and a virtual environment. It is most convenient to understand that these protocols operate to ensure equivalence between these two environments through an examination of the processes that govern the localization and control of the vehicle 104 on both environments.

  Considering the basic case of a single vehicle 104 on the runnable surface 601, at the start, the vehicle 104 may not have operational information about the nature of the raceway layout or its position on the course. Extremely expensive. The host device 108 that controls the events occurring in the virtual environment 202 may not have a position with respect to the travelable surface 601 on which the vehicle 104 is mounted or the current position of the vehicle 104.

  In at least one embodiment, the user 109 provides raceway and location information as part of the setup process through means that allow the host device 108 to make a decision (eg, a digital photograph of the driveable surface 601 including the vehicle 104). Means are provided to enable this. In an embodiment of the invention that does not allow the user 109 to provide such raceway information or vehicle position on the runnable surface 601 or otherwise provide such information prior to operation, Information for establishing the road and location may be collected by the vehicle 104 itself. Specifically, the vehicle 104 may travel along the course for a distance sufficient to read out data about the identification of the particular segment 602 of the travelable surface 601 and the position of the vehicle 104 on the segment 602. After the vehicle 104 obtains this information, the information is communicated to the host device 108, which uses the information to refer to the segment 602 index and to identify the unique segment 602 identification information in the segment 602 and location. Align with identification. As a result, the host device 108 approximates the position on the travelable surface 601 of the vehicle 104.

  Identification of the segment 602 by the host device 108 provides further data about the operational limits of the vehicle 104 that can then be communicated to the vehicle 104. This data includes localization information (eg, localization information in the case of a race track, a position along the race track of the vehicle 104 and a horizontal offset from the race track center line), and motion limitations (eg, race track information). In combination with data on the maximum possible speed along various sections or whether a turn is possible at the intersection on the raceway. An advantage of providing such data to the vehicle 104 is the ability to operate independently of commands from the host device 108 (eg, depending on maintaining lateral position on the course and changing course shape). There is a point that the vehicle 104 can maintain the speed adjustment) at a low level.

  Once the location of segment 602 and vehicle 104 is determined, host device 108 uses the motion model to track the position of vehicle 104 on an ongoing basis from an initial state established by the position of vehicle 104 in physical space. can do. Upon recognizing the error source described above that causes the difference between the actual position of the vehicle 104 and the position predicted by the motion model in the virtual environment 202, the vehicle 104 in the physical environment 201 can communicate with the host device 108. Update information of the vehicle position can be obtained through regular communication. Examples of such update information data include current information about road piece IDs and position IDs, and direct position data (for example, accurate lateral position of the vehicle 104 on the raceway). Using this data either directly or as input to a calculation to update the information virtual environment 202, the host device 108 can identify the movement of the vehicle 104 in the physical environment 201 and the corresponding vehicle representation 204 in the virtual environment 202. Maintain consistency between movements.

  In the described embodiment, the movement of the physical vehicle 104 takes precedence over the corresponding vehicle representation 204. This is because the update information is based on the characteristics of the movement of the physical vehicle 104 and is provided to the virtual model. Other embodiments may operate with the reverse protocol. Thus, the movement of the virtual vehicle representation 204 can have a priority, and the update information allows the physical vehicle 104 to match the movement of the counterpart. In this situation, it can be understood that there are scenarios other than the actual operation of the vehicle 104 or even scenarios that occur in the virtual environment 202 based on model movement. As an example, the movement and interaction of a vehicle in a previously played game is captured and saved for later replay. Even if the physical vehicle 104 is prioritized in the original game participation, to replay the game participation in the physical environment 201, the recorded event is replayed in virtual form to guide the corresponding physical vehicle 104. There may be a need.

Intermediate State In at least one embodiment of the present invention that maintains a bi-directional influence between the virtual environment 202 and the physical environment 201, when a difference occurs between the physical environment and the virtual environment, the priority may be set to The assignment of priority in a given situation may vary depending on the basic structure and manner in which the game participation objectives are supported. In other embodiments, priority is not assigned to one or the other, and the system may attempt to find a compromise between the two, with one or the other located partially between the states indicated separately. In such cases, a compromise can be met by adjusting behavior and movement in both the virtual environment 202 and the physical environment 201.

Capture and Save Game Participation In at least one embodiment, maintaining the equivalence of two states for a physical environment and a virtual environment, respectively, capturing, storing, distributing and / or subsequent game participation or whole game Remodeling is possible. The ability of the host device 108 to adjust the physical environment 201 to align the virtual environment 202 (when a state occurrence and priority difference occurs in the virtual environment 202) has been modified or recorded or mostly created by past games. It can also be applied to events that are played in the virtual environment 202 and replicated directly into the physical environment 201 and to an execution order that is replayed as an evolution of action. In at least one embodiment, upon modification of a recorded game scenario or live participation of a game, the action may involve a commentator that provides an explanation of the event. For example, when an event occurs in the physical environment 201, the automatic commentator can provide related information about the change in the operation state (and color annotation) in the game participation by coordinating the order of the virtual environment 202.

Virtual Accessories As described above with respect to FIG. 1, the agent 104 may take multiple forms regardless of direct user control or control of the AI planner 305 or some combination thereof. In at least one embodiment, a virtual accessory 106 may be provided to execute digitally rendered elements in physical space.

  Referring now to FIG. 5, an example of a virtual accessory 106 in the context of the present invention embodied in a racing and shooting game is illustrated. In this case, the virtual accessory 106 shows aiming and firing of the active turret 503. Like other agents in the game space, the virtual accessory 106 maintains its presence in both the physical environment 201 and the virtual environment 202. However, due to the physical presence, the host device provides a rendering of accessory 106. In FIG. 5, a display of a turret 503 is provided by a tablet 504 (eg, iPad (but any other electronic device is also available)). The tablet 504 displays the turret 503 in a position and state corresponding to the presence 501 in the virtual environment 202.

  In at least one embodiment where the agent is a vehicle 104 operating on a runnable surface 601 printed on the media, placing the tablet 504 in a designated location and / or in a direction marked on the media, thereby providing a virtual accessory. 106 can be integrated with game participation. Alternatively, an arbitrary arrangement may be performed relative to the physical game space.

  In at least one embodiment, a tablet 504 with a rear camera can be used to host the virtual accessory 106. When placing the tablet 504, the camera can be turned on to record the video. When the tablet 504 is positioned, the tablet position and direction relative to other elements of the game space can be estimated from the input data obtained by changing the field of view of the physical game space. With this method, there is no restriction on the arrangement of the tablet 504 with respect to the position and direction.

  In at least one embodiment, rendering of virtual accessory 106 can be performed on a background image of that portion of the print media on which it is mounted. This can be accomplished by the host device 108, which can provide an image of the coated portion of the media directly to the tablet 504. Alternatively, it can be performed by taking an image of an installation area under the tablet 504 at the time of arrangement. In the latter case, this approach is useful if the virtual accessory 106 is placed beyond the game space (eg, as defined by the print media geometry constraints).

  The function of the virtual accessory 106 in the integration into the game participation is the same as that of the other agents 104. In the scenario shown in FIG. 5, the vehicle representation 204 </ b> T passes near the turret representation 501 in the virtual environment 102. Correspondingly, the physical vehicle 104T passes through the turret 503 rendered on the virtual accessory 106 (tablet 504) in the physical environment 201. When the vehicle representation 204T advances during the line of turret representation 501, the turret representation 501 is fired, resulting in a collision with the vehicle representation 204T. In at least one embodiment, when viewed on a device that displays game participation in the virtual environment 202, the entire shooting and impact are visible. For observers observing the same behavior in the physical environment 201, the rendering of radiation from the turret gun 503 may be limited to the screen of the virtual accessory 106 (in this case, the tablet 504). However, due to the equivalence maintained between the virtual game environment 202 and the physical game environment 201, however, the impacted vehicle 104T and the vehicle representation 204T have the same result, the virtual vehicle representation 204T is damaged and this In some cases, the lack of control leads to deviation from the course. The corresponding vehicle 104T in the physical environment 201 may have the ability to indicate an impact through sound and / or by LED lighting and / or other means. In addition, the movement and behavior of the vehicle 104T mirrors the lack of control as shown in the virtual environment 202.

  The invention has been described in detail with respect to possible embodiments. Those skilled in the art will appreciate that the present invention may be practiced in other embodiments. First, the specific names of components, capitalization of terms, attributes, data structures, or any other programming or structural aspects are not essential or important, and structures that implement the invention or its features are different names, It may have a format or protocol. Further, the system can be implemented via a combination of hardware and software as described, or entirely hardware elements or entirely software elements. Also, the specific division of functionality between the various system components described herein is merely illustrative and not essential. That is, a function performed by a single system component may be performed by a plurality of components, and a function performed by a plurality of components may be performed by a single component.

  In various embodiments, the present invention can be implemented as a system or method for performing the techniques described above alone or in any combination. In another embodiment, the present invention may be implemented as a computer program product. The computer program product includes a non-transitory computer readable storage medium and computer program code encoded on the medium for causing a processor in a computing device or other electronic device to perform the techniques.

  As used herein, "one embodiment" or "embodiment" means a particular feature, structure, or characteristic described with an embodiment included in at least one embodiment of the invention. In various places in the present specification, the phrase “in at least one embodiment” does not necessarily refer to the same embodiment.

  Some of the above are presented for algorithms and symbolic representations of operations on data bits in the memory of a computing device. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. The algorithm here is generally considered a consistent order of steps (instructions) that leads to the desired result. These steps are those requiring physical manipulation of physical quantities. Usually, though not required, these quantities take the form of electrical, magnetic or optical signals that can be stored, transferred, combined, compared and otherwise manipulated. In some cases, these signals may be represented as bits, values, elements, symbols, characters, expressions, numbers, etc., mainly for general usage reasons. Further, in some cases, it may be convenient to indicate a specific arrangement of steps that require physical manipulation of physical quantities as modules or code devices without loss of generality.

  However, it should be noted that all of these terms and similar terms are to be associated with the appropriate physical quantities and are merely labels that are conveniently added to these quantities. Unless otherwise specified below, throughout this description, when using terms such as “processing” or “computing” or “calculation” or “display” or “decision”, computer system memory or registers or other It is understood to refer to the operations and processes of a computer system or similar electronic computing module and / or device that manipulates and transfers data expressed as physical (electronic) quantities in such information storage, transfer or display devices. Should be.

  Certain aspects of the present invention include the process steps and instructions described herein in the form of an algorithm. The process steps and instructions of the present invention can be implemented in software, firmware and / or hardware, and when implemented as software, can be downloaded and reside on different platforms used by various operating systems. Note that it can be operated from this platform.

  The present invention also relates to a device that operates internally. This apparatus may be specially constructed for the required purposes, or may include a general purpose computing device. The general purpose computing device is reconfigured by a computer program that is selectively activated or stored in the computing device. Such a computer program may be stored in a computer-readable storage medium, and includes, but is not limited to, any type of disk (eg, floppy disk, optical disk, CD- ROM, magneto-optical disk, read only (ROM), random access memory (RAM), EPROM, EEPROM, flash memory, solid state drive, magnetic or optical card, application specific integrated circuit (ASIC)) or electronic instructions Any type of medium that is suitable for storage and each connected to a computer system bus may be mentioned. Further, the computing devices referred to herein may include a single processor or may be an architecture using multiple processor designs for high computing capabilities.

  The algorithms and displays described herein are not inherently related to any particular computing device, virtual system, or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or may be more convenient by building more specialized equipment for performing the required method steps. The required structure for a variety of these systems will appear from the description herein. In addition, the present invention is not described with reference to any particular programming language. A variety of programming languages can be used to implement the teachings of the invention as described herein, and the above description of a particular language can be used to describe the enabling and optimizing aspects of the invention. It is understood that the above description has been made.

  Thus, in various embodiments, the invention may be implemented as software, hardware and / or other elements for controlling a computer system, computing device or other electronic device, or any combination or plurality thereof. Such electronic devices include, for example, processors, input devices (eg, keyboards, mice, touchpads, raceway pads, joysticks, raceway balls, microphones, and / or the like according to techniques well known in the art. Any combination), output devices (eg, screen, speakers, etc.), memory, long-term storage (eg, magnetic storage, optical storage, etc.) and / or network connectivity. Such electronic devices may be portable or non-portable. Examples of electronic devices that can be used to practice the present invention include mobile phones, personal digital assistants, smartphones, kiosks, server computers, corporate computing devices, desktop computers, laptop computers, tablet computers, consumers Electronic devices, televisions, set top boxes, etc. An electronic device for carrying out the present invention may be any operating system (eg, Linux®, Microsoft Windows® (available from Microsoft Corporation, Redmond, Washington), MacOSX (Cupartyno, California). Apple Inc.) (available from Apple Inc.), iOS (available from Apple Inc., Cuepartino, Calif.) And / or any other operating system adapted for use on the device.

  Although a limited number of embodiments have been described for the present invention, those skilled in the art having the above advantages will recognize other embodiments without departing from the scope of the present invention as described herein. Understand that can come up with. In addition, the terminology used herein is selected primarily for readability and illustrative purposes and is not selected to describe or limit the contents of the invention. Please note that. Accordingly, the disclosure of the present invention is illustrative and is not intended to limit the scope of the invention as recited in the claims.

Claims (27)

A system,
A plurality of physical agents, wherein the physical agents are configured for interaction with each other;
Receive the signal,
Perform operations in the physical environment in response to received signals;
Sending a signal indicating the state of the physical agent in the physical environment;
A physical agent configured as follows:
At least one controller configured to receive user input for controlling at least one of the physical agents and further configured to transmit a signal for controlling at least one of the physical agents; When,
A host device,
Receiving a signal indicating a physical state of the physical agent;
Sending a signal specifying an action to be taken by the physical agent;
Storing a virtual representation of the physical agents, the virtual representations of the physical agents being configured for interaction with each other;
For at least one of the physical agents,
Receiving a signal from the physical agent, the signal indicating a position of the physical agent;
In response to a received signal indicating a position different from the position of the stored virtual representation of the physical agent, the physical agent changes its position to match the position of the stored virtual representation of the physical agent. Sending a signal for
To maintain substantial equivalence between the physical state of the physical agent and the corresponding stored virtual representation state,
A host device configured to:
Including the system,
The physical agents include mobile agents, and each mobile agent
A propulsion mechanism configured to provide motive power to the mobile agent;
A sensor configured to detect a location of the mobile agent;
A mobile radio transceiver,
A microcontroller operably connected to the propulsion mechanism, the sensor, and the mobile radio transceiver, the microcontroller configured to control movement of the mobile agent;
Including
The system further includes a runnable surface having a plurality of machine readable codes, the machine readable code indicating a position on the runnable surface;
The mobile agent includes a vehicle configured to move along the travelable surface, and a sensor of each mobile agent that detects the position of the mobile agent moves the vehicle along the travelable surface. And a sensor that detects the machine-readable code . The host device maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
The system of claim 1, comprising: For two physical agents with similar physical characteristics, the virtual representation of the two physical agents has at least one characteristic different from each other;
The system of claim 2, wherein interactions between virtual representations of the physical agent reflect a substantial difference in the at least one characteristic. For two physical agents with similar mass, the virtual representation of the two physical agents has different masses,
The system of claim 2, wherein the interaction between the virtual representations of the physical agent reflects a substantial difference in mass. The host device maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
Sending a signal to the physical agent to change their physical state to match the state of the stored virtual representation;
The system of claim 1, comprising: For two physical agents with similar physical characteristics, the virtual representation of the two physical agents has at least one characteristic different from each other;
The system of claim 5, wherein interactions between virtual representations of the physical agent reflect a substantial difference in the at least one characteristic. The host device maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
The system of claim 1, further comprising: The host device includes a wireless transceiver;
Receiving the signal by the host device includes receiving a signal by the wireless transceiver;
The system of claim 1, wherein the host device transmitting a signal comprises the wireless transceiver transmitting a signal. Each controller
Mobile computing devices,
smartphone,
Tablet computer,
Desktop computers,
Laptop computer,
Video game consoles, and kiosks,
Including at least one selected from the group consisting of:
The host device is
Mobile computing devices,
smartphone,
Tablet computer,
Desktop computers,
Laptop computer,
Video game consoles, and kiosks,
The system of claim 1, comprising at least one selected from the group consisting of: The system further includes an accessory that can be placed in the physical environment;
The host device stores a virtual representation of the accessory, and the accessory is configured to affect the state of the virtual representation of the physical agent;
In response to an interaction between a virtual representation of a physical agent and a virtual representation of the accessory, the host device changes the physical state of the physical agent to reflect the result of the interaction The system of claim 1, wherein The system further includes a mobile computing device that is deployable within the physical environment and configured to display a dynamic representation of an accessory agent;
The host device stores a virtual representation of the accessory agent, wherein the virtual representation of the accessory agent is configured for interaction with the virtual representation of the physical agent;
In response to an interaction between the virtual representation of the accessory agent and a physical agent, the host device changes the physical state of the physical agent to reflect the result of the interaction. The system of claim 1, wherein the system transmits a signal for. The at least one controller is configured to receive user input for controlling the accessory agent and further configured to transmit a signal for controlling the accessory agent;
The system of claim 11 , wherein the host device adjusts a state of the stored virtual representation of the accessory agent in response to receiving a signal to control the accessory agent. The system of claim 12 , wherein the accessory agent includes a weapon and the virtual representation of the accessory agent is configured to fire on a virtual representation of a physical agent. The virtual representation of at least one physical agent includes a weapon, the virtual representation of the physical agent configured to fire on a virtual representation of another physical agent;
In response to a virtual representation of an agent being fired,
The host device responds as if the virtual representation was fired,
The system of claim 1, wherein the host device sends a signal to the corresponding physical agent to change its physical state to respond as if it were fired.   The system of claim 1, wherein the at least two controllers are configured to control a common physical agent. The system further includes an automated computing system configured to automatically control at least one of the physical agents;
At least one of the physical agents is configured to receive a signal from the automated computing system and perform an operation in the physical environment in response to the received signal, the physical agent configured to The system of claim 1, configured for interaction with at least one physical agent that is not controlled by a computing system.   The said at least one controller sending a signal to control at least one of said physical agents comprises said at least one controller sending a signal over the Internet. system. At least one configured to receive a signal from the host device indicating a state of the virtual representation of the physical agent, and further configured to display a dynamic representation of the physical agent based on the received signal Display devices,
The system of claim 1, further comprising: That the host device maintains substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation for at least one of the physical agents;
Receiving a signal from the physical agent, the signal indicating a position of the physical agent;
Adjusting the position of the stored virtual representation of the physical agent to match the indicated position of the physical agent;
The system of claim 1, further comprising: A method for using a mobile computing device to control a physical agent in a robotic system, wherein the physical agents are configured for interaction with each other, the method comprising:
Receiving a signal indicative of a physical state of the physical agent in a physical environment;
Sending a signal specifying a physical action to be taken by the physical agent in the physical environment;
Storing a virtual representation of the physical agents, the virtual representations of the physical agents being configured for interaction with each other;
For at least one of the physical agents,
Receiving a signal from the physical agent, the signal indicating a position of the physical agent;
In response to a received signal indicating a position different from the position of the stored virtual representation of the physical agent, the physical agent changes its position to match the position of the stored virtual representation of the physical agent. Sending a signal for
Maintaining a substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation of the physical agent;
Including performing
The physical agent is further configured to perform a physical operation in response to a signal received from a mobile computing device that functions as a controller operated by a user ;
The physical agents include mobile agents, and each mobile agent
A propulsion mechanism configured to provide motive power to the mobile agent;
A sensor configured to detect a location of the mobile agent;
A mobile radio transceiver,
A microcontroller operably connected to the propulsion mechanism, the sensor, and the mobile radio transceiver, the microcontroller configured to control movement of the mobile agent;
Including
The system further includes a runnable surface having a plurality of machine readable codes, the machine readable code indicating a position on the runnable surface;
The mobile agent includes a vehicle configured to move along the travelable surface, and a sensor of each mobile agent that detects the position of the mobile agent moves the vehicle along the travelable surface. Including a sensor for detecting the machine-readable code . Maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation,
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
21. The method of claim 20 , further comprising: Maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation,
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
Sending a signal to the physical agent to change their physical state to match the state of the stored virtual representation;
21. The method of claim 20 , comprising: Maintaining substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation,
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
21. The method of claim 20 , further comprising: A computer program product for using a mobile computing device to control a physical agent within a robotic system, wherein the physical agents are configured for interaction with each other, the computer program product comprising:
A non-transitory computer readable storage medium;
Computer program code encoded on the medium, the computer program code being transmitted to at least one processor in a host device,
Receiving a signal indicative of a physical state of the physical agent in a physical environment;
Sending a signal specifying a physical action to be taken by the physical agent in the physical environment;
Storing a virtual representation of the physical agents, the virtual representations of the physical agents being configured for interaction with each other;
For at least one of the physical agents,
Receiving a signal from the physical agent, the signal indicating a position of the physical agent;
In response to a received signal indicating a position different from the position of the stored virtual representation of the physical agent, the physical agent changes its position to match the position of the stored virtual representation of the physical agent. Sending a signal for
Maintaining a substantial equivalence between the physical state of the physical agent and the state of the corresponding stored virtual representation of the physical agent;
Computer program code configured to execute
Including
The physical agent is further configured to perform a physical operation in response to a signal received from a mobile computing device that functions as a controller operated by a user ;
The physical agents include mobile agents, and each mobile agent
A propulsion mechanism configured to provide motive power to the mobile agent;
A sensor configured to detect a location of the mobile agent;
A mobile radio transceiver,
A microcontroller operably connected to the propulsion mechanism, the sensor, and the mobile radio transceiver, the microcontroller configured to control movement of the mobile agent;
Including
The system further includes a runnable surface having a plurality of machine readable codes, the machine readable code indicating a position on the runnable surface;
The mobile agent includes a vehicle configured to move along the travelable surface, and a sensor of each mobile agent that detects the position of the mobile agent moves the vehicle along the travelable surface. A computer program product comprising a sensor for detecting the machine-readable code . Computer program code configured to cause at least one processor to maintain substantial equivalence between a physical state of the physical agent and a state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
25. The computer program product of claim 24 , further comprising computer program code configured to cause at least one processor to execute. Computer program code configured to cause at least one processor to maintain substantial equivalence between a physical state of the physical agent and a state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on the interaction between the virtual representations;
Sending a signal to the physical agent to change their physical state to match the state of the stored virtual representation;
25. The computer program product of claim 24 , further comprising computer program code configured to cause at least one processor to execute. Computer program code configured to cause at least one processor to maintain substantial equivalence between a physical state of the physical agent and a state of the corresponding stored virtual representation;
Adjusting the state of the stored virtual representation based on a received signal indicative of a physical state of the physical agent;
25. The computer program product of claim 24 , further comprising computer program code configured to cause at least one processor to execute.

Images