KR101793189B1 - Integration of a robotic system with one or more mobile computing devices - Google Patents

Abstract

The robotic system is integrated with one or more mobile computing devices. Under the control of the user or users, the physical configurations, or agents, of the individual components of the system in physical space are replicated in representation in virtual space. Some real-time equivalence is maintained between the physical and virtual spaces to implement a virtual environment that reflects the physical environment. Events that occur in one environment can directly affect the outcome of the progress of events that occur in the other environment and cause the outcome. By doing so, the elements of virtual space become precisely interdependent and unified with respect to peers that relate to elements in physical space. In at least one embodiment, the system of the present invention is implemented as an application of entertainment, such as the appearance of a video game in physical space.

Description

[0001] INTEGRATION OF A ROBOTIC SYSTEM WITH ONE OR MORE MOBILE COMPUTING DEVICES [0002]

Cross-reference of related application - reference

This application claims priority from U.S. Provisional Application No. 61 / 693,687, entitled " Integration of a Robotic System with One or More Mobile Computing Devices "(Attorney Docket ANK002-PROV), filed on August 27, Which is incorporated herein by reference.

The present invention relates to robotic systems that can be integrated with mobile computing devices.

One of the challenges of building robotic systems that can be mobile, autonomous, and / or otherwise operating with a certain perception of the environment is that costs can be important. Producing such systems at a cost that can support sales to the consumer market has been a historic obstacle to the development of consumer robots. Market opportunities to give a certain level of intelligence or capability based on artificial intelligence on products are enormous, but the cost structure of these products, especially commodity products, It makes it expensive.

A particular cause of such costs is the use of dedicated hardware and firmware for such robotic systems, including control systems and autonomous components. Given the cost-competitive nature of commodity-based product markets, in certain systems related to entertainment products, it may not be feasible to produce and sell products for such robotic applications in a profitable manner.

According to various embodiments of the present invention, mobile computing devices such as smart phones and / or tablets have an impact as a platform for supporting and controlling robotic systems. The rapid growth of such devices among consumers has made the assumption that a sufficient number of such devices are available for use by consumers in connection with the robotic system is feasible and realistic.

By using mobile computing devices, such as smartphones, to support at least a portion of the operation of the robotic system, the techniques of the present invention provide significant potential reductions in hardware requirements in the robotic system. In addition, such devices typically have rich user interfaces, notable computer performance, and built-in wireless connectivity, thus allowing devices to be used as a robot control system for robotic systems as described herein Ideally.

According to various embodiments of the present invention, a robotic system is integrated with one or more mobile computing devices. Such computing devices may include, for example, one or more smartphones, tablet computers, laptop computers, gaming consoles, kiosks, and / or the like. In other embodiments, techniques of the present invention may be implemented using any other suitable computing devices, such as desktop computers, web-based computing architectures, and / or the like. Such computing components may be networked with each other, and / or with physical components such as agents under the control of a user or users; Such networking may utilize Bluetooth, WiFi, and / or other wireless networking technologies.

In various embodiments as described herein, under the control of a user or users, the physical configurations, or agents, of individual components of the system in physical space are replicated in representation in virtual space. Some real-time equivalence can be maintained between physical and virtual spaces to implement a virtual environment that reflects the physical environment. In this way, the present invention supports a system in which events occurring in one environment can affect the outcome of the progress of events occurring in another environment and cause the results. By doing so, the elements of the virtual space can be precisely interdependent and unified with respect to the peers that are associated with the elements of physical space.

The integration of virtual and physical was the so-called tenants of augmented reality; Up to now, augmented reality applications have produced many examples of single-directional action (one that affects virtually, or vice versa), but they have not been able to compose a robotic bi-directional system. In at least one embodiment of the invention, an exactly symbiotic system is implemented in which the virtual environment affects the physical environment in a coherent manner, and vice versa.

In at least one embodiment, the system of the present invention is implemented as an application of entertainment, such as the appearance of a video game in physical space. Games and entertainment are an interesting use of the present invention for its inclusion of interaction among multiple users. In at least one embodiment, the system of the present invention provides an environment in which users may control one or more agents of the system, while one or more other agents may be controlled by artificial intelligence.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings illustrate a number of embodiments of the invention and, together with the description, serve to explain the principles of the invention in accordance with the embodiments. Those skilled in the art will recognize that the specific embodiments illustrated in the drawings are merely exemplary and are not intended to limit the scope of the invention.
1 is a block diagram illustrating an architecture for implementing the invention in accordance with one embodiment.
FIG. 2 illustrates an example of the integration of events and functions in physical and virtual space, according to one embodiment.
3 illustrates aspects of the role of a host device (base station) in accordance with at least one embodiment and includes blocks that illustrate the various components and structures that implement the relationships between users controlling their vehicles and their own vehicles .
4 illustrates an example of body covers on vehicles being used to reflect differences in the characteristics of corresponding representations of vehicles in a virtual environment, and thus to enhance the relationship between physical and virtual environments, according to one embodiment .
5 illustrates an example of the use of a virtual accessory for implementing an element that is digitally rendered within a physical space, according to one embodiment.
Figure 6 illustrates an example of an embodiment of the present invention for implementing a game play environment in which vehicles race on a race track, according to one embodiment.

For illustrative purposes, the present invention will be described largely in the context of a system herein embodying a car racing game, in which the agents under user control compete on a physical track, physical vehicles or accessories associated with game-play. Further details regarding such systems and their implementations for integrating virtual and physical environments are provided in "Distributed System of Autonomously Controlled Mobile Agents" (Attorney Docket No. ANK001CONT), filed December 6, 2012 U.S. Patent Application Serial No. 13 / 707,512, which is hereby incorporated by reference herein. However, those of ordinary skill in the art will recognize that the techniques described herein may be implemented in other contexts and environments, and need not be limited to vehicles on physical tracks. Thus, the term "vehicle" as described herein can be controlled and operated in the manner described herein, while also extending to any mobile agent that can be represented in a virtual environment as described herein Will be taken.

Although the present invention is described herein primarily in the context of an application of entertainment, one of ordinary skill in the art will recognize that the present invention may be implemented in many different contexts, including contexts that need not be associated with entertainment something to do.

System architecture

Referring now to Figure 1, there is shown an architecture for implementing the invention in accordance with one embodiment. 1, the game play is hosted by a host device 108, which may be, for example, a smartphone, tablet, laptop computer, or other, and / or Or any combination thereof, such as any combination thereof. In at least one embodiment, the host device 108 supports and drives various algorithms included in software that implements game operations. The host device 108 and associated software are collectively referred to herein as a base station or central control device.

Any of a variety of different devices may serve as the host device 108; Examples include smartphones, tablet computers, laptop computers, desktop computers, video game consoles, and / or any other computing device capable of supporting control software for the system. In at least one embodiment, such a device may be, for example and without limitation: iOS or MacOS available from Apple of Cupertino, CA; Android available from Google Inc., Mountain View, California; Or any available operating system, including Windows available from Microsoft Corporation of Redmond, Washington. In at least one embodiment, host device 108 is an iPhone or iPad available from Apple Inc. of Cupertino, Calif., Which drives an appropriate software application ("app"). In at least one embodiment, the software for controlling the host device 108 includes software for controlling, coordinating and executing game play according to rules, user-controlled operations, and / or artificial intelligence, Such as a downloadable application ("app") that includes appropriate functionality and a game display structure for operating application programs 104A through 104F. In at least one embodiment, the host device 108 maintains the state of the agents 104A through 104F and sends and receives commands to and from the agents 104A through 104F. The host device 108 may also include a suitable user interface for facilitating user interaction with the system.

In at least one embodiment, agents 104A-104F are mobile agents, such as vehicles, and so are referred to herein, although they may be other objects or components.

In at least one embodiment, the host device 108 is configured to determine whether the instructions originate from algorithms running on the host device 108 or are routed through the host device 108, but are physically present or remotely located, All activities that are sent to other components, such as agents 104A-104F and / or accessories 105,106, regardless of whether they originate from control devices 101D-101K, It is a central node for control commands. In other embodiments, a more distributed architecture may be implemented in which the host device 108 need not be a central node for all activity and control commands.

The example shown in FIG. 1 includes a specific number of controllers 101D through 101K, agents 104B through 104H, accessories 105 and 106 (which may also be considered as a type of agent), AI- (Which may also be considered as an agent type), and other components. Those skilled in the art will appreciate that certain quantities of these components shown in FIG. 1 and described herein are only exemplary, and that the invention may be practiced using any other quantity and / But may be implemented with some of the elements.

In the architecture of FIG. 1, system 100 is implemented in a centralized manner, where controllers 101D through 101K and agents (including vehicles 104A through 104F), along with other components, And communicates with the device 108. As shown, in at least one embodiment, multiple users 109 (or players) may control multiple agents in the form of vehicles 104A-104F, while other agents / (104J) can be controlled by artificial intelligence.

As shown in FIG. 1, any number of external devices may be connected to the host device 108 via any suitable communication protocol, such as, for example, a cellular / internet connection 107. The various external devices may or may not be the same as the host device 108. Some or all of the external devices act as player controllers. 1 shows a game console 101B having an arbitrary number of controllers 101J and 101K (controlled by users 109J and 109K, respectively); Laptop computer 101D (controlled by user 109D); Stand alone controller 101E (controlled by user 109E); And smartphones 101F, 101G, and 101H (controlled by users 109F, 109G, and 109H), as well as various devices that can be used as player controllers. In at least one embodiment, any or all of the controllers 101 may be an iPhone or iPad available from Apple Inc. of Cupertino, Calif., Which drives an appropriate software application ("app"). The controllers 101J, 101K, 101E may be of any suitable type, including for example controllers for use with common console game devices, for example.

In the embodiment shown in FIG. 1, the game is hosted on the host device 108. The host device 108 supports game play in the virtual space under the direction of the software as well as physical space in the physical environment (such as a race track); The state of the virtual environment is maintained in memory on the host device 108 and / or elsewhere.

Referring now also to FIG. 6, in accordance with an embodiment, there is shown an embodiment of the present invention for implementing a game play environment in which vehicles 104 (race cars) race on a drivable surface 601 (such as a race track) An example of an embodiment is shown. However, one of ordinary skill in the art will recognize that such an embodiment is only one example of an implementation of the present invention; For example, the system may be implemented in a different physical environment, with agents other than vehicles, and / or with different types of tracks or with no tracks at all.

As described in related U.S. Utility No. 13 / 707,512, entitled "Distributed System of Autonomously Controlled Mobile Agents ", the driveable surface 601 is, in at least one embodiment, a physical model of one or more roads, , Traffic lights 105, railroad crossings, and / or the like. The vehicles 104 are mobile agents that can move independently. The vehicles 104 may be physically modeled following vehicles, trucks, ambulances, animals or any other desired form. In at least one embodiment, each vehicle may receive commands from host device 108 / host device 108 via one or more sensors 604 that can read information from a driveable surface 601 and via, for example, (Not shown) that can transmit and receive information and / or other information.

As shown in Figure 6, the driveable surface 601 may include any number of partitions 602, although such a partitioned arrangement is optional. Such segments 602 may be concatenated at certain connection points and may be reconfigured to construct any desired structure. This structure is referred to as a driveable surface 601. Road segments 602 include consecutive zones in which one or more vehicles 104 can search for so-called drivable sections and connect them across each other using a simple click-in mechanism present at each junction. Each road segment 602 may also selectively transmit power to an adjacent road piece 602 and may optionally include a microcontroller for advanced functions, such as traffic lights 603 and others. have. The driveable surface 601 may also be provided as a single piece. The driveable surface 601 (and / or partitions 602) may be collapsible, collapsible, spoken, or otherwise miniaturized for storage.

In at least one embodiment, the base station software, running on the host device 108, updates events stored in the physical environment by updating stored information about the vehicle 104 location, direction, speed, and other aspects that characterize game events. Equivalence works on virtual versions of physical games that keep on going. In at least one embodiment, the host device 108 may determine that the game state in the physical environment and the virtual environment at any time in time is the same (or substantially the same), or at least the game state in the virtual environment At least to a sufficient degree of accuracy.

In at least one embodiment, the AI software runs on the host device 108 and issues commands for controlling one or more vehicles 104J operating on the track 601 (including wireless communication mechanisms or other mechanisms through). In other embodiments, the software for controlling the vehicles 104J may be located elsewhere and / or may be run on their own vehicles 104J.

In at least one embodiment, the host device 108 may serve as a control unit for a human user 109A that simultaneously controls the vehicle 104 (in the example shown, the human user 109A is a vehicle 104A) Lt; / RTI > the host device 108). Such a function may be implemented on the host device 108 while the host device 108 also serves as a forwarder and interpreter for controlling incoming commands from other devices 101D through 101K controlling other vehicles 104B through 104F. Can be provided. In another embodiment, the host device 108 does not serve as a control unit for the human user 109, but rather acts as a dedicated central control device.

In at least one embodiment, agents under user control (such as vehicles 104B through 104F) need not be formally or functionally consistent. For example, the users 109 may control objects or elements (traffic lights, railroad crossings, gun turrets, drawbridges, pedestrians, and / or the like) There is a chance to be given.

The player controllers 101D through 101K may communicate directly with the host device 108 or the player controllers may communicate through the intermediary devices. For example, in FIG. 1, controllers 101J and 101K communicate with host device 108 via game console 101B. Similarly, any number of knots of connections may be configured between the host device and the player controllers, such as one or more smartphones connecting to the host devices through a series of devices that are networked to the host device again.

Fig. 1 shows an example in which the vehicles 104B to 104F are controlled by the human users 109B to 109F, respectively. Additional agents, referred to as accessories 105 and 106, may also be controlled by human users 109, or additional agents may operate automatically (e.g., at host device 108 Under the direction of artificial intelligence software running elsewhere). Each accessory 105,106 may be electrically or mechanically identical and may be a physical or virtual item that may be used to directly affect gameplay environments and / or aspects of other agents 104. [ In this example, the accessory 105 is a physical traffic light as shown in FIG. Other examples of physical accessories may be obstructions, crossbar breakers, doorways, and / or the like; Such devices may be communicatively coupled to the host device 108 to control their operation in association with game play. In at least one embodiment, the user 109 may interact with the game display by altering the physical state of the accessory 105 and thereby doing so.

Smart accessories 105 may also be present as passive components in physical space. An example of a passive smart accessory is a decal that is intended to indicate a possible risk on a drivable surface 601 that is placed at some location before or during play. Such decals may include encoding techniques that allow their unique identification by vehicles 104 (e.g., as vehicles avoid it). In at least one embodiment, the user 109 controlling one or more such passive smart accessories may selectively activate or deactivate the accessories individually or in groups. Even in a physical environment, such as in a virtual environment, such as accessories, for example:

• oil slicks that can stop steering and braking of vehicles that avoid them;

● Landmines;

● nails;

Power supplies that facilitate the performance of the first vehicle 104 to avoid them physically when they are activated

And the like.

Although the passive accessories 105 do not necessarily need to indicate a state change in a physical way, the host device 108 can report changes in virtual state and, via the controllers 101 in real time, Lt; RTI ID = 0.0 > 109 < / RTI > The behavior of the vehicles 104 may also be altered in response to changes in the virtual state of the accessories 105.

The accessory 106 is an example of a virtual accessory, which has no physical components other than a computing device (such as a smartphone or tablet computer or the like) having an appropriate output device (such as a display screen). The virtual accessory 106 may be physically located at a particular location in the physical game environment to render the accessory appropriately in both appearance and state. Referring now to FIG. 5, an example of a virtual accessory (tablet 106) that implements a digitally rendered element (image 503 of a turret) within a physical space is shown, according to one embodiment. The image 503 can be moved and animated so that it appears to interact with the physical vehicles 104 in the game play area; For example, an image may be made visible to the physical vehicle 104, and the physical vehicle 104 may be made to respond as if it were shot. These behaviors and interactions also reflect the virtual environment, as shown in FIG. 5, where the turret 501 is firing at the virtual representation 502 of the vehicle 104. [ In various embodiments, the accessory 105,106 may operate under the control of artificial intelligence software running on the host device 108 and / or elsewhere, without the need for relying on a human user for operation.

It will be appreciated by those skilled in the art that as the number of users 109 and the number of AI-control partners increases, the performance requirements for host device 108 increase similarly. Depending on the number of agents 104 and the capabilities of the host device 108, increases in computer requirements may affect game performance, for example. In at least one embodiment of the invention, the system is implemented in a distributed environment, where, for example, the host device 108 may be coupled to any number of devices that it may support to execute the logic, And the like. Examples of these may include smartphones, tablet computers, laptops, game consoles, and / or the like, but may also be any suitable devices capable of providing the support needed to drive the logic assigned to it . In at least one embodiment, for example, some of the processing tasks associated with operating system 100 may be distributed to one or more controllers 101D through 101H.

The variance does not need to remain local; In at least one embodiment; The logic may be distributed, for example, to one or more remotely located servers. The module design for the structure of the host device 108 may provide itself for convenient distribution of logic and the type of logic processes that are offloaded from the host device 108 need be one particular type of function or process none. In at least one embodiment, for example, the variance of the logic may be prioritized according to computer and memory requirements, and therefore the resources of such a heavily loaded host device 108 should be allocated elsewhere.

The wireless interface employed for communication between the controllers 101D through 101H and / or controllers need not be the same as the interface used to connect to the agents 104A through 104F under the control of the user 109. [ For example, while the host device 108 communicates with the controllers 101D through 101H via Wi-Fi, the host device 108 communicates with the agents 104A through 104F via Bluetooth Quot;). In such a case, the host device 108 may serve as a bridge between a high-power protocol (such as Wi-Fi) and a low-power protocol (such as Bluetooth). The advantage of such an approach is that the vehicles 104 (in the case of vehicles 104J under AI control) controlled by the users 109 via the host device 108 or directly controlled by the host device 108 And can be recognized in instances with limited reserve power.

In certain Bluetooth low energy (BTLE or BLE) or similar capable wireless protocols, another benefit imposed by the use of Bluetooth is that agents 104 use wireless protocols to communicate with similarly enabled BTLE / wireless devices It is possible. In one embodiment, for example, a user 109 who wants to take control of a particular vehicle 104 or active smart accessory 105 may control the intended controller 101 (e.g., BTLE- Equipped smartphone). By exercising the influence of the BTLE capability to determine proximity or relative distance to another BTLE-enabled device, the user 109 can bring two BTLE-equipped devices within a critical range of distance. In at least one embodiment, this may lead to a data exchange between the smartphone (e.g., 101F) and the vehicle 104, while presenting the option of the selected vehicle 104 for play to the user 109. [ The selection is then communicated to the host device 108 by displaying a pair between the smartphone 101 of the vehicle 104 and the user 109 and is now designated as the controlling device of the vehicle 104. [

In various embodiments, BTLE data may be exchanged between vehicles 104 and / or similarly similarly wirelessly enabled agents may be used in other ways. For example, users or observers may receive information about the situation of the agent 104 regarding gameplay, overall lifetime usage, and / or historical accomplishments, or they may be able to perform diagnostics, It can be customized.

As described above, the controllers 101D through 101H may be implemented using any suitable devices. Again, less complex controllers 101J, 101K, such as wireless game pads or joysticks, may be used. In instances where a game pad or joystick 101J, 101K is used that is not equipped with a wireless communication module that supports direct communication with the host device 108, the connection to the host device 108 may be controlled by the game console 101B or other intermediary (Not shown) that plugs into an appropriate port on the host device 108, or through the use of a dongle (not shown) that plugs into the appropriate port on the host device 108. Such a dongle connects wirelessly to the controller 101 and communicates communications over the port to which it is plugged. Alternative embodiments of the dongle may include devices that implement bridges between a wireless protocol compatible with the controller 101 and a wireless protocol compatible with the host device 108.

In addition to communicating the commands of the users 109 to the vehicles 104B through 104F via the host device 108, the controllers 101D through 101H may also communicate commands from the host device 108 that reflects the current state of the game Updates can be received. In at least one embodiment, some or all of the controllers 101D through 101H may include one or more output devices (such as a display, speaker, haptic output mechanisms, and / or display devices) to improve the play experience based on such received status information. / RTI > and / or the like). Such enhancements may include, for example, renderings, haptic outputs (e.g., vibrations, gestures, etc.) that display and / or increase the motion on the game track to increase the sense of reality or provide details not visible to the user 109 ) And / or audio.

In at least one embodiment, the visual, haptic, and / or audio information presented to the users 109 through some or all of the controllers 101D-101H may be unique for each device. The various agents 104 can be in different states at any given time (e.g., in terms of location, speed, situation, operation, and the like); Further, in the case of multiple users 109 that control a single agent 104, user roles or control may be different. Thus, the various cues and data presented to each user 109 can be tailored to the current state of the agent 104 and the specific role of the user 109. [ For example, in at least one embodiment, the racing game may present the renderings of the racing course in terms of the vehicle 104 while displaying the vehicle-specific data to the user 109. [ In an embodiment where more than one user 109 may share control of a single agent 104, such as a vehicle in which a user 109 takes the role of a driver while a weapon that actuates the weapon is mounted, It may be appropriate that the various types of information transmitted to the agent 101 may differ in one or more aspects for each user 109 associated with the agent 104. [

In various embodiments, users of controllers 101D through 101H may physically exist such that controllers 101D through 101H communicate with host device 108 directly (via a wireless protocol such as Bluetooth). Alternatively, users of the controllers 101D through 101H may be located remotely and connected via a host network (such as network 107). The controllers 101D through 101H may rely on information reported from the host device 108 again regarding the game situation.

In at least one embodiment, maintaining tight coupling and equality between states of the virtual play environment and the physical play environment enables users 109 to control vehicles 104 on a track 601 located remotely . Such an architecture permits locally non-existent users to participate through relying on the virtual representation of the game. In at least one embodiment, the remote users 109 may have the same degree of control over their vehicles 104 as the local users 109 are, and they may be functionally identical to the local users 109 You can have a view of events.

In at least one embodiment, the crowd can view game events at a remote location by utilizing an external network connection. In the example of FIG. 1, the spectator, or observer 110, observes game events via the tablet 103 that communicates with the host device 108 via the cellular / internet connection 107. Because the actions occurring in the physical space are reflected in real time within the virtual environment, the tablet 103 (or any other device used to display the game play activity) is able to display rich renderings rich renderings and users 109, additional information about the situation and other possible details of interest to the observer 110.

In at least one embodiment, the remote user 109 (or a local user) has an auxiliary device (not shown) for displaying a game independent of the controller 101 that he or she is using to control the physical agent 104. [ Can be additionally used. For example, if the remote user's controller 101 has a small screen, it may be useful to have such an auxiliary device for use as a second screen, thereby providing a more complete and realistic gaming experience. For example, the user-specific information may be displayed on one display (such as a display on the controller 101), while a visualization of the game play may be presented on the secondary display. Using separate devices or devices as monitors will mitigate potential display restrictions and provide a more familiar experience for console gamers who are familiar with using handheld game pads to control console gamers' behavior on separate screens .

It is entirely possible to reproduce the game play in physical space in real time on the duplicated physical track, although the examples discussed herein illustrate the dependence on rendering or other representations of game play in virtual space. Since equivalence is maintained between virtual and physical, the remote user 109 can directly reproduce the play on the physical track that sufficiently matches the physical track 601 hosting the game play through reflecting the state of the virtual model have. Wireless communication between the controller and the host device 108 for replicated physical tracks can be used to cause vehicles on the replicated tracks to move and behave in a manner that replicates (or at least approximates) the operations.

Host device 108 and / or other components may provide additional functionality. For example, the one or more controller (s) 101 may provide additional information related to events occurring in the game and / or may serve as a direct platform for additional game play. Such devices include, for example:

A user interface capable of responding to touch, button control, orientation with respect to basic directions or device tilt / acceleration, and / or any other suitable user input;

● display screen;

• Include one or more audio components such as speakers and microphone, and may have any suitable components for providing a desired degree of interaction.

Those of ordinary skill in the art will recognize that the list is representative and not exhaustive.

One set of such enhancements may be generically referred to as non-operable information. For example, they include audio, video, and / or haptic output, separately or in any suitable combination. Certain types of output may include, for example, sound effects, music, vibrations, images, animations, film clips, and / or the like. Any of these types may be used alone or in any suitable combination to improve the sense of reality or enrich the sense aspects of the gaming experience. Alternatively, these enhancements may include summary or additional information about aspects of the game and / or its users 109 such as differential performance data, racer rankings over time, competition leaderboards, and / or other data And can take the form of data to be provided.

In at least one embodiment, highly interactive game controllers 101, such as smartphones and tablets, are capable of playing games or agents side by side in games or agents that operate within it, And / or to provide improvements to the game experience by exerting the influence of control and connection capabilities that can generate play scenarios other than the direct actions that occur in the physical environment. For example, in the context of a racing game, the vehicle 104 may arrive at a pit stop on a physical race track; The game play is then switched to the controller 101 if the user 109 is faced with the task of servicing the vehicle 104 as a member of a pit crew in an entirely digital context similar to the conventional video game format . As another example, the vehicle 104 may be driven to a particular point in the physical space, such as a building located on a play set, at which point the game play may be initiated by the driver 104, The display in the game controller is switched to the display in the case of continuing inside the building in the rendered environment. Such examples illustrate scenarios in which game play can evenly switch between a physical play environment and a digital play environment. The digital environment may be supported by the controller 101 and / or the highly interactive controller 101 implemented by an independent display console (not shown) that communicates with the controller 101 and / or the host device 108 . In such scenarios, the integration of parts or sections of the game display occurring in a purely digital space in the form of a traditional video game for the entire system may require that these parts of the experience be disconnected from, or affect, Does not mean not. Considering the example of the user 109 that causes the physical vehicle 104 to drive to a location that provides access to the digital portion of the play, such as an oil refueling station, the user 109 is no longer actively moving the physical vehicle 104 Although not controllable, the digital space may provide opportunities for mounting or repairing the vehicle 104 and thereby affecting the performance of the vehicle 104 when the user 109 resumes its control. Other examples may include a user 109 similarly entering a pure digital environment that provides control of physical accessories such as a bridge on a race course, which may allow users 109 to move from within a purely digital space to a physical environment Lt; RTI ID = 0.0 > 602 < / RTI > Similarly, such control may also be provided for virtual accessories 106, such as those hosted by an Android- or iOS-based device that is played in a physical play environment, They interact and / or influence their behavior.

In at least one embodiment in which the present invention is implemented as a competing racing game, the controller 101 may provide a function for operating the speed of the vehicle 104 and the steering device. However, further exploitation of the capabilities of a smartphone or similarly capable device as controller 101 may enable a wider range of game controls. For example, in the "death race" scenario, the vehicles 104 may be equipped with weapons that can be directed to the competing vehicles 104. [ In such an embodiment, the highly interactive controller 101 may be used to support basic vehicle speeds and functions beyond the steering device. Additional functions, such as, for example, selecting the target vehicle 104 from among the active representations of all the vehicles 104 racing on the drivable surface 601, or aiming the weapons mounted on the vehicle 104 Can be provided.

As mentioned above, in at least one embodiment, multiple users 109 may control a single agent 104 in the game. For example, in a racing game embodiment, three users 109 may be configured to allow one user 109 to provide control and speed control, another user 109 to aim and fire a forward cannon, 109 can control a single vehicle 104 operating the lateral cannon. In such a scenario, the controllers 101 provide information tailored to the role fulfilled by each user 109. For example, the user 109 driving the vehicle 104 may find display information and control techniques similar to those of a typical racing game, while the users 109 operating the cannons may find that the displayed viewpoint And a rendered view of the physical play environment including the virtual representations of the track elements and competing vehicles 104 on and on trajectories relative to their actual positions in the physical play environment can see.

For users 109 operating weapons in this example, the resulting control and interaction experience provided by the smart device may be similar to a first person shooter game.

The cannons need not (but can be) represented by any physical structure or element on their own physical vehicles 104; Rather, they can be represented in a virtual environment instead of (or in addition to) having such a physical structure. In at least one embodiment, when the cannon is fired, the rendering of such a foam may be presented in a virtual environment, and optionally the physical vehicles 104 may include some visual and / or audible indicia of activation of such weapons , Movement of physical cannon guns, sounds, and / or the like). The target of foaming (e.g., the other vehicle 104) may react as if it were hit by, for example, changing its course, turning it over, becoming disabled, and / or the like; Visual feedback can be provided in virtual and / or physical environments, for example, by target emitters that make it appear as if it burns out or otherwise. LEDs or other visual and / or audible components can be installed on the vehicles 104 to provide sounds and fog lights reproducing the noise of the machine cannon, It can be generated simultaneously with the cannon firing.

In at least one embodiment, the vehicles 104 may be designed to exhibit (impaired) damage in physical space. For example, the LED may indicate that the vehicle 104 is wearing damage, or the color change of the LED array may indicate the current damage status of the vehicle 104. [ In at least one embodiment, more complex approaches can be used to simulate or impose damage to vehicle 104 regardless of weapon attacks, impacts, or other means. These may include mechanisms that impart kinetic energy, such as may be the result of a collision or explosive force operating on the vehicle 104, or portions that are gradually shrunk from the vehicle 104 when triggered. In at least one embodiment, the vehicles 104 exhibit impaired damage by altering their behavior, for example, to pretend to be a punctured tire, a damaged steering system or engine components, or the like.

Bi-directional interaction between physical and virtual environments

In at least one embodiment, the system of the present invention maintains equivalence in game spaces occurring simultaneously in virtual and physical spaces by coordinating events at a lower level to allow bi-directional behavior between physical and virtual environments. Referring now to FIG. 2, an example of such integration of events and functions in both physical space and virtual space is shown. In this example, the mobile agents are vehicles competing in physical space; In addition, vehicles are equipped with a racing game equipped with virtual weapons.

The figure shows a series of events involving virtual events that affect physical events and vice versa. While the cars are racing on a physical course, the base station maintains a virtual representation of the race condition in real time, so that the position, velocity, acceleration, course and other metering characteristics of the moving vehicles are stored in memory In the present invention. In such a scenario, the virtual representations of the vehicles may have many functions or characteristics not present on the physical vehicles. One example is a cannon that the user 109 can use to fire to other vehicles in virtual space, but it does not exist in the physical space. Since virtual states and physical states are tightly coupled, events occurring in one can affect the other state.

The series of events depicted in the example of FIG. 2 occurs as follows. Vehicle 104K is represented in virtual environment 202 by vehicle representation 204K. In the virtual environment 202, the vehicle representation 204K fires a virtual cannon into the vehicle representation 204L, which represents the vehicle 104L in the physical environment 201. [ Although the vehicle 104K in the physical environment 201 may not have the actual artillery that fires the projectile, the host device 108 operating the game is allowed to enter the cannon, if given the relative positions of the vehicle representations 204K and 204L, It can be determined that medicines can produce an attack on the vehicle representation 204L at position 2 of the drawing as well as the orientation of the vehicle. The host device 108 also determines that the resulting virtual energy imparted upon attack and collision pushes the target vehicle representation 204L from its course in the virtual environment 202. [

As discussed above, in at least one embodiment, the system maintains equivalence between virtual and physical environments. Thus, in response to the above-described weapon attacks on the vehicle representation 204L, the control algorithms of the host device 108 reproduce the virtual movement of the vehicle representation 204L in the physical environment 201. [ Thus, the physical vehicle 104L is artificially propelled to move in a manner that mimics the movement of the vehicle expression 204L in the virtual environment 202. [ In the example of FIG. 2, the physical vehicle 104L, which is shot by the virtual weapon at position 2, is artificially deflected from its current course in physical space. This causes a collision in physical space with another vehicle 104M in position 3. [

The collision between physical vehicles 104L and 104M at location 3 involves the transfer of actual energy physically pushing vehicle 104M at its original course vector as shown at location 4. [ Upon detecting such a change in the course in the physical environment 201, the host device 108 causes the vehicle environment 204L and 204M to move the virtual environment 202 to the position of the vehicle 104L and 104M in the physical space 201 So that the motion is continuously reflected. Thus, the vehicle expression 204M is made to deviate from the course in the same way.

In this way, a series of events in the crossing states occurs, where the occurrences in the virtual environment 202 have results in the physical environment 201, and the resulting effects in the physical environment 201 are consequently And affects the order or dynamics of events in the virtual environment 202. The above-described scenario illustrates the tightly coupled nature of the physical and virtual environments 201, 202 in the system of the present invention. Instead of simply connecting the virtual components with the physical components, the various embodiments of the present invention are exactly symbiotic and bi-directional, so that events and changes that occur in one state (environment) And changes. ≪ RTI ID = 0.0 >

In at least one embodiment, the system of the present invention need not maintain an accurate balance of mutual influences across states, but may also be set to maintain a preferred dominant state. For example, in at least one embodiment, the system may be configured such that the virtual environment 202 dominates the physical environment 201 and that the physical environment 201 simply reflects events that occur in the virtual environment 202 ; In at least one embodiment, an opposite configuration may be implemented. Any suitable prioritization technique may be established between the physical and virtual environments 201, 202.

In at least one embodiment, the system of the present invention further provides benefits through the use of the host device 108 and / or the controller 101 with the capability to connect to external server networks (not shown) User experience. In at least one embodiment, a user 109 that controls one or more agents 104 via controller (s) 101 may download virtual products for use in a game and / Download digital content such as light pattern sequences or sound effects for the LEDs. Any suitable virtual accessories or digital content may be made available via any suitable e-commerce mechanism, such as downloading of applications and / or resources. Such content may be made available for installation via installation or other means for enabling users to tailor or improve their experience or for downloading via remote servers. In at least one embodiment, any aspect of the various aspects of the game experience may include, for example, individuals for AI-controlled vehicles, new narrators, new scenarios for existing race courses, Content methods for rendering events, and / or customization and enhancements of vehicles.

In addition to communicating with player controllers 101 and agents 104, host device 108, in at least one embodiment, coordinates operations from the game's operating framework and users 109, And adjusts the events that occur in the physical environment 201 and the events that occur in the virtual environment 202.

3, there is shown some aspects of the role of the host device 108 (base station) in accordance with at least one embodiment, and the user 104 (using the controllers 101) controlling the vehicles 104 A block diagram illustrating components and structures that may be implemented between the vehicle 109 and their own vehicles 104 is shown. Figure 3 also illustrates that in at least one embodiment, users 109 may use artificial intelligence (AI) planners 305A, 305B that use other users 109 or that run on host device 108 Or control both of the vehicles 104 that are assisted by, or both. In the example shown in the figure, four users 109M through 109Q are connected to the controllers 101M through 103M to control four agents (three vehicles 104M, 104N, 104P and one smart accessory 105) To 101Q) is used. Both AI planners 305A and 305B are also provided. Between the controllers 101 and the agents 104M, 104N, 104P, 105 are a number of structures that operate within the framework of the host device 108 (base station). The arrows indicate the direction of the information flow.

For illustrative purposes, and for clarity, FIG. 3 does not necessarily illustrate the complete flow of all structures or information, but merely emphasizes information related to vehicle control. For example, information is shown as moving from the controllers 101 to the filters 301 in one direction. Those skilled in the art will appreciate that the control mechanisms used in connection with the present invention may also be used by the host device 108 to retrieve, for example, race performance reports, sounds, images, animations, and / May include the transmission of additional data not shown in FIG. 3, including data transmitted to various other components of the system.

In at least one embodiment, vehicle commands originating from the user 109 are transmitted to the filter 301 via the controller 101; The filter 301 may improve the instructions received from the controller 101 according to the current circumstances of the user 104 under control of the user 109 or other aspects of the game. For example, in at least one embodiment, a rate filter 301 may be implemented. Under normal circumstances, the vehicle 104 may have an acceptable range of rates at which it can travel. The speed filter 301 can reduce this range, for example, by removing the top of the range if the vehicle 104 is (virtually) damaged by it as part of the gameplay. Other types of filters 301 may also have a number of different filters 104 where each vehicle 104 affects different aspects of its performance; The filters 104 may be provided to control or modify parameters such as speed, lane change rate, and / or equipment usage (e.g., ammunition of built-in weapons), and / or the like.

In at least one embodiment, an effect system 302 is provided that is capable of directly transmitting inputs associated with specific agents 104M, 104N, 104P, The effects system 302 has a broad capability to act on vehicle behavior and acts as a global actor. It can do so in any of several ways. In various embodiments, effects system 302 masks the possible outcomes of interaction of vehicle 104 with each other, as well as external factors that may affect the performance of one or more vehicles 104.

In at least one embodiment, a game engine 303 is provided that includes rule sets and scenarios and typically directs play. Game engine 303 is tightly coupled to effect system 302, which is usually a trigger effect. In addition, effect system 302 may notify game engine 303 of events that may trigger results on the progress of events, such as those determined by game engine 303, . For purposes of clarity, Figure 3 omits the lines connecting users 109 to effects system 302; However, one of ordinary skill in the art will appreciate that the actions taken by the users 109 may trigger the effect system 302 either directly or indirectly.

An example of an action taken directly by the user 109 in a modified racing scenario is as follows: In at least one embodiment, the user 109 may place a virtual oil film behind his or her vehicle 104 (More precisely, it is actually placed behind the vehicle representation 204 in the virtual environment 202, since the oil film is not present in the physical space). The trailing vehicle 104 avoiding the virtual oil film can experience (potentially) loss of control; This is implemented, for example, by an effect system 302 that (temporarily) reduces or eliminates the ability of the user 109 to control the trailing vehicle 104 to steer or brak.

An example of an indirect trigger for effect system 302 is as follows: In at least one embodiment, the user 109 may be able to determine, under typical automatic racing rules, that the game engine 303 has applied the yellow flag conditions Crashes to the vehicle of his or her vehicle 104 or other user 109 to indicate that it is to be enforced, the effect system 302 may initiate rate limits in accordance with the parameters of the yellow flag conditions. Such restrictions may remain, for example, until the game engine 303 sequentially releases the yellow flag conditions after the track is cleared.

3, if the command information is passed over the filter 301, the arbitrator 304 will notify the agents, regardless of the vehicles 104 or the accessories 105, Directly issue direct instructions effectively. In at least one embodiment, the arbiter 304 optimizes direct communications to the agent or agents 104M, 104N, 104P, 105 to which it is connected. Also, in some cases, arbitrator 304 may share the role of some filters 301 in adjusting or modifying user commands.

For example, arbitrator 304 may serve to reduce or eliminate redundant commands incoming from user 109. Arbitrator 304 may also be of value in situations such as those shown for vehicle 104N where both users 109N and 109P and AI planner 305A control a single vehicle 104N have. This is an example of a situation in which more than one user 109 is controlling the vehicle 104. As another example, as described above, the vehicle 104 may include a user 109 that controls the vehicle 104 and a second user 109 that operates its weapons, Weapons can be mounted. In such a case, arbitrator 304 consolidates and arranges the appropriate set of independent instructions in order to execute the operation (s) in agent 104 as required by independent users 109.

In another embodiment, the user 109 may pair with the AI planner 305 such that the AI planner 305 is the driver and the user 109 operates the weapon of the vehicle 104, or vice versa. The ability to rely on the AI planner 305 to assist the user 109 to control the vehicle 104 is dependent on the circumstances under which a single user 109 may have the capability to operate all systems on the other vehicle 104 Also offers many benefits. For example, planner 305 may be relied upon to smartly navigate around slower opponents on the course or to provide steering, leaving the user 109 for controlling the speed of the vehicle 104 as is. One advantage of designating the portion of control for the planner 305 is that it can be used by beginners or younger competitors who may not have motor skills needed to compete against older or more skilled users 109 Wide participation is possible. In this manner, the planner 305 may provide assistance to the users 109 controlling the vehicle 104, which may be arranged in a number of different ways depending on the game rules and / I can do that.

In another embodiment, an example of a planner 305 that takes control of the vehicle 104 may be reversed, such that the planner 305 controls the speed while the user 109 maintains control of the control. In more advanced scenarios of the jointly controlled vehicle 104 (or other agent), the user 109 may be comfortable with the steering and speed of control on the open course, You can be less confident about; Thus, the system may be configured to direct intervention by the planner 305 when facing situations or obstacles that require active avoidance.

In at least one embodiment, it may be possible for a single user 109 to control multiple agents 104 (which may include vehicles 104 and / or accessories 105). For example, the user 109 may control one agent 104 while a series of other agents are configured to follow the agent. In other embodiments, the user 109 commands may be replicated across multiple agents 104 so that they can act collectively (e.g., all change orientations or speeds simultaneously). In a more complex scenario, the user 109 may provide high-level commands to a group of agents 104. In the case of a racing game, for example, a plurality of vehicles 104 under such control may be responsive to the high-level target-oriented direction of the user 109, such as crashing to another user's vehicle 104, can do. In response to such commands, the agents 104 may position themselves around the target vehicle 104 to partially or completely surround the target vehicle and push the target vehicle away from the course together. In these cases, there may be sets of operations that are pushed in sequence or in sequence by multiple agents 104 operating in unison.

The control of the vehicle 104 is maintained by a single user 109, a single AI planner 305 or by a plurality of users 109 or users 109 and planners 305 Regardless of whether they are jointly held between the combinations, the arbitrator 304 selects independent and possibly independent commands intended for the vehicle 104, from among them, and / As shown in FIG. Many aspects of the behavior may be imparted to the AI planner 305 that controls the vehicle 104. For example, in addition to techniques related to drive control and strategy, AI planners 305 may implement individual characteristics that affect their general behavior. For example, the AI planners 305 may be aggressive drivers that tend to lean other vehicles 104 away from the driveable surface 601, or alternatively, (109). ≪ / RTI > In at least one embodiment, such AI schemes can be combined with specific arrangements jointly or intentionally with specific events occurring on the drivable surface 601 to provide messages to other users for provoking messages, for example, (109).

In at least one embodiment, some capabilities for assistive control are present on their own vehicles 104. For example, without the assistance of the host device 108, in at least one embodiment, the vehicles 104 are capable of driving through curves and turns without requiring an active steering device from the user 109, Lt; RTI ID = 0.0 > a < / RTI > relative lateral position. Similarly, the vehicles 104 are capable of adjusting their speed throughout the course in accordance with the speeds defined as part of the track identification (e. G., Encoded in a readable form on the track segment 602) Lt; / RTI > Also, by the same means for position measurement through the recognition of information encoded on the blocks 602, the vehicles 104 can determine whether they will remain on the course or whether such information in their field of detection for other information It is possible to decide whether or not to remain on the course due to the presence of the member. In such cases, in at least one embodiment, the user 109 may be moved to the vehicle 104, even though such control may be limited to a subset of the total capabilities of the vehicle 104 when that control is located on the driveable surface 601. [ May still be possible. In some embodiments, the reaction of the vehicle 104 to the control of the user 109 may be similar to that of a conventional remote-controlled car.

In other embodiments, the host device 108 may control the game in other ways than the actual game play. For example, the host device 108 may control various features of the vehicles 104 operating in the physical environment, as defined in the software. These features may include, for example, performance characteristics and vehicle capabilities. In an ongoing example of a vehicle-based game, for example, other physical aspects of the vehicle's acceleration profile, top speed, handling, and vehicle performance are dependent on physical constraints that exist for it in the real world, Can be controlled. In at least one embodiment, the system considers how the definitions result in game play results in both physical and virtual spaces.

Differences between agents in a virtual environment

The ability of soft algorithms to control critical aspects of agent 104's capabilities and behaviors provides the opportunity to translate these capabilities and behavioral differences across agents 104 to the same hardware. Considering the benefits of the consistency of the hardware design from the manufacturing simplicity and cost perspective, the benefits of the same building agents 104 can be recalled from a business perspective. While the present invention provides unique advantages in providing the ability to grant some degree of contrast between individual agents 104 in a group by controlling the differences through software-based means, Maintain consistency and thus reduce manufacturing costs.

For example, in the context of agents that are vehicles 104 that are adapted to compete in racing or competing driving environments, as described herein, it may be desirable to manufacture vehicles 104 with the same hardware components Which enables differences in the manner in which the vehicles 104 are driven, behaved, and reacted. For example, one vehicle 104 may be intended to perform like a high performance sports car while the other vehicle may be a heavy truck with poor acceleration, but high inertia, or a lightweight mini vehicle with a tight turning radius. Such differences may or may not be visually presented by different bodies or covers that fit on the undercarriage (but this may be merely an outline and, in at least one embodiment, And does not physically affect the performance characteristics of the vehicle). In software, such characteristics may be easily returned to vehicles 104, so that the basic capabilities of the physical agents may implement the same system, while the vehicles 104 may have characteristics . ≪ / RTI >

As described above, the symbiotic nature of the combination of physical and virtual environments 201, 202 may be controlled primarily (or entirely) to control the intended (impersonated) physical differences of the vehicles 104 to exist in the virtual environment 202 . For example, if two vehicles of the same weight and capacity are provided to the physical environment, the behavior of the vehicles 104 as delivered through the software may be very different from the physical world in terms of mass, power, maneuverability, Can be imitated. In at least one embodiment, the system of the present invention includes a physical environment 201, a physical representation of the vehicle representations 204 and their interactions in the virtual environment 202 to control the movement and behavior of the vehicles 104, Lt; / RTI >

For example, two vehicles 104 having the same or similar physical mobility profiles may be provided such that their weight, acceleration, maneuverability and similar metrics are the same or comparable to each other in the physical world. The main difference between the cars 104 is that one is intended to represent a glove tank (slow and very heavy) and the other a sedan (fast and light) in a virtual environment 202. In at least one embodiment, some visual contrasts may be made to the physical vehicles 104 to enhance the differences in the characteristics of the corresponding virtual representations 204; For example, body covers on the undercarriage may have different markings, designs, and / or colors to reflect differences in virtual characteristics.

4, body covers on vehicles 104R, 104S in physical environment 201 may have characteristics of corresponding vehicle representations 204R, 204S in virtual environment 202, according to one embodiment, Are used to reflect the differences between the physical and virtual environments 201, 202, thereby enhancing the relationship between the physical and virtual environments 201, The vehicle 104S is given a cover that makes it look like a tank, while the vehicle 104R is made to look like a sedan vehicle. In at least one embodiment, the vehicle representations 204R and 204S illustrate that even though physical vehicles 104R and 104S are actually comparable in mass and size, heavy-weight heavy-duty vehicles, such as tanks, It is configured to interact with each other in a way that reflects and falsifies the fact that it can be several times larger. Although the two vehicles 104R and 104S shown in Fig. 4 can operate in the physical environment 201 to maintain consistency of their interactions, .

In the example of FIG. 4, the vehicle 104R (representing a sedan vehicle) is moving at high speed on a path that causes it to collide with the vehicle 104S (representing a tank). As described above, the physical and virtual environments 201, 202 can be tightly coupled to each other. Since the mass properties of the two physical vehicles 104R and 104S are the same or substantially similar, the Newton mechanics may determine that the collision at position 2 may cause the result as shown in position (3) Where the vehicle 104S (representing the tank) is pushed and rotated and the vehicle 104R (representing the vehicle for the sedan), because the collision involves energy transfer to the vehicle 104S, It continues to travel on a course similar to its pre-crash heading, though at a speed. However, in at least one embodiment, the system maintains consistency with the characteristics of the two vehicles' representations 204R, 204S in the virtual environment 202, 104R, 104S are artificially made to follow the physics of the collision in the virtual environment 202. [ Specifically, since the mass of the vehicle representation 204S is significantly larger than the mass of the vehicle representation 204R, the vehicle representation 204R (sedan vehicle) effectively looks at the reaction to the vehicle representation 204S (tank) The result shown in position 4, which has little effect on the acceleration and position of the vehicle expression 204S, occurs. As shown in position 5 of physical environment 201, physical vehicles 104R and 104S are artificially created to follow the trajectory of corresponding vehicle representations 204R and 204S, Expand the seed for.

Then, essentially, in the collision between two such vehicles 104, which have vehicle representations 204 with such rapidly different masses, the vehicle 104R determines that the inertia of the vehicle representation 204S has an interaction Because they dominate, they experience considerably larger changes in speed and heading. According to the actual physics of the collision between the actual vehicles 104R, 104S, the result may be the same collision and the sedan pushes the tank in such a way that it may not be realistic considering the mass differences between the automotive and armored military vehicles . Thus, in at least one embodiment, the results of the collision are dependent on the vehicle parameters defined in the physics and software controlling interactions in the virtual environment, in order to maintain the differences that are returned to the vehicles in the virtual space.

Thus, in at least one embodiment, the order of certain details and events at the time of the collision is determined according to the algorithms that control the movement in the virtual environment 202. In at least one embodiment, consistency is maintained between physical and virtual environments. One possible approach for enforcing such consistency is to allow vehicles 104 to move through collision events in the same (or similar) manner as vehicle expressions 204 do, It is possible that the algorithms that determine the outcome of the collision are able to act directly on the control of the vehicles 104 in the physical environment 201. In at least one embodiment, the system of the present invention includes virtual and physical environments 202,201, 202 to maintain a desired degree of equivalence between events occurring in the virtual environment 202 and events occurring in the physical environment 201 ≪ / RTI >; < / RTI > In this manner, the system of the present invention provides bi-directional behavior of events, as described above.

Thus, in at least one embodiment, the vehicles 104R, 104S in the physical environment 201 are simultaneously responding with their opponents in the virtual environment 202. By doing so, the vehicles 104R and 104S perform and respond according to the parameters defined in the virtual environment 202 and according to the laws of physics as applied to those parameters in the virtual environment 202.

In this particular example, the conflict occurs in both the virtual and real environments 202,201. Because the mass characteristics imposed on the vehicle representations 204 in the virtual environment 202 do not match the mass characteristics of the physical vehicles 204, the outcome of the collision may be different in the two environments. Here, the operational protocol gives priority to the physics of the collision as determined in the virtual environment 202; The physical vehicles 104 are made to react according to their physics. However, in other situations, it may be desirable to give priority to the behaviors and physics of the physical environment 201, and to allow the virtual environment 202 to be adjusted accordingly.

While the example shown in Fig. 4 describes the application of virtual parameters to conventional Newtonian physics, one of ordinary skill in the art will recognize that any set of rules may be influenced by the motion of virtual bodies or other physical world physics at the time of the collision Can be defined to control any other aspect of movement. Other instances include, for example, physical vehicles 104 that are driven through corresponding portions of driveable surface 601 are inertial; Or part of the virtual environment 202 that loses its ability to steer, stop, and move in accordance with the warped slopes when the vehicle speed and acceleration are reduced as if the physically drivable surface 601 were kept flat, 0.0 > non-friction < / RTI > Many types of imaginary forces can act artificially and act on the motion of the vehicles 104 differently than they can be the forces of the real world acting according to real-world physics. In this way, the system of the present invention can implement and implement behaviors that do not follow the real-world laws of physics, but that can follow other rules.

Error correction and track information

Even in situations where the virtual and physical environments 202 and 201 are intended to operate as reflecting one another (i. E. Where physics and dynamics are always meant to produce and match the same results each time) There may be situations that can be introduced, causing differences between events in the physical environment 202,

For example, when modeling the motion of the vehicle 104 that begins to drive around the circular course, the acceleration, the undercarriage geometry and mass of the vehicle 104, the course adjustment, the distance between the wheels and the driveable surface 601 Differences between actual and modeled approximations for aspects such as friction and the like can introduce errors. Without correction, such errors can be exacerbated. As a result, if there are no protocols establishing and maintaining the equivalence between the events occurring in the physical environment 201 and the events in the virtual environment 202, then the two events will likely divide rapidly.

In at least one embodiment, the system may be configured to correct such errors based on an enforced priority scheme that specifies whether they should be corrected for physical or virtual environments. For example, in at least one embodiment, the system of the present invention can establish protocols that ensure equivalency maintained between physical and virtual environments. It may be most convenient to understand how these protocols operate to ensure equality between the two environments through review of processes controlling the control and location of vehicles 104 over both environments.

Considering the basic case of a single vehicle 104 on a drivable surface 601, it is initially believed that the vehicle 104 is entirely predicted not to be capable of operating on information about its position on the track layout, do. It is also possible that the host device 108 controlling events occurring in the virtual environment 202 does not have information about the current position of the vehicle 104 on or above the driveable surface 601 on which the vehicle 104 is located Do.

Via at least one embodiment, by means of direct input or by means that enables the host device 108 to determine it, such as a digital photograph of a driveable surface 601 that includes the vehicle 104, ) Is provided as means for enabling track and location information to be provided as part of the setup process. In the practice of the present invention, which does not allow the user 109 to provide such track information or vehicle position on the drivable surface 601 or to provide such information prior to other operations, May be collected by the vehicle 104 itself. Specifically, the vehicle 104 has a course for a distance sufficient to read data relating to the identification of the specific segment 602 of the surface 601 on which it is drivable and the identification of the position of the vehicle 104 on the segment 602 Can be driven. Once the vehicle 104 has obtained this information, the information is communicated to the host device 108, which has indexes of the compartments 602 and the compartments 602 for matching the location identification Use it to refer. In doing so, the host device 108 generates an approximation of the position of the vehicle 104 on the driveable surface 601.

Identification of the compartment 602 by the host device 108 further yields a set of data relating to operational limitations of the vehicle 104. [ Such data may include a combination of position measurement information (e.g., in the case of a racetrack, its horizontal offset from the centerline of the track and the position of the vehicle 104 along the track) Lt; / RTI > may be data relating to operational constraints such as are allowed at intersections on this track. The benefit of providing such data to the vehicle 104 is that it maintains its lateral position on the course at a level where the vehicle 104 is low and adjusts its speed in response to changes in course geometry , It is possible to maintain some ability to operate without regard to commands from the host device 108.

Once the location of the compartment 602 and the vehicle 104 is identified, the host device 108 tracks the location of the vehicle 104 on an ongoing basis from an initial state established by the location of the vehicle 104 in physical space A motion model can be employed. Recognizing the above-mentioned causes of error in calculating the actual position of the vehicle 104 and the position differences predicted by the motion model in the virtual environment 202, And may provide updates of that location through regular communication with the device 108. [ The update data may include, for example, current information about the road piece ID and location ID, as well as current information about the direct location, such as the exact lateral location of the vehicle 104 on the track. Using these data as inputs or directly for calculations that serve to update the virtual environment 202, the host device 108 can be configured to determine the movement of the vehicle 104 in the physical environment 201 and the movement of the vehicle 104 in the virtual environment 202 Lt; RTI ID = 0.0 > 204 < / RTI >

In the described embodiment, the movements of the physical vehicle 104 have priority over the movement of the corresponding vehicle representation 204, since updates are based on the characteristics of the movement of the physical vehicle 104 and are provided to the virtual model . Other embodiments may operate with a reverse order protocol. Thus, the motion of the virtual vehicle representation 204 may take precedence, and updates may be made to ensure that the physical vehicle 104 is matched to its motion. It can be appreciated that such situations may include scenarios in which scenarios other than the live motion of the vehicle 104 or the vehicles 104, or even events developed in the virtual environment 202, are based on model motion. An example may be that vehicle motions and interactions in a previously played game are stored and captured for subsequent replay. Recreating such game play in the physical environment 201 may be performed in a virtual form that leads the corresponding physical vehicle 104 or vehicles 104, even if the physical vehicles 104 have priorities in the original game play Lt; RTI ID = 0.0 > events. ≪ / RTI >

Intermediate states

In at least one embodiment of the present invention that maintains a bi-directional behavior between the virtual and physical environments 202, 201, when physical and virtual differences are present, the priority is determined by assigning a priority in situations and any given situation You can rely on this basic structure and the way you support your gameplay goals. In other embodiments, preferences may not be assigned to one side or the other, but rather a compromise between them in a state in which the system may partly depend on one side or the other side separately. In such cases, behavior and movement in both virtual and physical environments 202, 201 may be adjusted to follow a tradeoff condition.

Game frame capture  And save

In at least one embodiment, the maintenance of two equivalent states, i.e., physical state and virtual state, enables capture, storage, distribution, and / or subsequent reproduction of game play or entire games. The ability of the host device 108 to adjust the physical environment 201 to match the virtual environment 202 (when differences in state appearance and priority are given to the virtual environment 202) also occurs in the virtual environment 202 And playing back or reproducing sequences of events that are directly replicated in the physical environment 201, as well as in the sequence of play that is recorded, fully made and replayed as the progress of the actions. In at least one embodiment, the action may be accompanied by a narrator providing a narration of the events during the progress of the game being played in the play or during the recreation of the recorded game scenario. For example, while events occur in the physical environment 201, the coordinated sequences in the virtual environment 202 provide the auto narrator with relevant information (as well as color explanations) about the state of change in the game play . ≪ / RTI >

Virtual accessory

As described above in connection with FIG. 1, agents 104 may take many forms, regardless of direct user control or control of AI planner 305, or some combination thereof. In at least one embodiment, the virtual accessory 106 may be provided to implement a digitally rendered element in physical space.

Referring now to FIG. 5, there is shown an example of a virtual accessory 106 implemented in a racing and shooting game in the context of the present invention. In this case, the virtual accessory 106 represents an actively aimed and foam gun turret 503. Similar to other agents in the game space, the virtual accessory 106 maintains its presence in both the physical and virtual environments 201,202. However, for physical presence, the host device provides rendering of the accessory 106. 5, the illustration of the turret 503 is provided by a tablet 504 (such as an iPad, although any other electronic device may be used), which in its presence 501 in the virtual environment 202 And displays the turret cloth 503 at and in the corresponding state.

In at least one embodiment, where the agents operate on a driveable surface 601 that is printed on the media, the accessory 106 may be positioned in the orientation shown on the media and / Can be integrated into play. Alternatively, any arrangement may be allowed in relation to the physical game space.

In at least one embodiment, the tablet 504 may be equipped with a rear camera that can be used with the host virtual accessory 106. During placement of the tablet 504, the camera can be switched on video and recorded. As the tablet 504 is positioned, the movement view of the physical game space provides input data from which the position and orientation of the tablet with respect to other elements of the game space can be estimated. This approach eliminates the restriction on placement of the tablet 504 with respect to position and orientation.

In at least one embodiment, the rendering of the virtual accessory 106 may be done in the background against a background image of such portion of the overprinted media. This can be accomplished by the host device 108 providing an image of the cover portion of the media directly to the tablet 504 or by the tablet 504 taking an image of the footprint area next to it during placement. In the latter case, this approach is useful if the virtual accessory 106 is intended for placement beyond the game space, such as is defined by the geometric constraints of the printed medium.

The functionality of the virtual accessory 106 with regard to its integration into game play is similar to that of other agents 104. In the scenario shown in FIG. 5, vehicle representation 204T passes near turret representation 501 in virtual environment 102; Correspondingly, the physical vehicle 104T passes through the turret cloth 503 which is rendered on the virtual accessory 106 (tablet 504) in the physical environment 201. As the vehicle representation 204T passes over the foam line of the turret representation 501, the turret representation 501 is fired, causing a collision with the vehicle representation 204T. In at least one embodiment, when observed on a device displaying game play in the virtual environment 202, full shots and collisions are seen. For viewers seeing the same behavior in the physical environment 201, rendering of the firing from the turret shell 503 may be limited to the screen of the virtual accessory 106, in this case tablet 504. However, depending on the equivalency maintained between the virtual and physical game environments 202, 201, the results of the struck vehicle 104T and the vehicle representation 204T may be the same, and the virtual vehicle representation 204T may be damaged And in this instance, the loss of control can lead to its veering away from the course. The corresponding vehicle 104 in the physical environment 201 may have the capability to display impacts through sounds and / or by illuminating LEDs and / or other means. In addition, the movement and behavior of the vehicle 104T reflects the resulting loss of control as seen in the virtual environment 202.

The present invention has been described in detail with respect to possible embodiments. Those skilled in the art will appreciate that the present invention may be practiced in other embodiments. First, the specific naming of components, capitalization of terms, attributes, data structures, or any other programming or architectural aspects are not essential or important, and the present invention or mechanisms implementing the features may be implemented in different names, , Or protocols. Further, the system may be implemented as a combination of hardware and software, or entirely as hardware elements, or entirely as software elements, as described. Moreover, the specific distribution of functionality among the various system components described herein is merely exemplary and not essential; The functions performed by a single system component may instead be performed by a plurality of components and the functions performed by the plurality of components may be performed by a single component instead.

In various embodiments, the present invention may be implemented as a system or method for performing the above-described techniques, alone or in any combination. In another embodiment, the present invention is directed to a computer-readable storage medium having computer-executable instructions for performing the above-described techniques on a computing device or other electronic device, May be implemented as a computer program product.

Reference in the specification to " one embodiment "or" an embodiment " means that a particular feature, structure, or characteristic is included in at least one embodiment of the invention in the context of the embodiments. The appearances of the phrase "at least one embodiment" in various places in the description are not necessarily all references to the same embodiment.

Some portions of the above content are presented with respect to symbolic representations and algorithms of operations on in-memory data bits of a computing device. The descriptions and representations of these algorithms are the means described by the skilled artisan in the data processing arts to most effectively convey the nature of the work of the skilled artisan in the data processing arts to other artisans skilled in the art. The algorithm is here, and generally, considered to be a self-consistent sequence of steps (instructions) leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electronic, magnetic, or optical signals that can be stored, transferred, combined, compared, and otherwise manipulated. It is sometimes convenient to refer to such signals as bits, values, elements, signs, characters, terms, numbers, or the like, primarily for general usage reasons. In addition, it is sometimes convenient to also refer to modules or code devices, without losing generality, to particular arrangements of steps that require physical manipulations of physical quantities.

It should be borne in mind, however, that all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless expressly stated otherwise as apparent from the following discussion, it is appreciated that throughout the description discussions utilizing terms such as " processing "or" computing & Refers to the operations and processes of a computer system, or similar electronic computing module and / or device, which may store data represented by physical (electronic) quantities in computer system memories or registers or other information storage, transmission or display devices Manipulate and convert.

Certain aspects of the invention include process steps and instructions described herein in the form of algorithms. The process steps and instructions of the present invention may be implemented in software, firmware and / or hardware, and when implemented in software, be operated on different platforms used by various operating systems and downloaded to exist on them Should be mentioned.

The present invention also relates to an apparatus for performing the operations herein. The device may be specifically configured for the required purposes, or it may comprise a general purpose computing device selectively activated or reconfigured by a computer program stored on the computing device. Such computer programs may include, but are not limited to, floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs) , EEPROMs, flash memory, solid state drives, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of disk or computer system bus, May be stored in a computer-readable storage medium, such as any suitable type of medium. In addition, computing devices referred to herein may include a single processor or may be architectures employing multiple processor designs for increased computing capacity.

The algorithms and displays presented herein are not inherently related to any particular computing device, virtualization system, or other device. Various general purpose systems may also be used with the programs in accordance with the teachings herein, or it may be convenient to construct a more specialized apparatus for performing the required method steps. The structure required for various of these systems will be apparent from the description provided herein. Furthermore, the present invention is not described with reference to any particular programming language. It is to be understood that various programming languages may be used to implement the teachings of the present invention as described herein and that any of the above references to particular languages are provided for the disclosure of the best mode and enablement of the present invention It will be understood.

Thus, in various embodiments, the invention may be implemented as software, hardware, and / or other elements for controlling a computer system, a computing device, or other electronic device, or any combination or plurality of them. Such an electronic device may be, for example, a processor, an input device (keyboard, mouse, touch pad, trackpad, joystick, trackball, microphone, and / (Such as a screen, speakers, and / or the like), memory, long-term storage devices (such as magnetic storage devices, optical storage devices, and / or the like), and / or network connections. Such an electronic device may be portable or non-portable. Examples of electronic devices that can be used to implement the present invention are mobile phones, personal digital assistants, smart phones, kiosks, server computers, enterprise computing devices, desktop computers, laptop computers, tablet computers, consumer electronic devices, televisions, - a top box, or the like. Electronic devices for implementing the invention include, for example: Linux; Microsoft Windows available from Microsoft Corporation of Redmond, Washington; Such as Mac OS X available from Apple Inc. of Cupertino, Calif., IOS available from Apple, Inc. of Cupertino, Calif., And / or any other operating system adapted for use on the device .

Although the present invention has been described with respect to a limited number of embodiments, those skilled in the art, having the benefit of the description above, may contemplate that other embodiments may be devised which do not depart from the scope of the invention as described herein . It should also be mentioned that the language used in the specification is mainly chosen for readability and instructional purposes and has not been selected to describe or limit the claimed subject matter of the invention. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the claims.

Claims (58)

A plurality of physical agents, wherein the physical agents are configured for interaction with one another, each physical agent comprising:
Receiving signals;
Perform operations in a physical environment in response to the received signals;
The plurality of physical agents configured to transmit signals indicative of a physical state of the physical agent in the physical environment;
At least one controller, wherein the at least one controller is further configured to receive user input for controlling at least one of the physical agents, and to transmit signals for controlling at least one of the physical agents; And
As a host device:
Receive signals indicative of the physical state of the physical agents;
Transmit signals that specify actions to be taken by the physical agents;
Store virtual representations of the physical agents configured for interaction with one another;
For at least one of the physical agents:
Receiving a signal from the physical agent indicating a location of the physical agent; and
A signal for causing the physical agent to change its position to coincide with the location of the stored virtual representation of the physical agent in response to the received signal indicating a location different than the location of the stored virtual representation of the physical agent Wherein the host device is configured to maintain substantial equivalence between the physical states of the physical agents and the corresponding stored states of the stored virtual representations by a transmitting operation. The method according to claim 1,
Wherein the host device maintains substantial equivalence between the physical states of the physical agents and the corresponding stored states of the stored virtual representations comprising:
Adjusting the states of the virtual representations stored based on received signals representing the physical states of the physical agents; And
And adjusting the states of the virtual representations stored based on interactions between the virtual representations. The method of claim 2,
For two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in the at least one characteristic. The method of claim 2,
For two physical agents having similar masses, the virtual representations of the two physical agents have a mass that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in mass. The method according to claim 1,
Wherein the host device maintains substantial equivalence between the physical states of the physical agents and the corresponding stored states of the stored virtual representations comprising:
Adjusting the states of the virtual representations stored based on interactions between the virtual representations; And
And sending signals to cause the physical agents to change their physical states to conform to the stored states of the stored virtual representations. The method of claim 5,
For two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in the at least one characteristic. The method according to claim 1,
Wherein the host device maintains substantial equivalence between the physical states of the physical agents and the corresponding stored states of the stored virtual representations comprising:
And adjusting the states of the virtual representations stored based on received signals representing the physical states of the physical agents. The method according to claim 1,
The physical agents comprising mobile agents, each mobile agent comprising:
A propulsion mechanism configured to provide a motive force to the mobile agent;
A sensor configured to detect a position of the mobile agent;
Mobile wireless transceiver; And
A microcontroller operatively coupled to the propulsion mechanism, the sensor, and the mobile radio transceiver, the microcontroller being configured to control movement of the mobile agent. The method of claim 8,
Further comprising a driveable surface having a plurality of machine-readable codes indicative of positions on the surface;
Wherein the mobile agents include vehicles configured to move along the driveable surface and wherein the sensor of each mobile agent detecting the position of the mobile agent is responsive to movement of the vehicle along the surface, And the sensor for detecting codes. The method according to claim 1,
The host device including a wireless transceiver,
The host device receiving signals comprises the wireless transceiver to receive signals;
Wherein the host device transmitting signals comprises the wireless transceiver transmitting signals. The method according to claim 1,
Each controller is:
Mobile computing devices;
Smartphone;
Tablet computer;
Desktop computers;
A laptop computer;
Video game consoles; And
At least one selected from the group consisting of kiosks,
The host device comprising:
Mobile computing devices;
Smartphone;
Tablet computer;
Desktop computers;
A laptop computer;
Video game consoles; And
A kiosk, and a kiosk. The method according to claim 1,
The accessory being capable of being located within the physical environment;
The host device storing a virtual representation of the accessory, the accessory being configured to affect the state of the virtual representations of the physical agents;
Responsive to an interaction between the virtual representation of the physical agent and the virtual representation of the accessory, the host device sends a signal to cause the physical agent to change its physical state to reflect the result of the interaction. system. The method according to claim 1,
The mobile computing device being further configured to display an ephemeral representation of an accessory agent that can be located within the physical environment;
The host device storing a virtual representation of the accessory agent, the virtual representation of the accessory agent being configured for interaction with virtual representations of the physical agents;
Responsive to an interaction between the virtual representation of the accessory agent and the virtual representation of the physical agent, the host device sends a signal to cause the physical agent to change its physical state to reflect the result of the interaction , system. 14. The method of claim 13,
At least one controller is configured to receive a user input for controlling the accessory agent and is further configured to transmit signals for controlling the accessory agent;
Wherein the host device adjusts the state of the stored virtual representation of the accessory agent in response to receiving a signal for controlling the accessory agent. 15. The method of claim 14,
Wherein the accessory agent comprises a weapon and the virtual representation of the accessory agent is configured to fire into a virtual representation of a physical agent. The method according to claim 1,
Wherein the virtual representation of the at least one physical agent comprises a weapon and the virtual representation of the physical agent is configured to fire into a virtual representation of another physical agent;
In response to the virtual representation of the sparked agent:
The host device causing the virtual representation to respond as if it were fired;
The host device sends a signal to cause the corresponding physical agent to change its physical state to respond as if it were foamed. The method according to claim 1,
Wherein the at least two controllers are configured to control a common physical agent. The method according to claim 1,
Further comprising an automatic computing system configured to automatically control at least one of the physical agents;
Wherein at least one of the physical agents is configured to receive signals from the automatic computing system and to perform operations in the physical environment in response to the received signals, And is configured for interaction with one physical agent. The method according to claim 1,
Wherein the at least one controller transmitting signals for controlling at least one of the physical agents comprises the at least one controller for transmitting signals over the Internet. The method according to claim 1,
Configured to receive signals indicative of the status of the virtual representations of the physical agents from the host device and to display a mechanical representation of the physical agents based on the received signals, ≪ / RTI > The method according to claim 1,
Wherein, for at least one of the physical agents, the host device maintains substantial equivalence between the physical states of the physical agents and the corresponding stored states of the stored virtual representations:
Receiving a signal from the physical agent indicating a location of the physical agent; And
And adjusting the location of the stored virtual representation of the physical agent to conform to the indicated location of the physical agent. CLAIMS 1. A method for using mobile computing devices for controlling physical agents in a robotic system, the physical agents being configured for interaction with one another, the method comprising:
Receiving signals indicative of a physical state of the physical agents in a physical environment;
Sending signals specifying physical actions to be taken by the physical agents in the physical environment;
Storing virtual representations of the physical agents, wherein the virtual representations of the physical agents are configured for interaction with one another; And
For at least one of the physical agents:
Receiving a signal from the physical agent indicating a location of the physical agent; And
A signal for causing the physical agent to change its position to coincide with the location of the stored virtual representation of the physical agent in response to the received signal indicating a location different than the location of the stored virtual representation of the physical agent Maintaining a substantially equivalent between the physical states of the physical agents and the corresponding stored states of the virtual agents by the transmitting operation,
Wherein the physical agents are further configured to perform physical operations in response to signals received from mobile computing devices operating as user-actuated controllers. 23. The method of claim 22,
Maintaining substantially equivalent between the physical state of the physical agent and the corresponding stored state of the virtual representation comprises:
Adjusting the state of the virtual representations stored based on received signals representing the physical state of the physical agents; And
And adjusting the state of the virtual representations stored based on interactions between the virtual representations. 23. The method of claim 22,
Maintaining substantially equivalent between the physical state of the physical agent and the corresponding stored state of the virtual representation comprises:
Adjusting the state of the virtual representations stored based on interactions between the virtual representations; And
And sending signals to cause the physical agents to change their physical state to match the state of the stored virtual representations. 23. The method of claim 22,
Maintaining substantially equivalent between the physical state of the physical agent and the corresponding stored state of the virtual representation comprises:
Further comprising adjusting the state of the virtual representations stored based on received signals indicative of the physical state of the physical agents. 17. A non-transitory computer-readable storage medium storing computer-executable instructions for controlling physical agents in a robotic system using mobile computing devices, the physical agents being configured to interact with one another, the computer- The command is:
At least one processor in a host device is:
Receiving signals indicative of a physical state of the physical agents in a physical environment;
Sending signals specifying physical actions to be taken by the physical agents in the physical environment;
Storing virtual representations of the physical agents configured for interaction with one another; And
For at least one of the physical agents:
Receiving a signal from the physical agent indicating a location of the physical agent; And
A signal for causing the physical agent to change its position to coincide with the location of the stored virtual representation of the physical agent in response to the received signal indicating a location different than the location of the stored virtual representation of the physical agent The method comprising the steps of: maintaining, by an act of transmitting, substantially equivalent between the physical states of the physical agents and the corresponding states of the corresponding stored virtual representations of the physical agents; Computer-executable instructions,
Wherein the physical agents are further configured to perform physical operations in response to signals received from mobile computing devices operating as user-actuated controllers. 27. The method of claim 26,
Wherein the at least one processor is configured to maintain substantial equivalence between the physical state of the physical agent and the corresponding stored state of the virtual representation of the physical agent,
Adjusting the state of the virtual representations stored based on received signals representing the physical state of the physical agents; And
Further comprising computer-executable instructions configured to perform the step of adjusting the state of the virtual representations stored based on interactions between the virtual representations. 27. The method of claim 26,
Wherein the at least one processor is configured to maintain substantial equivalence between the physical state of the physical agent and the corresponding stored state of the virtual representation of the physical agent,
Adjusting the state of the virtual representations stored based on interactions between the virtual representations; And
Further comprising computer-executable instructions configured to cause the physical agents to send signals to cause their physical state to change to conform to the state of the stored stored virtual representations, Possible storage medium. 27. The method of claim 26,
Wherein the at least one processor is configured to maintain substantial equivalence between the physical state of the physical agent and the corresponding stored state of the virtual representation of the physical agent,
Further comprising computer-executable instructions configured to perform the step of adjusting the state of the virtual representations stored based on received signals indicative of the physical state of the physical agents. Possible storage medium. A plurality of physical agents, wherein the physical agents are configured for interaction with one another, each physical agent comprising:
Receiving signals;
Perform operations in a physical environment in response to the received signals;
The plurality of physical agents being configured to transmit signals indicative of the status of the physical agent in the physical environment;
At least one controller, wherein the at least one controller is further configured to receive user input for controlling at least one of the physical agents, and to transmit signals for controlling at least one of the physical agents; And
As a host device:
Receive signals indicative of a physical state of the physical agents;
Transmit signals that specify actions to be taken by the physical agents;
Store virtual representations of the physical agents configured for interaction with one another;
Wherein the host device is configured to maintain substantial equivalence between the physical states of the physical agents and the corresponding stored states of the physical agents;
Maintaining substantially equivalent between said physical states of said physical agents and said corresponding states of said virtual representations of said physical agents comprising:
Determining whether there is a difference between the physical states of the physical agents and the corresponding stored states of the virtual agents; And
In response to the difference:
Sending a signal to the at least one physical agent to change the physical state of the at least one physical agent to reduce the difference; And
And automatically modifying the difference by performing at least one action selected from the group consisting of modifying at least one stored virtual representation of the physical agent to reduce the difference. 32. The method of claim 30,
The act of modifying at least one stored virtual representation of a physical agent to reduce the difference comprises:
Adjusting the states of the virtual representations stored based on received signals representing the physical states of the physical agents; And
Adjusting the states of the virtual representations stored based on interactions between the virtual representations;
Wherein the act of sending a signal to the at least one physical agent to change the physical state of the at least one physical agent to reduce the difference is characterized in that the physical agents send their physical states to the states And sending signals to cause the device to change to conform to a predetermined condition. 32. The method of claim 31,
For two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in the at least one characteristic. 32. The method of claim 31,
For two physical agents having similar masses, the virtual representations of the two physical agents have a mass that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in mass. 32. The method of claim 30,
The operation of automatically reducing the difference is:
Adjusting the states of the virtual representations stored based on interactions between the virtual representations; And
And sending signals to cause the physical agents to change their physical states to conform to the stored states of the stored virtual representations. 35. The method of claim 34,
For two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that is substantially different from each other;
Wherein the interaction between the virtual representations of the physical agents reflects a substantial difference in the at least one characteristic. 32. The method of claim 30,
The operation of automatically reducing the difference is:
And adjusting the states of the virtual representations stored based on received signals representing the physical states of the physical agents. 32. The method of claim 30,
The physical agents comprising mobile agents, each mobile agent comprising:
A propulsion mechanism adapted to power the mobile agent;
A sensor configured to detect a position of the mobile agent;
Mobile wireless transceiver; And
A microcontroller operatively coupled to the propulsion mechanism, the sensor, and the mobile radio transceiver, the microcontroller being configured to control movement of the mobile agent. 37. The method of claim 37,
Further comprising a driveable surface having a plurality of machine-readable codes indicative of positions on the surface;
Wherein the mobile agents include vehicles configured to move along the driveable surface and wherein the sensor of each mobile agent detecting the position of the mobile agent is responsive to movement of the vehicle along the surface, And the sensor for detecting codes. 32. The method of claim 30,
The host device including a wireless transceiver,
The host device receiving signals comprises the wireless transceiver to receive signals;
Wherein the host device transmitting signals comprises the wireless transceiver transmitting signals. 32. The method of claim 30,
Each controller is:
Mobile computing devices;
Smartphone;
Tablet computer;
Desktop computers;
A laptop computer;
Video game consoles; And
At least one selected from the group consisting of kiosks,
The host device comprising:
Mobile computing devices;
Smartphone;
Tablet computer;
Desktop computers;
A laptop computer;
Video game consoles; And
A kiosk, and a kiosk. 32. The method of claim 30,
The accessory being capable of being located within the physical environment;
The host device storing a virtual representation of the accessory, the accessory being configured to affect the state of the virtual representations of the physical agents;
Responsive to an interaction between the virtual representation of the physical agent and the virtual representation of the accessory, the host device sends a signal to cause the physical agent to change its physical state to reflect the result of the interaction. system. 42. The method of claim 41,
Wherein the at least one controller is configured to receive user input for controlling an accessory agent and is further configured to transmit signals for controlling the accessory agent;
Wherein the host device adjusts the state of the stored virtual representation of the accessory agent in response to receiving a signal for controlling the accessory agent. 43. The method of claim 42,
Wherein the accessory agent comprises a weapon and the virtual representation of the accessory agent is configured for foaming in a virtual representation of a physical agent. 32. The method of claim 30,
The mobile computing device being further configured to display an ephemeral representation of an accessory agent that can be located within the physical environment;
The host device storing a virtual representation of the accessory agent, the virtual representation of the accessory agent being configured for interaction with virtual representations of the physical agents;
Responsive to an interaction between the virtual representation of the accessory agent and the virtual representation of the physical agent, the host device sends a signal to cause the physical agent to change its physical state to reflect the result of the interaction , system. 32. The method of claim 30,
Wherein the virtual representation of the at least one physical agent comprises a weapon and the virtual representation of the physical agent is configured to fire into a virtual representation of another physical agent;
In response to the virtual representation of the sparked agent:
The host device causing the virtual representation to respond as if it were fired;
The host device sends a signal to cause the corresponding physical agent to change its physical state to respond as if it were foamed. 32. The method of claim 30,
Wherein the at least two controllers are configured to control a common physical agent. 32. The method of claim 30,
Further comprising an automatic computing system configured to automatically control at least one of the physical agents;
Wherein at least one of the physical agents is configured to receive signals from the automated computing system and to perform operations in the physical environment in response to the received signals, And is configured for interaction with at least one physical agent. 32. The method of claim 30,
Wherein the at least one controller transmitting signals for controlling at least one of the physical agents comprises the at least one controller for transmitting signals over the Internet. 32. The method of claim 30,
Configured to receive a signal from the host device indicating the status of the virtual representations of the physical agents, and to display a mechanical representation of the physical agents based on the received signals. ≪ / RTI > 32. The method of claim 30,
The act of determining whether a difference exists includes receiving a signal indicative of the position from a physical agent;
Wherein the act of automatically reducing the difference comprises adjusting the position of the stored virtual representation of the physical agent to conform to the indicated location of the physical agent. CLAIMS 1. A method for using mobile computing devices for controlling physical agents in a robotic system, the physical agents being configured for interaction with one another, the method comprising:
Receiving signals indicative of a physical state of the physical agents in a physical environment;
Sending signals specifying physical actions to be taken by the physical agents in the physical environment;
Storing virtual representations of the physical agents, wherein the virtual representations of the physical agents are configured for interaction with one another; And
And maintaining substantially equivalent between the physical states of the physical agents and the corresponding stored states of the physical agents,
The physical agents are further configured to perform physical operations in response to signals received from mobile computing devices operating as user-actuated controllers,
Maintaining substantial equivalence between the physical states of the physical agents and the corresponding stored virtual representations of the physical agents comprising:
Determining whether there is a difference between the physical states of the physical agents and the corresponding stored states of the virtual agents; And
In response to the difference:
Sending a signal to the at least one physical agent to change the physical state of the at least one physical agent to reduce the difference; And
And modifying at least one stored virtual representation of the physical agent to reduce the difference. ≪ Desc / Clms Page number 22 > 54. The method of claim 51,
Wherein modifying the at least one stored virtual representation of the physical agent to reduce the difference comprises:
Adjusting the state of the virtual representations stored based on received signals representing the physical state of the physical agents; And
Adjusting the state of the virtual representations stored based on interactions between the virtual representations;
Wherein the step of transmitting a signal to the at least one physical agent to change the physical state of the at least one physical agent to reduce the difference causes the physical agents to match their physical state to the stored state of the stored virtual representations ≪ / RTI > the method comprising: 54. The method of claim 51,
The step of automatically reducing the difference comprises:
Adjusting the state of the virtual representations stored based on interactions between the virtual representations; And
And sending signals to cause the physical agents to change their physical state to conform to the state of the stored virtual representations. 54. The method of claim 51,
The step of automatically reducing the difference comprises:
And adjusting the state of the virtual representations stored based on received signals indicative of the physical state of the physical agents. 17. A non-transitory computer-readable storage medium storing computer-executable instructions for controlling physical agents in a robotic system using mobile computing devices, the physical agents being configured to interact with one another, the computer- The command is:
At least one processor in a host device is:
Receiving signals indicative of a physical state of the physical agents in a physical environment;
Sending signals specifying physical actions to be taken by the physical agents in the physical environment;
Storing virtual representations of the physical agents, wherein the virtual representations of the physical agents are configured for interaction with one another; And
And to maintain substantially equivalent between the physical states of the physical agents and the corresponding stored states of the physical agents, the computer-executable instructions being encoded on the medium ,
Wherein the physical agents are further configured to perform physical operations in response to signals received from mobile computing devices operating as user-actuated controllers;
The computer-executable instructions being configured to maintain substantial equivalence between the physical states of the physical agents and the corresponding stored states of the physical agents,
Determining whether there is a difference between the physical states of the physical agents and the corresponding stored states of the virtual agents; And
In response to the difference:
Sending a signal to the at least one physical agent to change the physical state of the at least one physical agent to reduce the difference; And
And modifying the at least one stored virtual representation of the physical agent to reduce the difference. The computer-executable instructions configured to perform the step of automatically reducing the difference by performing at least one step selected from the group consisting of: Gt; computer-readable < / RTI > storage medium. 55. The method of claim 55,
The computer-executable instructions configured to cause at least one processor to change at least one stored virtual representation of a physical agent to reduce the difference comprises: at least one processor configured to:
Adjusting the state of the virtual representations stored based on received signals representing the physical state of the physical agents; And
And adjusting the state of the stored virtual representations based on interactions between the virtual representations;
Wherein the computer-executable instructions configured to send a signal to at least one physical agent to change the physical state of the at least one physical agent to reduce the difference, Computer-executable instructions configured to send signals to cause the computer to change to conform to the state of the virtual representations. 55. The method of claim 55,
The computer-executable instructions configured to cause the at least one processor to automatically reduce the difference comprises: at least one processor:
Adjusting the state of the virtual representations stored based on interactions between the virtual representations; And
And to send signals to cause the physical agents to change their physical state to conform to the state of the stored stored virtual expressions. ≪ RTI ID = 0.0 > Storage medium. 55. The method of claim 55,
Wherein the computer-executable instructions configured to cause at least one processor to automatically reduce the difference comprises: at least one processor:
And computer-executable instructions configured to perform the step of adjusting the state of the virtual representations stored based on received signals indicative of the physical state of the physical agents. Storage medium.

Images