To make an interactive environment upon the given ones

[sprakashdash]

I am planning to make an interactive environment just like issacgym/python/examples/projectiles.py where with the click of a mouse button we can throw projectiles toward robots (in my case unitree a1 robot). I tried including the steps onto the _create_envs method in legged_robot.py but I could only include this much of code into it:

self.proj_env = self.gym.create_env(self.sim, env_lower, env_upper, int(np.sqrt(self.num_envs)))
proj_asset_options = gymapi.AssetOptions()
proj_asset_options.density = 0.01
proj_asset = self.gym.create_sphere(self.sim, 0.05, proj_asset_options)
self.projectiles = []
for i in range(self.num_projectiles):
    proj_pose = gymapi.Transform()
    proj_pose.p = gymapi.Vec3(i * 0.5, 1.0, 10)
    proj_pose.r = gymapi.Quat(0, 0, 0, 1)

    # create actors which will collide with actors in any environment
    proj_handle = self.gym.create_actor(self.proj_env, proj_asset, proj_pose, "projectile_" + str(i), -1, 0)

    # set each projectile to a different, random color
    proj_color = 0.5 + 0.5 * np.random.random(3)
    self.gym.set_rigid_body_color(self.proj_env, proj_handle, 0, gymapi.MESH_VISUAL_AND_COLLISION, gymapi.Vec3(proj_color[0], proj_color[1], proj_color[2]))
    self.projectiles.append(proj_handle)

Now when I include the next part that is:

proj_index = 0
for event in self.gym.query_viewer_action_events(self.viewer):
    if event.action == 'space_shoot' or event.action == 'mouse_shoot' and event.value > 0:
        pos = self.gym.get_viewer_mouse_position(self.viewer)
    cam_pose = self.gym.get_viewer_camera_transform(self.viewer, self.proj_env)
    cam_fwd = cam_pose.r.rotate(gymapi.Vec3(0,0,1))
    spawn = cam_pose.p
    speed = 25
    vel = cam_fwd * speed
    ang_vel = 1.57 - 3.141*np.random.random(3)

    proj_handle = self.projectiles[proj_index]
    state = self.gym.get_actor_rigid_body_states(self.proj_env, proj_handle, gymapi.STATE_NONE)
    state['pose']['p'].fill((spawn.x, spawn.y, spawn.z))
    state['pose']['r'].fill((0, 0, 0, 1))
    state['vel']['linear'].fill((vel.x, vel.y, vel.z))
    state['vel']['angular'].fill((ang_vel[0], ang_vel[1], ang_vel[2]))
    self.gym.set_actor_rigid_body_states(self.proj_env, proj_handle, state, gymapi.STATE_ALL)
    proj_index = (proj_index + 1) % len(self.projectiles)

I get an AtrributeError as self.viewer is not a part of LeggedRobot class.
But I tried to put the second part into step() method and I get an Isaac Gym error: [Error] [carb.gym.plugin] Function GymGetActorRigidBodyStates cannot be used with the GPU pipeline after simulation starts. Please use the tensor API if possible. See docs/programming/tensors.html for more info.
I got to know from the Isaac gym docs that gym.get_actor_rigid_body_states can be used for CPU rendering but not for GPU.
IS THERE A WAY I CAN DO THE WHOLE INTERACTIVE SIMULATION VIA GPU?

[nikitardn]

the equivalent in the tensor api is rigid_body_state_tensor = self.gym.acquire_rigid_body_state_tensor(self.sim) for reading the values and self.gym.set_actor_root_state_tensor_indexed for setting root states. You should be able to use those to set positions and velocities of the projectiles. however you won't be able to create new rigid bodies on the fly, so you will need to create all of them before the simulation starts and store them somewhere on the side. Hope this helps

[sprakashdash]

@nikitardn Thanks for the help.. I was able to do this like last week ago. I have one more question: can we make a dynamic terrain in Issac Gym? Like shifting platforms or something like that?

[nikitardn]

You should be able to add shifting platforms as boxes, but our current height scanner won't work with that since we get the heights from the static terrain.