Wrong Axis Unity Import

[andreilica]

Hello!

Firstly, greatly appreciate all the work you have put into this project.

I would have two questions for you:

1. I want to import an .fbx into Unity but the axis are wrong. The fbx was converted using the Blender python script you have provided. Even in Blender the fbx has wrong axis. If I try to rotate the Armature in Blender, when I play the animation, it gets back to the original wrong pose. In Unity it seems that it should be rotated with 90 degrees around the Z axis and in Blender it should be rotated 90 degrees around the Y Axis.

2. Also, is it possible for us to remove the camera distance from the FBX? As you can see in the Blender picture, the armature is translated with some distance from the center of the world, would it be possible to center it?

Thank you!

[Arthur151]

Yes, of course, we can.
For current version of ROMP, to remove the translation, please just change this line from
bones[bone_name_from_index[0]].location = Vector((100*trans[1], 100*trans[2], 100*trans[0])) - pelvis_position
to
bones[bone_name_from_index[0]].location = pelvis_position
or
bones[bone_name_from_index[0]].location = Vector((0, 0, 0))

To rotate the body mesh, please just change this line from
bone.rotation_quaternion = (quat_x_90_cw @ quat_z_90_cw) @ bone_rotation
to, for example,
# rotate -90 degree along y axis
bone.rotation_quaternion = (quat_x_90_cw @ quat_z_90_cw @ quat_y_90_cw) @ bone_rotation
using the rotation we provided here.
Follow this way, you can rotate the mesh in the way you want.

Or you can wait for us to update the new commit. But it is really simple to solve it in the way above.
Please let me know if there is any question.

[andreilica]

Hello! Thank you, it is working as it should now!

Just one more little question so I won't have to open another issue. In order to apply the OneEuroFilter on top of the outputs, is it only necessary to run verts = self.mesh_smoother.process(verts) on results[img_path][subject_idx]['verts'] ? Or something else is needed for that?

Thanks,
Have a great day!

[Arthur151]

I'm glad to see it works now.
Yes, applying the OneEuroFilter on the vertex of the human mesh is the most simple and straightforword way to smooth the results. Later, ROMP will add more elegant way to smooth the output, such as the 'body pose', which will make the motion more smooth.

[andreilica]

I have added results[img_path][subject_idx]['verts'] = self._mesh_smoother.process(verts_results[batch_idx]) in the reorganize_results() function but there is nothing changed in the output, there is still jitter present.

Here is the original video:
https://user-images.githubusercontent.com/48161631/123106114-095d3c00-d441-11eb-9eb2-a6f047eb1dae.mp4

Here is the output:
https://user-images.githubusercontent.com/48161631/123106256-272aa100-d441-11eb-9d42-40f06b27358c.mp4

[Arthur151]

In this case, pose rotation is the one that you should smooth.
You can try to smooth it before ROMP convert it to mesh, for example at
here
before/after params_dict['global_orient'] = rot6D_to_angular(params_dict['global_orient']).
Run in batch size =1, and smooth it via
params_dict['global_orient']=self._mesh_smoother.process(params_dict['global_orient'][0]).unsqueeze(0)
I haven't tried this, but maybe you can try it. I can't promise it works well because the motion is so fast and the image is blurr.

[liuhaorandezhanghao]

我已经添加results[img_path][subject_idx]['verts'] = self._mesh_smoother.process(verts_results[batch_idx])了reorganize_results()函数，但输出中没有任何变化，仍然存在抖动。

这是原始视频：https :
//user-images.githubusercontent.com/48161631/123106114-095d3c00-d441-11eb-9eb2-a6f047eb1dae.mp4

这是输出：https :
//user-images.githubusercontent.com/48161631/123106256-272aa100-d441-11eb-9d42-40f06b27358c.mp4

Hello, I imported the fbx file of the smpl model into unity. I want to modify the pose of the smpl through the pose parameters of the algorithm regression. What should I do? I currently write the pose parameter through the axis angle function, but several joints are wrong.

[Arthur151]

Could you please provide more detailed describtion of the bugs?
Please check that the name of joints are matched with the ones in fbx of smpl model

[liuhaorandezhanghao]

您好，请问如何输出更平滑的运动呢？我的算法结果传到unity的人物模型身上，整体还是抖动很明显的，尤其是脚底，我已经对pose参数用过一次卡尔曼滤波了。谢谢

[Arthur151]

可以调整滤波器的参数，获得更加平滑的动作，smooth coeff越小，越平滑。