Support for FBX and BVH

[felixchenier]

As suggested by @zybermonk in #252 (comment)

[felixchenier]

@zybermonk

Since you asked for supporting it, I guess you use these files in your work at the moment. What is your current workflow, do you use a Python library, or you open those in a separate software to convert it to another, supported file format?

[zybermonk]

The source of the .fbx files are typically from motion capture. However, the deep learning pipeline cannot work with them directly. The DL pipeline has several sources; .c3d, .npy, .bvh, .csv ,etc. and .fbx as well. Usually, for the first stage of visualisation several platforms and libraries are used; among which KTK is the main (for .c3d and .npy especially). For, the .fbx and .bvh, blender is the current go to software, along with other biomechanics viewers like Mokka

The present bottleneck is in converting back the outputs from DL pipeline to desired formats based on applications. And here, the process to get the motions in .bvh and .fbx are rigorous and require calling blender in the background, working with the meshes and so on.

Most researchers in these areas follow similar pattern. So far I have only found one python library to load .fbx i.e is pyfbx, which is outdated (last release 2019). Additionally; there are only methods to convert FROM .fbx to other formats and a severe lack of conversion techniques from other formats TO .fbx

Researchers working in this area work a lot with SMPL format too, hence using .fbx and .bvh is essential to build meshes and retarget rigs for a variety of use cases.

This is where I believe KTK can be beneficial to handle all the conversions and biomechanics, TO and FROM multiple formats, to produce ready to use .fbx and .bvh, maybe by internally handling the rigging using an other libraries.

I am not sure how complex or relevant these areas are for KTK, but whenever the support is considered, would hugely benefit 3D human motion research.

[felixchenier]

@zybermonk Thanks for your explanation. Unfortunately due to time constraint I need to prioritize, and the near-future new features will be related to musculoskeletal modelling instead of file reading. But I do think this feature request is very good.

For starters, I think that instead of either using Blender manually or write fbx/bvh-related import/export scripts from scratch, it should be possible to install Blender, then script Blender to convert a fbx or bvh into a simpler intermediate file that can be read after by KTK. A bit less nice because we need to install Blender, but still.

To run a Python script in Blender's workspace:

https://blender.stackexchange.com/questions/100497/use-blenders-bpy-in-projects-outside-blender

To load a fbx in a Blender's workspace:

https://blender.stackexchange.com/questions/139003/import-fbx-using-script

The only thing that is missing is of course the most important part of the script: reading what we need and write it back as an intermediary file.

I put these links for my future self but maybe also as a starter for someone who could be interested in coding it.