arteq code release

LICENSE

@@ -1,21 +1,58 @@
-MIT License
-
-Copyright (c) 2023 Haven Feng
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+License
+
+Software Copyright License for non-commercial scientific research purposes
+Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use the SGNify model, data and software, (the "Model & Software"), including 3D meshes, blend weights, blend shapes, textures, software, scripts, and animations. By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License
+
+Ownership / Licensees
+The Software and the associated materials has been developed at the
+
+Max Planck Institute for Intelligent Systems (hereinafter "MPI").
+
+Any copyright or patent right is owned by and proprietary material of the
+
+Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (hereinafter “MPG”; MPI and MPG hereinafter collectively “Max-Planck”)
+
+hereinafter the “Licensor”.
+
+License Grant
+Licensor grants you (Licensee) personally a single-user, non-exclusive, non-transferable, free of charge right:
+
+To install the Model & Software on computers owned, leased or otherwise controlled by you and/or your organization;
+To use the Model & Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects;
+Any other use, in particular any use for commercial, pornographic, military, or surveillance, purposes is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artifacts for commercial purposes. The Data & Software may not be used to create fake, libelous, misleading, or defamatory content of any kind excluding analyses in peer-reviewed scientific research. The Data & Software may not be reproduced, modified and/or made available in any form to any third party without Max-Planck’s prior written permission.
+
+The Data & Software may not be used for pornographic purposes or to generate pornographic material whether commercial or not. This license also prohibits the use of the Software to train methods/algorithms/neural networks/etc. for commercial, pornographic, military, surveillance, or defamatory use of any kind. By downloading the Data & Software, you agree not to reverse engineer it.
+
+No Distribution
+The Model & Software and the license herein granted shall not be copied, shared, distributed, re-sold, offered for re-sale, transferred or sub-licensed in whole or in part except that you may make one copy for archive purposes only.
+
+Disclaimer of Representations and Warranties
+You expressly acknowledge and agree that the Model & Software results from basic research, is provided “AS IS”, may contain errors, and that any use of the Model & Software is at your sole risk. LICENSOR MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE MODEL & SOFTWARE, NEITHER EXPRESS NOR IMPLIED, AND THE ABSENCE OF ANY LEGAL OR ACTUAL DEFECTS, WHETHER DISCOVERABLE OR NOT. Specifically, and not to limit the foregoing, licensor makes no representations or warranties (i) regarding the merchantability or fitness for a particular purpose of the Model & Software, (ii) that the use of the Model & Software will not infringe any patents, copyrights or other intellectual property rights of a third party, and (iii) that the use of the Model & Software will not cause any damage of any kind to you or a third party.
+
+Limitation of Liability
+Because this Model & Software License Agreement qualifies as a donation, according to Section 521 of the German Civil Code (Bürgerliches Gesetzbuch – BGB) Licensor as a donor is liable for intent and gross negligence only. If the Licensor fraudulently conceals a legal or material defect, they are obliged to compensate the Licensee for the resulting damage.
+Licensor shall be liable for loss of data only up to the amount of typical recovery costs which would have arisen had proper and regular data backup measures been taken. For the avoidance of doubt Licensor shall be liable in accordance with the German Product Liability Act in the event of product liability. The foregoing applies also to Licensor’s legal representatives or assistants in performance. Any further liability shall be excluded.
+Patent claims generated through the usage of the Model & Software cannot be directed towards the copyright holders.
+The Model & Software is provided in the state of development the licensor defines. If modified or extended by Licensee, the Licensor makes no claims about the fitness of the Model & Software and is not responsible for any problems such modifications cause.
+
+No Maintenance Services
+You understand and agree that Licensor is under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Model & Software. Licensor nevertheless reserves the right to update, modify, or discontinue the Model & Software at any time.
+
+Defects of the Model & Software must be notified in writing to the Licensor with a comprehensible description of the error symptoms. The notification of the defect should enable the reproduction of the error. The Licensee is encouraged to communicate any use, results, modification or publication.
+
+Publications using the Model & Software
+You acknowledge that the Model & Software is a valuable scientific resource and agree to appropriately reference the following paper in any publication making use of the Model & Software.
+
+Citation:
+@misc{feng2023generalizing,
+ title={Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance},
+ author={Haiwen Feng and Peter Kulits and Shichen Liu and Michael J. Black and Victoria Abrevaya},
+ year={2023},
+ eprint={2304.10528},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+Commercial licensing opportunities
+For commercial uses of the Software, please send email to [email protected]
+
+This Agreement shall be governed by the laws of the Federal Republic of Germany except for the UN Sales Convention.

README.md

@@ -0,0 +1,92 @@
+## Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance
+
+\[[Project Page](https://arteq.is.tue.mpg.de/)\]
+\[[arXiv](https://arxiv.org/abs/2304.10528)\]
+
+![Teaser](https://arteq.is.tue.mpg.de/media/upload/artieq_teaser2.png)
+
+## Table of Contents
+
+- [License](#license)
+- [Description](#description)
+ - [Setup](#setup)
+ - [Training](#training)
+ - [Eval](#eval)
+- [Citation](#citation)
+- [Acknowledgments](#acknowledgments)
+- [Contact](#contact)
+
+## License
+
+Software Copyright License for **non-commercial scientific research purposes**.
+Please read carefully the [terms and conditions](https://github.com/MPForte/sgnify/blob/master/LICENSE) and any accompanying documentation before you download and/or use the SGNify model, data and software, (the "Model & Software"), including 3D meshes, blend weights, blend shapes, textures, software, scripts, and animations. By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this [License](./LICENSE).
+
+## Description
+
+This repository contains the training code used for the experiments in [Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance](https://arteq.is.tue.mpg.de/).
+
+### Setup
+
+1. Create an account at https://arteq.is.tue.mpg.de/
+2. Run `./install.sh`
+3. Activate the environment `conda activate arteq`
+
+### Training
+
+Run the following command to execute the code:
+
+```Shell
+python src/train.py \
+ --EPN_input_radius 0.4 \
+ --EPN_layer_num 2 \
+ --aug_type so3 \
+ --batch_size 2 \
+ --epochs 15 \
+ --gt_part_seg auto \
+ --i 0 \
+ --kinematic_cond yes \
+ --num_point 5000
+```
+
+### Eval
+
+Run the following command to evaluate the model:
+
+```Shell
+python src/eval.py \
+ --EPN_input_radius 0.4 \
+ --EPN_layer_num 2 \
+ --aug_type so3 \
+ --epoch 15 \
+ --gt_part_seg auto \
+ --i 0 \
+ --kinematic_cond yes \
+ --num_point 5000
+```
+
+or with `--paper_model`.
+
+## Citation
+
+If you find this Model & Software useful in your research we would kindly ask you to cite:
+
+```
+@misc{feng2023generalizing,
+ title={Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance},
+ author={Haiwen Feng and Peter Kulits and Shichen Liu and Michael J. Black and Victoria Abrevaya},
+ year={2023},
+ eprint={2304.10528},
+ archivePrefix={arXiv},
+ primaryClass={cs.CV}
+}
+```
+
+## Acknowledgments
+
+@TODO
+
+## Contact
+
+For questions, please contact [[email protected]](mailto:[email protected]).
+
+For commercial licensing (and all related questions for business applications), please contact [[email protected]](mailto:[email protected]).

environment.yml

@@ -0,0 +1,25 @@
+name: arteq
+channels:
+ - nvidia
+ - pytorch
+ - conda-forge
+dependencies:
+ - python==3.10.*
+ - pip
+ - git
+ - joblib
+ - numpy
+ - plyfile
+ - pre-commit
+ - pyg::pytorch-scatter
+ - pytorch::pytorch
+ - pytorch::pytorch-cuda=11.7
+ - scipy
+ - scikit-learn
+ - tqdm
+ - trimesh
+ - pip:
+ - smplx
+ - webdataset
+ - -e external/vgtk/vgtk
+ - -e external/vgtk

external/vgtk/__init__.py

@@ -0,0 +1,27 @@
+import sys
+TRAIN_PATH = "../"
+sys.path.insert(0, TRAIN_PATH)
+
+from models.agent_ae import EquivariantPose
+from models.so3conv import *
+from models import enc_so3net
+
+
+def get_agent(cfg):
+ if cfg.arch_type == 'ae':
+ return EquivariantPose(cfg)
+ else:
+ raise ValueError
+
+def set_requires_grad(nets, requires_grad=False):
+ """Set requies_grad=Fasle for all the networks to avoid unnecessary computations
+ Parameters:
+ nets (network list) -- a list of networks
+ requires_grad (bool) -- whether the networks require gradients or not
+ """
+ if not isinstance(nets, list):
+ nets = [nets]
+ for net in nets:
+ if net is not None:
+ for param in net.parameters():
+ param.requires_grad = requires_grad