Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…lenderProc into develop
  • Loading branch information
themasterlink committed May 15, 2020
2 parents cdba502 + 303b2bb commit 0a2f630
Show file tree
Hide file tree
Showing 216 changed files with 20,173 additions and 2,076 deletions.
87 changes: 87 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@

# Auto detect text files and perform LF normalization
* text=auto

#
# The above will handle all files NOT found below
#

# Source files
# ============
*.pxd text diff=python
*.py text diff=python
*.py3 text diff=python
*.pyw text diff=python
*.pyx text diff=python
*.pyz text diff=python
*.hdf5 binary

# Documents
*.gitattributes text
.gitignore text
*.md text
*.bibtex text diff=bibtex
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
*.md text
*.tex text diff=tex
*.adoc text
*.textile text
*.mustache text
*.csv text
*.tab text
*.tsv text
*.txt text
*.sql text

# Graphics
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.tif binary
*.tiff binary
*.ico binary
*.svg binary
*.eps binary

# Scripts
*.bash text eol=lf
*.fish text eol=lf
*.sh text eol=lf
# These are explicitly windows files and should use crlf
*.bat text eol=crlf
*.cmd text eol=crlf
*.ps1 text eol=crlf

# Serialisation
*.json text
*.toml text
*.xml text
*.yaml text
*.yml text

# Archives
*.7z binary
*.gz binary
*.tar binary
*.tgz binary
*.zip binary

# Text files where line endings should be preserved
*.patch -text

#
# Exclude files from exporting
#

.gitattributes export-ignore
.gitignore export-ignore
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ blender
.idea/
.vscode/
output/
debug/
resources/cctextures
resources/scenenet/*zip
resources/scenenet/SceneNetData
.vscode
*.pyc
*.swp
docs/build/
Expand Down
201 changes: 201 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Contributing to BlenderProc

Thank you for your interest in BlenderProc.

The following is a short set of guidelines for contributing to BlenderProc. These are guidelines, not rules. If you feel like this set can be improved, feel free to propose changes in a PR.

## Contents

[Getting started](#getting-started)
* [BlenderProc Source Code](#blenderproc-source-code)
* [BlenderProc Design Decisions](#blenderproc-design-decisions)

[Ways to contribute](#ways-to-contribute)
* [Reporting bugs](#reporting-bugs)
* [Suggesting enchancements](#suggesting-enchancements)
* [Pull Requests](#pull-requests)

[Styleguides](#styleguides)
* [Git Commit Message and Branch Names](#git-commit-message-and-branch-names)
* [Python Styleguide](#python-styleguide)
* [BlenderProc Module Documentation Styleguide](#blenderproc-module-documentation-styleguide)
* [BlenderProc Example Styleguide](#blenderproc-example-styleguide)

## Getting started

### BlenderProc Source Code

BlenderProc is a small open source project: just one repository. So when you decide to contribute to our ongoing effort to improve this tool, it is expected that you are familiar with the [source code](src/) and made your way through the relevant [examples](examples/).

### BlenderProc Design Decisions

At this point BlenderProc has a well-established project-wide code structure with key elements like modules, composite-modules, providers, utilities, examples, configuration files, etc. with the intent of keeping BlenderProc easy as an to use and easy to extend pipeline tool. The way the majority of this elements are developed is goverened by the [styleguides](#styleguides). But yes, it may happen such that the current way of organizing and developing is not suitable for your case, so use your best judgement.

## Ways to contribute

### Reporting bugs

Bugs are tracked as [GitHub issues](https://guides.github.com/features/issues/).
Create an issue, explain the problem and include additional details to help maintainers reproduce the problem:
* Use a clear and descriptive title.
* Describe the exact steps which reproduce the problem in as many details as possible.
* Provide specific examples to demonstrate the steps: part of the config file, copy/pasteable code snippets, etc.
* Explain which behavior you expected to see instead and why.
* Include screenshots if possible.

### Suggesting enchancements

Enhancement suggestions are tracked as [GitHub issues](https://guides.github.com/features/issues/).
Create an issue and provide the following information:
* Use a clear and descriptive title.
* Provide specific examples to demonstrate the steps: part of the config file, copy/pasteable code snippets, etc.
* Describe the current behavior and explain which behavior the enchancement will introduce.
* Explain why this enhancement would be useful.

### Pull Requests

In order to increase a pace of project-wide decision-making and maintain of the BlenderProc, please follow these steps to have your contribution considered by the maintainers:
* Link to the issue that your change relates to. If there is not yet an issue for your bug or issue for an enchancement request, please open a new issue and then link to that issue in your pull request.
* Fix description: short walk-through the concept of the solution.

If it is a bug fix PR:
* Verify that examples that use the fixed modules are working and their READMEs are updated if that is needed.

If it is an enchancement/feature PR:
* Provide a new example if major feature or enchancement is introduced.

The contents of the PR (i.e. code, documentation, examples) must follow the [styleguides](#styleguides).
While the prerequisites above must be satisfied prior to having your pull request reviewed, the reviewer(s) may ask you to complete additional design work, tests, or other changes before your pull request can be ultimately accepted.

## Styleguides

### Git Commit Message and Branch Names

Following is a simple pattern for Git Commit messages:

```
<type>(<scope>): <subject>
<body>
<footer>
```
* `type` - type of the change, lowercase.

| type | meaning |
| ------ | ------- |
| `doc` | changes to the external or internal documentation|
| `feat` | new, removed or changed feature |
| `fix` | bug fix |
| `refactor` | refactoring/restructuring of the code |
| `revert` | reverted commit or parts of it |
| `style` | code cosmetica: formatting with no code change |
| `chore` | updating grunt tasks/maintaining with no production code change, cleanup |

* `subject` - the headline of the change, all lowercase.
* `scope` - scope of the change, all lowercase.
* `body` - optional text that describes your commit.
* `footer` - optional referencing of the issues, also all lowercase.

that may look like this:

```
feat(loader): add wavefront object loader, all lower case
New module allows loading .obj files and setting custom properties like `physics` to the loaded objects.
closes #123
```

For branch names please stick to the ussie number pattern of:

```
iss_300_some_short_name
```

### Python styleguide

We are trying to write good python code, but we are not hardcore Python evangelists. So use your best judgement, but remember that [PEP8](https://www.python.org/dev/peps/pep-0008/) dictates how to do things in general.

### BlenderProc Module Documentation Styleguide

We are using [Sphinx](https://www.sphinx-doc.org/en/master/index.html) for automatic generation of the API's documentation.
This way you are required to apply the following patterns:

* Pipeline Module Description

Depending on the amount of the parameters required/available to configure your module, and on the way they are required to be organized in the config file, provide csv tables with parameter description/explanation along with general module description.
After the general description, please give a couple of examples with short explanation, try to show off all config parameters in them. For every parameter give a short explanation of its type, its default value, its range or available values, if applicable. The exact order is represented below.

```python
class MyNewPipelineModule(Loader):
""" This module is doing this, this, and this, while incorporating that.
Example 1: Explain what happens
{
"some": "config",
"example": "here"
}
Example 2: Explain this more complex example
{
"some": "more",
"complex": "example"
}
**Table for a part of config**:
.. csv-table::
:header: "Keyword", "Description"
"param_a", "Used for this/means this. Type: type. Default: value. Available: [some_value, value, another_value]."
"param_b", "Used for that/means that. Type: type. Range: [min, max]. Default: B."
**Table for another part of config if needed**:
.. csv-table::
:header: "Keyword", "Description"
"param_c", "Used for this. Contains that. Type: type."
"param_c/param_d", "Used for this/means this. Type: type."
"param_c/param_e", "Used for that/means that. Type: type. Default: value. Available: [value, value1, value2]."
"""
```

* Pipeline Module Method Description

This one is pretty self-explanatory.

```python
def run(self):
""" Doing this big thing in the following steps:
1. Step 1
2. Step 2
...
N. Profit!!!
"""
pass

def _foo(self, bar_a, bar_b):
""" Sums bar_a and bar_b together and returns the result.
:param a: Used for this/means this. Type: type.
:param b: used for that/means that. Type: type.
:return: A sum of bar_a and bar_b.
"""
return bar_a + bar_b
```

### BlenderProc Example Styleguide

When you are proposing a new module or significant changes to the existing modules, new example as a part of your PR may be expected.
To create a good example, please follow this steps:
* Create a folder with a clear descriptive name in [examples](examples/) folder.
* In this folder provide at least a configuration config.yaml file and a README.md file.
* And any other files that may be necessary (like rendering images, .obj files, text files with some data required by the pipeline, etc), but keep it as clean as possible and **do not include any copyrighted materials**.

For the README.md of the example, please follow the [template](examples/EXAMPLE_README_TEMPLATE.md). If the proposed changes are not including some new modules or substantial changes in the existing ones, but an example is still required, then follow your best judgement.

Also remember, that when making changes to the existing modules it is up to you to verify that existing examples that are using this module are valid and working. Fix the configuration files and update READMEs of these examples, if the example requires it.
34 changes: 27 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,31 @@
# BlenderProc

![](readme.png)
<p align="center">
<img src="readme.jpg" alt="Front readme image" width=430>
</p>

A procedural blender pipeline for image generation for deep learning.
A procedural Blender pipeline for image generation for Deep Learning.

Check out our arXiv paper (we are updating it from time to time): https://arxiv.org/abs/1911.01911

## Contents

* [General](#general)
* [Functionality](#functionality)
* [Examples](#examples)
* [First step](#first-step)
* [Source code](#source-code)
* [Contributions](#contributions)
* [Change log](#change-log)

## General

In general, one run of the pipeline first loads or constructs a 3D scene, then sets some camera positions inside this scene and renders different types of images (rgb, depth, normals etc.) for each of them.
The blender pipeline consists of different modules, where each of them performs one step in the described process.
The modules are selected, ordered and configured via a .yaml file.

To run the blender pipeline one just has to call the `run.py` script in the main directory together with the desired config file and any additional arguments.

An exemplary `config.yaml` can be found in the respective example folder.
```
python run.py config.yaml <additional arguments>
```
Expand All @@ -24,7 +36,7 @@ This runs all modules specified in the config file in a step-by-step fashion in

The following modules are already implemented and ready to use:

* Loading: *.obj, SunCG, Replica scenes.
* Loading: *.obj, SunCG, Replica scenes, BOP datasets.
* Lighting: Set, sample lights, automatic lighting of SunCG scenes.
* Cameras: set, sample or load camera poses from file.
* Rendering: RGB, depth, normal and segmentation images.
Expand All @@ -37,15 +49,23 @@ For advanced usage which is not covered by these modules, own modules can easily
* [Basic scene](examples/basic/): Basic example
* [Simple SUNCG scene](examples/suncg_basic/): Loads a SUNCG scene and camera positions from file before rendering color, normal, segmentation and a depth images.
* [SUNCG scene with camera sampling](examples/suncg_with_cam_sampling/): Loads a SUNCG scene and automatically samples camera poses in every room before rendering color, normal, segmentation and a depth images.
* [Replica dataset](examples/replica-dataset): Load a replica room, sample camera poses and render normal images.
* [Replica dataset](examples/replica_dataset): Load a replica room, sample camera poses and render normal images.
* [COCO annotations](examples/coco_annotations): Write to a .json file containing COCO annotations for the objects in the scene.

... And much more!

## First step

Now head on to the [examples](examples) and check the README there: get some basic understanding of the config files, start exploring our examples and get a gist about writing yor own modules!
Now head on to the [examples](examples) and check the README there: get some basic understanding of the config files, start exploring our examples and get an idea about the power of BlenderProc.

## Source Code

Now it's a good time to take a look at the [source code](src): all modules are there. Explore and look at the short guide about writing your own modules.

## Contributions

Found a bug? help us by reporting it. Want a new feature in the next BlenderProc release? Create an issue. Made something useful or fixed a bug? Show it, then. Check the [contributions guidelines](CONTRIBUTING.md).

## Change log

See our [change log](change_log.md).
See our [change log](change_log.md).
Loading

0 comments on commit 0a2f630

Please sign in to comment.