refactor code and readme (#3)

.github/workflows/formatter.yml

@@ -0,0 +1,15 @@
+name: Formatter CI
+
+on:
+ push:
+ branches:
+ - main
+ pull_request:
+ types: [opened, reopened, synchronize]
+
+jobs:
+ formatter:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - uses: psf/black@stable

.vscode/settings.json

@@ -1,5 +1,9 @@
 {
- "editor.formatOnSave": true,
- "editor.formatOnSaveMode": "file",
- "files.trimTrailingWhitespace": true
+ "editor.formatOnSave": true,
+ "editor.formatOnSaveMode": "file",
+ "files.trimTrailingWhitespace": true,
+ "python.formatting.provider": "none",
+ "[python]": {
+ "editor.defaultFormatter": "ms-python.black-formatter"
+ }
 }

README.md

@@ -1,124 +1,197 @@
 # CamTools
 
-Tools for handling pinhole camera parameters and plotting cameras.
+Camtools: camera tools for computer vision. Useful for plotting, converting,
+projecting, and ray casting with camera parameters.
 
-## Install
+<a href="https://github.com/yxlao/camtools/actions/workflows/formatter.yml">
+<img src="https://github.com/yxlao/camtools/actions/workflows/formatter.yml/badge.svg" alt="Formatter">
+</a>
 
-```bash
-# For development.
-pip install -e .
-
-# For install.
-pip install .
-```
-
-<!-- ```bash
-mkdir build && cd build
-cmake ..
-make pip-install -j$(nproc)
-python -c "import camtools as ct; print(ct.__version__)"
-``` -->
-
-## Matrix conventions
-
-```
-K : (3, 3) # Camera intrinsic matrix.
- # [[fx, s, cx],
- # [ 0, fy, cy],
- # [ 0, 0, 1]]
- # x: goes from top-left to top-right.
- # y: goes from top-left to bottom-left.
-R : (3, 3) # Rotation matrix.
-Rc : (3, 3) # Rc = R.T = R.inv().
-t : (3,) # Translation.
-T : (4, 4) # Extrinsic matrix with (0, 0, 0, 1) row below.
- # T = [R | t
- # 0 | 1]
- # T projects world space coordinate to the camera space
- # (a.k.a view space or eye space). The camera center in world
- # space projected by T becomes [0, 0, 0, 1]^T, i.e. the camera
- # space has its origin at the camera center.
- # T @ [[|], = [[0],
- # [C], [0],
- # [|], [0],
- # [1]] [1]]
-P : (3, 4) # World-to-pixel projection matrix. P = K @ [R | t] = K @ T[:3, :].
-W2P : (4, 4) # World-to-pixel projection matrix. It is P with (0, 0, 0, 1)
- # row below. When using W2P @ point_homo, the last
- # element is always 1, thus it is ignored.
-pose : (4, 4) # Camera pose. pose = T.inv(). pose[:3, :3] = R.T = Rc. pose[:3, 3] = C.
-C : (3,) # Camera center.
-```
-
-## Coordinate conventions
-
-### 3D to 2D projection
-
-Project 3D point `[X, Y, Z, 1]` to 2D `[x, y, 1]` pixel, e.g. with
-`pixels = ct.project.points_to_pixel(points, K, T)`.
-
-```python
-# 0 -------> 1 (x)
-# |
-# |
-# v (y)
-
-cols = pixels[:, 0] # cols, width, x, top-left to top-right
-rows = pixels[:, 1] # rows, height, y, top-left to bottom-left
-cols = np.round(cols).astype(np.int32)
-rows = np.round(rows).astype(np.int32)
-cols[cols >= width] = width - 1
-cols[cols < 0] = 0
-rows[rows >= height] = height - 1
-rows[rows < 0] = 0
-```
-
-It can be confusing to use `x, y, u, v`. Prefer `row` and `col`.
-
-
-### UV coordinates
-
-```python
-# OpenGL convention:
-# 1 (v)
-# ^
-# |
-# |
-# 0 -------> 1 (u)
-
-# The following conversion accounts for pixel size
-us = 1 / width * (0.5 + cols)
-vs = 1 / height * (0.5 + (height - rows - 1))
-```
+## Installation
 
-## Notes on vector vs. matrix
-
-We choose to use 1D array for vector values like `t` and `C`. For example, `t`
-is of shape `(3, )` instead of `(3, 1)`.
-
-```python
-# The `@` operator can be directly used to dot a matrix and a vector
-# - If both arguments are 2-D they are multiplied like conventional matrices.
-# - If either argument is N-D, N > 2, it is treated as a stack of matrices
-# residing in the last two indexes and broadcast accordingly.
-# - If the first argument is 1-D, it is promoted to a matrix by prepending a 1
-# to its dimensions. After matrix multiplication the prepended 1 is removed.
-# - If the second argument is 1-D, it is promoted to a matrix by appending a 1
-# to its dimensions. After matrix multiplication the appended 1 is removed.
-
-# t is (3, ) and it is promoted to be (3, 1).
-C = - R.T @ t
-```
+```bash
+# Option 1: install from pip.
+pip install camtools
 
-## Unit tests
+# Option 2: install from git.
+pip install git+https://github.com/yxlao/camtools.git
 
-```bash
-pytest . -s
-pytest camtools -s
+# Option 3: install from source.
+git clone https://github.com/yxlao/camtools.git
+cd camtools
+pip install -e . # Dev mode, if you want to modify camtools.
+pip install . # Install mode, if you want to use camtools only.
 ```
 
+## What can you do with CamTools?
 
-## TODO
+1. Plot cameras. Useful for debugging 3D reconstruction and NeRFs!
 
-- Full unit tests
-- PyTorch/Numpy wrapper (e.g. with `eagerpy`)
+ ```python
+ import camtools as ct
+ import open3d as o3d
+ cameras = ct.camera.create_camera_ray_frames(Ks, Ts)
+ o3d.visualization.draw_geometries([cameras])
+ ```
+
+ <p align="center">
+ <img src="./camtools/assets/camera_frames.png" width="360" />
+ </p>
+
+2. Convert camera parameters.
+
+ ```python
+ pose = ct.convert.T_to_pose(T) # Convert T to pose
+ R, t = ct.convert.T_to_R_t(T) # Convert T to R and t
+ C = ct.convert.pose_to_C(pose) # Convert pose to camera center
+ K, T = ct.convert.P_to_K_T(P) # Decompose projection matrix to K and T
+ # And more...
+ ```
+
+3. Projection and ray casting.
+
+ ```python
+ # Project 3D points to pixels.
+ pixels = ct.project.points_to_pixel(points, K, T)
+
+ # Back-project depth image ot 3D points.
+ points = ct.project.im_depth_to_points(depth, K, T)
+
+ # Ray cast a triangle mesh to depth image.
+ im_depth = ct.raycast.mesh_to_depths(mesh, Ks, Ts, height, width)
+
+ # And more...
+ ```
+
+4. Image I/O and depth I/O with no surprises.
+
+ ```python
+ ct.io.imread()
+ ct.io.imwrite()
+
+ ct.io.imread_detph()
+ ct.io.imwrite_depth()
+ ```
+
+ Strict type checks and range checks are enforced. These APIs are specifically
+ designed to solve the following pain points:
+
+ - Is my image `float32` or `uint8`?
+ - Does it has range `[0, 1]` or `[0, 255]`?
+ - Is it RGB or BGR?
+ - Do my image have alpha channel?
+ - When saving depth image as integer-based `.png`, is it correctly scaled?
+
+5. Useful command-line tools (run in terminal).
+
+ ```bash
+ # Crop image boarders.
+ ct crop-boarders *.png --pad_pixel 10 --skip_cropped --same_crop
+
+ # Draw synchronized bounding boxes interactively.
+ ct draw-bboxes path/to/a.png path/to/b.png
+
+ # For more help.
+ ct --help
+ ```
+
+ <p align="center">
+ <img src="https://user-images.githubusercontent.com/1501945/241416210-e11ff3bf-22e6-46c0-8ba0-d177a0015323.png" width="400" />
+ </p>
+
+6. And more.
+ - Solve line intersections.
+ - COLMAP tools.
+ - Points normalization.
+ - ...
+
+## Camera conventions
+
+<p align="center">
+ <img src="./camtools/assets/camera_coordinates.svg" width="360" />
+</p>
+
+We follow the standard pinhole camera model:
+
+- **Camera coordinate:** right-handed, with $Z$ pointing away from the camera
+ towards the view direction and $Y$ axis pointing down. Note that this is
+ different from the Blender convention, where $Z$ points towards the opposite
+ view direction and the $Y$ axis points up.
+- **Image coordinate:** starts from the top-left corner of the image, with $x$
+ pointing right (corresponding to the image width) and $y$ pointing down
+ (corresponding to the image height). This is also consistent with OpenCV, but
+ pay attention that the 0-th dimension in the image array is the height (i.e.,
+ $y$) and the 1-th dimension is the width (i.e., $x$). That is:
+ - $x$ <=> width <=> column <=> the 1-th dimension
+ - $y$ <=> height <=> row <=> the 0-th dimension
+- `K`: `(3, 3)` camera intrinsic matrix.
+ ```python
+ K = [[fx, s, cx],
+ [ 0, fy, cy],
+ [ 0, 0, 1]]
+ ```
+- `T` or `W2C`: `(4, 4)` camera extrinsic matrix.
+ ```python
+ T = [[R | t = [[R_01, R_02, R_03, t_0],
+ 0 | 1]] [R_11, R_12, R_13, t_1],
+ [R_21, R_22, R_23, t_2],
+ [ 0, 0, 0, 1]]
+ ```
+ - `T` is also known as the world-to-camera `W2C` matrix, which transforms a
+ point in the world coordinate to the camera coordinate.
+ - `T`'s shape is `(4, 4)`, not `(3, 4)`.
+ - `T` must be invertible, where `np.linalg.inv(T) = pose`.
+ - The camera center `C` in world coordinate is projected to `[0, 0, 0, 1]` in
+ camera coordinate, i.e.,
+ ```python
+ T @ C = np.array([0, 0, 0, 1]).T
+ ```
+- `R`: `(3, 3)` rotation matrix.
+ ```python
+ R = T[:3, :3]
+ ```
+ - `R` is a rotation matrix. It is an orthogonal matrix with determinant 1, as
+ rotations preserve volume and orientation.
+ - `R.T == np.linalg.inv(R)`
+ - `np.linalg.norm(R @ x) == np.linalg.norm(x)`, where `x` is a `(3, )` vector.
+- `t`: `(3,)` translation vector.
+ ```python
+ t = T[:3, 3]
+ ```
+ - `t`'s shape is `(3,)`, not `(3, 1)`.
+- `pose` or `C2W`: `(4, 4)` camera pose matrix. It is the inverse of `T`.
+ ```python
+ pose = T.inv()
+ ```
+ - `pose` is also known as the camera-to-world `C2W` matrix, which transforms a
+ point in the camera coordinate to the world coordinate.
+ - `pose` is the inverse of `T`, i.e., `pose == np.linalg.inv(T)`.
+- `C`: camera center.
+ ```python
+ C = pose[:3, 3]
+ ```
+ - `C`'s shape is `(3,)`, not `(3, 1)`.
+ - `C` is the camera center in world coordinate. It is also the translation
+ vector of `pose`.
+- `P`: `(3, 4)` the camera projection matrix.
+ - `P` is the world-to-pixel projection matrix, which projects a point in the
+ homogeneous world coordinate to the homogeneous pixel coordinate.
+ - `P` is the product of the intrinsic and extrinsic parameters.
+ ```python
+ # P = K @ [R | t]
+ P = K @ np.hstack([R, t[:, None]])
+ ```
+ - `P`'s shape is `(3, 4)`, not `(4, 4)`.
+ - It is possible to decompose `P` into intrinsic and extrinsic matrices by QR
+ decomposition.
+ - Don't confuse `P` with `pose`.
+- For more details, please refer to the following blog posts:
+ [part 1](https://ksimek.github.io/2012/08/14/decompose/),
+ [part 2](https://ksimek.github.io/2012/08/22/extrinsic/),
+ and [part 3](https://ksimek.github.io/2013/08/13/intrinsic/).
+
+## Future works
+
+- Refined APIs.
+- Full PyTorch/Numpy compatibility.
+- Unit tests.

camtools/__init__.py

@@ -13,4 +13,7 @@
 from . import sanity
 from . import solver
 from . import stat
-from .version import __version__
+
+import pkg_resources
+
+__version__ = pkg_resources.get_distribution("camtools").version