Skip to content

Source for the "Making Python 100x faster with less than 100 lines of Rust" blog post

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT
Notifications You must be signed in to change notification settings

ohadravid/poly-match

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

The repo contains the source code for the Making Python 100x faster with less than 100 lines of Rust blog post (there's a copy in this repo as well).

It's a demo library in Python that we can convert parts of to Rust, to improve its performance.

Here's a table summarizing the perf gains:

Version Avg time per iteration (ms) Multiplier
v1 - Baseline implementation (Python) 293.41 1x
v2 - Naive line-to-line translation of find_close_polygons 23.44 12.50x
v3 - Polygon implementation in Rust 6.29 46.53x
v4 - Optimized allocations 2.90 101.16x
v5 - Move outer loop to Rust (*) 0.81 362.23x

(*) v5 was added after the post was published, showing how to reduce Python to Rust overhead even more.

There are also two versions which use "vectorizing" (doing more of the work directly in numpy). While not applicable to the original library, they also show major performance gains:

Version Avg time per iteration (ms) Multiplier
v1 - Baseline implementation (Python) 293.41 1x
v1.5 - A bit of numpy vectorization 41.79 7.02x
v1.6 - A lot of numpy vectorization, by @dangrie158 5.18 56.68x

Setup

The code should work on all supported platforms (Python & Rust), but --native profiling is only supported by py-spy on x86 Linux and Windows.

(macOS / Arm Linux can still generate profiles viewing just the Python code.)

For the complete Rust setup, you'll need Rust (use rustup).

To setup with rye, run:

rye sync

To build the native extension, run:

source .venv/bin/activate
(cd poly_match_rs && maturin develop --release)

To run the benchmarks, run:

rye run python ./measure_all.py

For profiling,

sudo rye run py-spy record -o profile.svg -- python measure.py

Exploring more optimizations

There are a few more optimizations we can try.

For example, we could use (dx^2 + dy^2) < dist^2 (calculating dist^2 only once, saving a sqrt). According to the profiler outputs, can we expect this to make a big difference?

We could build a Rust bench using criterion, which would require us to "split" the Python & pure-Rust parts (Maybe using AsRef<Polygon>).

We could also try to convert the list of Polygons only once to Rust at the start of the run. According to the profiler outputs, can we expect this to make a big difference? (See v5)

License

This repo is licensed under either of

Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)

at your option.

About

Source for the "Making Python 100x faster with less than 100 lines of Rust" blog post

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published