Skip to content
forked from binpash/pash

PaSh: Light-touch Data-Parallel Shell Processing

License

Notifications You must be signed in to change notification settings

DanielSongShen/pash

 
 

Repository files navigation

PaSh: Light-touch Data-Parallel Shell Processing

A system for parallelizing POSIX shell scripts.

Quick Jump: Running PaSh | Installation | Testing | Repo Structure | Community & More

Running PaSh

To parallelize, say, ./evaluation/intro/hello-world.sh with parallelization width of 2, from the top-level directory of the repository run:

./pa.sh ./evaluation/intro/hello-world.sh

Run ./pa.sh --help to get more information about the available commands. See docs/tutorial for a longer tutorial.

Installation

On Ubuntu, Fedora, Debian, or Arch, run curl up.pash.ndr.md | sh to setup PaSh. Alternatively, you can clone the repo and run ./scripts/distro-deps.sh; ./scripts/setup-pash.sh

You can also built the Docker container from scratch by running

git clone [email protected]:binpash/pash.git
cd pash/scripts
docker build -t "pash/18.04" .
# Then launch container:
docker run --name pash-play -it pash/18.04

More installation instructions in the tutorial. Remember to export PASH_TOP in your startup scripts!

Testing

To execute the current tests before committing and pushing code, simply run:

./scripts/run_tests.sh

Repo Structure

This repo hosts the core pash development. The structure is as follows:

  • annotations: DSL characterizing commands, parallelizability study, and associated annotations.
  • compiler: Shell-Dataflow translations and associated parallelization transformations.
  • docs: Design documents, tutorials, installation instructions, etc.
  • evaluation: Shell pipelines and example scripts used for the evaluation.
  • runtime: Runtime component — e.g., eager, split, and assocaited combiners.
  • scripts: Scripts related to continuous integration, deployment, and testing.

Community & More

Mailing Lists:

  • pash-users: Join this mailing list for discussing all things pash
  • pash-commits: Join this mailing list for commit notifications

Development/contributions:

About

PaSh: Light-touch Data-Parallel Shell Processing

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 33.4%
  • Shell 29.2%
  • OCaml 19.9%
  • C 13.2%
  • JavaScript 2.6%
  • C++ 1.1%
  • Other 0.6%