Skip to content

Latest commit

 

History

History
187 lines (153 loc) · 16.3 KB

Super-Computing.md

File metadata and controls

187 lines (153 loc) · 16.3 KB
Raw

Super Computing: HPC, Distributed Computing, Cloud computing, Cluster computing, Grid computing, Parallel computing, Hardware arch (ARM, CUDA, GPU, MIPS), Kernels, Compilers (source-to-source compiler, transcompiler/ transpilers)

ARM-CUDA

GPU

  • GPU-benchmarking :: GPU benchmarking on Julia.
  • MXNet.jl :: The dmlc/mxnet Julia package that brings flexible and efficient GPU computing and state-of-art deep learning to Julia.
  • Titan.jl :: Write GPU kernels using pure Julia.

Org-JuliaGPU

Mailing list : https://groups.google.com/forum/#!forum/julia-gpu

  • AbstractGPUArray.jl :: An abstract interface for a GPU array, must be implemented by a back-end like OpenCL or OpenGL.
  • CLBLAS.jl :: CLBLAS integration for Julia.
  • CUBLAS.jl :: Julia interface to CUBLAS.
  • CUDA.jl :: This package wraps key functions in CUDA Driver API.
  • CUDArt.jl :: Wrapper for CUDA runtime API.
  • CUDNN.jl :: Julia wrapper for the NVIDIA cuDNN GPU deep learning library.
  • CURAND.jl : Wrapper for NVidia's cuRAND library.
  • HSA.jl :: Julia Bindings for the HSA Runtime.
  • julia-CuMatrix :: CUDA Matrix library.
  • julia-kernels :: A small suite of tools aimed at being able to write kernels in Julia, which could be executed on the CPU, or as GPU kernels.
  • MXNet.jl :: The dmlc/mxnet Julia package for flexible and efficient GPU computing and state-of-art deep learning in Julia.
  • OpenCL.jl :: OpenCL 1.2 Julia bindings - a cross platform parallel computation API for programming parallel devices, with implementations from AMD, Nvidia, Intel, and others, similar in scope to PyOpenCL.
  • UberSignals.jl :: Concept for a fast event signal system, using JIT and GPU acceleration, loosely inspired by Reactive.jl.
Resources

BENCHMARKS

COMPILERS

  • Clang.jl :: Julia interface to libclang and C wrapper generator and its fork CIndex.jl to access the libclang interface of the LLVM Clang compiler.
  • CompilerTools.jl :: The CompilerTools package, part of the High Performance Scripting project at Intel Labs.
  • CompilerOptions.jl :: A Julia package for reading compiler options.
  • Eglib.jl :: Clang.jl wrapping example, C code from @kindlmann.
  • JITTools.jl :: Tools for working with in-memory object.
  • julia-icc-travis :: Build Julia using icc on the Travis CI.
  • LLVM.jl :: A Julia package for LLVM.
  • llvm-cbe :: A resurrected LLVM C Backend, with improvements for Julia.

Preprocessor

  • Precompile.jl :: Precompilation in Julia base, and its documentation
  • SnoopCompile.jl :: Make your packages work faster with more extensive precompilation - Spy on the JIT-compiler and learn which functions & types it's compiling.

Transpiler

  • Julia2C :: A source-to-source translator from Julia to C. This initial version converts basic Julia types and expressions into the corresponding C types and statements.

Computer Performance

Code Analyzer

  • IACA.jl :: A wrapper around intel's arch code analyzer for Julia.

Time

CONCURRENCY

  • Actors.jl :: An Actor Model implementation in Julia.
  • RawMutex.jl :: A __MUT__ual __EX__clusion program object in Julia that allows multiple program threads to share the same resource, such as file access, but not simultaneously.

Distributed Computing

Cloud/ Cluster

  • AWS.jl :: supports the EC2 and S3 API's, letting you start and stop EC2 instances dynamically.
  • AWSCore.jl :: Amazon Web Services Core Functions and Types.
  • AWSS3.jl :: AWS S3 Simple Storage Service interface for Julia.
  • ChainedVectors.jl :: Few utility types over Julia Vector type.
  • CloudArray.jl :: Easy big data programming in the cloud.
  • ClusterDicts.jl :: Global and Distributed dictionaries for Julia.
  • Flume.jl :: A port of the Google Flume Data-Parallel Pipeline system.
  • FunHPC.jl :: Functional High-Performance Computing - A high-level API for distributed computing, implemented on top of MPI. Also on Bitbucket.
  • GCloud.jl :: Tools for working with Google Compute engine via the cloud CLI.
  • HavenOnDemand.jl :: Julia package to access HPE Haven OnDemand API.
  • HPAT.jl :: High Performance Analytics Toolkit (HPAT) is a Julia-based framework for big data analytics on clusters.
  • OCAWS.jl :: An AWS library.
  • ParallelAccelerator.jl :: The ParallelAccelerator package, part of the High Performance Scripting project at Intel Labs.
  • ParallelGLM.jl :: Parallel fitting of GLMs using SharedArrays.
  • PTools.jl :: A collection of utilities for parallel computing in Julia.
  • SGEArrays.jl :: SGEArray implements a simple iterator in Julia to efficiently handle Sun Grid Engine task arrays.

Grid Computing

  • IBFS.jl :: Grid simulation solver.

Job Scheduler

  • ClusterManagers.jl :: Support for different clustering technologies.
  • Collectl.jl :: Plotting information from Collectl in julia.
  • LCJC.jl :: Loosely Coupled Julia Clusters.
  • LoraMPI.jl :: MPI Job Manager for Lora Parralel-Centric Runners.
  • MatlabCluster.jl :: Julia cluster manager for Matlab Job Scheduler.
  • Persist.jl :: The package Persist allows running jobs independent of the Julia shell.
  • SimJulia.jl :: A combined continuous time / discrete event process oriented simulation framework written in Julia inspired by the Simula library DISCO and the Python library SimPy.

Parallel Computing

  • ArrayFire.jl by @hshindo :: Julia bindings for ArrayFire.
  • ArrayFire.jl by @JuliaComputing :: Julia Wrapper for the ArrayFire library.
  • Dagger.jl :: A framework for out-of-core and parallel computation and hierarchical Scheduling of DAG Structured Computations.
  • MT-Workloads :: Multi-threaded workloads in Julia.
  • Slurm.jl :: Experimental Julia interface to slurm.schedmd.com.

SIMD Computing

  • SIMD.jl :: Explicit SIMD vector operations for Julia.
  • SIMDVectors.jl :: An experimental package that uses the PR #15244 to create a stack allocated fixed size vector which supports SIMD operations and very similar in spirit to the SIMD.jl package.

Org-JuliaParallel

  • Blocks.jl :: A framework to represent chunks of entities and parallel methods on them.
  • ClusterManagers.jl :: Support for different clustering technologies.
  • DistributedArrays.jl :: Distributed Arrays in Julia.
  • Elly.jl :: Hadoop HDFS and Yarn client.
  • HDFS.jl :: An interface wrapper over the Hadoop HDFS library that wraps the HDFS C library libhdfs and provides APIs similar to Julia Filesystem APIs which can be used for direct access to HDFS files.
  • hwloc.jl :: The Portable Hardware Locality (hwloc) package wraps the hwloc library to provide a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures, including NUMA memory nodes, sockets, shared caches, cores and simultaneous multithreading.
  • MessageUtils.jl :: A collection of utilities for messaging.
  • MPI.jl :: A basic Julia wrapper for the portable message passing system Message Passing Interface (MPI).
    • JuliaMPIMonteCarlo :: Illustrative examples using Julia and MPI to do Markov Chain Monte Carlo (MCMC) methods.
  • ScaLAPACK.jl :: Scalable Linear Algebra PACKage.
Resources

Org-JuliaLang

  • Yeppp.jl :: A low level, high performance library for vectorized operations, elementwise operation on arrays, supports the x86(-64), ARM and MIPS architectures, and takes advantage of a lot of SIMD extensions (from SSE to AVX2 on x86, NEON on ARM). The New BSD(3-clause BSD)-licensed source code is hosted on Bitbucket.

Publications