Skip to content

Tkunl/hash-worker

Repository files navigation

Hash Worker plugin-react version codecov GitHub License

Introduce

中文文档

Hash-worker is a library for fast calculation of file hashes. It is based on hash-wasm and utilizes WebWorkers for parallel computation, which speeds up computation when processing file blocks.

Hash-worker supports two hash computation algorithms: MD5 and CRC32.

Both browser and node.js are supported.

Unit testing using Jest achieved 97% line coverage.

Warning

The merkleHash computed by the Hash-worker is the root hash of a MerkleTree constructed based on file block hashes. Note that this is not directly equivalent to a hash of the file itself.

Install

$ yarn add hash-worker
# or
$ npm install hash-worker

Usage

CDN (Global)

<script src="https://unpkg.com/hash-worker/dist/global.js"></script>
<script>
  HashWorker.getFileHashChunks()
</script>

ESM

import { getFileHashChunks, destroyWorkerPool, HashChksRes, HashChksParam } from 'hash-worker'

function handleGetHash(file: File) {
  const param: HashChksParam = {
    file: file,
    config: {
      workerCount: 8,
      strategy: Strategy.md5
    }
  }

  getFileHashChunks(param).then((data: HashChksRes) => {
    console.log('chunksHash', data.chunksHash)
  })
}

/**
 * Destroy Worker Thread
 */
function handleDestroyWorkerPool() {
  destroyWorkerPool()
}

[WARNING]

If you are using Vite as a build tool and are experiencing dependency optimization issues with the hash-worker package, you can exclude the hash-worker package from dependency optimization in the vite.config.js file.

Attention: Old version of vite may not emit errors.

// vite.config.js
import {defineConfig} from 'vite';
import vue from '@vitejs/plugin-vue'

export default defineConfig({
  plugins: [vue()],
  optimizeDeps: {
    exclude: ['hash-worker'] // new added
  }
})

Options

HashChksParam

HashChksParam is used to configure the parameters needed to calculate the hash.

filed type default description
file File / Files that need to calculate the hash (required for browser environments)
filePath string / Path to the file where the hash is to be calculated (required for Node environments)
config Config Config Parameters for calculating the Hash

Config

filed type default description
chunkSize number 10 (MB) Size of the file slice
workerCount number 8 Number of workers turned on at the same time as the hash is calculated
strategy Strategy Strategy.mixed Hash computation strategy
borderCount number 100 The cutoff for the hash calculation rule in 'mixed' mode
isCloseWorkerImmediately boolean true Whether to destroy the worker thread immediately when the calculation is complete
// strategy.ts
export enum Strategy {
  md5 = 'md5',
  crc32 = 'crc32',
  mixed = 'mixed',
}

When Strategy.mixed strategy is used, if the number of file fragments is less than borderCount, the md5 algorithm will be used to calculate the hash value to build the MerkleTree. Otherwise, it switches to using the crc32 algorithm for MerkleTree construction.

HashChksRes

HashChksRes is the returned result after calculating the hash value.

filed type description
chunksBlob Blob[] In a browser environment only, the Blob[] of the file slice is returned
chunksHash string[] Hash[] for file slicing
merkleHash string The merkleHash of the file
metadata FileMetaInfo The metadata of the file

FileMetaInfo

filed type description
name string The name of the file used to calculate the hash
size number File size in KB
lastModified number Timestamp of the last modification of the file
type string file extension
Wroker Count Speed
1 234 MB/s
4 610 MB/s
8 851 MB/s
12 1011 MB/s
  • These measurements were made with Chrome v126 on a Zen3 Desktop CPU

LICENSE

MIT

Contributions

Contributions are welcome! If you find a bug or want to add a new feature, please open an issue or submit a pull request.

Author and contributors

Tkunl Kanno Eternal-could

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages