Skip to content

A python module for generating Rabin fingerprints

Notifications You must be signed in to change notification settings

neoblackcap/pyrabin

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

50 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PyRabin

A python module for generating rabin fingerprints.

The C extension implementation is based on rabin-fingerprint-c with some bugs fixed.

Features

  • Fast fingerprint generation.
  • Split file into blocks according to the rabin fingerprints.
  • Window/Chunk size is configurable.

API

Module methods

  • set_prime(integer)
    • Set Rabin polynomial prime, default is 3.
  • set_window_size(integer)
    • Set Rabin polynomial sliding window size, default is 48 bytes.
  • set_max_block_size(integer)
    • Set Rabin polynomial max block size, default is 64K bytes.
  • set_min_block_size(integer)
    • Set Rabin polynomial min block size, default is 2K bytes
  • set_average_block_size(integer)
    • Set Rabin polynomial average block size, average is 8K bytes.
  • get_file_fingerprints(filename)
    • Returns a list of tupple, each tuple represents (offset, length, fingerprint)
  • split_file_by_fingerprints(filename)
    • Returns a list of tupple, each tuple represents (offset, length, fingerprint, filename).
    • The default filename is sha1sum of the chunk with '.blk' suffix.

class Rabin

  • register(callback)
    • Register a callback for calling when a block is reached
    • The callback is called with three parameters: (offset, length, fingerprint)
  • update(data)
    • Update Rabin fingerprint list by adding a block of data
  • clear()
    • Clear fingerprints and reset status
  • fingerprints()
    • Return fingerprints of all data passed by the update method

Examples

  • See test/test.sh for example

Bugs

  • Please report bugs / mis-formatted pages to the github issure tracker.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

About

A python module for generating Rabin fingerprints

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C 71.5%
  • Python 19.9%
  • C++ 5.7%
  • Shell 2.9%