Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gmp optional #5

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Conversation

NCGThompson
Copy link

@NCGThompson NCGThompson commented Apr 30, 2021

Fixes #4

damgård-jurik will take advantage of either gmpy2 or gmpy but doesn't require either. PyCryptodome is used for random prime generation.

Explanation

@int_to_mpz

@int_to_mpz existed before this pull request. It takes a function and its inputs as inputs. It goes through each input and casts to mpz if appropriate. With this update, if not GMP_AVAILABLE, then the given function is returned with out any modifications.

mpzCast()

Prior to this update, explicit casts to mpz() existed throughout the source code. With the exception of inside @int_to_mpz, all mpz()s were replaced with the the function mpzCast() that only casts to mpz if possible.

I chose a function rather than a type alias, so that if a user inputs a custom integral type, the output will be in that type unless mpz is available, which takes priority.

gen_prime() and gen_safe_prime()

[outdated:]In gen_saf_prime() calls to gen_prime() are replaced with Crypto...getPrime() and calls to Gmpy2...isPrime() is replaced with Crypto...isPrime(). gen_prime() has been kept for backwards compatibility. It is now merely a wrapper for Crypto...getPrime().

PyCrypto requirements

There are two similar libraries, PyCrypto and PyCryptodome. PyCryptodome is overall better than PyCrypto, however both of them suit our purposes, and damgard-jurik/prime_gen.py will work with either of them. PyCrypto's package name is pycrypto and module name is Crypto. PyCryptodome has two variants, with package names pycryptodome and pycryptodomex, and the only difference is the module names. pycryptodome's module name is Crypto, and pycryptodomex's name is Cryptodome. Since pycryptodome and pycrypto have the same name, they will break if installed in the same environment at the same time. With setup.py only one package name can be specified. Since some users may either already have pycrypto or pycryptodome installed, pycryptodomex is the only package that is guaranteed to not break anything. If pycryptodomex is uninstalled after damgard-jurik is installed, then damgard-jurik will keep working as long as either pycryptodome or pycrypto is installed.

Performance

[outdated:]As of the opening of this pull request, test.py runs an order of magnitude slower than in the master branch whether or not gmpy2 library is available. Debugging has shown that the library correctly detects gmpy2 and uses it if possible, but the slowing still exists. I have two hypotheses:

  • Key generation runs slower because Crypto.getPrime() doesn't use mpz. Cryptodome has an undocumented prime generator that does use mpz for internal use only.
  • mpzCast() calls take longer than a simple cast, and the sheer number of them slow computation down.

Neither sound likely, but I don't have any other ideas. Until I figure out how to speed up computation, I will keep this pull request a draft.

mpz can now be imported from either gmpy2 or gmpy.

Created mpzCast() function, which will behave as an mpz cast only if available.
Replaced mpz()s with mpzCast()s in utils.py, except in int_to_mpz.

int_to_mpz will return the wrapped function if gmp is not available.
In crypto.py and shamir.py, gmpy2 is no longer explicitely imported, and not needed.
pycryptodome is added as a requirement in setup.py; gmpy2 is kept as a requirement.
gen_prime() is kept as a wrapper to getPrime(), but getPrime() is called directly in gen_safe_prime().
If import Crypto... throws ImportError, then fall back to import Cryptodome...
rational: pycryptodome and pycrypto are imported as "Crypto". Installing both at once is a problem.
pycryptodomex is equivalent to pycryptodome, exceot it is imported as "Cryptodome".
The requirement was switched to pycryptodomex to break computers with pycrypto installed.

Despite the requirement list containing only one of the packages, damgard_jurik can use either of the three.
@NCGThompson
Copy link
Author

This article suggests the overhead of a function call with an input variable may be 350ns. test.py calls mpzCast() a widely varying number of times, but it tends to be around 14000. Multiplying those figures together gives about 5ms, which is much less than the 15000ms extra time unaccounted for. Clearly, the delay is not purely from function call overhead.

If GMP is not available, try importing Crypto or Cryptodome and use that for prime checking and generating.
In gmp_prime (previously gen_prime), it was possible that the output had a higher length than the requested. Adds check for this.

In gen_safe_prime, the outputed number has a more thorough primality check when not using GMP. When using GMP, primality test is thorough by default.
@NCGThompson
Copy link
Author

NCGThompson commented Apr 30, 2021

My first hypothesis turned out to correct:

Key generation runs slower because Crypto.getPrime() doesn't use mpz. Cryptodome has an undocumented prime generator that does use mpz for internal use only.

This is probably due to the computationally expensive Miller-Rabin tests being done without GMP.

I reverted damgard-jurik/prime_gen.py to its state before this pull request and started over. Now damgard-jurik requires a gmpy based library OR a PyCrypto based library. When GMP is available, test.py runs just as quickly as before this pull request, otherwise it takes >1 second on my computer.

@NCGThompson NCGThompson marked this pull request as ready for review April 30, 2021 03:47
A previous commit should have the main primality test in gen_safe_prime() set to 15. Added the 15.
@@ -27,6 +27,8 @@ pip install -e .
```
*Note that the `-e` flag will instruct pip to install the package as "editable". That is, when changes are made to any part of the package during development, those changes will immediately be available system-wide on the activated python environment.*

For best performance, [install gmpy2](https://gmpy2.readthedocs.io/en/latest/intro.html#installation).
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nickboucher Do you think this documentation change is adequate?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make gmpy2 optional
1 participant