Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

softhsm2 leaks memory #684

Closed
oerdnj opened this issue Sep 27, 2022 · 5 comments
Closed

softhsm2 leaks memory #684

oerdnj opened this issue Sep 27, 2022 · 5 comments

Comments

@oerdnj
Copy link

oerdnj commented Sep 27, 2022

Hey, I've implemented a custom memory tracking in BIND 9 for the external libraries, including OpenSSL, and it found that softhsm2 module is leaking following memory (lines should match OpenSSL 3.0.5):

	ptr 0x56056e775b50 size 264 file ../crypto/evp/pmeth_lib.c line 130
	ptr 0x56056e708570 size 120 file ../crypto/ec/ec_kmeth.c line 184
	ptr 0x56056e708310 size 120 file ../crypto/rsa/rsa_meth.c line 48
	ptr 0x56056e708280 size 24 file ../crypto/rsa/rsa_meth.c line 71
	ptr 0x56056e774c10 size 264 file ../crypto/evp/pmeth_lib.c line 130
	ptr 0x56056e6f6020 size 56 file ../crypto/threads_pthread.c line 50

It's following methods:

  1. ptr 0x55dd947bb8f0 size 24 file ../crypto/rsa/rsa_meth.c line 71 --> RSA_meth_set1_name()
  2. ptr 0x55dd947bbbe0 size 120 file ../crypto/ec/ec_kmeth.c line 184 --> EC_KEY_METHOD_new()
  3. ptr 0x55dd947bb980 size 120 file ../crypto/rsa/rsa_meth.c line 48 --> RSA_meth_dup()
  4. ptr 0x55dd948287b0 size 264 file ../crypto/evp/pmeth_lib.c line 130 --> EVP_PKEY_meth_new()
@oerdnj
Copy link
Author

oerdnj commented Sep 27, 2022

The leak can be triggered just by loading and freeing engine, e.g. ENGINE_free(ENGINE_by_id('pkcs11')); will trigger the memory leak...

@oerdnj
Copy link
Author

oerdnj commented Sep 27, 2022

Thinking about it a little more, it could also be a bug in engine_pkcs11.

@oerdnj
Copy link
Author

oerdnj commented Sep 27, 2022

Which is in fact true...

@oerdnj oerdnj closed this as completed Sep 27, 2022
@oerdnj
Copy link
Author

oerdnj commented Sep 27, 2022

Reported here, if anyone is interested: OpenSC/libp11#475

@rijswijk
Copy link
Contributor

Thanks for the write-up @oerdnj (and finding the place where the leak is)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants