Cannot specify private key ID inside PKCS#11 URI, key not found

[johndoe31415]

Hi there. I'm using OpenSC 0.17.0-3 and OpenSSL 1.1.0g-2ubuntu4.1 on Ubuntu Mate 18.04 x86_64 and use it with a NitroKey HSM for signing. It works nicely as long as I specify my key IDs in the legacy "slot:id" format, e.g.:

$ OPENSSL_CONF=test.cnf openssl req -new -x509 -days 365 -sha256 -keyform engine -engine pkcs11 -subj "/CN=Test CA" -out test.crt -key 0:1

The OpenSSL config is very unspectacular:

openssl_conf = openssl_conf

[openssl_conf]
engines = engines_config

[engines_config]
pkcs11 = engine_pkcs11_parameters

[engine_pkcs11_parameters]
engine_id = pkcs11
dynamic_path = /usr/lib/x86_64-linux-gnu/engines-1.1/libpkcs11.so
MODULE_PATH = /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
init = 0

[req]
distinguished_name = default
x509_extensions = extensions
req_extensions = extensions

[extensions]
basicConstraints = critical,CA:TRUE
keyUsage = critical,digitalSignature,keyCertSign,cRLSign
subjectKeyIdentifier = hash

It also works perfectly when specifying a PKCS#11 URI and a key label, i.e.:

$ OPENSSL_CONF=test.cnf openssl req -new -x509 -days 365 -sha256 -keyform engine -engine pkcs11 -subj "/CN=Test CA" -out test.crt -key 'pkcs11:object=my_key_label'

However, when I try to specify the key by its's key ID (01) using a PKCS#11 URI, it fails:

$ OPENSSL_CONF=test.cnf openssl req -new -x509 -days 365 -sha256 -keyform engine -engine pkcs11 -subj "/CN=Test CA" -out test.crt -key 'pkcs11:id=1;type=private'
engine "pkcs11" set.
Enter PKCS#11 token PIN for UserPIN (SmartCard-HSM):
Key not found.
PKCS11_get_private_key returned NULL
cannot load Private Key from engine
139964866924992:error:80067065:pkcs11 engine:ctx_load_privkey:object not found:eng_back.c:876:
139964866924992:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:crypto/engine/eng_pkey.c:78:
unable to load Private Key

Using OPENSC_DEBUG=3, I have created two logs:

$ OPENSSL_CONF=test.cnf openssl req -new -x509 -days 365 -sha256 -keyform engine -engine pkcs11 -subj "/CN=Test CA" -out test.crt -key 0:1 >found 2>&1
$ OPENSSL_CONF=test.cnf openssl req -new -x509 -days 365 -sha256 -keyform engine -engine pkcs11 -subj "/CN=Test CA" -out test.crt -key 'pkcs11:id=1;type=private' >notfound 2>&1

Then diffed them. Disregarding timestamps and pointer values, the relevant difference appears ~30 lines after the 11 bytes APDU is sent to the device containing my PIN (no worries, I will change it). Log when everything works (legacy "-key 0:1"): https://gist.github.com/johndoe31415/d0473110d44c036cd014392282583cb1

Log when it can't find the key (-key pkcs11:id=1): https://gist.github.com/johndoe31415/9b7f3cf742a1e5f58a8b581c7b258d05

Not sure if I'm doing something wrong here. Tried to specify id=01, id=1, id=id_01 on the command line as well, but neither worked.

Thanks for looking at this.
All the best,
Joe

[dengert]

https://tools.ietf.org/html/rfc7512 says: in "3. Examples of PKCS #11 URIs":
'Also note that the "id" attribute value is entirely percent-encoded, as recommended."
and gives an example.

try -key 'pkcs11:id=%01;type=private'

[johndoe31415]

Aaaaaaaaaaaaaaah! You're brilliant! Thank you so much, I completely missed that. Works perfectly.

Thank you again so much and have a great weekend :-)

All the best, Joe