pkcs15-crypt signing fails on release 0.25.0 w/ a YubiKey-bound RSA key

[ashort96]

Problem Description

When trying to sign with a YubiKey 5c nano on release 0.25.0, I get the following error:

> pkcs15-crypt --sign --sha-256 --pkcs1 --raw --input <(openssl dgst -sha256 -binary file.txt)
Using reader with a card: Yubico YubiKey FIDO+CCID
Enter PIN [PIN]:
No input file specified. Reading from stdin
Compute signature failed: Internal error

I believe this was introduced in this PR: #2975

The pad algorithm output from the logs is 0xC0, which lines up with SC_ALGORITHM_RSA_PAD_PKCS1 from:

OpenSC/src/libopensc/opensc.h

Lines 118 to 120 in d2cbd63

	#define SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_01 0x00000040 /* PKCS#1 v1.5 padding type 1 */
	#define SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_02 0x00000080 /* PKCS#1 v1.5 padding type 2 */
	#define SC_ALGORITHM_RSA_PAD_PKCS1 (SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_01 | SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_02) /* PKCS#1 v1.5 (type 1 or 2) */

Operating System: macOS Sonoma 14.0 (23A344)
Machine: 2021 Macbook Pro M1 Max

Proposed Resolution

Steps to reproduce

Update to the latest opensc (0.25.0), try to sign with pkcs15-crypt using an RSA key on a YubiKey.

Logs

Logs from running the above command on v0.25.0

P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] pkcs15-sec.c:609:sc_pkcs15_compute_signature: called
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] pkcs15-sec.c:662:sc_pkcs15_compute_signature: supported algorithm flags 0x1, private key usage 0x2E
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:683:sc_get_encoding_flags: called
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:687:sc_get_encoding_flags: iFlags 0x20C0, card capabilities 0x1
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:748:sc_get_encoding_flags: pad flags 0x20C0, secure algorithm flags 0x1
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:749:sc_get_encoding_flags: returning with: 0 (Success)
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] pkcs15-sec.c:734:sc_pkcs15_compute_signature: DEE flags:0x000020c0 alg_info->flags:0x00000001 pad:0x000020c0 sec:0x00000001
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:608:sc_pkcs1_encode: called
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:614:sc_pkcs1_encode: hash algorithm 0x2000, pad algorithm 0xC0
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] padding.c:675:sc_pkcs1_encode: returning with: -1400 (Internal error)
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] pkcs15-sec.c:743:sc_pkcs15_compute_signature: Unable to add padding: -1400 (Internal error)
P:48614; T:0x8043991808 15:33:16.870 [pkcs15-crypt] pkcs15-sec.c:786:sc_pkcs15_compute_signature: returning with: -1400 (Internal error)

Logs from running the same command on v0.24.0

P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:622:sc_get_encoding_flags: iFlags 0x2002, card capabilities 0x1
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:678:sc_get_encoding_flags: pad flags 0x2002, secure algorithm flags 0x1
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:679:sc_get_encoding_flags: returning with: 0 (Success)
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] pkcs15-sec.c:732:sc_pkcs15_compute_signature: DEE flags:0x00002002 alg_info->flags:0x00000001 pad:0x00002002 sec:0x00000001
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:544:sc_pkcs1_encode: called
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:550:sc_pkcs1_encode: hash algorithm 0x2000, pad algorithm 0x2
P:66864; T:0x7944100928 16:11:48.815 [pkcs15-crypt] padding.c:574:sc_pkcs1_encode: returning with: 0 (Success)

[dengert]

padding.c:614:sc_pkcs1_encode: hash algorithm 0x2000, pad algorithm 0xC0 calls the switch statement:

OpenSC/src/libopensc/padding.c

Lines 628 to 675 in d2cbd63

	switch(pad_algo) {
	case SC_ALGORITHM_RSA_PAD_NONE:
	/* padding done by card => nothing to do */
	if (out != tmp)
	memcpy(out, tmp, tmp_len);
	*out_len = tmp_len;
	LOG_FUNC_RETURN(ctx, SC_SUCCESS);
	case SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_01:
	/* add pkcs1 bt01 padding */
	rv = sc_pkcs1_add_01_padding(tmp, tmp_len, out, out_len, mod_len);
	LOG_FUNC_RETURN(ctx, rv);
	case SC_ALGORITHM_RSA_PAD_PSS:
	/* add PSS padding */
	#ifdef ENABLE_OPENSSL
	mgf1_hash = flags & SC_ALGORITHM_MGF1_HASHES;
	if (hash_algo == SC_ALGORITHM_RSA_HASH_NONE) {
	/* this is generic RSA_PKCS1_PSS mechanism with hash
	* already done outside of the module. The parameters
	* were already checked so we need to adjust the hash
	* algorithm to do the padding with the correct hash
	* function.
	*/
	hash_algo = hash_len2algo(tmp_len);
	}
	/* sLen is by default same as hash length */
	if (!(md = hash_flag2md(ctx, hash_algo)))
	return SC_ERROR_NOT_SUPPORTED;
	sLen = EVP_MD_size(md);
	sc_evp_md_free(md);
	/* if application provide sLen, use it */
	if (pMechanism != NULL) {
	CK_MECHANISM *mech = (CK_MECHANISM *)pMechanism;
	CK_RSA_PKCS_PSS_PARAMS *pss_params;
	if (mech->pParameter && sizeof(CK_RSA_PKCS_PSS_PARAMS) == mech->ulParameterLen) {
	pss_params = mech->pParameter;
	sLen = pss_params->sLen;
	}
	}
	rv = sc_pkcs1_add_pss_padding(ctx, hash_algo, mgf1_hash,
	tmp, tmp_len, out, out_len, mod_bits, sLen);
	#else
	rv = SC_ERROR_NOT_SUPPORTED;
	#endif
	LOG_FUNC_RETURN(ctx, rv);
	default:
	/* We shouldn't be called with an unexpected padding type, we've already
	* returned SC_ERROR_NOT_SUPPORTED if the card can't be used. */
	LOG_FUNC_RETURN(ctx, SC_ERROR_INTERNAL);

But both SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_01 and SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_02 are both set. Only one pad_algo bit should be set.

This may be related to e8883b1

@xhanulik any ideas?

[xhanulik]

Yes, when extracting pad_algo from flags, the code now does not handle when both SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_01 and SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_02 are set. In pad_algo the SC_ALGORITHM_RSA_PAD_PKCS1_TYPE_02 should be excluded.
I will create PR with patch for this.