Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pcscd fails to read future yubikeys after removing a yubikey, until restarted #125

Closed
oddlama opened this issue May 1, 2022 · 25 comments
Closed

Comments

@oddlama
Copy link

oddlama commented May 1, 2022

Versions

  • ccid-1.5.0
  • pcsc-lite-1.9.5
  • libusb 1.0.26
pcsc-lite version 1.9.5.
Copyright (C) 1999-2002 by David Corcoran <[email protected]>.
Copyright (C) 2001-2018 by Ludovic Rousseau <[email protected]>.
Copyright (C) 2003-2004 by Damien Sauveron <[email protected]>.
Report bugs to <[email protected]>.
Enabled features: Linux x86_64-pc-linux-gnu libsystemd serial usb libudev usbdropdir=/usr/lib64/readers/usb ipcdir=/run/pcscd filter configdir=/etc/reader.conf.d

Platform

  • gentoo (~amd64, updated 01.05.2022)
  • (Smart card middleware name and version) Unclear to me what this means.
  • Yubico Yubikey
  • Yubikey 5C NFC

Issue

  • What do you do?

I replug my yubikey, or wake my system from sleep (which reenumerates the USB device). Now I want to use it with gpg and for simplicity, we assume that I just want to execute gpg --card-status.

  • What result do you expect?

The yubikey is detected and read.

  • What result do you get instead?

Generally, when I insert a yubikey and try to read it with gpg --card-status, the first attempt ALWAYS fails. Regardless of the situation. The second try and all successive tries work exactly if pcscd was freshly restarted. If pcscd is already running and had seen a yubikey previously, it always fails until restarted.

Here are the relevant syslogs for the described actions. At the end of this issue I have included a more detailed pcscd debug log. When I just plug-in the yubikey for the first time after a fresh system restart:

May 01 13:07:51 kernel: pcieport 0000:06:02.0: enabling device (0000 -> 0002)
May 01 13:07:51 kernel: pci 0000:3d:00.0: enabling device (0000 -> 0002)
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: xHCI Host Controller
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: new USB bus registered, assigned bus number 3
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
May 01 13:07:51 kernel: usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.16
May 01 13:07:51 kernel: usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
May 01 13:07:51 kernel: usb usb3: Product: xHCI Host Controller
May 01 13:07:51 kernel: usb usb3: Manufacturer: Linux 5.16.18-gentoo-dist xhci-hcd
May 01 13:07:51 kernel: usb usb3: SerialNumber: 0000:3d:00.0
May 01 13:07:51 kernel: hub 3-0:1.0: USB hub found
May 01 13:07:51 kernel: hub 3-0:1.0: 2 ports detected
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: xHCI Host Controller
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: new USB bus registered, assigned bus number 4
May 01 13:07:51 kernel: xhci_hcd 0000:3d:00.0: Host supports USB 3.1 Enhanced SuperSpeed
May 01 13:07:51 kernel: usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.16
May 01 13:07:51 kernel: usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
May 01 13:07:51 kernel: usb usb4: Product: xHCI Host Controller
May 01 13:07:51 kernel: usb usb4: Manufacturer: Linux 5.16.18-gentoo-dist xhci-hcd
May 01 13:07:51 kernel: usb usb4: SerialNumber: 0000:3d:00.0
May 01 13:07:51 kernel: hub 4-0:1.0: USB hub found
May 01 13:07:51 kernel: hub 4-0:1.0: 2 ports detected
May 01 13:07:51 kernel: usb 3-1: new full-speed USB device number 2 using xhci_hcd
May 01 13:07:51 kernel: usb 3-1: New USB device found, idVendor=1050, idProduct=0407, bcdDevice= 5.27
May 01 13:07:51 kernel: usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
May 01 13:07:51 kernel: usb 3-1: Product: YubiKey OTP+FIDO+CCID
May 01 13:07:51 kernel: usb 3-1: Manufacturer: Yubico
May 01 13:07:51 kernel: input: Yubico YubiKey OTP+FIDO+CCID as /devices/pci0000:00/0000:00:1d.0/0000:05:00.0/0000:06:02.0/0000:3d:00.0/usb3/3-1/3-1:1.0/0003:1050:0407.006F/input/input94
May 01 13:07:51 kernel: hid-generic 0003:1050:0407.006F: input,hidraw0: USB HID v1.10 Keyboard [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:3d:00.0-1/input0
May 01 13:07:51 kernel: hid-generic 0003:1050:0407.0070: hiddev96,hidraw1: USB HID v1.10 Device [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:3d:00.0-1/input1
May 01 13:07:51 systemd[1]: Started PC/SC Smart Card Daemon.
May 01 13:07:53 kernel: pci_bus 0000:07: Allocating resources
May 01 13:07:53 kernel: pci_bus 0000:3d: Allocating resources

Afterwards, the first gpg --card-status fails and produces these additional logs (scdaemon segfault):

May 01 13:08:05 kernel: pipe-connection[1877414]: segfault at 0 ip 00007f39626622de sp 00007f3962434b18 error 4 in libc.so.6[7f396251c000+169000]
May 01 13:08:05 kernel: Code: 00 00 00 00 00 0f 1f 80 00 00 00 00 89 f8 31 d2 c5 c1 ef ff 09 f0 25 ff 0f 00 00 3d 80 0f 00 00 0f 8f 56 03 00 00 c5 fe 6f 0f <c5> f5 74 06 c5 fd da c1 c5 fd 74 c7 c5 fd d7 c8 85 c9 74 7e f3 0f
May 01 13:08:05 systemd[1]: Started Process Core Dump (PID 1878669/UID 0).
May 01 13:08:05 systemd-coredump[1878670]: Resource limits disable core dumping for process 1877413 (scdaemon).
May 01 13:08:05 systemd-coredump[1878670]: [🡕] Process 1877413 (scdaemon) of user 1000 dumped core.
May 01 13:08:05 systemd[1]: [email protected]: Deactivated successfully.

And from then, gpg --card-status works without producing additional logs.

Now the more severe problem is that as soon as I remove my yubikey, future yubikeys are never detected. And from the point of removal onwards, libusb errors are periodically written to the syslog. These errors continue for 1 minute, until systemd deactivates pcscd. This also temporarily resolves the issue, as pcscd is then restarted when plugging in a yubikey again.

May 01 13:13:27 kernel: usb 3-1: USB disconnect, device number 2
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: xHCI host controller not responding, assume dead
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: HC died; cleaning up
May 01 13:13:27 kernel: pcieport 0000:00:1d.0: pciehp: Slot(16): Link Down
May 01 13:13:27 kernel: pcieport 0000:00:1d.0: pciehp: Slot(16): Card not present
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: remove, state 4
May 01 13:13:27 kernel: usb usb4: USB disconnect, device number 1
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: USB bus 4 deregistered
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: remove, state 1
May 01 13:13:27 kernel: usb usb3: USB disconnect, device number 1
May 01 13:13:27 kernel: pcieport 0000:06:00.0: can't change power state from D3cold to D0 (config space inaccessible)
May 01 13:13:27 kernel: pcieport 0000:06:01.0: can't change power state from D3cold to D0 (config space inaccessible)
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: Host halt failed, -19
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: Host not accessible, reset failed.
May 01 13:13:27 kernel: xhci_hcd 0000:3d:00.0: USB bus 3 deregistered
May 01 13:13:27 kernel: pci_bus 0000:07: busn_res: [bus 07] is released
May 01 13:13:27 kernel: pci_bus 0000:08: busn_res: [bus 08-3c] is released
May 01 13:13:27 kernel: pci_bus 0000:3d: busn_res: [bus 3d] is released
May 01 13:13:27 kernel: pci_bus 0000:06: busn_res: [bus 06-3d] is released
May 01 13:13:27 pcscd[1878508]: 00000000 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:13:27 pcscd[1878508]: 00000071 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 01 13:13:27 pcscd[1878508]: 00400173 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:13:27 pcscd[1878508]: 00000048 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 01 13:13:28 pcscd[1878508]: 00400179 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:13:28 pcscd[1878508]: 00000042 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:13:28 pcscd[1878508]: 00000014 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 01 13:13:28 pcscd[1878508]: 00400183 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:13:28 pcscd[1878508]: 00000046 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
( 1 minute ...)
May 01 13:14:28 pcscd[1878508]: 00400374 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 01 13:14:28 systemd[1]: pcscd.service: Deactivated successfully.
  • Is your Yubikey or system faulty?

No, I have confirmed this issue both with a different yubikey and on a fresh install of my distribution.

Log

(https://gist.githubusercontent.com/oddlama/59e90fc6ad47a8069180a69d94fe095c/raw/1f7da7963220040eae3bbf278164e46a934f123a/log.txt)
I have generated a debug log as requested. The exact actions I did while pcscd was logging are the following:

  • Start pscsd
  • Wait some seconds
  • Insert Yubikey
  • Wait some seconds
  • gpg --card-status (fails)
  • Wait some seconds
  • gpg --card-status (succeeds)
  • Wait some seconds
  • Remove Yubikey (libusb errors begin)
  • Wait some seconds
  • Re-insert same Yubikey
  • Wait some seconds
  • Execute gpg --card-status three times with short waiting time in-between, but all fail.

EDIT: It should be noted, that all gpg --card-status commands were executed as an unprivileged user with access to the yubikey usb device (granted via udev). If using root, the first attempt also fails but other attemps succeed. Not sure whether gpg uses the integrated scdaemon as a fallback for the root user.

@LudovicRousseau
Copy link
Owner

scdaemon is a process used by GnuPG. it is problematic.
See https://ludovicrousseau.blogspot.com/2019/06/gnupg-and-pcsc-conflicts.html for a solution.
Maybe you can also report the scdaemon crash to the GnuPG project.

To fix the repeating log ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE please try the patch bellow for the CCID driver:

diff --git a/src/ifdhandler.c b/src/ifdhandler.c
index 54c647e..f320f13 100644
--- a/src/ifdhandler.c
+++ b/src/ifdhandler.c
@@ -1956,7 +1956,7 @@ EXTERNAL RESPONSECODE IFDHICCPresence(DWORD Lun)
 
 	if (IFD_NO_SUCH_DEVICE == return_value)
 	{
-		return_value = IFD_ICC_NOT_PRESENT;
+		return_value = IFD_NO_SUCH_DEVICE;
 		goto end;
 	}

@oddlama
Copy link
Author

oddlama commented May 1, 2022

Thanks a lot for the quick response! I'll report the scdaemon segfault to GnuPG as soon as they approve my account for reporting.

If I really only needed use my Yubikey with GnuPG, which one of scdaemon or pcsc-lite would you recommend me to keep? Is there some fundamental difference in their implementations, or can one be considered a superior implementation?

Also, am I correctly assuming that I need pcsc-lite if I wanted to use my yubikey with PAM?

@LudovicRousseau
Copy link
Owner

If you only need to use GnuPG I would suggest to remove the pcscd package (I don't know how pcsc-lite is packaged on Gentoo) or just disable it.
Then you see what service is broken on your system.

I would say pcsc-lite is better but I am biased here :-)

@oddlama
Copy link
Author

oddlama commented May 2, 2022

I've just noticed, that I already had disable-ccid in ~/.gnupg/scdaemon.conf (and permissions are all set correctly). If I interpret your FAQ correctly, shouldn't this have prevented this issue from happening?

@LudovicRousseau
Copy link
Owner

Try the patch I proposed. Maybe scdaemon is innocent here.

@oddlama
Copy link
Author

oddlama commented May 2, 2022

I've applied the patch and re-tested everything, unfortunately everything is exactly like before. I'm getting the scdaemon segfault on first gpg --card-status and also the repeated libusb errors when removing the yubikey.

The only actual difference I observed is that the logged message change slightly (after removing the yubikey):

May 02 22:27:22 pcscd[16918]: 00000000 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 02 22:27:22 pcscd[16918]: 00000063 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/ifdwrapper.c:364:IFDStatusICC() Card not transacted: 617
May 02 22:27:23 pcscd[16918]: 01000221 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/eventhandler.c:336:EHStatusHandlerThread() Error communicating to: Yubico YubiKey OTP+FIDO+CCID 00 00
May 02 22:27:23 pcscd[16918]: 00000074 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 02 22:27:24 pcscd[16918]: 00400204 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 02 22:27:24 pcscd[16918]: 00000047 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/ifdwrapper.c:364:IFDStatusICC() Card not transacted: 617
May 02 22:27:25 pcscd[16918]: 01000210 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/eventhandler.c:336:EHStatusHandlerThread() Error communicating to: Yubico YubiKey OTP+FIDO+CCID 00 00
May 02 22:27:25 pcscd[16918]: 00000031 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 02 22:27:25 pcscd[16918]: 00400196 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 02 22:27:25 pcscd[16918]: 00000047 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/ifdwrapper.c:364:IFDStatusICC() Card not transacted: 617
May 02 22:27:26 pcscd[16918]: 01000264 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 02 22:27:26 pcscd[16918]: 00000047 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/ifdwrapper.c:364:IFDStatusICC() Card not transacted: 617
May 02 22:27:27 pcscd[16918]: 01000224 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/eventhandler.c:336:EHStatusHandlerThread() Error communicating to: Yubico YubiKey OTP+FIDO+CCID 00 00
May 02 22:27:27 pcscd[16918]: 00000072 ccid_usb.c:1488:InterruptRead() libusb_submit_transfer failed: LIBUSB_ERROR_NO_DEVICE
May 02 22:27:28 pcscd[16918]: 00400206 ccid_usb.c:886:WriteUSB() write failed (3/2): LIBUSB_ERROR_NO_DEVICE
May 02 22:27:28 pcscd[16918]: 00000066 /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/ifdwrapper.c:364:IFDStatusICC() Card not transacted: 617

@oddlama
Copy link
Author

oddlama commented May 2, 2022

Info: Opened a related gnupg bug report (https://dev.gnupg.org/T5963)

@LudovicRousseau
Copy link
Owner

The patch I proposed will not solve the problem. You can ignore it.

Your problem is similar to #57
Can you:

  • run lsusb
  • connect the Yubikey token
  • run lsusb
  • disconnect the token
  • run lsusb

And send me all the outputs?

Can you also apply this patch and generate a new pcscd log

diff --git a/src/hotplug_libudev.c b/src/hotplug_libudev.c
index 5048689..29c011f 100644
--- a/src/hotplug_libudev.c
+++ b/src/hotplug_libudev.c
@@ -343,7 +343,10 @@ static void HPRemoveDevice(struct udev_device *dev)
 	parent = udev_device_get_parent_with_subsystem_devtype(dev, "usb",
 		"usb_device");
 	if (!parent)
+	{
+		Log1(PCSC_LOG_ERROR, "USB Device parent is NULL.");
 		return;
+	}
 
 	devpath = udev_device_get_devnode(parent);
 	if (!devpath)
@@ -360,6 +363,7 @@ static void HPRemoveDevice(struct udev_device *dev)
 		return;
 	}
 
+	Log3(PCSC_LOG_INFO, "Removing: %s %s", devpath, sysname);
 	for (i=0; i<PCSCLITE_MAX_READERS_CONTEXTS; i++)
 	{
 		if (readerTracker[i].fullName && !strcmp(sysname, readerTracker[i].sysname))

@oddlama
Copy link
Author

oddlama commented May 3, 2022

In my gpg bug report i speculated it might have somethin to do with a disappearing usb host controller. Note sure if that's really the case though. Here are the requested lsusb outputs:

Bus 002 Device 002: ID 0bda:0328 Realtek Semiconductor Corp. USB3.0-CRW
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 1bcf:2c6b Sunplus Innovation Technology Inc. HD WebCam
Bus 001 Device 005: ID 1044:7a39 Chu Yuen Enterprise Co., Ltd USB-HID Keyboard
Bus 001 Device 004: ID 8087:0a2b Intel Corp. Bluetooth wireless interface
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

[connect yubikey]

Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 002: ID 1050:0407 Yubico.com Yubikey 4/5 OTP+U2F+CCID
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0bda:0328 Realtek Semiconductor Corp. USB3.0-CRW
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 1bcf:2c6b Sunplus Innovation Technology Inc. HD WebCam
Bus 001 Device 005: ID 1044:7a39 Chu Yuen Enterprise Co., Ltd USB-HID Keyboard
Bus 001 Device 004: ID 8087:0a2b Intel Corp. Bluetooth wireless interface
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

[remove yubikey]

Bus 002 Device 002: ID 0bda:0328 Realtek Semiconductor Corp. USB3.0-CRW
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 1bcf:2c6b Sunplus Innovation Technology Inc. HD WebCam
Bus 001 Device 005: ID 1044:7a39 Chu Yuen Enterprise Co., Ltd USB-HID Keyboard
Bus 001 Device 004: ID 8087:0a2b Intel Corp. Bluetooth wireless interface
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Here's the new log from patched pcsc-lite (https://gist.github.com/oddlama/821cbc24763141927aeb1d452276a6ff)

@LudovicRousseau
Copy link
Owner

It is the same problem as in #57
Connecting your Yubikey creates a new USB bus. In you example it is bus 003.
When the token is removed the device disappears (as expected) but the USB bus also disappears.

It is confirmed by the new pcscd log you generated.
pcscd is "surprised" by the missing bus and the device is not removed.

Remove the previous patch and try this one (generate a new psccd log please):

diff --git a/src/hotplug_libudev.c b/src/hotplug_libudev.c
index 5048689..3a3ad78 100644
--- a/src/hotplug_libudev.c
+++ b/src/hotplug_libudev.c
@@ -331,7 +331,7 @@ static LONG HPReadBundleValues(void)
 static void HPRemoveDevice(struct udev_device *dev)
 {
 	int i;
-	const char *devpath;
+	const char *devpath = "[NULL]";
 	struct udev_device *parent;
 	const char *sysname;
 
@@ -342,16 +342,17 @@ static void HPRemoveDevice(struct udev_device *dev)
 	   tree, but the function will find it.*/
 	parent = udev_device_get_parent_with_subsystem_devtype(dev, "usb",
 		"usb_device");
-	if (!parent)
-		return;
-
-	devpath = udev_device_get_devnode(parent);
-	if (!devpath)
+	if (parent)
 	{
-		/* the device disapeared? */
-		Log1(PCSC_LOG_ERROR, "udev_device_get_devnode() failed");
-		return;
+		devpath = udev_device_get_devnode(parent);
+		if (!devpath)
+		{
+			/* the device disapeared? */
+			Log1(PCSC_LOG_ERROR, "udev_device_get_devnode() failed");
+		}
 	}
+	else
+		Log1(PCSC_LOG_ERROR, "USB Device parent is NULL.");
 
 	sysname = udev_device_get_sysname(dev);
 	if (!sysname)
@@ -360,6 +361,7 @@ static void HPRemoveDevice(struct udev_device *dev)
 		return;
 	}
 
+	Log3(PCSC_LOG_INFO, "Removing: %s %s", devpath, sysname);
 	for (i=0; i<PCSCLITE_MAX_READERS_CONTEXTS; i++)
 	{
 		if (readerTracker[i].fullName && !strcmp(sysname, readerTracker[i].sysname))

With this patch the device should be removed even if the device parent (USB bus) is no more present.

@oddlama
Copy link
Author

oddlama commented May 3, 2022

I've applied your new patch and generated a new pcscd log: https://gist.github.com/oddlama/860637c3fb01c04eb3f565e58f3a7286 (FYI: the general behavior is the same)

@LudovicRousseau
Copy link
Owner

Much better. The Yubikey device is correctly removed now:

00000029 [140311589492288] ccid_usb.c:1532:InterruptRead() InterruptRead (3/2): LIBUSB_TRANSFER_NO_DEVICE
00011825 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/hotplug_libudev.c:663:HPEstablishUSBNotifications() USB Device removed
00000166 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/hotplug_libudev.c:355:HPRemoveDevice() USB Device parent is NULL.
00000005 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/hotplug_libudev.c:364:HPRemoveDevice() Removing: [NULL] 3-1:1.2
00000001 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/hotplug_libudev.c:369:HPRemoveDevice() Removing USB device[0]: Yubico YubiKey OTP+FIDO+CCID at /dev/bus/usb/003/002

I see you inserted again the device. But adding the device failed because it is already in use:

00000001 [140311606568512] ccid_usb.c:573:OpenUSBByName() Trying to open USB bus/device: 3/2
00000029 [140311606568512] ccid_usb.c:672:OpenUSBByName() Can't claim interface 3/2: LIBUSB_ERROR_BUSY
00000164 [140311606568512] ccid_usb.c:204:close_libusb_if_needed() libusb_exit
00000056 [140311606568512] ifdhandler.c:160:CreateChannelByNameOrChannel() failed
00000004 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/readerfactory.c:1138:RFInitializeReader() Open Port 0x200000 Failed (usb:1050/0407:libudev:2:/dev/bus/usb/003/002)
00000001 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/readerfactory.c:380:RFAddReader() Yubico YubiKey OTP+FIDO+CCID init failed.
00000001 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/readerfactory.c:632:RFRemoveReader() UnrefReader() count was: 1
00000001 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/readerfactory.c:1151:RFUnInitializeReader() Attempting shutdown of Yubico YubiKey OTP+FIDO+CCID 00 00.
00000001 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/readerfactory.c:1009:RFUnloadReader() Unloading reader driver.
00000044 [140311606568512] /var/tmp/portage/sys-apps/pcsc-lite-1.9.5/work/pcsc-lite-1.9.5/src/hotplug_libudev.c:540:HPAddDevice() Failed adding USB device: Yubico YubiKey OTP+FIDO+CCID

Maybe that is a side effect of scdaemon that also opened the device.

I would suggest to remove (or disable scdaemon) and check this solved the problem when reconnecting.

@oddlama
Copy link
Author

oddlama commented May 3, 2022

Scdaemon is disabled with disable-ccid in ~/.gnupg/scdaemon.conf. Am I missing another config option? And thanks for your dedication to this issue :)

@LudovicRousseau
Copy link
Owner

It should be disabled. But I suspect scdaemon is still accessing the token.
I have no other explanation for the LIBUSB_ERROR_BUSY error.

@oddlama
Copy link
Author

oddlama commented May 3, 2022

Unfortunately my gpg bug report has not been answered yet. Maybe they face a related issue.
On gentoo i can simply compile gpg without usb support, so I guess that might fix it for me. Thanks for your help.

Are you positive that pcscd is not involved anymore?

@LudovicRousseau
Copy link
Owner

The patch I pushed should fix the problem when you remove the token.
I don't know what is the problem when you insert again the token.

Building gpg without USB support is a good idea. Tell me if you still have problems after that.

@oddlama
Copy link
Author

oddlama commented May 3, 2022

I can confirm that re-inserting the yubikey is now working properly, thank you. Unfortunately, any time the yubikey is inserted, the first attempt to use it with gpg fails, any subsequent tries succeed. In the failure case, gpg --card-status reports:

GPG: selecting card failed: End of file
GPG: OpenPGP card not available: End of file

Yet, pcsc_scan always picks up the yubikey on first try, so I think this is only a gnupg related problem now. So I guess there's nothing to do here anymore, but I'll still report back when the gpg issue is resolved.

@LudovicRousseau
Copy link
Owner

You can generate a pcscd log including a gpg --card-status failure so I can have a look.
Maybe the token is not started or configured correctly.

@LudovicRousseau
Copy link
Owner

One more question: what is your computer brand & model? Or the mother board brand & model if it is a computer you made yourself?
Have you tried to upgrade the BIOS?

I would like to know why it failed only on some systems. For example I was not able to reproduce the problem with the Yubikey tokens I have.

@oddlama
Copy link
Author

oddlama commented May 4, 2022

It's a AERO 15X (i7-8750H) laptop from 2017 or 2018. I have none of the issues on my desktop PC. It might have something to do with the fact that the only USB-C slot is a thunderbolt port. BIOS is up to date, never had issues before they started appearing around January 2022, yet I cannot remember the month with certainty. I didn't change anything about my system except for updating software.

All logs that I made already follow the procedure outlined in the initial post, but to be certain I've created another one (https://gist.github.com/oddlama/60fbdf13245c568dfa84d15393652f2e). These were my actions:

  • Insert Yubikey
  • gpg --card-status (fails)
  • gpg --card-status (succeeds)
  • Remove Yubikey (libusb errors begin)
  • Re-insert same Yubikey
  • gpg --card-status (fails)
  • gpg --card-status (succeeds)

The problem is 100% reproducible on this machine, so if you need any other information be sure to let me know.

@oddlama
Copy link
Author

oddlama commented May 4, 2022

I've just noticed an additional issue, might be related. When I manually start pcscd as root, the above behavior occurs. But when pcscd is started as root by systemd automatically (when the yubikey is being plugged in), the card is never detected by gpg --card-status (it reports service not running, yet pcscd is alive). I've changed the systemd service to also include the advanced logging output and this is what I get then: (https://gist.github.com/oddlama/0300073b0823532d3ea0928057b77312)

EDIT: Found out that a gpg-agent running as root seems to interfere somehow. Killing it solves the inconsistency. Still strange. Related pcscd log here (https://gist.github.com/oddlama/1325c4f32d1f54827c75759e38f81dfc)

It gets even more weird. It generally seems that gpg --card-status succeeds on second try, when using a manually started pcscd from the command given by you above. ssh-add -L (gpg-agent) then correctly queries the ssh key from the yubikey. Just actually using the by invoking ssh somehost reports that gpg-agent refuses to work.

EDIT2: You might also be interested in my gnupg bug report. Meanwhile I have generated a valgrind report of the segfault, and right at the beginning it includes an issue seemingly in libpcsclite. Excerpt from https://dev.gnupg.org/T5963:

SERIALNO
==35605== Thread 2:
==35605== Conditional jump or move depends on uninitialised value(s)
==35605==    at 0x58BCFAE: SCardListReaders (in /usr/lib64/libpcsclite.so.1.0.0)
==35605==    by 0x11D78A: apdu_dev_list_start (apdu.c:2031)
==35605==    by 0x1245EF: select_application (app.c:817)
==35605==    by 0x117AFF: open_card_with_request (command.c:281)
==35605==    by 0x117AFF: cmd_serialno (command.c:358)
==35605==    by 0x4A20839: ??? (in /usr/lib64/libassuan.so.0.8.5)
==35605==    by 0x4A20C28: assuan_process (in /usr/lib64/libassuan.so.0.8.5)
==35605==    by 0x11854C: scd_command_handler (command.c:2521)
==35605==    by 0x114C03: start_connection_thread (scdaemon.c:1202)
==35605==    by 0x4A2F49D: ??? (in /usr/lib64/libnpth.so.0.1.2)
==35605==    by 0x4ABD3E9: start_thread (pthread_create.c:442)
==35605==    by 0x4B3F0DF: clone (clone.S:100)

@LudovicRousseau
Copy link
Owner

I have not found any PC/SC issue in your logs.
Your problems are now only related to GnuPG.

I will have a look at the problem with SCardListReaders reported by valgrind.

@oddlama
Copy link
Author

oddlama commented May 5, 2022

Thanks a lot for all the help. I'll sort out the rest with the GnuPG folks.

@LudovicRousseau
Copy link
Owner

The problem with SCardListReaders() is in fact in GnuPG.
I reported it at https://lists.gnupg.org/pipermail/gnupg-devel/2022-May/035063.html

@LudovicRousseau
Copy link
Owner

I saw no feedback on the gnupg-dev mailing list.
Si I also reported upstream at https://dev.gnupg.org/T5979

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants