sshfs seems crashing: socket is not connected

[sztyler]

Dear Developers,

I am not sure how to summaries things but sshfs keeps "crashing" without noticing it (i.e. I am using the parameter "reconnect" but it does not have any effect).

"journalctl" just provides the following information:

Jun 19 01:59:53 kernel: audit: type=1701 audit(1560902393.278:50): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=428 comm="mount.fuse.sshf" exe="/usr/bin/sshfs" sig=6 res=1
Jun 19 01:59:53 audit[428]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 pid=428 comm="mount.fuse.sshf" exe="/usr/bin/sshfs" sig=6 res=1

as a result, the system starts a "systemd-coredump" which results in a kernel panic:

Jun 19 02:04:53  kernel:  kthread+0x112/0x130
Jun 19 02:04:53  kernel:  nfsd+0xed/0x150 [nfsd]
Jun 19 02:04:53  kernel:  svc_process+0xde/0x110 [sunrpc]
Jun 19 02:04:53  kernel:  ? nfsd_destroy+0x60/0x60 [nfsd]
Jun 19 02:04:53  kernel:  svc_process_common.isra.6+0x33c/0x790 [sunrpc]
Jun 19 02:04:53  kernel:  nfsd_dispatch+0x9e/0x210 [nfsd]
Jun 19 02:04:53  kernel:  nfsd4_proc_compound+0x445/0x690 [nfsd]
Jun 19 02:04:53  kernel:  nfsd4_encode_operation+0x9b/0x1b0 [nfsd]
Jun 19 02:04:53  kernel:  nfsd4_encode_readdir+0xd9/0x1c0 [nfsd]
Jun 19 02:04:53  kernel:  ? nfsd_finish_read+0x190/0x190 [nfsd]
Jun 19 02:04:53  kernel:  ? nfsd4_encode_getattr+0x30/0x30 [nfsd]
Jun 19 02:04:53  kernel:  nfsd_readdir+0x222/0x240 [nfsd]
Jun 19 02:04:53  kernel: Call Trace:
Jun 19 02:04:53  kernel: CR2: 00007f6b04003000 CR3: 000000007acac003 CR4: 00000000001606e0
Jun 19 02:04:53  kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 02:04:53  kernel: FS:  0000000000000000(0000) GS:ffff9d853db80000(0000) knlGS:0000000000000000
Jun 19 02:04:53  kernel: R13: 0000000000000000 R14: 0000000000000062 R15: 00000000ffffff99
Jun 19 02:04:53  kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffff9d85364f3498
Jun 19 02:04:53  kernel: RBP: ffffb90b00917d40 R08: 0000000000000235 R09: 0000000000000001
Jun 19 02:04:53  kernel: RDX: 0000000000000007 RSI: 0000000000000082 RDI: 00000000ffffffff
Jun 19 02:04:53  kernel: RAX: 0000000000000000 RBX: 0000000000000062 RCX: 0000000000000000
Jun 19 02:04:53  kernel: RSP: 0018:ffffb90b00917ca0 EFLAGS: 00010286
Jun 19 02:04:53  kernel: Code: 00 00 b8 00 00 00 05 74 04 c3 31 c0 c3 48 83 ec 08 89 fe 48 c7 c7 b7 b0 4f c0 c6 05 e5 2f 05 00 01 89 44 24 04 e8 c6 0f fc fa <0f> 0b 8b 44 24 04 48 83 c4 08 c3 90 0f 1f 44 00 00 48 83>
Jun 19 02:04:53  kernel: RIP: 0010:nfserrno+0x64/0x70 [nfsd]
Jun 19 02:04:53  kernel: Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Jun 19 02:04:53  kernel: CPU: 3 PID: 445 Comm: nfsd Not tainted 5.1.9-arch1-1-ARCH #1
Jun 19 02:04:53  kernel: Modules linked in: fuse ip6t_REJECT nf_reject_ipv6 xt_state ip6table_filter ip6_tables ipt_rpfilter iptable_raw ipt_REJECT nf_reject_ipv4 xt_recent xt_tcpudp xt_conntrack nf_conntrack nf_def>
Jun 19 02:04:53  kernel: WARNING: CPU: 3 PID: 445 at fs/nfsd/nfsproc.c:820 nfserrno+0x64/0x70 [nfsd]
Jun 19 02:04:53  kernel: audit: type=1131 audit(1560902693.353:52): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@0-705-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? termi>
Jun 19 02:04:53  kernel: nfsd: non-standard errno: -103
Jun 19 02:04:53  kernel: ------------[ cut here ]------------

My distribution is "Arch Linux" (clean/fresh install):

# sshfs -V
SSHFS version 3.5.2
FUSE library version 3.5.0
using FUSE kernel interface version 7.29
fusermount3 version: 3.5.0

However, the overall setup is more complicated...

1. The system outlined above mounts a remote share by using sshfs:
   [email protected]:/ /opt/folder fuse.sshfs port=12345,umask=0022,uid=0,gid=104,allow_other,_netdev,reconnect 0 0
2. Not sure if this matters but "1.2.3.4" is "Debian Stretch":
   Linux cert 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64 GNU/Linux
3. The system which mounts the sshfs device, provides at the same time the mounted sshfs device as a NFS-share (i.e. a nfs-server is running, v4.x only)

# cat /etc/exports
/opt/folder            1.2.3.5(ro,sync,no_subtree_check,secure,all_squash,anonuid=0,anongid=1000,fsid=3)

Overall I could not narrow down what causes the crash but I observed a couple of things:

1. I can reproduce the crash with one certain nfs-client. Overall, I have three nfs clients but the crash only occurs with one certain client -entering the mounted nfs-directory and performing a "ls -l" is enough for causing a crash (i.e. sshfs reports that the socket is not available when I try to enter the directory directly on the main server). The major difference between the nfs-client is the operation system. The one which causes the crash is also using "Arch Linux" where the other two are using "CentOS7"; however, all clients use the same nfs4 protocol version (4.1)
2. I have another server (Debian Strech) having an older sshfs version. Using instead this server as intermediated server, i.e., mounting the sshfs device and providing the nfs4 share, works like a charm -no problems, no crashs.

# sshfs -V
SSHFS version 2.8
FUSE library version: 2.9.7
fusermount version: 2.9.7
using FUSE kernel interface version 7.19

I guess that NFS crashs the sshfs mount, from my point of view, this should not happen.

Please let me know if you need further information. I am really curious what is causing this problem.

Thank you!

[Nikratio]

SSHFS does not have any active, regular contributors or developers. The current maintainer continues to apply pull requests and tries to make regular releases, but unfortunately has no capacity to do any development beyond addressing high-impact issues. When reporting bugs, please understand that unless you are including a pull request or are reporting a critical issue, you will probably not get a response.

To prevent the issue tracker from being flooded with issues that no-one is intending to work on, and to give more visibilty to critical issues that users should be aware of and that most urgently need attention, I will also close most bug reports once they've been inactive for a while.

Please note that this isn't meant to imply that you haven't found a bug - you most likely have and I'm grateful that you took the time to report it. Unfortunately, SSHFS is a purely volunteer driven project, and at the moment there simply aren't any volunteers.