path/filepath: add Glob variant taking a context.Context?

[bradfitz]

From internal bug report:

filepath.Glob can take a lot of time, and can't be canceled.

Add a Context variant?

[robpike]

That seems like a domain-crossing change. It makes me uncomfortable. Lots of things take a long time, and it would be lousy if every possible long-running operation got a Context variant.

[luna-duclos]

Perhaps a more general cancellation mechanism than contexts is in order ? It'd be very nice if most long running operations in Go could be cancelled, but I don't think context is the right tool for the job.

[minux]

Which part of Glob is taking a long time? If it's the syscall, then we cannot do anything about it even if Glob supports cancellation. (Sending signals might not work if the process is in uninterruptible state due to underlying file system.)

[ianlancetaylor]

Now that we have the context package in the standard library, I don't think we want to introduce another, different, cancellation mechanism.

I don't know but I wouldn't guess that the problem is the system call. I suspect the problem is that Glob is being used to traverse a large directory tree.

I don't think it makes sense to add a context.Context argument to Glob, though. If it's really necessary to cancel the directory traversal, then I think Glob is probably the wrong function to call. The program should call a different function that streams the files to a channel (and takes a Context argument). I don't know that that function belongs in the standard library.

[mdempsky]

@minux POSIX says all file system calls that block indeterminately due to waiting on the underlying disk/network storage device are interruptible (though I would expect in practice that kernel bugs and things like FUSE regularly break that).

[minux]

I frequently encounter processes in D state due to flaky NFS in the past. Our Glob doesn't support **, so even if it's traversaling huge directories, if the majority time is not spent in syscall and GC, I think we should optimize Glob more.

[bradfitz]

Yeah, I assumed our glob supported "**" when I filed the bug because I remembered #11862 and thought it was already fixed.

We can do nothing here. The internal Google glob can be fixed instead.