Related, If you copy a file via the OS's copy function the system knows the file was scanned and you get fast copies. If you copy the file by opening a new destination file for write, opening the source file for read, and copying bytes, then of course you trigger the virus scanner.
So for example I was using a build system and part of my build needed to copy ~5000 files of assets to the "out" folder. It was taking 5 seconds on other OSes and 2 minutes on Windows. Turned out the build system was copying using the "make a new file and copy bytes" approach instead of calling the their language's library copy function, which, at least on Windows, calls the OS copyfile function. I filed a bug and submitted a PR. Unfortunately while they acknowledged the issue they did not take the PR nor fix it on their side. My guess is they don't really care about devs that use Windows.
Note that python's copyfile does this wrong on MacOS. It also uses the open, read bytes, write bytes to new file method instead of calling into the OS. While it doesn't have the virus scanning issue (yet) it does mean files aren't actually "copied" so metadata is lost.
> Note that python's copyfile does this wrong on MacOS. It also uses the open, read bytes, write bytes to new file method instead of calling into the OS.
It doesn't, since 3.8. It tries fcopyfile() and only if it fails, does the read/write dance.
That's different thing; the copied data includes only file data itself, not metadata. From the documentation
- on shutil.copyfile:
> Copy the contents (no metadata) of the file named src to a file named dst and return dst in the most efficient way possible. src and dst are path-like objects or path names given as strings.
- on "Platform-dependent efficient copy operations":
> On macOS fcopyfile is used to copy the file content (not metadata).
On top of the shutil module:
> Warning
> Even the higher-level file copying functions (shutil.copy(), shutil.copy2()) cannot copy all file metadata.
> On POSIX platforms, this means that file owner and group are lost as well as ACLs. On Mac OS, the resource fork and other metadata are not used. This means that resources will be lost and file type and creator codes will not be correct. On Windows, file owners, ACLs and alternate data streams are not copied.
So for example I was using a build system and part of my build needed to copy ~5000 files of assets to the "out" folder. It was taking 5 seconds on other OSes and 2 minutes on Windows. Turned out the build system was copying using the "make a new file and copy bytes" approach instead of calling the their language's library copy function, which, at least on Windows, calls the OS copyfile function. I filed a bug and submitted a PR. Unfortunately while they acknowledged the issue they did not take the PR nor fix it on their side. My guess is they don't really care about devs that use Windows.
Note that python's copyfile does this wrong on MacOS. It also uses the open, read bytes, write bytes to new file method instead of calling into the OS. While it doesn't have the virus scanning issue (yet) it does mean files aren't actually "copied" so metadata is lost.