Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix #14574, cp on files >2GB #30989

Merged
merged 1 commit into from
Feb 10, 2019
Merged

fix #14574, cp on files >2GB #30989

merged 1 commit into from
Feb 10, 2019

Conversation

JeffBezanson
Copy link
Sponsor Member

@JeffBezanson JeffBezanson commented Feb 7, 2019 •
edited
Loading

There may be a better way to do this, but this is a silly bug to have so here is a simple fix for now.

Also, any ideas on how to test this? Creating multiple GB of files seems like the kind of thing that could disagree with CI systems.

@JeffBezanson JeffBezanson added domain:io Involving the I/O subsystem: libuv, read, write, etc. kind:bugfix This change fixes an existing bug labels Feb 7, 2019
@ViralBShah
Copy link
Member

Copy from /dev/zero to /dev/null?

vtjnash
vtjnash reviewed Feb 7, 2019
View reviewed changes
base/file.jl Outdated
bytes = filesize(stat(src_file))
sendfile(dst_file, src_file, Int64(0), Int(bytes))
bytes = Int(filesize(stat(src_file)))
offs = Int64(0)
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set this to const -1

@JeffBezanson
Copy link
Sponsor Member Author

Ok, we can now use libuv's copyfile, but there's a problem. Previously we created the destination file with just open, which respects the process umask. But libuv copyfile copies the source file's mode exactly, independent of the umask. Should we change this behavior, or do an extra chmod?

@JeffreySarnoff
Copy link
Contributor

When copying a file, keeping that file's mode seems more obvious (when I choose to edit a read-only file, it ought require some overtly permissive act).

@nalimilan
Copy link
Member

FWIW Python 3 preserves permissions. The fact that uv_copy_file also preserves them is probably intentional too.

Though for a backport we would probably need to preserve the previous behavior.

@JeffBezanson
Copy link
Sponsor Member Author

Good point, I'll do a backwards-compatible fix first.

@JeffBezanson JeffBezanson added the status:triage This should be discussed on a triage call label Feb 8, 2019
@JeffBezanson JeffBezanson merged commit 05785f9 into master Feb 10, 2019
@JeffBezanson JeffBezanson deleted the jb/bigcopy branch February 10, 2019 22:42
@nalimilan
Copy link
Member

Maybe we could have tests that are only run manually, before tagging a release? It sounds problematic to have code paths that are never exercized at all.

@vtjnash vtjnash mentioned this pull request Apr 10, 2019
Closed
@ararslan
Copy link
Member

So can this be backported?

@JeffBezanson
Copy link
Sponsor Member Author

Yes I think so.

JeffBezanson added a commit that referenced this pull request Jun 6, 2019
@JeffBezanson
fix #14574, cp on files >2GB (#30989)
77162f9 
(cherry picked from commit 05785f9)
@ararslan
Copy link
Member

Looks like this didn't make it into 1.1.1. I'll apply the label for 1.2, then hopefully we can get it into rc2. I just hit this issue again today.

@JeffBezanson
Copy link
Sponsor Member Author

I believe this is already in 1.2.

@ararslan
Copy link
Member

Hm, yeah, it does appear to be on the release-1.2 branch. There don't appear to be any references to this PR though, so I guess backporting isn't including a PR reference as recommended by DISTRIBUTING.md?

@KristofferC
Copy link
Sponsor Member

KristofferC commented Jun 27, 2019 •
edited
Loading

What do you mean? This wasn't backported, it was already merged when branching.

@ararslan
Copy link
Member

Oh sorry, I'm just confused. Please ignore me.

@asinghvi17 asinghvi17 mentioned this pull request Jul 7, 2019
Closed
@KristofferC KristofferC removed the status:triage This should be discussed on a triage call label Aug 26, 2019
KristofferC pushed a commit that referenced this pull request Aug 26, 2019
@JeffBezanson @KristofferC
fix #14574, cp on files >2GB (#30989)
85879e8 
(cherry picked from commit 05785f9)
@KristofferC KristofferC mentioned this pull request Aug 26, 2019
Merged
55 tasks
KristofferC pushed a commit that referenced this pull request Aug 26, 2019
@JeffBezanson @KristofferC
fix #14574, cp on files >2GB (#30989)
1788b07 
(cherry picked from commit 05785f9)
KristofferC pushed a commit that referenced this pull request Feb 20, 2020
@JeffBezanson @KristofferC
fix #14574, cp on files >2GB (#30989)
fdc78be 
(cherry picked from commit 05785f9)
@jaakkor2 jaakkor2 mentioned this pull request Jun 1, 2020
Closed
@KristofferC KristofferC mentioned this pull request Sep 14, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vtjnash vtjnash vtjnash left review comments

Assignees
No one assigned
Labels
domain:io Involving the I/O subsystem: libuv, read, write, etc. kind:bugfix This change fixes an existing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@JeffBezanson @ViralBShah @JeffreySarnoff @nalimilan @ararslan @KristofferC @vtjnash