Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃悰 fix bus error on smaller readonly file in unix #44354

Merged
merged 10 commits into from
Aug 21, 2023

Conversation

xgdgsc
Copy link
Contributor

@xgdgsc xgdgsc commented Feb 26, 2022

fix #28245 in unix

stdlib/Mmap/src/Mmap.jl Outdated Show resolved Hide resolved
stdlib/Mmap/src/Mmap.jl Outdated Show resolved Hide resolved
@xgdgsc
Copy link
Contributor Author

xgdgsc commented Mar 1, 2022

This seems clearer to me. But don' t know if it would have any performance issue.

@brenhinkeller brenhinkeller added the kind:bugfix This change fixes an existing bug label Jul 24, 2022
stdlib/Mmap/src/Mmap.jl Outdated Show resolved Hide resolved
stdlib/Mmap/src/Mmap.jl Outdated Show resolved Hide resolved
@vtjnash vtjnash added the status:merge me PR is reviewed. Merge when all tests are passing label Nov 8, 2022
@giordano
Copy link
Contributor

giordano commented Nov 8, 2022

Tests are extremely unhappy on almost all platforms

@vtjnash
Copy link
Sponsor Member

vtjnash commented Nov 9, 2022

We may be able to emulate (aka greatly accelerate) the filesize test by doing a seekend instead and checking the position?

@DilumAluthge DilumAluthge removed the status:merge me PR is reviewed. Merge when all tests are passing label Nov 9, 2022
@xgdgsc
Copy link
Contributor Author

xgdgsc commented Jan 12, 2023

As grow! does nothing to Anonymous. I did the above commit. What do you think of the macOS failure? I looked at the c code and have no idea. Would it be fine to keep the old behaviour for macOS? Should SharedArrays.jl be changed?

@KristofferC
Copy link
Sponsor Member

@vtjnash, could you take another look at this?

@vtjnash
Copy link
Sponsor Member

vtjnash commented Feb 13, 2023

Tests are still failing, so there is not much more I can really do here for review. My previous suggestion for a fix seems to have been unsuccessful, since the seekend is failing.

Error in testset SharedArrays:
Error During Test at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-2.0/build/default-macmini-x64-2-0/julialang/julia-master/julia-926f1dde71/share/julia/test/testdefs.jl:21
  Got exception outside of a @test
  LoadError: IOError: filesize: Illegal seek for <fd 21>
  Stacktrace:
    [1] filesize(s::IOStream)
      @ Base ./iostream.jl:226

@vtjnash
Copy link
Sponsor Member

vtjnash commented Feb 13, 2023

Actually, IIUC, this is a very weird file on macOS (from shm_open). I can read and write, but not seek the fd. And stat is a bit awkward, as Julia cannot return it:

julia> Base.StatStruct(stat_buf)
StatStruct for ""
   size: 4096 bytes
 device: 0
  inode: 0
   mode: 0o000600 (-rw-------)
  nlink: 0
    uid: 501 (jameson)
    gid: 20 (staff)
   rdev: 0
  blksz: 0
 blocks: 0
  mtime: 1969-12-31T19:00:00-0500 (19401 days ago)
  ctime: 1969-12-31T19:00:00-0500 (19401 days ago)

julia> stat(s)
ERROR: stat returned zero type for a valid path

implemented here: https://github.com/apple/darwin-xnu/blob/a1babec6b135d1f35b2590a1990af3c5c5393479/bsd/kern/posix_shm.c#L695-L700

This seems to be an implementation quality issue with IOStream, where we attempt to use seekend as a fast-path for filesize, but we should fall back to attempting fstat when that fails.

Unrelatedly, there appears to be a minor issue in SharedArrays forgetting to close the fd_mem handle in this functions caller.

@xgdgsc
Copy link
Contributor Author

xgdgsc commented Jul 15, 2023

Not sure whether the InteractiveUtils test failure is related. Would it be fine to fix it only for linux first like this and wait for macOS solution later? Guess not many would encounter this bug on macOS.

@vtjnash vtjnash added status:merge me PR is reviewed. Merge when all tests are passing backport 1.10 Change should be backported to the 1.10 release labels Jul 18, 2023
@DilumAluthge
Copy link
Member

CI is all green on this.

@vtjnash Is this good to merge?

@vtjnash vtjnash merged commit ead627e into JuliaLang:master Aug 21, 2023
6 checks passed
@DilumAluthge DilumAluthge removed the status:merge me PR is reviewed. Merge when all tests are passing label Aug 21, 2023
KristofferC pushed a commit that referenced this pull request Feb 6, 2024
Fixes: #28245
Co-authored-by: Jameson Nash <[email protected]>
Co-authored-by: Stefan Karpinski <[email protected]>
(cherry picked from commit ead627e)
@KristofferC KristofferC removed the backport 1.10 Change should be backported to the 1.10 release label Feb 6, 2024
@mkitti
Copy link
Contributor

mkitti commented Apr 22, 2024

As reported in #54203 this fix introduced another issue since filesize seeks which may not be appropriate for certain kinds of IO.

Drvi pushed a commit to RelationalAI/julia that referenced this pull request Jun 7, 2024
Fixes: JuliaLang#28245
Co-authored-by: Jameson Nash <[email protected]>
Co-authored-by: Stefan Karpinski <[email protected]>
(cherry picked from commit ead627e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bugfix This change fixes an existing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mmap bus error
8 participants