Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make IOStream thread-safe #32421

Merged
merged 2 commits into from
Jul 17, 2019
Merged

make IOStream thread-safe #32421

merged 2 commits into from
Jul 17, 2019

Conversation

JeffBezanson
Copy link
Sponsor Member

Unfortunately this has a hefty ~5x slowdown for small I/O operations.

@JeffBezanson JeffBezanson added domain:io Involving the I/O subsystem: libuv, read, write, etc. domain:multithreading Base.Threads and related functionality labels Jun 26, 2019
@JeffBezanson
Copy link
Sponsor Member Author

We decided to make this thread-safe by default, then if it turns out to be a problem consider adding an option like open(filename, locked=false) to disable the lock. Note the overhead is only significant (2x-4x) in the case of very small I/O operations like bytes or small integers. Any code doing large numbers of small IOs can probably be greatly sped up anyway.

@JeffBezanson
Copy link
Sponsor Member Author

Note one reason this is important is that when julia's stdout is redirected to a file, stdout is an IOStream.

@JeffBezanson JeffBezanson changed the title WIP: make IOStream thread-safe make IOStream thread-safe Jul 9, 2019
vtjnash
vtjnash reviewed Jul 15, 2019
View reviewed changes
NEWS.md Outdated Show resolved Hide resolved
@JeffBezanson @vtjnash
Update NEWS.md
5088719 
Co-Authored-By: Jameson Nash <[email protected]>
@JeffBezanson JeffBezanson merged commit 261e2b9 into master Jul 17, 2019
@JeffBezanson JeffBezanson deleted the jb/iostreamlock branch July 17, 2019 16:33
@baggepinnen
Copy link
Contributor

I believe this causes quite the slowdown in WAV.jl
dancasimiro/WAV.jl#52 (comment)
Is the locked=false option on the horizon?

@StefanKarpinski
Copy link
Sponsor Member

Is it possible to read/write in larger units than individual bytes?

@JeffBezanson
Copy link
Sponsor Member Author

Right; while this was a ~5x slowdown; WAV.jl is 40x slower than other wav readers.

@JeffBezanson JeffBezanson mentioned this pull request Jan 2, 2020
Closed
@jyjemily jyjemily mentioned this pull request May 29, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vtjnash vtjnash vtjnash approved these changes

Assignees
No one assigned
Labels
domain:io Involving the I/O subsystem: libuv, read, write, etc. domain:multithreading Base.Threads and related functionality
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@JeffBezanson @baggepinnen @StefanKarpinski @vtjnash