Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Implementation of create file #1282

Merged
merged 9 commits into from
Dec 12, 2018
Merged

Conversation

bartlomieju
Copy link
Member

@bartlomieju bartlomieju commented Dec 5, 2018

Trying to implement simple logger in deno I found that creating a file is still impossible.

I saw work done in #908, but I turns out we can simply use tokio::fs::OpenOptions to configure what mode we want to open file in.

This is very rough minimal implementation that still needs a lot of work.

TODO:

  • settle on OpenModes and implement as enum in files.ts
  • factor out setup code of tokio::fs::OpenOptions in ops.ts
  • tests

@bartlomieju
Copy link
Member Author

bartlomieju commented Dec 6, 2018

File open modes in Python (taken from https://docs.python.org/3/library/io.html#io.FileIO)

The mode can be 'r', 'w', 'x' or 'a' for reading (default), writing, exclusive creation or appending. The file will be created if it doesn’t exist when opened for writing or appending; it will be truncated when opened for writing. FileExistsError will be raised if it already exists when opened for creating. Opening a file for creating implies writing, so this mode behaves in a similar way to 'w'. Add a '+' to the mode to allow simultaneous reading and writing.

Which translates to roughly this:

enum OpenMode {
    Read = 'r',
    Write = 'w', // <- truncates the file if it has content
    ExclusiveOpen = 'x',
    Append = 'a',
    ReadWrite = 'r+' // 'w+' would also work
    ReadAppend = 'a+'
    ReadExclusiveOpen = 'x+' // <- better name needed
}

Opinions welcome.

CC: @ry @kitsonk @piscisaureus

@kitsonk
Copy link
Contributor

kitsonk commented Dec 6, 2018

@bartlomieju I would totally defer to Ryan or @piscisaureus on this one. I am not really good at having well informed opinions about OS APIs.

@bartlomieju
Copy link
Member Author

Sure, thanks for letting know.

@bartlomieju
Copy link
Member Author

Go's and Ruby's file mode for reference:
https://golang.org/pkg/os/#pkg-constants

const (
        // Exactly one of O_RDONLY, O_WRONLY, or O_RDWR must be specified.
        O_RDONLY int = syscall.O_RDONLY // open the file read-only.
        O_WRONLY int = syscall.O_WRONLY // open the file write-only.
        O_RDWR   int = syscall.O_RDWR   // open the file read-write.
        // The remaining values may be or'ed in to control behavior.
        O_APPEND int = syscall.O_APPEND // append data to the file when writing.
        O_CREATE int = syscall.O_CREAT  // create a new file if none exists.
        O_EXCL   int = syscall.O_EXCL   // used with O_CREATE, file must not exist.
        O_SYNC   int = syscall.O_SYNC   // open for synchronous I/O.
        O_TRUNC  int = syscall.O_TRUNC  // if possible, truncate file when opened.
)

https://ruby-doc.org/core-2.2.2/IO.html#method-c-new-label-IO+Open+Mode

"r"  Read-only, starts at beginning of file  (default mode).

"r+" Read-write, starts at beginning of file.

"w"  Write-only, truncates existing file
     to zero length or creates a new file for writing.

"w+" Read-write, truncates existing file to zero length
     or creates a new file for reading and writing.

"a"  Write-only, each write call appends data at end of file.
     Creates a new file for writing if file does not exist.

"a+" Read-write, each write call appends data at end of file.
     Creates a new file for reading and writing if file does
     not exist.

"b"  Binary file mode
     Suppresses EOL <-> CRLF conversion on Windows. And
     sets external encoding to ASCII-8BIT unless explicitly
     specified.

"t"  Text file mode

@ry
Copy link
Member

ry commented Dec 7, 2018

This looks great! r w x a with optional plus is the time tested interface.

Can you add some more tests? Add TODOs for the missing ones.

@bartlomieju
Copy link
Member Author

bartlomieju commented Dec 7, 2018

@ry I added OpenMode enum with explanation for each mode.

Please check how OpenOptions::append behaves. Do we need to care about position of reading?

I will write full tests for each mode tomorrow morning. I expect to find some quirky behavior along the way that will have to be pinpointed.

EDIT: grammar

panic!("Unknown file open mode.");
}
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I move this whole operation somewhere else?

Probably some comments explaining why each option is used would be appropriate as well.

js/files.ts Show resolved Hide resolved
@bartlomieju
Copy link
Member Author

I got a bit confused, I found that Deno already has writeFile and readFile implemented. Does it make sense to have separate implementations of them and open in Rust?

cc @ry

@ry
Copy link
Member

ry commented Dec 9, 2018

@bartlomieju Regarding writeFile and readFile - these ops are fully buffered. They're fine for small files, but if you have a 1 gigabyte file, you might not want to load it all into memory just to read the first few bytes from it. This is the purpose of having raw file handles.

js/files.ts Outdated Show resolved Hide resolved
js/files.ts Outdated Show resolved Hide resolved
js/files_test.ts Outdated Show resolved Hide resolved
js/files_test.ts Show resolved Hide resolved
@bartlomieju
Copy link
Member Author

@ry I resolved your remarks. Please let know if OpenMode comments are alright.

To finish tests we need to have seek working, but that's for another PR.

Copy link
Member

@ry ry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for seeing this through!

@ry ry merged commit c2b91a3 into denoland:master Dec 12, 2018
@bartlomieju bartlomieju deleted the create_file branch December 12, 2018 17:16
ry added a commit to ry/deno that referenced this pull request Dec 14, 2018
- console.assert should not throw error (denoland#1335)
- Support more modes in deno.open (denoland#1282, denoland#1336)
- Simplify code fetch logic (denoland#1322)
- readDir entry mode (denoland#1326)
- Use stderr for exceptions (denoland#1303)
- console.log formatting improvements (denoland#1327, denoland#1299)
- Expose TooLarge error code for buffers (denoland#1298)
@ry ry mentioned this pull request Dec 14, 2018
ry added a commit that referenced this pull request Dec 14, 2018
- console.assert should not throw error (#1335)
- Support more modes in deno.open (#1282, #1336)
- Simplify code fetch logic (#1322)
- readDir entry mode (#1326)
- Use stderr for exceptions (#1303)
- console.log formatting improvements (#1327, #1299)
- Expose TooLarge error code for buffers (#1298)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants