Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Minor request] keyword flags when opening. #222

Open
ryofurue opened this issue Jul 31, 2023 · 2 comments
Open

[Minor request] keyword flags when opening. #222

ryofurue opened this issue Jul 31, 2023 · 2 comments

Comments

@ryofurue
Copy link

NCDatasets uses "c":

NCDatasets("filename.nc", "c")

to create the file, whereas Julia's open function doesn't have a "c" flag. It seems that "w" is the right flag for Julia'sopen function, but it's not clear to me whether this is always equivalent to "create".

This situation is confusing enough for me so that I never use the "w" flag for Julia's open function and I instead use

open("filename"; create = true)

( https://docs.julialang.org/en/v1/base/io-network/#General-I/O )

So, my request would be: It would be nice if the NCDataset function faithfully mirrored open's keywords.

(Purely out of curiosity, where did "c" come from? Julia's "w", "r", . . . seem to be the same as those of fopen() the C library function.)

Environment

  • operating system: macOS 13.4.1 (c)
  • Julia version: julia 1.9.2 from juliaup
  • Output of the julia command versioninfo()
Julia Version 1.9.2
Commit e4ee485e909 (2023-07-05 09:39 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 10 × Apple M2 Pro
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
  Threads: 1 on 6 virtual cores
  • NCDatasets version: v0.12.17
  • Output of using Pkg; Pkg.status(mode=PKGMODE_MANIFEST)

Full output
Status ~/.julia/environments/v1.9/Manifest.toml
[621f4979] AbstractFFTs v1.4.0
[79e6a3ab] Adapt v3.6.2
[bf4720bc] AssetRegistry v0.1.0
[d1d4a3ce] BitFlags v0.1.7
[ad839575] Blink v0.12.8
[179af706] CFTime v0.1.2
[944b1d66] CodecZlib v0.7.2
[35d6a980] ColorSchemes v3.22.0
[3da002f7] ColorTypes v0.11.4
[c3611d14] ColorVectorSpace v0.10.0
[5ae59095] Colors v0.12.10
[1fbeeb36] CommonDataModel v0.2.4
⌅ [34da2185] Compat v3.46.2
[f0e56b4a] ConcurrentUtilities v2.2.1
[d38c429a] Contour v0.6.2
[717857b8] DSP v0.7.8
[9a962f9c] DataAPI v1.15.0
[864edb3b] DataStructures v0.18.14
[e2d170a0] DataValueInterfaces v1.0.0
[8bb1440f] DelimitedFiles v1.9.1
[ffbed154] DocStringExtensions v0.9.3
[460bff9d] ExceptionUnwrapping v0.1.9
[c87230d0] FFMPEG v0.4.1
[7a1cc6ca] FFTW v1.7.1
[d8f44d74] FilterHelpers v0.2.0
[53c48c17] FixedPointNumbers v0.8.4
[1fa38f19] Format v1.3.2
[59287772] Formatting v0.4.2
[de31a74c] FunctionalCollections v0.5.0
[28b8d3ca] GR v0.72.9
[6aaf26a4] Geodesics v0.2.4 https://github.com/anowacki/Geodesics.jl#master
[c96ba436] GeographicLib v0.4.1 https://github.com/anowacki/GeographicLib.jl#master
[9a22fb26] GibbsSeaWater v0.1.2
[c27321d9] Glob v1.3.1
[42e2da0e] Grisu v1.0.2
[cd3eb016] HTTP v1.9.14
[9fb69e20] Hiccup v0.2.2
[18e54dd8] IntegerMathUtils v0.1.2
[8197267c] IntervalSets v0.7.4
[92d709cd] IrrationalConstants v0.2.2
[c8e1da08] IterTools v1.8.0
[82899510] IteratorInterfaceExtensions v1.0.0
[1019f520] JLFzf v0.1.5
[692b3bcd] JLLWrappers v1.4.1
[97c1335a] JSExpr v0.5.4
[682c06a0] JSON v0.21.4
[b964fa9f] LaTeXStrings v1.3.0
[23fbe1c1] Latexify v0.16.1
[50d2b5c4] Lazy v0.15.1
[2ab3a3ac] LogExpFunctions v0.3.24
[e6f89c97] LoggingExtras v1.0.0
[1914dd2f] MacroTools v0.5.10
[dbb5928d] MappedArrays v0.4.2
[739be429] MbedTLS v1.1.7
[442fdcdd] Measures v0.3.2
[e1d29d7a] Missings v1.1.0
[5cb8414e] ModuleInterfaceTools v1.0.1
[ffc61752] Mustache v1.0.17
[a975b10e] Mux v1.0.1
[85f8d34a] NCDatasets v0.12.17
[77ba4419] NaNMath v1.0.2
[510215fc] Observables v0.5.4
[6fe1bfb0] OffsetArrays v1.12.10
[4d8831e6] OpenSSL v1.4.1
[bac558e1] OrderedCollections v1.6.2
[d96e819e] Parameters v0.12.3
[69de0a69] Parsers v2.7.1
[fa939f87] Pidfile v1.3.0
[b98c9c47] Pipe v1.3.0
[ccf2f8ad] PlotThemes v3.1.0
[995b91a9] PlotUtils v1.3.5
[a03496cd] PlotlyBase v0.8.19
[f0f68f2c] PlotlyJS v0.18.10
[91a5bcdd] Plots v1.38.16
[f27b6e38] Polynomials v3.2.13
[aea7be01] PrecompileTools v1.1.2
[21216c6a] Preferences v1.4.0
[27ebfcd6] Primes v0.5.4
[3cdcf5f2] RecipesBase v1.3.4
[01d81517] RecipesPipeline v0.6.12
[189a3867] Reexport v1.2.2
[05181044] RelocatableFolders v1.0.0
[ae029012] Requires v1.3.0
[6c6a2e73] Scratch v1.2.0
[e8f3a9d7] SearchSortedNearest v0.1.1
[992d4aef] Showoff v1.0.3
[777ac1f9] SimpleBufferStream v1.1.0
[a2af1166] SortingAlgorithms v1.1.1
[276daf66] SpecialFunctions v2.3.0
[90137ffa] StaticArrays v1.6.2
[1e83bf80] StaticArraysCore v1.4.2
[82ae8749] StatsAPI v1.6.0
[2913bbd2] StatsBase v0.34.0
[b5087856] StrFormat v1.0.1
[68059f60] StrLiterals v1.1.0
[3783bdb8] TableTraits v1.0.1
[bd369af6] Tables v1.10.1
[62fd8b95] TensorCore v0.1.1
[3bb67fe8] TranscodingStreams v0.9.13
[5c2747f8] URIs v1.4.2
[3a884ed6] UnPack v1.0.2
[1cfade01] UnicodeFun v0.4.1
[1986cc42] Unitful v1.15.0
[45397f5d] UnitfulLatexify v1.6.3
[41fe7b60] Unzip v0.2.0
[0f1e0344] WebIO v0.8.21
[104b5d7c] WebSockets v1.6.0
[cc8bc4a8] Widgets v0.6.6
[6e34b625] Bzip2_jll v1.0.8+0
[83423d85] Cairo_jll v1.16.1+1
[2e619515] Expat_jll v2.5.0+0
[b22a6f82] FFMPEG_jll v4.4.2+2
[f5851436] FFTW_jll v3.3.10+0
[a3f928ae] Fontconfig_jll v2.13.93+0
[d7e528f0] FreeType2_jll v2.13.1+0
[559328eb] FriBidi_jll v1.0.10+0
[0656b61e] GLFW_jll v3.3.8+0
[d2c73de3] GR_jll v0.72.9+0
[78b55507] Gettext_jll v0.21.0+0
[6727f6b2] GibbsSeaWater_jll v3.5.2+0
[7746bdde] Glib_jll v2.74.0+2
[3b182d85] Graphite2_jll v1.3.14+0
⌅ [0234f1f7] HDF5_jll v1.12.2+2
[2e76f6c2] HarfBuzz_jll v2.8.1+1
[1d5cc7b8] IntelOpenMP_jll v2023.1.0+0
[aacddb02] JpegTurbo_jll v2.1.91+0
[f7e6163d] Kaleido_jll v0.2.1+0
[c1c5ebd0] LAME_jll v3.100.1+0
[88015f11] LERC_jll v3.0.0+1
[1d63c593] LLVMOpenMP_jll v15.0.4+0
[dd4b983a] LZO_jll v2.10.1+0
⌅ [e9f186c6] Libffi_jll v3.2.2+1
[d4300ac3] Libgcrypt_jll v1.8.7+0
[7e76a0d4] Libglvnd_jll v1.6.0+0
[7add5ba3] Libgpg_error_jll v1.42.0+0
[94ce4f54] Libiconv_jll v1.16.1+2
[4b2f31a3] Libmount_jll v2.35.0+0
[89763e89] Libtiff_jll v4.5.1+1
[38a345b3] Libuuid_jll v2.36.0+0
[856f044c] MKL_jll v2023.1.0+0
⌃ [7243133f] NetCDF_jll v400.902.5+1
[e7412a2a] Ogg_jll v1.3.5+1
⌅ [458c3c95] OpenSSL_jll v1.1.21+0
[efe28fd5] OpenSpecFun_jll v0.5.5+0
[91d4177d] Opus_jll v1.3.2+0
[30392449] Pixman_jll v0.42.2+0
[c0090381] Qt6Base_jll v6.4.2+3
[a2964d1f] Wayland_jll v1.21.0+0
[2381bf8a] Wayland_protocols_jll v1.25.0+0
[02c8fc9c] XML2_jll v2.10.3+0
[aed1982a] XSLT_jll v1.1.34+0
[ffd25f8a] XZ_jll v5.4.3+1
[4f6342f7] Xorg_libX11_jll v1.8.6+0
[0c0b7dd1] Xorg_libXau_jll v1.0.11+0
[935fb764] Xorg_libXcursor_jll v1.2.0+4
[a3789734] Xorg_libXdmcp_jll v1.1.4+0
[1082639a] Xorg_libXext_jll v1.3.4+4
[d091e8ba] Xorg_libXfixes_jll v5.0.3+4
[a51aa0fd] Xorg_libXi_jll v1.7.10+4
[d1454406] Xorg_libXinerama_jll v1.1.4+4
[ec84b674] Xorg_libXrandr_jll v1.5.2+4
[ea2f1a96] Xorg_libXrender_jll v0.9.10+4
[14d82f49] Xorg_libpthread_stubs_jll v0.1.1+0
[c7cfdc94] Xorg_libxcb_jll v1.15.0+0
[cc61e674] Xorg_libxkbfile_jll v1.1.2+0
[12413925] Xorg_xcb_util_image_jll v0.4.0+1
[2def613f] Xorg_xcb_util_jll v0.4.0+1
[975044d2] Xorg_xcb_util_keysyms_jll v0.4.0+1
[0d47668e] Xorg_xcb_util_renderutil_jll v0.3.9+1
[c22f9ab0] Xorg_xcb_util_wm_jll v0.4.1+1
[35661453] Xorg_xkbcomp_jll v1.4.6+0
[33bec58e] Xorg_xkeyboard_config_jll v2.39.0+0
[c5fb5394] Xorg_xtrans_jll v1.5.0+0
[3161d3a3] Zstd_jll v1.5.5+0
⌅ [214eeab7] fzf_jll v0.29.0+0
[a4ae2306] libaom_jll v3.4.0+0
[0ac62f75] libass_jll v0.15.1+0
[f638f0a6] libfdk_aac_jll v2.0.2+0
[b53b4c65] libpng_jll v1.6.38+0
[f27f6e37] libvorbis_jll v1.3.7+1
[1270edf5] x264_jll v2021.5.5+0
[dfaa095f] x265_jll v3.5.0+0
[d8fb68d0] xkbcommon_jll v1.4.1+0
[0dad84c5] ArgTools v1.1.1
[56f22d72] Artifacts
[2a0f44e3] Base64
[ade2ca70] Dates
[8ba89e20] Distributed
[f43a241f] Downloads v1.6.0
[7b1f6079] FileWatching
[b77e0a4c] InteractiveUtils
[4af54fe1] LazyArtifacts
[b27032c2] LibCURL v0.6.3
[76f85450] LibGit2
[8f399da3] Libdl
[37e2e46d] LinearAlgebra
[56ddb016] Logging
[d6f4376e] Markdown
[a63ad114] Mmap
[ca575930] NetworkOptions v1.2.0
[44cfe95a] Pkg v1.9.2
[de0858da] Printf
[3fa0cd96] REPL
[9a3f8284] Random
[ea8e919c] SHA v0.7.0
[9e88b42a] Serialization
[1a1011a3] SharedArrays
[6462fe0b] Sockets
[2f01184e] SparseArrays
[10745b16] Statistics v1.9.0
[fa267f1f] TOML v1.0.3
[a4e569a6] Tar v1.10.0
[8dfed614] Test
[cf7118a7] UUIDs
[4ec0a83e] Unicode
[e66e0078] CompilerSupportLibraries_jll v1.0.5+0
[deac9b47] LibCURL_jll v7.84.0+0
[29816b5a] LibSSH2_jll v1.10.2+0
[c8ffd9c3] MbedTLS_jll v2.28.2+0
[14a3606d] MozillaCACerts_jll v2022.10.11
[4536629a] OpenBLAS_jll v0.3.21+4
[05823500] OpenLibm_jll v0.8.1+0
[efcefdf7] PCRE2_jll v10.42.0+0
[bea87d4a] SuiteSparse_jll v5.10.1+6
[83775a58] Zlib_jll v1.2.13+0
[8e850b90] libblastrampoline_jll v5.8.0+0
[8e850ede] nghttp2_jll v1.48.0+0
[3f19e933] p7zip_jll v17.4.0+0
Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use status --outdated -m

@Alexander-Barth
Copy link
Owner

The option "c" comes from the NC_CLOBBER:
https://docs.unidata.ucar.edu/netcdf-c/current/group__datasets.html#gaccbdb128a1640e204831256dbbc24d3e

see also:
https://en.wikipedia.org/wiki/Clobbering

I guess that the NetCDF authors want to make it clear that existing content will be overwritten.

I agree that the boolean parameters (read, write, create, append) are easier to read.
For some combinations (write = false, append = true) we would need to raise an error.

For reference here is the logic for the default values in base julia:
https://github.com/JuliaLang/julia/blob/e4ee485e90961018b7e53ce14b8b99335953e176/base/io.jl#L344

@ryofurue
Copy link
Author

ryofurue commented Aug 1, 2023

The option "c" comes from the NC_CLOBBER:

I see! I thought it was "create".

https://docs.unidata.ucar.edu/netcdf-c/current/group__datasets.html#gaccbdb128a1640e204831256dbbc24d3e

Hmm . . . This makes me wonder what happens if you nc_open an existing netCDF file with NC_CLOBBER and immediately close it. Will the file remain intact or be truncated to become zero-sized? It's not clear from the documentation.

If you are familiar with Unix-shell programming, you know "clobber" means "truncate" & "write", but the description in the netCDF documentation, "overwrite existing file", can mean "if you write something into it, the existing contents will be overwritten" and it's not clear whether "truncate" is implied or not. Did the netCDF developers adopt the shell's clobbering?

If you say "create", it would be clear that the existing file will be truncated.

So, among the following three methods:

NCDatasets: "r", "w", "c"
julia's mode strings: "r", "w", "a", . . .
julia's boolean keywords: write, truncate, create, append . . .

the last set is the clearest. The julia code you linked to (thanks!) serves as a "flow chart" of the logic but even without that, what will happen is fairly clear from the names "truncate", "append", "create", etc.

Anyway, this is just a very minor issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants