Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand DatasetAccessProperties coverage, add helpers #949

Merged
merged 3 commits into from
Jun 10, 2022

Conversation

mkitti
Copy link
Member

@mkitti mkitti commented Jun 6, 2022

Expanded DatasetAccessProperties to cover

  • chunk_cache: Chunk cache parameters as (nslots, nbytes, w0).
    Default: (521, 0x100000, 0.75)
  • efile_prefix: Path prefix for reading external files.
    The default is the current working directory.
    • :origin: alias for raw"\$ORIGIN" will make the external file relative to
      the HDF5 file.
  • virtual_prefix: Path prefix for reading virtual datasets.
  • virtual_printf_gap: The maximum number of missing source files and/or
    datasets with the printf-style names when getting the extent of an unlimited
    virtual dataset
  • virtual_view: Influences whether the view of the virtual dataset includes
    or excludes missing mapped elements
    • :first_missing: includes all data before the first missing mapped data
    • :last_available: includes all available mapped data

Pending tasks:

  • Add tests

@mkitti
Copy link
Member Author

mkitti commented Jun 6, 2022

The only missing dataset access property is append_flush:
https://portal.hdfgroup.org/display/HDF5/H5P_GET_APPEND_FLUSH
https://portal.hdfgroup.org/display/HDF5/H5P_SET_APPEND_FLUSH

This has a signature unlike the others.

@mkitti
Copy link
Member Author

mkitti commented Jun 6, 2022

julia> dapl = HDF5.init!(HDF5.DatasetAccessProperties())
HDF5.DatasetAccessProperties(
  chunk_cache     = (nslots = 0x0000000000000209, nbytes = 0x0000000000100000, w0 = 0.75),
  efile_prefix    = "",
  virtual_prefix  = "",
  virtual_printf_gap = 0x0000000000000000,
  virtual_view    = :last_available,
)

@@ -411,6 +419,11 @@ function h5p_get_efile_prefix(plist)
return String(buffer)
end

function h5p_set_efile_prefix(plist, sym::Symbol)
sym === :origin ? h5p_set_efile_prefix(plist, raw"$ORIGIN") :
error("The only valid symbol for h5p_set_efile_prefix is :origin.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

throw(ArgumentError(.... ?

src/api/helpers.jl Outdated Show resolved Hide resolved
src/api/helpers.jl Outdated Show resolved Hide resolved
@mkitti mkitti marked this pull request as ready for review June 9, 2022 21:09
@mkitti
Copy link
Member Author

mkitti commented Jun 10, 2022

This should be good to go for now. I think virtual_printf_gap and virtual_view testing will need to be deferred until we actually implement virtual datasets.

@musm musm merged commit 3a805bf into master Jun 10, 2022
@musm musm deleted the mkitti/dapl_properties branch June 10, 2022 14:25
@musm
Copy link
Member

musm commented Jun 10, 2022

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants