Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Win32 memfs support. #1

Closed
wants to merge 14 commits into from

Conversation

jbuonagurio
Copy link

Thank you for your work to support Windows! I am able to build the dtip-windows branch with no issues, other than adding some additional linker dependencies when compiling with statically-linked netCDF (specifically hdf5_hl, szip and zlib). That is probably something to address in ecbuild.

This PR adds memfs support. As there is no API equivalent to fmemopen or funopen for FILE pointers, this is implemented with a temporary file, but the file flags FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE hint to the filesystem that it shouldn't be flushed to disk so it still acts like a memory buffer. See this MSDN blog post for a discussion of this technique: It's only temporary

I originally suggested this patch in Jira issue SUP-2586. All of my other suggestions for Windows support have been incorporated (and more!), so this is my only remaining contribution at the moment.

I'm also working on adding support for efficient subsetting of remote files using something similar to GDAL virtual file systems, but that will take some time.

@dtip
Copy link
Contributor

dtip commented Jan 2, 2019

Thanks for your work on this, John!

I believe your MEMFS patch from SUP-2586 was added to the develop branch in commit 0ec1a0b a few days ago. I've rebased the dtip-windows branch onto it and also added commit 02ee12b (which relies on the dtip-windows branch of ecBuild) so your changes should now be included. I'll close the pull request for now, but if you feel there's still part of the patch missing, please re-open it.

Apologies for blasting your pull request with the rebase and subsequent force-push. I wasn't expecting any requests to merge with this branch and missed that you'd opened one!

@dtip dtip closed this Jan 2, 2019
@jbuonagurio
Copy link
Author

Awesome, thanks. I haven't moved to the develop branch yet, but in the meantime I started working on some code to provide an abstract I/O interface for FTP/HTTP files using libcurl, in this commit:

jbuonagurio/eccodes@ec41774

Would you or someone else at ECMWF be interested in providing some feedback or helping with this feature? Should I open a PR or do you prefer an issue on Jira?

@dtip
Copy link
Contributor

dtip commented Jan 3, 2019

I'm with ECMWF on a short-term contract, so not the best person to help out, but I've passed your message on to the code owners. My guess is that Jira is preferable.

shahramn pushed a commit that referenced this pull request Jun 21, 2022
RaineAllDay referenced this pull request in RaineAllDay/eccodes Aug 9, 2023
updating ci for fork use only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants