Support Iris on Windows #467
Comments
|
+1, I have no access to linux unfortunately but I'm very interested in becoming an Iris user. |
|
👍 - I know @munslowa had a look at this some time ago, but I have no idea how far he got. I have no expertise in this, but I can't imagine its too hard (if you know what you're doing with compiling on Windows, which I don't...). |
|
@pelson, I'm sure you are right, but I think the problem is that there are only a few people who really know how to compile python binaries on Windows. Nearly everyone relies on @cgohlke binaries or scientific distributions for windows where lots of packages are already built (Enthought Python Distribution, Anaconda, Python(x,y)). But wait. I just checked and @cgohlke has windows binaries for Cartopy, and Pyke is pure python, right? Christoph and the other distributions cover everything else except for Udunits. So perhaps that only leaves udunits as a prerequisite for Iris? |
|
Which package requires UDUNITS? |
|
You remember well @pelson. udunits is where I got stuck. In lib/iris/unit.py (line 180) it needs to load the shared library and then go on to load the xml unit database. Searching around I found http:https://cran.rakanu.com/web/packages/udunits2/index.html. |
|
The above seems to be a R package. |
|
Grrr... Just remembered the hard way that many (all?) of Gohlke's current Python Windows binaries will NOT work with the latest EPD, because Gohlke went to numpy 1.7 and EPD is still at numpy 1.6.1. So Gohlke's Cartopy windows binary, for example, doesn't work with the EPD. |
Sadly, this also requires the UDUNITS-2 library to be installed. |
|
It took a few hours as I got my head around Python on Windows and installed dependencies, but by sidestepping udunits in unit.py I now have basic loading and plotting functionality under windows. |
|
Would anyone be interested in evaluating an "alpha quality" version of the ECMWF GRIB API for MSVC 2008? I cannot currently find time to do this. Ideally, we would check if the Iris grib tests pass or not. If so, I might be able to arrange access to the pre-release. |
|
Sure, I'd be willing to test this out. On Mon, May 13, 2013 at 6:22 AM, bblay [email protected] wrote:
Dr. Richard P. Signell (508) 457-2229 |
|
I'm also happy to help test. |
|
Thanks Rich, contact details sent. Thanks @scw, I attempted to contact you privately with the email address of a developer at ECMWF who will help you get the api. Please could you let me know if you got that email? |
|
@bblay Yes, I did get your message, thanks -- I'll let you know once I've heard back and have the API in-hand. cheers, Shaun |
|
It's just been revealed, the Windows version does not include a python interface, and there is seemingly nothing on the horizon 😟. |
No harm in everyone with an interest nudging the ECMWF to see if we can get it moved up (or even, onto) their agenda. |
|
One potential way forward:
We could finish writing this: https://github.com/bblay/gribby |
|
@esc24 What did you do to "sidestep" udunits? Thanks, |
|
I put a collection of ugly |
|
Great. Also, @semmerson at Unidata is working on getting UDUNITS going on Windows. Hopefully we'll hear good news on that front soon! |
|
I'd like to add a note of support for the effort to get a Windows version of IRIS and Cartopy built. I have a user base that could benefit from the effort but who predominantly use Windows. Thanks for your work and please let me know if there's any way I can help. |
|
There is a windows binary of cartopy v0.7 at http:https://www.lfd.uci.edu/%7Egohlke/pythonlibs/#cartopy. The blocker for Iris on windows is udunits (although grib support might also be a challenge). I'm very encouraged by the news that @semmerson is looking into udunits. |
|
@semmerson is still working on the CMAKE build for UDUNITS, but there is a pull request on UDUNITS by @sdebionne that has a native Windows build for Visual Studio: Does that sound like something that you guys could use to get UDUNITS going on Windows? P.S. Another option would be to use a different units package. Of the 18 units packages (aka physical quantities packages) for Python that Trevor Bekolay looked into during his SciPY 2013 talk (http:https://www.youtube.com/watch?v=N-edLdxiM40 ), only one of them appears to be pure python -- the "magnitude" package by @juanre at |
|
That sounds very promising. I'll give it a go. I have a little bit of breathing room in my diary next week so I'll put this on the list. Sorry for letting this topic slip lately. |
|
Excellent. Thanks for the update. |
Did anyone try this out? I couldn't get it to compile... |
|
@ajdawson Andrew, I would be glad to help you. Can you tell me more about what goes wrong ? |
I can but not until I'm at a windows machine, I'll investigate and get back to you. Thanks @sdebionne. |
|
@sdebionne - I loaded the solution into Visual Studio Express 2012 and tried to build the solution but I got 3 errors: I've omitted the other details from the output. I also tried usingVisual Studio Professional 2012 but I got the same errors. I haven't used Visual Studio for many years so I'm a beginner really, it may be something really simple but I'm afraid I'm stuck! Your help is much appreciated. |
|
@ajdawson Andrew, You should have something like this line 29 of parser.y : #if !defined(_MSC_VER)
# include <strings.h>
#else
# define snprintf _snprintf
# define vsnprintf _vsnprintf
# define strcasecmp _stricmp
# define strncasecmp _strnicmp
#endifAnd that should prevent |
|
@ajdawson And in #if !defined(_MSC_VER)
# include <strings.h>
#else
# define snprintf _snprintf
# define vsnprintf _vsnprintf
# define strcasecmp _stricmp
# define strncasecmp _strnicmp
# define strdup _strdup
#endif
|
|
Thanks @sdebionne my source tree had become polluted from previous attempts. I did a fresh checkout of your branch, opened the solution file in Visual Studio, and selected build solution and I got the following errors: These are because I haven't generated parser.c right? Do I need to get Visual Studio to do that or do I copy it from somewhere? You mentioned the source tarball... Sorry for so many questions! |
|
OK so I copied parser.c form udunits source tarball and applied the patch you gave me and that seems to work. Is that what I am meant to do? I'm now having problems related to scanner.l/scanner.c. First it said couldn't find scanner.c, so I copied it over from the source tarball(?) and then I get an error in scanner.l saying |
|
@sdebionne - I ran into exactly the same parser.c issue as @ajdawson, but rather than copying I tried to generate it using bison (I can't remember the error - this was a month or two back) and failed. I expect your patch will solve that. Any ideas for the scanner.c issue? Any help would be much appreciated, as it would be great to get udunits building under windows. |
@ajdawson Yes, exactly. I agree that it's not really practical though. @esc24 Well I'm not very acquainted with bison so what I did, which is of course not recommanded, is edit the generated files. That explain why I gave a patch for |
|
Looks like the latest release of udunits2 supports windows: http:https://www.unidata.ucar.edu/software/udunits/udunits-2.2.4/doc/udunits/udunits2.html#Windows. I don't know how useful that will be, I don't have time to test it myself. |
|
Andrew, I'm working on it. Currently, it only supports building from source using Regards, On Sat, Jan 18, 2014 at 9:08 AM, Andrew Dawson [email protected]:
|
|
Steve, |
|
Hi Rich, Work on a binary Windows release is currently suspended as I'm waiting for I also have to work on a 2-year, 2M$ NSF grant on which I'm a PI (it deals All that aside, you should be able to build the UDUNITS version on the --Steve On Thu, Feb 13, 2014 at 9:53 AM, Rich Signell [email protected]:
|
|
@ajdawson Andrew, I guess you could not managed to use my PR... Could you tell me where you get stucked ? I may have time int the next weeks to improve this. I probably need to rebase my PR first if you are using the master branch... @semmerson While I'm not a fan of CMake, official Windows support (through MinGW) is an excellent news ! |
|
Sdebionne, I decided to use CMake because that's what the netCDF developers use -- and the UDUNITS package is heavily used by netCDF folks. The reason I didn't get around to your pull request has nothing to do with the quality of your work -- I was simply too constrained by time and the decision to use CMake. |
|
@semmerson, any status update here on building native windows version of udunits? |
|
Rich, There's been no progress since my last comment because I've been working on We also need to create some persistent Windows virtual machines in order to It should still be possible, however, to build the HEAD commit of the --Steve |
|
@semmerson , can you provide a scope estimate for completing this project (e.g. is this a several hour, several day, or several week job?) |
|
Rich, All told, establishing a Windows package repository on persistent virtual --Steve Regards, On Wed, May 21, 2014 at 10:04 AM, Rich Signell [email protected]:
|
|
thanks! |
|
UDUNITS now builds on Windows using CMAKE, thanks to @WardF and @semmerson: Unidata/UDUNITS-2#17 |
|
I put experimental Windows binaries at http:https://www.lfd.uci.edu/~gohlke/pythonlibs/#iris using the following changes ( diff --git a/lib/iris/unit.py b/lib/iris/unit.py
index 0a022c6..40dc111 100644
--- a/lib/iris/unit.py
+++ b/lib/iris/unit.py
@@ -169,11 +169,25 @@ _ut_set_error_message_handler = None
########################################################################
#
+# load the shared libraries on Windows
+#
+if sys.platform.startswith('win32'):
+ try:
+ _lib_c = ctypes.cdll.msvcr90
+ except Exception:
+ _lib_c = ctypes.cdll.msvcr100
+ _lib_ud = iris.config.get_option(
+ 'System', 'udunits2_path',
+ default=os.path.join(sys.prefix, 'share', 'udunits', 'udunits2.dll'))
+ _lib_ud = ctypes.CDLL(_lib_ud, use_errno=True)
+
+#
# load the libc shared library
#
if _lib_c is None:
_lib_c = ctypes.CDLL(ctypes.util.find_library('libc'))
+if _lib_c:
#
# cache common shared library functions
#
@@ -189,6 +203,7 @@ if _lib_ud is None:
default=ctypes.util.find_library('udunits2'))
_lib_ud = ctypes.CDLL(_lib_ud, use_errno=True)
+if _lib_ud:
#
# cache common shared library functions
#
diff --git a/setup.py b/setup.py
index 93d0ab3..eddae8b 100644
--- a/setup.py
+++ b/setup.py
@@ -27,7 +27,7 @@ def find_package_tree(root_path, root_package):
if dir_name in exclude_dirs or not contains_init_file:
dir_names.remove(dir_name)
if dir_names:
- prefix = dir_path.split('/')[root_count:]
+ prefix = dir_path.split(os.path.sep)[root_count:]
packages.extend(['.'.join([root_package] + prefix + [dir_name])
for dir_name in dir_names])
return packages
@@ -258,7 +258,7 @@ setup(
['fileformats/_pyke_rules/*.k?b'] + \
['tests/stock*.npz']
},
- data_files=[('iris', ['CHANGES', 'COPYING', 'COPYING.LESSER'])],
+ data_files=[('Lib\site-packages\iris', ['CHANGES', 'COPYING', 'COPYING.LESSER'])],
tests_require=['nose'],
features={
'unpack': setuptools.Feature(Anyway, most/all examples fail with |
|
Fix for the diff --git a/lib/iris/io/__init__.py b/lib/iris/io/__init__.py
index e4b08b6..5b3ada9 100644
--- a/lib/iris/io/__init__.py
+++ b/lib/iris/io/__init__.py
@@ -182,7 +182,7 @@ def load_files(filenames, callback, constraints=None):
# Create default dict mapping iris format handler to its associated filenames
handler_map = collections.defaultdict(list)
for fn in all_file_paths:
- with open(fn) as fh:
+ with open(fn, 'rb') as fh:
handling_format_spec = iris.fileformats.FORMAT_AGENT.get_spec(os.path.basename(fn), fh)
handler_map[handling_format_spec].append(fn) |
|
@cgohlke, this is exciting! I tried to see if I could get this going with Anaconda on Win32, so I created a new "iris" environment for python 2.7, numpy 1.8 with I discovered that I need to install When I import iris, I get: I'm not sure what this error means. I'm also not sure what I was supposed to do about |
|
You guys absolutely rock! I installed the binaries on my windows machine with a minimum of fuss and A few minor things I ran into in installing it. I needed biggus and pyke Thanks for making my day. Colin [image: Inline image 2] Colin Talbert USGS Fort Collins Science Center USGS North Central Climate Science Center Work schedule: On Tue, Aug 19, 2014 at 7:20 AM, Rich Signell [email protected]
|
|
I put |
|
@cgohlke , does it look like my problem above is indeed a udunits problem, or some issue with ctypes? |
|
Okay, I figured how to get it going with Anaconda. The Thanks @cgohlke !!! |
|
Thanks also to @WardF and @semmerson at Unidata for getting the udunits2 package to build on native Windows that made this all possible! |
|
Glad this is proving useful, and it's interesting to see what it has taken to get this working on Windows with other projects/tools. It's good information that may need to make it in to the cmake/windows documentation, when it appears :). |
|
I've been away for a while so only just seen this fantastic news! 😀 Thank you everyone! |
It would be great if Iris was supported on Windows. Most US Government Employees are on Windows computers -- at the USGS, this is partly because ArcGIS is a Windows application.
The text was updated successfully, but these errors were encountered: