gmtread fails

[hearnsj]

I really am not sure this is the correct place to file this issue. So apologies if GMT.jl is not at fault!
Environment - Julia 1.1.0 Windows 10 OS
I installed GMT version 6.0.0 from https://w3.ualg.pt/~jluis/mirone/downloads/gmt.html
I run :
I = gmtread("world.tif", img=true);

then it crashes

[Command: julia C:\Users\hearn\Documents\Satellite\Crawley\gmtread.jl]
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0xb30e4c0 -- getindex at .\array.jl:730 [inlined]
macro expansion at .\multidimensional.jl:671 [inlined]
macro expansion at .\cartesian.jl:64 [inlined]
macro expansion at .\multidimensional.jl:666 [inlined]
_unsafe_getindex! at .\multidimensional.jl:662 [inlined]
_unsafe_getindex at .\multidimensional.jl:656
in expression starting at C:\Users\hearn\Documents\Satellite\Crawley\gmtread.jl:3
macro expansion at .\multidimensional.jl:669 [inlined]
macro expansion at .\cartesian.jl:64 [inlined]
macro expansion at .\multidimensional.jl:666 [inlined]
_unsafe_getindex! at .\multidimensional.jl:662 [inlined]
_unsafe_getindex at .\multidimensional.jl:656
getindex at .\multidimensional.jl:642 [inlined]
getindex at .\abstractarray.jl:927
unknown function (ip: 000000000B30E1CA)
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1864
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
get_image at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:519
GMTJL_Get_Object at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:850
gmt at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:264
#gmtread#46 at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmtreadwrite.jl:120
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1864
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219 [inlined]
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1571 [inlined]
jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:56
#gmtread at .\none:0
unknown function (ip: 000000000B2E104A)
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1864
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
do_call at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:323
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:411
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:362 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:755
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:885
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 00000000062F2C6F)
unknown function (ip: 0000000000000004)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:713
jl_parse_eval_all at /home/Administrator/buildbot/worker/package_win64/build/src\ast.c:883
jl_load at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:826 [inlined]
jl_load at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:833
include at .\boot.jl:326 [inlined]
include_relative at .\loading.jl:1038
include at .\sysimg.jl:29
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
exec_options at .\client.jl:267
_start at .\client.jl:436
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1571 [inlined]
true_main at /home/Administrator/buildbot/worker/package_win64/build/ui\repl.c:96
wmain at /home/Administrator/buildbot/worker/package_win64/build/ui\repl.c:217
__tmainCRTStartup at /usr/src/debug/mingw64-x86_64-runtime-5.0.3-1/crt\crtexe.c:329
mainCRTStartup at /usr/src/debug/mingw64-x86_64-runtime-5.0.3-1/crt\crtexe.c:212
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 12000481 (Pool: 11997593; Big: 2888); GC: 26
[Finished in 9.899s]

[joa-quim]

Hi. Yes, this is the right place for reporting. Though the problem is apparently occurring inside the gmt.dll lib itself. I mean the GMT program, not the GMT.jl wrapper.

Another thing is that I'll need an example file to debug this crash.

You can also do this other test. On the Windows command line (not Julia) please run this and tell me what it prints (assuming that you are running it from the same dir as the tif file)

grdinfo world.tif

[hearnsj]

No problems
PS C:\Users\hearn\Documents\Satellite\Crawley> grdinfo.exe world.tif
world.tif: Title: Grid imported via GDAL
world.tif: Command:
world.tif: Remark:
world.tif: Pixel node registration used [Cartesian grid]
world.tif: Grid file format: gd = Import/export through GDAL
world.tif: x_min: -180 x_max: 180 x_inc: 0.17578125 name: n_columns: 2048
world.tif: y_min: -90 y_max: 90 y_inc: 0.17578125 name: n_rows: 1024
world.tif: z_min: 0 z_max: 255 name: z
world.tif: scale_factor: 1 add_offset: 0
+proj=longlat +datum=WGS84 +no_defs
PS C:\Users\hearn\Documents\Satellite\Crawley>

[joa-quim]

Hmm, so all looks fine in that side. Now I really need a file to debug. My quick test case that I actually run before posting in the forum worked perfectly fine.

[hearnsj]

Thankyou. The file is world.tif
I downloaded this as it is used as an example in one of the packages I used yesterday.

Please rename world.jpg as world.tif - github only allows certain file types to be uploaded

[hearnsj]

This is not going very well! Looks like Github opened the file to display it.
I am really sorry that I Cannot tell you the URL for the original file. I thought it was a common example for testing.

[joa-quim]

You can try to zip it and post the compressed file.
The tests have a test like this. Does it work for you? It's just a plain tif file (no geotif) but it uses most of the same code flow.

		gmtwrite("lixo.tif", rand(UInt8,32,32,3), driver=:GTiff)
		I = gmtread("lixo.tif", img=true);

[joa-quim]

OK, I could download the image (renamed to aa.tif) ... and works fine here

julia> I = gmtread("aa.tif", img=true)
GMT.GMTimage("+proj=longlat +datum=WGS84 +no_defs ", "", [-180.0, 180.0, -90.0, 90.0, NaN, NaN], [0.175781, 0.175781], 1, NaN, "", "", "", "", [-180.0, -179.824, -179.648, -179.472, -179.297, -179.121, -178.945, -178.769, -178.593, -178.417  …  178.417, 178.593, 178.769, 178.945, 179.121, 179.297, 179.472, 179.648, 179.824, 180.0], [-90.0, -89.824, -89.6481, -89.4721, -89.2962, -89.1202, -88.9443, -88.7683, -88.5924, -88.4164  …  88.4164, 88.5924, 88.7683, 88.9443, 89.1202, 89.2962, 89.4721, 89.6481, 89.824, 90.0], UInt8[0x0c 0x0b … 0x0c 0x0b; 0x0b 0x44 … 0x0b 0x44; … ; 0x44 0x0c … 0x44 0x0c; 0x0c 0x0b … 0x0c 0x0b]

UInt8[0x44 0xff … 0x27 0x0c; 0x0c 0xe7 … 0x4f 0x0b; … ; 0xe0 0x96 … 0x0b 0x44; 0x99 0xb8 … 0x44 0x0c]

UInt8[0x0b 0x29 … 0xc5 0xb7; 0x44 0x6c … 0xb9 0xcd; … ; 0x68 0x0b … 0xcd 0xbd; 0x4b 0x44 … 0xc3 0xb3], "", "", "", Int32[0, 0, 0], 0, UInt8[0x00 0x00 … 0x00 0x00; 0x00 0x00 … 0x00 0x00; … ; 0x00 0x00 … 0x00 0x00; 0x00 0x00 … 0x00 0x00], "BRPa")

I find strange that in your stack frame I cannot find any reference to the gmt_w64.dll which is where the crash should be happening. What happens if you run the tests? (] test GMT)

[hearnsj]

Thankyou. I think GMT did not install properly.
I had to add the GMT directory to my PATH. I suspect I have to add the library also.
Sorry if this was a fault by me.

[hearnsj]

Aha! pkg> test GMT fails
ERROR: LoadError: BoundsError: attempt to access 0-element Array{Float32,1} at index [4290772993]
Stacktrace:
[1] getindex at .\array.jl:729 [inlined]
[2] get_grid(::Ptr{Nothing}, ::Ptr{Nothing}) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:428
[3] GMTJL_Get_Object(::Ptr{Nothing}, ::GMT.GMT_RESOURCE) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:842
[4] gmt(::String) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:264
[5] top-level scope at C:\Users\hearn.julia\packages\GMT\v32Hn\test\runtests.jl:17
[6] include at .\boot.jl:326 [inlined]
[7] include_relative(::Module, ::String) at .\loading.jl:1038
[8] include(::Module, ::String) at .\sysimg.jl:29
[9] include(::String) at .\client.jl:403
[10] top-level scope at none:0
in expression starting at C:\Users\hearn.julia\packages\GMT\v32Hn\test\runtests.jl:13
ERROR: Package GMT errored during testing

[hearnsj]

This fails also gmtwrite("lixo.tif", rand(UInt8,32,32,3), driver=:GTiff)

There must be something wrong in the way I installed the Windows gmt6 package.
I will look at this later - I have to travel home now.
Sorry again for flagging this up as an error - when it is a gmt6 / Windows problem.

[joa-quim]

Had to drive home too.

So all points out to a lack of communication between GMT.jl and the GMT lib and that is not your fault but I'm having difficulties imagining what is causing it.

Having to manually had the GMT bin directory to your PATH is a sad limitation of the NSIS program that is used to create the GMT installer but that should just be a minor annoyance and cannot explain the mystery since your test on Windows cmd (or was ir PowerShell?) worked fine.

Did you install GMT in the default location (c:\programs\gmt6)? I mean, you didn't install it in a directory with spaces in the name, right?

I also guess that this works fine for you

julia> run(`gmt --version`)
6.0.0
Process(`gmt --version`, ProcessExited(0))

but not

gmt("psxy -")

[hearnsj]

Works for me:

julia> run(gmt --version)
6.0.0_5747e39
Process(gmt --version, ProcessExited(0))

[hearnsj]

gmt fails

ulia> gmt("psxy -")
psxy [core] 6.0.0_5747e39 [64-bit] [MP] - Plot lines, polygons, and symbols on maps

usage: psxy [

] -J -R///[//][+r] [-A[m|p|x|y]]
[-B] [-C] [-D/] [-E[x|y|X|Y][+a][+c[l|f]][+n][+p][+w]] [-F] [-G]
[-I] [-K] [-L[+b|d|D][+xl|r|x0][+yb|t|y0][+p]] [-N[c|r]] [-O] [-P]
[-S[][[unit]]] [-T] [-U[][+c][+j][+o/]] [-V[]] [-W[][]]
[-X[a|c|r][]] [-Y[a|c|r][]] [-a=[,...]]
[-bi[][t][w][+l|b]] [-di] [-e[~]]
[-f[i|o]] [-g[a]x|y|d|X|Y|D|[]z[-|+][]] [-h[i|o][][+c][+d][+r][+t<title>]]
[-i[+l][+s][+o][,...]] [-p[x|y|z][/[/]][+w/[/][+v/]]
[-t] [-:[i|o]] [--PAR=]

ERROR: Something went wrong when calling the module. GMT error number = 1
Stacktrace:
[1] error(::String, ::Int32) at .\error.jl:42
[2] gmt(::String) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:247
[3] top-level scope at none:0

[hearnsj]

GMT is installed in c:\programs\gmt6

[joa-quim]

No, the gmt("psxy -") run fine. By printing the the usage message it means it was able to communicate with the library. The trailing error is something I never bother to try to fix.

When you run the tests it seemed to me that it failed in the first one. Can you please repeat it on the REPL. Just this

G = gmt("grdmath -R-2/2/-2/2 -I0.5 X Y MUL");

Confess that I'm confused. Tested the same GMT binaries in a different machine and it worked fine there too.

[hearnsj]

No problems!

julia> G = gmt("grdmath -R-2/2/-2/2 -I0.5 X Y MUL");
ERROR: BoundsError: attempt to access 0-element Array{Float32,1} at index [4290772993]
Stacktrace:
[1] getindex at .\array.jl:729 [inlined]
[2] get_grid(::Ptr{Nothing}, ::Ptr{Nothing}) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:428
[3] GMTJL_Get_Object(::Ptr{Nothing}, ::GMT.GMT_RESOURCE) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:842
[4] gmt(::String) at C:\Users\hearn.julia\packages\GMT\v32Hn\src\gmt_main.jl:264
[5] top-level scope at none:0

[hearnsj]

get_grid is being called with get_grid(::Ptr{Nothing}, ::Ptr{Nothing})

[hearnsj]

I did a pkg> remove GMT
Then pkg> add GMT #master
A build GMT then gives lots of warnings about Method definitions being overwritten
Then test GMT

┌ Warning: Unknown units request (lolo) Ignoring it
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\common_options.jl:1042
filter1d -Fm15
gmtconnect [ERROR]: Trouble: Pass 1 found 1 while pass 2 found 2 connections!
grdimage [COMPATIBILITY]: Option -D is deprecated; images are detected automatically
┌ Warning: Usage error, both difference and trend were required. Ignoring the trend request.
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\grdtrend.jl:59
grdimage [COMPATIBILITY]: Option -D is deprecated; images are detected automatically
┌ Warning: Justification code provided (bottomrigh) is not valid. Defaulting to TopRight
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\common_options.jl:1857
psbasemap [ERROR]: 1 external clip operations were not terminated!
psconvert [ERROR]: Error opening HKLM key
'gswin64c' is not recognized as an internal or external command,
operable program or batch file.
psconvert [ERROR]: Failed to parse response to GhostScript version query [n = -1 0 0].
ERROR: LoadError: Something went wrong when calling the module. GMT error number = 78
Stacktrace:
[1] error(::String, ::Int32) at .\error.jl:42
[2] gmt(::String) at C:\Users\hearn.julia\packages\GMT\vzfwQ\src\gmt_main.jl:247
[3] #psconvert#192(::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol},NamedTuple{(:adjust, :fmt),Tuple{Bool,String}}}, ::Function, ::String, ::Array{Any,1}) at C:\Users\hearn.julia\packages\GMT\vzfwQ\src\psconvert.jl:137
[4] (::getfield(GMT, Symbol("#kw##psconvert")))(::NamedTuple{(:adjust, :fmt),Tuple{Bool,String}}, ::typeof(psconvert), ::String) at .\none:0
[5] top-level scope at C:\Users\hearn.julia\packages\GMT\vzfwQ\test\runtests.jl:490
[6] include at .\boot.jl:326 [inlined]
[7] include_relative(::Module, ::String) at .\loading.jl:1038
[8] include(::Module, ::String) at .\sysimg.jl:29
[9] include(::String) at .\client.jl:403
[10] top-level scope at none:0
in expression starting at C:\Users\hearn.julia\packages\GMT\vzfwQ\test\runtests.jl:13
ERROR: Package GMT errored during testing

[hearnsj]

Arggh. Sorry for wasting your time. I did not have Ghostscript installed.
I still get these errors when pkg> test GMT

┌ Warning: Unknown units request (lolo) Ignoring it
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\common_options.jl:1042
filter1d -Fm15
gmtconnect [ERROR]: Trouble: Pass 1 found 1 while pass 2 found 2 connections!
grdimage [COMPATIBILITY]: Option -D is deprecated; images are detected automatically
┌ Warning: Usage error, both difference and trend were required. Ignoring the trend request.
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\grdtrend.jl:59
grdimage [COMPATIBILITY]: Option -D is deprecated; images are detected automatically
┌ Warning: Justification code provided (bottomrigh) is not valid. Defaulting to TopRight
└ @ GMT C:\Users\hearn.julia\packages\GMT\vzfwQ\src\common_options.jl:1857
psbasemap [ERROR]: 1 external clip operations were not terminated!
pscoast [ERROR]: Unable to find or open the Digital Chart of the World for GMT
pscoast [ERROR]: Perhaps you did not install this file in DIR_DCW, the shared dir, or the user dir?
pscoast [ERROR]: Use your package manager to install package dcw-gmt.
pscoast [ERROR]: Alternatively, get the latest dcw-gmt-.tar.gz or dcw-gmt-.zip from the ftp:https://ftp.soest.hawaii.edu/gmt.
pscoast [ERROR]: Unable to find or open the Digital Chart of the World for GMT
pscoast [ERROR]: Perhaps you did not install this file in DIR_DCW, the shared dir, or the user dir?
pscoast [ERROR]: Use your package manager to install package dcw-gmt.
pscoast [ERROR]: Alternatively, get the latest dcw-gmt-.tar.gz or dcw-gmt-.zip from the ftp:https://ftp.soest.hawaii.edu/gmt.
psrose -I1 -A20
[Session GMT (16)]: Error returned from GMT API: GMT_WRITTEN_ONCE (84)
[Session GMT (16)]: Error returned from GMT API: GMT_WRITTEN_ONCE (84)
[Session GMT (16)]: Error returned from GMT API: GMT_WRITTEN_ONCE (84)
[Session GMT (16)]: Error returned from GMT API: GMT_OBJECT_NOT_FOUND (59)
[Session GMT (17)]: Error returned from GMT API: GMT_BAD_GEOMETRY (6)
[Session GMT (17)]: Error returned from GMT API: GMT_BAD_GEOMETRY (6)
[Session GMT (17)]: Error returned from GMT API: GMT_BAD_GEOMETRY (6)
Testing GMT tests passed

[hearnsj]

Clearly I have to also download dcw-gmt and gshhg-gmt

[hearnsj]

The gmtread is working now. I have not unpacked and installed dcw-gmt or gshhg-gmt
Please leave the issue open till I complete that.
And thankyou for looking at this - even though I Was an idiot.
Maybe I should submit a change to the README - install Ghostscript you idiot

[joa-quim]

Fine. Don't understand why it was not working before (the same tests are run by Travis CI whenever I make a commit) but good that it's working now.
The dcw-gmt and gshhg-gmt are installed by the GMT installer and should not need any further download. Regarding the dcw, there used to an issue with the installer that did not select it by default and users often missed to do it manually. But even that should have been fixed in the latest installer. Anyway, you can check if you have them by looking into ...\gmt6\share for the subdirs coast and dcw.

About the ghostscritp ... the README warns that it's needed.

Maybe you could post in the forum that things are solved and it works?

[hearnsj]

Thankyou for the patience and all the help. Closed!