-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ERROR: GMT: Expects a Matrix for input #396
Comments
OK, I have found that adding " -H " in
|
Sorry, it's not listed in header help but it does exist. Line 93 of
|
Got it, thanks! I have been trying to use Here I can only provide some feedback... julia> include("test_movie.jl")
movie main_script.sh -C15x15x100 -Nglobe -Lf -A -P -T360
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
<horizon> is distance from center to perimeter (<= 90, default 90)
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
coast [WARNING]: ${MOVIE_WIDTH} not a valid number and may not be decoded properly.
coast [ERROR]: Could not decode ${MOVIE_FRAME}, return NaN.
coast [ERROR]: Option -J parsing failure. Correct syntax:
-Jg<lon0>/<lat0>[/<horizon>]/<scale> OR -JG<lon0>/<lat0>[/<horizon>]/<width>
<horizon> is distance from center to perimeter (<= 90, default 90)
<scale> is <1:xxxx> or <radius> (in cm)/<lat>, or use <width> in cm
coast [ERROR]: Offending option -JG${MOVIE_FRAME}/20/${MOVIE_WIDTH}
end [WARNING]: Figure # 1 (../globe_001) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_006) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_005) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_003) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_000) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_004) was registered but no matching PostScript-|+ file found - skipping
end [WARNING]: Figure # 1 (../globe_002) was registered but no matching PostScript-|+ file found - skipping The julia shell has being staying at this point and no further process... So I can't produce the gif as in the document... the content in the directory after finishing these commands drwx------ 2 t t 20480 Jul 15 18:23 globe
-rw-r--r-- 1 t t 112 Jul 15 18:23 main_script.sh
-rw-r--r-- 1 t t 524 Jul 15 12:15 Manifest.toml
-rw-r--r-- 1 t t 52 Jul 15 12:15 Project.toml
-rw-r--r-- 1 t t 300 Jul 15 18:22 test_movie.jl In globe directory, there are a lot of sh file, but the movie_params_* files start from Using sh files seems too complex to me. I think, for me, the best procedure might be this: gmtbegin()
gmtfig(:inmemory) # which means I want to store it in memory and then use it as a background picture
# some code balabal
tmp = gmtend() # the background tmp structure data is assigned to tmp
#===========================#
allframes = []
for ii in 1:360 # or whatever frames
gmtfig(:inmemory) # or maybe save to png
gmtbegin() # or gmtbegin(tmp) ?
this_background = copy(tmp) # start from the background
# some other code
thisframe = gmtend() # again store the data in memory when gmtend
push!(allframes, thisframe,)
end
#=============================#
movie(allframes,"filename.mp4")
# or maybe this can move to each frame: https://juliaio.github.io/VideoIO.jl/stable/writing/#Iterative-Encoding-1
It's only my personal thinking. The main feature is to provide a way to store the temporary figure data for later manipulation. Another thing is to provide function that user can define crop region (maybe the canvas, but I can't check it as the movie example above hasn't completed....). |
OK, I see already a problem. The movie examples in Unix use, e.g.
The idea with Start by using Vd=2 and run the command in the bash shell like I said before. This avoids hanging the Julia REPL in case things go wrong. You will probably also need to edit |
Installed GMT & Julia in my Centos8 VM. Fixed the names and slash issues (in master). And both of these worked
and running with Vd=2 plus running this on bash
|
It works also in Archlinux (@v1.4) pkg> activate .
Activating environment at `~/test_GMT/Project.toml`
julia> include("test_movie.jl")
movie main_script.sh -C15x15x100 -Nglobe -Lf -A -P -T360
movie [NOTICE]: Running: gm convert -delay 4 -loop 1 +dither globe/globe_*.png globe.gif
118.279850 seconds (3.60 M allocations: 169.211 MiB, 0.03% gc time)
julia> But the convertion from the figures to the gif consumes a lot of memory It might related to that I also don't have a clear idea how to input complex data for each frame when all the plotting are done in sh.... All axes involve contour interpolation of different data (stored in sqlite data file ~120GB). And the interpolation is very slow. Even I use the HPC at my laboratory the plotting can last for days...
So I run three julia at the same time on the HPC 😄 For converting from pngs to mp4, I use Iterative-Encoding which could save a lot of memory... Here is the final output (after hours of plotting) |
That example is using gif but you could change the format to mpeg by using So you never tried to try to reuse the mesh in contour to do further interpolations. You can raise this issue in the forum. |
I think that the procedure of current movie function is very similar to my code, plot each figure and then combine them together. In general, I prefer to do all things in one language (Julia now). So I may try the function later... I think I ran some example code related to interpolation (triangulate) but I haven't tried to implement something based on it (the output data is complex for me 🤣 ) |
Since the document in master branch has been updated ( |
What the movie program has is the facilities to calculate perspective, viewpoint and so on that allow doing movies like those in the animations much easier. But I agree that it should be possible to reason Julia only ... but that TTFP delay is deadly. |
Nothing really complicated. You have two files. One with the data triplets (x,y,z) and another with 3 indices per row of each traingle. The indices in second file refer to line numbers in first file. |
I have tested data preparationusing GMT
using BenchmarkTools
R = :g
G = gmt("grdmath -R$R -I0.5 X Y MUL =");
xyz = grd2xyz(G)
t1 = triangulate(xyz, )
fn_tg = "triangulate.txt"
gmtwrite(fn_tg, t1)
fn_ds = "xyz.txt"
gmtwrite(fn_ds, xyz)
cb_jet = makecpt(color=:jet, range=(-4e4, 4e4, 1e3))
some benchmark
julia> @benchmark t1 = triangulate(xyz, )
BenchmarkTools.Trial:
memory estimate: 17.83 MiB
allocs estimate: 240
--------------
minimum time: 229.609 ms (0.00% GC)
median time: 239.570 ms (0.00% GC)
mean time: 244.705 ms (2.01% GC)
maximum time: 289.117 ms (16.21% GC)
--------------
samples: 21
evals/sample: 1
julia> @benchmark xyz_read = gmtread(fn_ds)
BenchmarkTools.Trial:
memory estimate: 5.97 MiB
allocs estimate: 205
--------------
minimum time: 114.579 ms (0.00% GC)
median time: 116.363 ms (0.00% GC)
mean time: 116.408 ms (0.03% GC)
maximum time: 119.399 ms (0.22% GC)
--------------
samples: 43
evals/sample: 1 So the main part of time might be used to case 1 no mesh information: 374.942 ms @benchmark contour(xyz,
pen="0.15p,+c",
# mesh="0.01p",
# index=t1,
# index=fn_tg,
annot=(labels = (font = 1, prefix = "test ",),),
color=cb_jet,
# Vd = 1,
region=R,
# show=true,
# fmt=:png
)
BenchmarkTools.Trial:
memory estimate: 12.19 MiB
allocs estimate: 3317
--------------
minimum time: 360.554 ms (0.16% GC)
median time: 374.942 ms (0.00% GC)
mean time: 376.999 ms (0.03% GC)
maximum time: 422.016 ms (0.11% GC)
--------------
samples: 14
evals/sample: 1 case 2 mesh information in file: 318.378 ms @benchmark contour(xyz,
pen="0.15p,+c",
# mesh="0.01p",
# index=t1,
index=fn_tg,
annot=(labels = (font = 1, prefix = "test ",),),
color=cb_jet,
# Vd = 1,
region=R,
# show=true,
# fmt=:png
)
BenchmarkTools.Trial:
memory estimate: 12.19 MiB
allocs estimate: 3321
--------------
minimum time: 312.193 ms (0.14% GC)
median time: 318.378 ms (0.00% GC)
mean time: 321.183 ms (0.03% GC)
maximum time: 339.904 ms (0.00% GC)
--------------
samples: 16
evals/sample: 1 case 3 mesh information in Mx3 array: Segmentation fault @time contour(xyz,
pen="0.15p,+c",
# mesh="0.01p",
index=t1[1].data,
# index=fn_tg,
annot=(labels = (font = 1, prefix = "MLAT ",),),
# fill = true,
color=cb_jet,
# Vd = 1,
region=R,
# show=true,
# fmt=:png
)
signal (11): Segmentation fault
in expression starting at REPL[170]:1
gmt_strtok at /usr/lib/libgmt.so (unknown line)
GMT_Create_Options at /usr/lib/libgmt.so (unknown line)
unknown function (ip: 0x353435322f302e35)
Allocations: 98529301 (Pool: 98521771; Big: 7530); GC: 108
[1] 1111131 segmentation fault (core dumped) julia It seems that if the mesh information is provided, the contour would be some speedup (maybe the time would be spent in gmtread instead of triangulate). If the Mx3 array (as said at doc) could be passed directly, the time could be decreased more(?), but when passing the array to contour, a |
OK, will investigate the crash. The |
Thank you~ I think I almost become a bug trigger 😂 |
I have that gift myself as well. |
One big reason for slowness is the file reading. GMT is slow reading files because it does check an a lot of possible cases. Speeding it up has been one thing I push but it's still waiting.
The julia version is supposed to work too but that's something I have not tested.
which is almost half the time I get with the |
And after fixing it
Will commit to master this afternoon |
You should now be able to do
but dumb of me. I think you could already have written
|
Thank you!!! BenchmarkTools.Trial:
memory estimate: 24.06 MiB
allocs estimate: 3452
--------------
minimum time: 113.849 ms (0.00% GC)
median time: 122.205 ms (0.31% GC)
mean time: 124.151 ms (0.18% GC)
maximum time: 148.674 ms (0.27% GC)
--------------
samples: 41
evals/sample: 1 About three times faster (374.942/122.205), 👍 |
Which ones are you not able to plot with GMT? (I know not everything is possible) |
One of the things I have tried but haven't got any results is this one: |
The plot looks trivial (plot3d) but the not sure if the field lines could be computed with |
Maybe start a new issue with this, and probably in GMT itself since the plotting should be easy as long as we have all necessary data. |
The data can be produced by some other program (a lot of simulation code in ionosphere need to use grids based on geomagnetic field). I will try to prepare an example (with data and code in Julia) and open a new issue later. Thank you~ |
Have you looked at |
I have just checked it (and IGRF and CM4/CM5 and Dst and F10.7...) Actually all these things are needed in my research! But I write my own Julia program to download, read, manage them.... For this function, I think it's very useful (but... I really want to do all the things in one language 😂 ) |
Nothing stops you to. See the |
OK, no problem I will check them! Still the GMT.jl (gmt) can surprise me when I though I have enough feelings about it!! What's more, it's completely free!!! Thank you~ |
It seems that users can't use
makecpt
betweengmtbegin()
andgmtend()
ERROR: GMT: Expects a Matrix for input
works fine
This might imply that one must do all
makecpt
before starting a session?But in my case, I usually plot several axes in one figure (session), and it is necessary to use different cpts, especially when making movies.
I found that the last line of makecpt begins with
global current_cpt
. The gmt document says"In classic mode we write the CMT to standard output, while under modern mode we simply save the CPT as the current session CPT (but see -H). " ref. However, I haven't found the corresponding H parameter in
GMT.jl
'smakecpt
...Is this the problem?
The text was updated successfully, but these errors were encountered: