Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let GMT_Put_Vector accept GMT_DATETIME as type #3396

Merged
merged 10 commits into from
May 29, 2020
Merged

Conversation

PaulWessel
Copy link
Member

See this issue for context. This PR allows for calling GMT_Put_Vector with type = GMT_DATETIME and the array as an array of strings (char **) and we convert ISO time-stamps to internal doubles for internal column representation.
I did not add the same for GMT_Get_Vector. Some issues arose:

  1. Because input vector is not numerical, GMT needs to allocate an array to hold the internal representation. This array was allocated by GMT, not the external user. Hence, when GMT quits there will currently be a memory leak. Probably not the end of the world.
  2. Ideally, we should flag that this array is internally allocated, but the GMT_VECTOR structure only as a scalar alloc_mode variable that applies to all columns. This was short-sighted. However, that variable is in the hidden section and not accessible to users, so we can change.
  3. For GMT_Get_Vector, the question is do we need that capability? Is it not easier to convert the double array with internal time to some string-representation in python via library function on your end? If not, then GMT would have to allocate a string array and pass it out, having no mechanism to free it.

I suggest you first try this branch to see if you can get those values in there and test this first. Meanwhile, I will go back to master and let the alloc_mode be an array with one entry per column, just like we have for data type. Then we can free those columns that were internally allocated and ignore the others. OK?

This means we expect an array of text strings (char **) and we convert ISO time-stamps to internal doubles.
@seisman
Copy link
Member

seisman commented May 27, 2020

A working script works with the master branch but crashes using this branch. The error message is:

Error returned from GMT API: GMT_NOT_A_SESSION (29)

@PaulWessel
Copy link
Member Author

Sorry about that, try now.

@PaulWessel
Copy link
Member Author

Merged the master in so now no more memory leak for your case.

src/gmt_api.c Outdated Show resolved Hide resolved
@PaulWessel PaulWessel changed the title WIP Let GMT_Put_Vector accept GMT_DATETIME as type Let GMT_Put_Vector accept GMT_DATETIME as type May 28, 2020
@joa-quim
Copy link
Member

In Julia

julia> D = text_record(["2008-01-01T00:00  5.0"; "2008-01-01T00:01 6.0"]);

julia> plot(D, region="2008-01-01T00:00/2008-01-01T00:01/5/6", f="T", Vd=1)
        psxy  -R2008-01-01T00:00/2008-01-01T00:01/5/6 -JX12c/8c -Baf -BWSen -fT -P -K > C:\TMP\GMTjl_tmp.ps
[Session GMT (0)]: Error returned from GMT API: GMT_READ_ONCE (77)
[Session GMT (0)]: Error returned from GMT API: GMT_READ_ONCE (77)
[Session GMT (0)]: Error returned from GMT API: GMT_READ_ONCE (77)

D is a dataset with text only.

@seisman
Copy link
Member

seisman commented May 29, 2020

GMT command line also supports input data like 123:27:15.120W, but it seems no way to pass these geographic coordinate strings to GMT API, too.

@PaulWessel
Copy link
Member Author

As you (may) know, GMT has several methods for reading its input. Command line is focused on files, but other modes are reading datasets from input matrices, sets of vectors, another dataset, etc. I seems you may be asking about a general method that expects an array of text records. You would then open a VirtualFile with a new modifier, e.g..,

GMT_Open_VirtualFile (API, GMT_IS_DATASET|GMT_VIA_STRINGS, GMT_IS_PLP, GMT_IN, T, input);

and then pass input as the virtual file to the module. Is that what is needed?

As for the original question about Put_Vector I think having the ability to pass datetime strings is reasonable since that is a common time representation, but to extend this to give text representations for longitude etc seems too much. Yet I am not using the API as you are so you tell me if the VirtualFile solution above would solve what you are trying to do. If we had to solve this via GMT_Put_Vector then I can imagine inventing a GMT_GEO_STRING type that is passed just like the GMT_DATETIME is now passed and we could do the conversion the same way.

@seisman
Copy link
Member

seisman commented May 29, 2020

I seems you may be asking about a general method that expects an array of text records. You would then open a VirtualFile with a new modifier, e.g..,

GMT_Open_VirtualFile (API, GMT_IS_DATASET|GMT_VIA_STRINGS, GMT_IS_PLP, GMT_IN, T, input);

and then pass input as the virtual file to the module.

It seems a good solution for complicated input types.

src/gmt_api.c Outdated Show resolved Hide resolved
Yes of course, sorry.

Co-authored-by: Dongdong Tian <[email protected]>
Copy link
Member

@seisman seisman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR now works well for PyGMT.

@joa-quim's test is using another API function?

@PaulWessel
Copy link
Member Author

Yes, I think he wants to feed an array of strings (i.e., a virtual text file) into GMT, but that is not implemented (yet). We will let @joa have the last word on that tomorrow, but unless he is useing GMT_Put_Vector I think this one will then be closed and merged.

@joa
Copy link

joa commented May 29, 2020

I assume this was ment for @joa-quim ,)

@PaulWessel
Copy link
Member Author

Yes, @joa-quim sorry.

@joa-quim
Copy link
Member

Yes, I think he wants to feed an array of strings (i.e., a virtual text file) into GMT

Yes, what I want is to pass a dataset with text and that being recognized.

@PaulWessel
Copy link
Member Author

Perhaps those who are most motivated to see this done should take the lead? I truly have my hands full these days. Because our modules either read via GMT_Read_Data or via GMT_Get_Record there needs to be two slightly different implementations added. Our gmt_ascii_input does all the work of parsing except that it reads the next record from a file pointer, whereas you will want to give it that record from a text array.. We probably need to rearrange gmt_ascii_input so that we dont duplicate all that code just to access an array.
I can tell you there is quite a bit work to get this implemented so I am ruling it out for 6.1.

@seisman
Copy link
Member

seisman commented May 29, 2020

I think it's a very low priority for PyGMT. It's OK for me to merge it as it is.

@PaulWessel
Copy link
Member Author

Yes, it would not go into this branch anyway. @joa OK for you to make a clear feature request for this.

@PaulWessel PaulWessel merged commit ddd829b into master May 29, 2020
@PaulWessel PaulWessel deleted the api-datetime branch May 29, 2020 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants