How to avoid precision loss?

[amarux]

Dear all,

I am new using GMT in Julia and I am asking for help about the precision loss if grdmath module is used for a given operation.

If for example, I used 8 bytes float array in Julia and make a GMT grid with mat2grid, and then I computed the spatial derivatives using DDX and DDY, I can introduce rounding errors.

I am based on the grdmath documentation (6.0.0) that indicated: Regardless of the precision of the input data, GMT programs that create grid files will internally hold the grids in 4-byte floating point arrays ... Data with higher precision (i.e., double precision values) will lose that precision once GMT operates on the grid or writes out new grids. To limit loss of precision when processing data you should always consider normalizing the data prior to processing.

So, I am wondering if you could help with an example to understand how to avoid loss of precision.

Thank you!

[joa-quim]

Can you please post this question in the GMT forum? This is not Julia specific and I have not much time to look/experiment right now.

…

Sent from my iDedo

No dia 01/08/2020, às 21:08, Amaru Marquez ***@***.***> escreveu:  Dear all, I am new using GMT in Julia and I am asking for help about the precision loss if grdmath module is used for a given operation. If for example, I used 8 bytes float array in Julia and make a GMT grid with mat2grid, and then I computed the spatial derivatives using DDX and DDY, I can introduce rounding errors. I am based on the grdmath documentation (6.0.0) that indicated: Regardless of the precision of the input data, GMT programs that create grid files will internally hold the grids in 4-byte floating point arrays ... Data with higher precision (i.e., double precision values) will lose that precision once GMT operates on the grid or writes out new grids. To limit loss of precision when processing data you should always consider normalizing the data prior to processing. So, I am wondering if you could help with an example to understand how to avoid loss of precision. Thank you! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[joa-quim]

Anyway, the normalization should be obtained by, if X is your grid, with x_min & x_max as extrema,

Xnorm = (X - x_min) ./ (x_max - x_min)

[amarux]

Thank you very much!