Update how CPT hinges are handled #2416

PaulWessel · 2020-01-07T03:38:18Z

Separate between soft and hard hinges. Hard hinges are data driven like those for bathymetry or magnetics, while soft hinges are ignored unless +h[hinge] is given. Explain this better in the CPT documentation.

Description of proposed changes

Fix the etopo1 CPT to have a hinge at 0, not some tiny nonzero number
Fix the srtm CPT to also have hinge at 0 but stay blue for a short while afterwards
Add optional modifier +h[hinge] to turn soft hinges into hard hinges
Remove the value for the HINGE given in the CPTs since they should (now) all be zero in the normalized -1/1 range
Remove change HINGE to HARD_HINGE for the bathy.topo and mag CPTs
Change remaining HINGE to SOFT_HINGE
Add triangles on GMT_App_M_1?.ps plots to indicate which CPTs have hard or soft hinges

Separate between soft and hard hinges. Hard hinges are data driven like those for bathymetry or magnetics, while soft hinges are ignored unless +h[hinge] is given.

PaulWessel · 2020-01-07T04:10:04Z

Notes to self: the normalized hinge value is always 0. When we internally normalize a CPT for 0-1 in gmt_sample_cpt the normalized hinge will be at 0.5. The user-supplied hinge will be in user coordinates [0].

Explain soft and hard hinges

PaulWessel · 2020-01-08T08:08:23Z

I believe this PR is ready for review. I have improved the documentation pertaining to CPTs by explaining the new soft and hard hinges, linking to the actual CPT plots in the appendix, adding triangles to indicate which ones have soft or hard hinges, and introduce two include rst_files that programs that produce (create_cpt.rst_) or use with grids (use_cpt_grd.rst_) includes to spell out what -C does. I hope to later do one for the remaining tools that read CPTs (plot, etc) but these are still too specific to the module. In the process I had to update a few PS since we basically reverted the hard hinges from many CPTS such as polar, split, etc. For instance, this demonstrates that polar no longer has a hard hinge:

gmt makecpt -Cpolar -T-1/3
-1	blue	1	white
1	white	3	red

but it can be activated by forcing a hinge:

gmt makecpt -Cpolar+h -T-1/3
-1	blue	0	white
0	white	3	red

and instead putting the hinge at 2:

gmt makecpt -Cpolar+h2 -T-1/3
-1	blue	2	white
0	white	1	red

remkos · 2020-01-08T09:30:19Z

There are a few things I do not understand and may need to be clarified:

Is "normalised units" the same as "units in the CPT file"?
Is in +h in normalised units, in the units of the CPT file, or scaled units? This is not at all indicated.
Why are CPTs with a (soft) hinge said to be normalised to -1/1 whereas others (if they still exist) normalised to 0/1? Is this information relevant to the user? Isn't that an internal issue?
It is not evident that in case you do not make discrete intervals out of a CPT that is continuous, the hole matter about hinges is totally irrelevant. In that case every value in the CPT file is a hinge.
What are +U and +u supposed to do? It is said to be a conversion from meters to "another unit". To Watts maybe? How do I specify that unit? And the CPTs are dimensionless; they are not in meters. I really do not get the point of this.

Personally, I think this is making things way too complicated with those hinges. I think users should not be allowed to set the intervals on output CPT files, i.e. changing from continuous to discrete or vice versa; then the whole issue becomes moot. If a CPT has, e.g., 16 discrete colours, like panoply then it should stay like that. It's how the colour map was designed, with every value in the CPT file being a "hinge". Yes, the user can alter the interval to their wishes, but not change whether it is discrete or continuous.

Conversely, a colour map like polar is meant to be continuous, with a hinge in the middle, and that should stay like that. If there is a real need for the polar-like colour map with say 16 discrete intervals, it should be made. Yes, that would extend the number of colour maps provided, e.g., adding polar16, but that be it.

True, the current implementation provides a lot of possibilities, but at the detriment of clarity, and opening the door to a lot of user-misuse.

PaulWessel · 2020-01-08T19:39:05Z

Thanks for the feedback. Here are some background before I address your specific concerns: In GMT 6 we added HINGE to several CPTs and in the process introduced backwards incompatibilities. The discussion in #2412 captures those problems. We realized there is a difference between what I now call a hard hinge (e.g., sea-level, specified by a feature of the intended data) versus a soft hinge (i.e., the white point in polar, which is not tied to a data feature per se). The 6.0 broke the ability to make a polar-like CPT that goes from 0 to 100 with a white point at 50, for instance. The solution was to restore backwards compatibility but recognize this second group of hinge-worthy CPTs by flagging them as having soft hinges that the user can activate if they so desire via +h[hinge] (e.g., when not specifying a symmetric range about zero or not selecting zero as their data hinge). OK, some answers:

Is "normalised units" the same as "units in the CPT file"?

Yes, and your feedback makes me realize that there is no point exposing the user to the issue of normalized z-values and the -1/+1 versus 0/1. It is irrelevant to them.

Is in +h in normalised units, in the units of the CPT file, or scaled units? This is not at all indicated.

The Cookbook makes this distinction I think by contrasting the normalized z' values with the users data values. So the hinge is in user data values. I will make this more clear in the documentation for -C.

Why are CPTs with a (soft) hinge said to be normalised to -1/1 whereas others (if they still exist) normalised to 0/1? Is this information relevant to the user? Isn't that an internal issue?

Yes, internal issue and I will remove that. It may be placed as comments for gurus who need to make new master CPTs in the future.

It is not evident that in case you do not make discrete intervals out of a CPT that is continuous, the hole matter about hinges is totally irrelevant. In that case every value in the CPT file is a hinge.

Yes, discrete CPTs have hinges everywhere; this distinction will be clarified. Nothing new here though. What we should add is that selecting discrete nodes may introduce aliasing of the original color table; I think we already warn when users create some of these.

What are +U and +u supposed to do? It is said to be a conversion from meters to "another unit". To Watts maybe? How do I specify that unit? And the CPTs are dimensionless; they are not in meters. I really do not get the point of this.

These modifiers goes back to at least 5.2 or before. Their documentation has been sketchy and only present in the cookbook. The initial reason for these modifiers was to have the ability to use a CPT scaled to meters (e.g., for a topography grid) but be able to use it with psscale to make a km CPT without having to make a second CPT just for that purpose. Furthermore, the same modifiers have also been available to grids from that time as well, converting the x and y values in Cartesian grids between different distance units (e|k|m|M|n|u|s). So I need to emphasize distance unit more clearly. Thus, +U|u is a shared and long-lived mechanism for both CPTs and grids.

I will implement the changes I mentioned next.

PaulWessel · 2020-01-08T20:03:23Z

I have update the documentation discussion of CPTs in the cookbook and the two new include rst_ files regarding hinges.

seisman · 2020-01-08T21:11:51Z

See the script below. The output seems incorrect.

gmt begin map png
gmt makecpt -Cpolar -T100/300/20
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar -T100/300/20 -Z
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h -T100/300/20
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h -T100/300/20 -Z
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h200 -T100/300/20
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h200 -T100/300/20 -Z
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h150 -T100/300/20
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt makecpt -Cpolar+h150 -T100/300/20 -Z
gmt colorbar -Baf -Dx0/0+w10c+h -Y2c

gmt end show

PaulWessel · 2020-01-08T21:19:50Z

Yes, I see the discrete output tables are wrong - it looks fine with most of the continuous. I have reverted this to WIP and will work on this a bit later.

PaulWessel · 2020-01-08T23:26:27Z

Unrelated to the script above but: Should we impose that all the GMT master CPTs be continuous? It is really treacherous to have users create discrete tables by resampling discrete CPTs, since the number of slices in the master vs the ones requested cause really problems.

joa-quim · 2020-01-08T23:36:26Z

At least not the mag.cpt. It's meant to be discrete.

PaulWessel · 2020-01-08T23:50:45Z

But would you want Lester to run
gmt makecpt -Cmag -T300/700/100
on it?

joa-quim · 2020-01-08T23:58:07Z

If it's going to be used in a FAA map maybe that's fine. What I want is to continue to use gmt grdimge mag.grd -Cmag with no need to re-create a discrete CPT everytime.

PaulWessel · 2020-01-08T23:59:14Z

I guess what I think we would like to avoid is bad user decisions in making discrete tables that badly alias the original tables, especially if they are discrete to start with.

joa-quim · 2020-01-09T00:01:59Z

You can bug them with a message saying "Resampling a discrete CPT is really a BAD idea".

remkos · 2020-01-09T08:39:29Z

I guess what I think we would like to avoid is bad user decisions in making discrete tables that badly alias the original tables, especially if they are discrete to start with.

That comes back to what I said before. Maybe we should not even allow to change the intervals from discrete CPTs. Just like mag.cpt, they are designed a certain way, and users should not be allowed to meddle with it.

I'm also not in favour of making discrete CPTs out of continuous ones, but that is way less dangerous.

joa-quim · 2020-01-17T20:47:30Z

I agree with rules 1-3 (that's what I've been trying to say)
I'm very lost with hinges. Are hard & soft hinges different or not. hinge = 0 is not a distinction

PaulWessel · 2020-01-17T20:53:46Z

I am also OK with rules 1-3. The issue I have is with hard versus soft hinges. A hard hinge means what it says, it is inflexible because it is tied to a physical feature (shoreline). A soft hinge is independent of data and instead reflects a point of rapid color change in the CPT that could be tied to a data value hinge, and the user can do so via +hz. I am fine with the idea of using polar to show deviation in seismic velocities from 6.0 km/s, for instance, via gmt makecpt -Cpolar+h6 -T4/8. But for a hard hinge I don't think gmt makecpt -Cgeo+h1000 -T2000/4000 should be allowed. It violates the intent of that particular CPT and potentially makes stupid GMT maps. Perhaps we could get @remkos and @leouieda to weigh in on these issues.

seisman · 2020-01-17T20:56:52Z

With rules 1-3, gmt makecpt -Cgeo -T2000/4000 give what you want exactly, right? Like what I said, users who explicitly use +h1000 are responsible for their own choices.

joa-quim · 2020-01-17T21:02:22Z

If we can change hard hinges ... they are no hard anymore and then we should only have hinges

PaulWessel · 2020-01-17T22:06:30Z

Yes. We either have hard vs soft hinges, or just hinges. The command above gives (with rule 2):

gmt makecpt -Cgeo -T2000/4000
2000	0/97/71	2020.40816327	16/122/47
2020.40816327	16/122/47	2204.08163265	232/215/125
2204.08163265	232/215/125	2489.79591837	161/67/0
2489.79591837	161/67/0	2693.87755102	100/50/25
2693.87755102	100/50/25	3142.85714286	110/109/108
3142.85714286	110/109/108	3632.65306122	255/254/253
3632.65306122	255/254/253	4000	255/254/253

but gmt makecpt -Cgeo+h3000 -T2000/4000 -V gives

makecpt [WARNING]: Cannot override hard hinges in CPT geo.cpt with +h. Modifier is ignored.
makecpt [WARNING]: gmtsupport_validate_cpt: CPT hard hinge is outside actual data range - range adjusted to start at hinge 2000 and below-hinge CPT ignored.

joa-quim · 2020-01-17T23:01:52Z

So we are not talking the same rules. 0/97/71 is color of z = 0 in geo. Not z = 2000.
This is not what you implemented at the time of #2416 (comment)

PaulWessel · 2020-01-17T23:14:37Z

Well, doing otherwise takes us to -G:

-G Truncate incoming CPT to be limited to the z-range <zlo>/<zhi>.
	   To accept one of the incoming limits, set that limit to NaN.

My first implementation stretched zmin or zmax to reach the hinge at 0. The above is the case where we leave zmin/zmax along and just toss the other half of the CPT. To strictly interpolate 2000/4000 to mean just that section of geo has not been done (but possible via -G).

PaulWessel · 2020-01-17T23:23:30Z

Should have said that CPTs with a hard hinge, like any CPT, are stretchable, hence the color at the users 2000 will be the color from the hinge. Otherwise the CPT would be completely static and unchangeable. You then run

gmt makecpt -Ggeo > mine.cpt

and use that.

PaulWessel · 2020-01-21T00:22:50Z

I would like this to get resolved so we can plan 6.1

CPTs with hard hinges and ranges (e.g., geo with range RANGE = -8000/8000) can obviously be stretched and resampled: We may want to go from -12000 to +9000 meters, for instance. The range of the CPT is never fixed; it is just a default range when none is given in makecpt. Hence, commands like gmt makecpt -Cgeo -T2000/4000 cannot simply pull out colors from the original 2000/4000 subset of the -8000/8000 range. That would mean we could not use this cpt outside the -8/8 km range. Instead, we resample the positive elevation CPT (hinge 0 to 8000) and compress it to fit the desired 2000/4000 meter output. So yes, the first color at 2000 is the one at 0 in the original CPT and the last color at 4000 is the color at 8000 in the original CPT.

There may be cases where you want to extract only a subset of the CPT colors. We added -G for that, but this is a very special operation that few people would ever need.

Thus, it seems the final decision we are stuck at has to do with the meaning of a hard hinge. I am arguing that it should be immutable: It is what it is, and the designer of the CPT wants it to be a fixed boundary tied to the physical properties it was intended for (topographic relief). In contrast, a soft hinge is mutable; you can change it with +hz. If it were not so and +h could apply to the hard hinges as well, then the only difference there would be between hard and soft is this:

hard: There is always a hinge (no +h needed to activate it) and we stretch the output differently, except when user range excludes the hinge where we discard the unused half of the CPT, then sample normally.
soft: It requires a +h modifier to activate the hinge. After that it behaves as a hard hinge.

joa-quim · 2020-01-21T16:01:48Z

Re-reading this again I have no strong opinion on the two alternatives, but the second (to let a hard hinge soften a bit and move) gives more freedom without increasing confusion. The last paragraph

hard: There is always a hinge (no +h needed to activate it) and we stretch the output differently, except when user range excludes the hinge where we discard the unused half of the CPT, then sample normally. soft: It requires a +h modifier to activate the hinge. After that it behaves as a hard hinge.

is what should go into the docs.

PaulWessel · 2020-01-21T17:40:35Z

OK, I can live with that. After all, it takes an unprecendented act on part of the user to move the geo hinge away from 0 via +h, so not likely to happen by accident. I will implement the movable hard hinge and see how it goes.

PaulWessel · 2020-01-21T18:48:37Z

Hi @joa-quim and @seisman. Let me know if this is the expected behavior. If you agree then I can adjust the docs and request a review.

seisman · 2020-01-21T18:51:25Z

Yes, this is exactly what I expected.

joa-quim · 2020-01-21T19:29:36Z

Yes, that's how I understood it. I assume that an equivalent of case d) for soft hinges, e.g. -Cpolar+h -T10/20 will give the same cpt as i) but with [10 20] limits.

PaulWessel · 2020-01-21T19:45:00Z

Yes:

gmt makecpt -Cpolar+h -T10/20 
10	white	20	red

PaulWessel · 2020-01-21T19:47:53Z

Hi @seisman, if you are happy with the latest merge in this PR please review and then we can fix any issues and merge this back to the mothership.

seisman · 2020-01-21T20:14:09Z

A crash by a user error:

$ gmt makecpt -Cwhite,blue -T0,1
makecpt [ERROR]: Mistmatch between number of entries in color list and z list
gmt(56804,0x106bcf5c0) malloc: *** error for object 0x7fd9d8700d40: pointer being freed was not allocated
gmt(56804,0x106bcf5c0) malloc: *** set a breakpoint in malloc_error_break to debug
[1]    56804 abort      gmt makecpt -Cwhite,blue -T0,1

seisman

This PR looks good to me, except the small crash in my previous comment.

seisman · 2020-01-21T21:00:25Z

@PaulWessel Remember to click the "Squash and merge" button when merging. Don't use the first one.

PaulWessel · 2020-01-21T21:12:45Z

Thanks, done.

Update how CPT hinges are done

89de734

Separate between soft and hard hinges. Hard hinges are data driven like those for bathymetry or magnetics, while soft hinges are ignored unless +h[hinge] is given.

PaulWessel added 7 commits January 7, 2020 13:01

Updates so far - in progress

5cbef76

Add warning that +h cannot affect hard hinges

7a759e9

Update scripts and PS to show short and hard hinges

70ffabd

Update GMT_tut_17.ps

15536f0

Update docs

f5ff933

Explain soft and hard hinges

Add include files explaining CPT syntax for creation and use

00fee51

Polish cpt docs

e2965f2

PaulWessel changed the title ~~WIP Update how CPT hinges are done~~ Update how CPT hinges are handled Jan 8, 2020

Fix handing of soft hinge assignment

394ea99

PaulWessel requested review from seisman and joa-quim January 8, 2020 08:08

Improve hinge documentationand CPT

e58278d

joa-quim approved these changes Jan 8, 2020

View reviewed changes

PaulWessel changed the title ~~Update how CPT hinges are handled~~ WIP Update how CPT hinges are handled Jan 8, 2020

Merge branch 'master' into cpthingework

7e89302

Fix print message in validate_cpt function

db1e6d5

PaulWessel added 2 commits January 17, 2020 15:13

Leave user zmin/zmax selection as is

be4f086

Merge branch 'master' into cpthingework

e4fb649

Add new test, update cpthinges to new behavior

267ce55

PaulWessel changed the title ~~WIP Update how CPT hinges are handled~~ Update how CPT hinges are handled Jan 21, 2020

seisman approved these changes Jan 21, 2020

View reviewed changes

PaulWessel added 2 commits January 21, 2020 10:32

Avoid freeing same array twice

9ee3e59

Merge branch 'master' into cpthingework

d8d676f

PaulWessel merged commit b8106c1 into master Jan 21, 2020

PaulWessel deleted the cpthingework branch January 21, 2020 21:12

seisman mentioned this pull request Jun 28, 2020

Re-caching GMT remote files GenericMappingTools/pygmt#495

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update how CPT hinges are handled #2416

Update how CPT hinges are handled #2416

PaulWessel commented Jan 7, 2020 •

edited

Loading

PaulWessel commented Jan 7, 2020

PaulWessel commented Jan 8, 2020

remkos commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

seisman commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 8, 2020 •

edited

Loading

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 9, 2020

remkos commented Jan 9, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

seisman commented Jan 17, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

PaulWessel commented Jan 21, 2020

joa-quim commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

seisman commented Jan 21, 2020

joa-quim commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

seisman commented Jan 21, 2020

seisman left a comment

seisman commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

Update how CPT hinges are handled #2416

Update how CPT hinges are handled #2416

Conversation

PaulWessel commented Jan 7, 2020 • edited Loading

PaulWessel commented Jan 7, 2020

PaulWessel commented Jan 8, 2020

remkos commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

seisman commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 8, 2020 • edited Loading

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 8, 2020

PaulWessel commented Jan 8, 2020

joa-quim commented Jan 9, 2020

remkos commented Jan 9, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

seisman commented Jan 17, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

joa-quim commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

PaulWessel commented Jan 17, 2020

PaulWessel commented Jan 21, 2020

joa-quim commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

seisman commented Jan 21, 2020

joa-quim commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

seisman commented Jan 21, 2020

seisman left a comment

Choose a reason for hiding this comment

seisman commented Jan 21, 2020

PaulWessel commented Jan 21, 2020

PaulWessel commented Jan 7, 2020 •

edited

Loading

joa-quim commented Jan 8, 2020 •

edited

Loading