Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow passing arguments containing spaces into pygmt functions #1487

Merged
merged 23 commits into from
Mar 13, 2022
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b5ad405
Replace spaces in arguments with octal code 040
weiji14 Sep 7, 2021
14648c2
Remove workarounds for spaces in fig.subplot's autolabel and title args
weiji14 Sep 7, 2021
924f439
Remove workaround for spaces in fig.text's -F argument
weiji14 Sep 7, 2021
707745f
Remove double quotes around legend label test examples
weiji14 Sep 7, 2021
03e4308
Edit test_rose_no_sectors to remove single quotes from title
weiji14 Sep 7, 2021
095449a
Remove workaround for spaces in fig.psconvert prefix
weiji14 Sep 7, 2021
d81b80b
Format using blackdoc and fix flake8 error
weiji14 Sep 7, 2021
dd849cb
Merge branch 'main' into arg_with_space
weiji14 Jan 10, 2022
c29e632
Ensure spaces in pygmt.config arguments can work
weiji14 Jan 10, 2022
83c8c3b
Manually handle prefix -F in psconvert
weiji14 Jan 13, 2022
2ba8420
Merge branch 'main' into arg_with_space
weiji14 Jan 13, 2022
b58af8b
Merge branch 'main' into arg_with_space
weiji14 Mar 6, 2022
3ec7727
Handle PROJ4 strings with spaces
weiji14 Mar 6, 2022
f19c41b
Use Modifier Letter Colon instead of regular colon to fix WIndows tests
weiji14 Mar 6, 2022
7a518a1
Merge branch 'main' into arg_with_space
weiji14 Mar 7, 2022
9774d5e
Try using underscore instead of Modifier Letter Colon
weiji14 Mar 7, 2022
ff40d27
Refactor the whole function using a single loop
weiji14 Mar 10, 2022
36fdec5
Merge branch 'main' into arg_with_space
weiji14 Mar 10, 2022
6c399ba
Merge branch 'main' into arg_with_space
weiji14 Mar 11, 2022
4770396
Merge branch 'main' into arg_with_space
weiji14 Mar 12, 2022
801ba01
Raise GMTInvalidInput if no prefix argument is passed to psconvert
weiji14 Mar 13, 2022
ecb580e
Merge branch 'main' into arg_with_space
weiji14 Mar 13, 2022
a526d45
Fix rst syntax in pygmt/helpers/utils.py
weiji14 Mar 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/gallery/embellishments/legend.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
pen="faint",
label="Apples",
)
fig.plot(data="@Table_5_11.txt", pen="1.5p,gray", label='"My lines"')
fig.plot(data="@Table_5_11.txt", pen="1.5p,gray", label="My lines")
fig.plot(data="@Table_5_11.txt", style="t0.15i", color="orange", label="Oranges")

fig.legend(position="JTR+jTR+o0.2c", box=True)
Expand Down
10 changes: 7 additions & 3 deletions pygmt/figure.py
Original file line number Diff line number Diff line change
Expand Up @@ -235,10 +235,14 @@ def psconvert(self, icc_gray=False, **kwargs):
kwargs["N"] = "+i"
else:
kwargs["N"] += "+i"
# allow for spaces in figure name
kwargs["F"] = f'"{kwargs.get("F")}"' if kwargs.get("F") else None

# Manually handle prefix -F argument so spaces aren't converted to \040
# by build_arg_string function. For more information, see
# https://github.com/GenericMappingTools/pygmt/pull/1487
prefix = kwargs.pop("F")
Copy link
Member

@maxrjones maxrjones Mar 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a check before this that prefix is provided so that the user doesn't get a KeyError here? Or you could raise GMTInvalidInput in an except statement.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Looking at the docs at https://docs.generic-mapping-tools.org/6.3/psconvert.html, it reads like prefix/-F is optional, but when I tried running psconvert without the -F option, it throws an error: psconvert [ERROR]: Modern GMT mode requires the -F option. See https://github.com/GenericMappingTools/gmt/blob/adb244afa51ca7246cc051080c9d47193087d6c2/src/psconvert.c#L1041-L1042

So since PyGMT is modern mode only, it should be fine to keep this line intact.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs say "If no input files are given, will convert the current active figure (see pygmt.figure). In this case, an output name must be given using parameter prefix." How does one provide other input files than the current figure using pygmt.Figure.psconvert?

If we chose not to support operations analogous to psconvert -Tg test.ps (which uses the same prefix as the original file name), I think we should make prefix required in the function signature or raise an invalid input exception to avoid the traceback below. I don't think it's obvious to that the error is caused by not using prefix.

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one provide other input files than the current figure using pygmt.Figure.psconvert?

We can't, because it has not been implemented yet. The current fig.psconvert is a method of the pygmt.Figure() class and is tied to the fig, and can only 'psconvert' the active figure.

If we chose not to support operations analogous to psconvert -Tg test.ps (which uses the same prefix as the original file name), I think we should make prefix required in the function signature or raise an invalid input exception to avoid the traceback below. I don't think it's obvious to that the error is caused by not using prefix.

Ok, I've added a try-except block in 801ba01 to raise a GMTInvalidInput if prefix/-F is not given.


with Session() as lib:
lib.call_module("psconvert", build_arg_string(kwargs))
lib.call_module("psconvert", f'-F"{prefix}" {build_arg_string(kwargs)}')

def savefig(
self, fname, transparent=False, crop=True, anti_alias=True, show=False, **kwargs
Expand Down
33 changes: 28 additions & 5 deletions pygmt/helpers/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def dummy_context(arg):


def build_arg_string(kwargs):
"""
r"""
Transform keyword arguments into a GMT argument string.

Make sure all arguments have been previously converted to a string
Expand All @@ -131,6 +131,11 @@ def build_arg_string(kwargs):
same command line argument. For example, the kwargs entry ``'B': ['xa',
'yaf']`` will be converted to ``-Bxa -Byaf`` in the argument string.

Note that spaces ` ` in arguments are converted to the equivalent octal
code `\040`, except in the case of -J (projection) arguments where PROJ4
weiji14 marked this conversation as resolved.
Show resolved Hide resolved
strings (e.g. "+proj=longlat +datum=WGS84") will have their spaces removed.
See https://github.com/GenericMappingTools/pygmt/pull/1487 for more info.

Parameters
----------
kwargs : dict
Expand All @@ -151,7 +156,7 @@ def build_arg_string(kwargs):
... A=True,
... B=False,
... E=200,
... J="X4c",
... J="+proj=longlat +datum=WGS84",
... P="",
... R="1/2/3/4",
... X=None,
Expand All @@ -160,7 +165,7 @@ def build_arg_string(kwargs):
... )
... )
... )
-A -E200 -JX4c -P -R1/2/3/4 -Z0
-A -E200 -J+proj=longlat+datum=WGS84 -P -R1/2/3/4 -Z0
>>> print(
... build_arg_string(
... dict(
Expand All @@ -176,6 +181,16 @@ def build_arg_string(kwargs):
Traceback (most recent call last):
...
pygmt.exceptions.GMTInvalidInput: Unrecognized parameter 'watre'.
>>> print(
... build_arg_string(
... dict(
... B=["af", "WSne+tBlank Space"],
... F='+t"Empty Spaces"',
... l="'Void Space'",
... ),
... )
... )
-BWSne+tBlank\040Space -Baf -F+t"Empty\040\040Spaces" -l'Void\040Space'
"""
gmt_args = []

Expand All @@ -185,11 +200,19 @@ def build_arg_string(kwargs):
if kwargs[key] is None or kwargs[key] is False:
pass # Exclude arguments that are None and False
elif is_nonstr_iter(kwargs[key]):
gmt_args.extend(f"-{key}{value}" for value in kwargs[key])
for value in kwargs[key]:
_value = str(value).replace(" ", r"\040")
gmt_args.append(rf"-{key}{_value}")
elif kwargs[key] is True:
gmt_args.append(f"-{key}")
else:
gmt_args.append(f"-{key}{kwargs[key]}")
if key != "J": # non-projection parameters
_value = str(kwargs[key]).replace(" ", r"\040")
else:
# special handling if key == "J" (projection)
# remove any spaces in PROJ4 string
_value = str(kwargs[key]).replace(" ", "")
gmt_args.append(rf"-{key}{_value}")
Comment on lines +209 to +215
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know almost nothing about PROJ4, but does a PROJ4 string always contains +proj=xxx or EPSG? If yes, then we can simplify the if-test to something like:

            if key != "J" or (key == "J" and "+proj" not in value and "EPSG" not in value):
                _value = str(kwargs[key]).replace(" ", r"\040")
            else:
                # special handling if -J + PROJ4 string
                # remove any spaces in PROJ4 string
                _value = str(kwargs[key]).replace(" ", "")
            gmt_args.append(rf"-{key}{_value}")

The motivation is, since PROJ4 is not commonly used, it seems a waste of time to do call the replace method for GMT-style -J.

Copy link
Member Author

@weiji14 weiji14 Mar 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if all PROJ-strings have either +proj=xxx or EPSG (see https://proj.org/operations/projections/index.html), but they probably do? I know that there's +init=EPSG:xxx which kinda works when I tried here, but that's actually deprecated syntax (see https://pyproj4.github.io/pyproj/stable/gotchas.html#init-auth-auth-code-should-be-replaced-with-auth-auth-code) and probably not something we want to allow.

Also unsure if 2 extra if-checks saves time, considering that either way, a str.replace still needs to happen for the space character - either to \040 or to nothing ``.

return " ".join(sorted(gmt_args))


Expand Down
2 changes: 1 addition & 1 deletion pygmt/src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ def __init__(self, **kwargs):
self.old_defaults[key] = lib.get_default(key)

# call gmt set to change GMT defaults
arg_str = " ".join([f"{key}={value}" for key, value in kwargs.items()])
arg_str = " ".join([f'{key}="{value}"' for key, value in kwargs.items()])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xref: https://forum.generic-mapping-tools.org/t/errors-in-pygmt-when-there-are-spaces-in-string-arguments/2494/3

Ok, I've fixed the FORMAT_DATE_MAP="o dd" issue reported in the forum by wrapping the arguments in double quotes here in commit c29e632. Didn't use \040 because o\040dd doesn't work. The problem with this solution is that the workaround suggested in the forum (use FORMAT_DATE_MAP='"o dd"') will break in PyGMT v0.6.0, but I think that's acceptable since that workaround isn't intuitive anyway.

with Session() as lib:
lib.call_module("set", arg_str)

Expand Down
6 changes: 0 additions & 6 deletions pygmt/src/subplot.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,10 +148,6 @@ def subplot(self, nrows=1, ncols=1, **kwargs):
{XY}
"""
kwargs = self._preprocess(**kwargs) # pylint: disable=protected-access
# allow for spaces in string without needing double quotes
if isinstance(kwargs.get("A"), str):
kwargs["A"] = f'"{kwargs.get("A")}"'
kwargs["T"] = f'"{kwargs.get("T")}"' if kwargs.get("T") else None

if nrows < 1 or ncols < 1:
raise GMTInvalidInput("Please ensure that both 'nrows'>=1 and 'ncols'>=1.")
Expand Down Expand Up @@ -222,8 +218,6 @@ def set_panel(self, panel=None, **kwargs):
{V}
"""
kwargs = self._preprocess(**kwargs) # pylint: disable=protected-access
# allow for spaces in string with needing double quotes
kwargs["A"] = f'"{kwargs.get("A")}"' if kwargs.get("A") is not None else None
# convert tuple or list to comma-separated str
panel = ",".join(map(str, panel)) if is_nonstr_iter(panel) else panel

Expand Down
2 changes: 1 addition & 1 deletion pygmt/src/text.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ def text_(
kwargs["F"] += f"+j{justify}"

if isinstance(position, str):
kwargs["F"] += f'+c{position}+t"{text}"'
kwargs["F"] += f"+c{position}+t{text}"

extra_arrays = []
# If an array of transparency is given, GMT will read it from
Expand Down
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_basemap_utm_projection.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: e6984efed2a94673754cc7f1f1d74832
size: 9069
path: test_basemap_utm_projection.png
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_config_format_date_map.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: 3619720cdfcd857cbdbb49ed7fe6e930
size: 1392
path: test_config_format_date_map.png
4 changes: 2 additions & 2 deletions pygmt/tests/baseline/test_rose_no_sectors.png.dvc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
outs:
- md5: 8e1c47b1cf6001dad3b3c0875af4562e
size: 150390
- md5: ce2d5cd1415b7c7bbeea5bf6ff39c480
size: 150288
path: test_rose_no_sectors.png
23 changes: 23 additions & 0 deletions pygmt/tests/test_basemap.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,29 @@ def test_basemap_winkel_tripel():
return fig


@pytest.mark.mpl_image_compare(filename="test_basemap_utm_projection.png")
@pytest.mark.parametrize(
"projection",
[
"EPSG_32723 +width=5",
"+proj=utm +zone=23 +south +datum=WGS84 +units=m +no_defs +width=5",
],
)
def test_basemap_utm_projection(projection):
"""
Create a Universal Transverse Mercator (Zone 23S) basemap plot.

Also check that providing the projection as an EPSG code or PROJ4 string
works.
"""
projection = projection.replace(
"EPSG_", "EPSG:" # workaround Windows not allowing colons in filenames
)
fig = Figure()
fig.basemap(region=[-52, -50, -12, -11], projection=projection, frame="afg")
return fig


@pytest.mark.mpl_image_compare
def test_basemap_rose():
"""
Expand Down
19 changes: 19 additions & 0 deletions pygmt/tests/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,25 @@ def test_config_font_annot():
return fig


@pytest.mark.mpl_image_compare
def test_config_format_date_map():
"""
Test that setting FORMAT_DATE_MAP config changes how the output date string
is plotted.

Note the space in 'o dd', this acts as a regression test for
https://github.com/GenericMappingTools/pygmt/issues/247.
"""
fig = Figure()
with config(FORMAT_DATE_MAP="o dd"):
fig.basemap(
region=["1969-7-21T", "1969-7-23T", 0, 1],
projection="X2.5c/0.1c",
frame=["sxa1D", "S"],
)
return fig


@pytest.mark.mpl_image_compare
def test_config_format_time_map():
"""
Expand Down
3 changes: 2 additions & 1 deletion pygmt/tests/test_figure.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,8 @@ def test_figure_savefig_filename_with_spaces():
fig = Figure()
fig.basemap(region=[0, 1, 0, 1], projection="X1c/1c", frame=True)
with GMTTempFile(prefix="pygmt-filename with spaces", suffix=".png") as imgfile:
fig.savefig(imgfile.name)
fig.savefig(fname=imgfile.name)
assert r"\040" not in os.path.abspath(imgfile.name)
assert os.path.exists(imgfile.name)


Expand Down
11 changes: 9 additions & 2 deletions pygmt/tests/test_grdproject.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,20 @@ def test_grdproject_file_out(grid, expected_grid):
xr.testing.assert_allclose(a=temp_grid, b=expected_grid)


def test_grdproject_no_outgrid(grid, expected_grid):
@pytest.mark.parametrize(
"projection",
["M10c", "EPSG:3395 +width=10", "+proj=merc +ellps=WGS84 +units=m +width=10"],
)
def test_grdproject_no_outgrid(grid, projection, expected_grid):
"""
Test grdproject with no set outgrid.

Also check that providing the projection as an EPSG code or PROJ4 string
works.
"""
assert grid.gmt.gtype == 1 # Geographic grid
result = grdproject(
grid=grid, projection="M10c", spacing=3, region=[-53, -51, -20, -17]
grid=grid, projection=projection, spacing=3, region=[-53, -51, -20, -17]
)
assert result.gmt.gtype == 0 # Rectangular grid
assert result.gmt.registration == 1 # Pixel registration
Expand Down
2 changes: 1 addition & 1 deletion pygmt/tests/test_legend.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def test_legend_entries():
pen="faint",
label="Apples",
)
fig.plot(data="@Table_5_11.txt", pen="1.5p,gray", label='"My lines"')
fig.plot(data="@Table_5_11.txt", pen="1.5p,gray", label="My lines")
fig.plot(data="@Table_5_11.txt", style="t0.15i", color="orange", label="Oranges")
fig.legend(position="JTR+jTR")

Expand Down
2 changes: 1 addition & 1 deletion pygmt/tests/test_rose.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ def test_rose_no_sectors(data_fractures_compilation):
region=[0, 500, 0, 360],
diameter="10c",
labels="180/0/90/270",
frame=["xg100", "yg45", "+t'Windrose diagram'"],
frame=["xg100", "yg45", "+tWindrose diagram"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "+t'Windrose diagram'" still work or not?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does still work, but the single quotes would be printed. I think it was a typo from whoever wrote that test.

pen="1.5p,red3",
transparency=40,
scale=0.5,
Expand Down