GUI for Automated plotting #2414

Preetam-Das26 · 2024-06-25T10:59:06Z

Purpose of PR?:

Fixes #

Does this PR introduce a breaking change?

If the changes in this PR are manually verified, list down the scenarios covered::

Additional information for reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Does this PR results in some Documentation changes?
If yes, include the list of Documentation changes

Checklist:

Bug fix. Fixes #
New feature (Non-API breaking changes that adds functionality)
PR Title follows the convention of <type>: <subject>
Commit has unit tests

Merge develop to stable == 9.0.0

Merge stable to develop

Occasionally the test runs on macos-13 are super slow and there is generally a lot of variation in the time required to run the test suite on those runners. I think they genuinely need more time sometimes, so this increases the timeout to accommodate.

* Run flake8 for all .py files tracked in git * Fix flake8 errors

…-MSS#2411)

this should keep it archived also when a server sets the threshold to a few years

* Add a codespell lint job * Fix misspellings found by codespell

* updated data base migration to current release * deprecation for manual migration added

…ot downloading

`rtd.io` is on longer operational, according to [the project page](https://readthedocs.org/projects/mss/) the correct domain is either `rtfd.io` or the full `readthedocs.io`. Also switch to https.

* Fixed an issue with units being ignored in vertical axis * Fixed an issue with *assuming* that vertical axis is in pressure if no standard name "air_pressure" was available * Fixed an issue with extrapolating data * Fixes an issue with interpolating in pressure instead of log-pressure * Fixes an issue with ignoring jumps/extrapolation in data for missing longitudes (now it is consistent with vertical section driver)

* doc updated * indenting level fixed * add details about the wsgi modules

matrss

It is unclear to me how this is supposed to be used.

I've tried loading the "mssautoplot.json" shared in Slack using the "cpath" button. There is no feedback on if that has worked or not, I would have expected some of the other widgets to be prefilled with content from that configuration.

When pressing the "Store plots directory" button after loading the configuration file, with or without first selecting some settings from the dropdowns, I get (an) "Invalid STYLE and/or URL requested" message(s) in the terminal and then the GUI just closes.

I also get those messages with python mslib/utils/mssautoplot.py --cpath mssautoplot.json as suggested by https://mss.readthedocs.io/en/stable/mssautoplot.html#examples, so maybe that is an issue on my end.

matrss · 2024-07-04T13:48:51Z

mslib/utils/autoplot_gui.py

Please add the file header from the template (adapt the copyright information therein): https://github.com/Open-MSS/MSS/blob/develop/new_file_template.py.

This isn't required for the other files, since they are tool-generated.

matrss · 2024-07-04T13:52:05Z

mslib/utils/autoplot_gui.py

+ self.url.remove(i.text())
+ self.urlListWidget.takeItem(self.urlListWidget.row(i))
+
+ def layersWindow(self):


Please use snake_case for functions and methods as is idiomatic for python, unless something else is required by outside constraints (e.g. PyQt5 uses camelCase for its stuff, which is outside our control).

matrss · 2024-07-04T13:56:00Z

mslib/msui/qt5/ui_mss_autoplot.py

+from PyQt5 import QtCore, QtGui, QtWidgets
+
+
+class Ui_Form(object):


These should probably have a more descriptive name.

@Preetam-Das26 this should get a better name

Merge stable to develop

Preetam-Das26 · 2024-07-04T15:27:27Z

I have not added the logging messages. I will do so in the next commit.
Please follow these steps:
1.Kindly fill the necessary parameters:
View
Cpath
Stime
Etime
Itime
Vtime
Interval
2. For login to the wms server:
Paste the wms server url provided in slack.
Next click on add and fill the username and password as mentioned in slack.
Again you have to click on add.
I think you are skipping this step.

Next click on store plots button.
Plots will be downloaded.

matrss · 2024-07-05T12:52:47Z

I get an error when trying to add the WMS server URL:

Traceback (most recent call last):
  File "/home/icg149/Projects/MSS/mslib/utils/autoplot_gui.py", line 235, in addURL
    self.check_url_in_keyring(text)
  File "/home/icg149/Projects/MSS/mslib/utils/autoplot_gui.py", line 242, in check_url_in_keyring
    cred = keyring.get_credential(link, "")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/core.py", line 73, in get_credential
    return get_keyring().get_credential(service_name, username)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/backends/chainer.py", line 69, in get_credential
    credential = keyring.get_credential(service, username)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/backends/kwallet.py", line 125, in get_credential
    return super().get_credential(service, username)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/backend.py", line 143, in get_credential
    password = self.get_password(service, username)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/backends/kwallet.py", line 108, in get_password
    if not self.connected(service):
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/icg149/Projects/MSS/.pixi/envs/dev/lib/python3.11/site-packages/keyring/backends/kwallet.py", line 99, in connected
    raise InitError(f'Failed to open keyring: {e}.') from e
keyring.errors.InitError: Failed to open keyring: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.19253 was not provided by any .service files.

We had this issue in the past already (#1815), you should be able to reuse the method from mslib.utils.auth instead of using keyring directly.

swsrkty · 2024-07-05T14:30:37Z

When pressing the "Store plots directory" button after loading the configuration file, with or without first selecting some settings from the dropdowns, I get (an) "Invalid STYLE and/or URL requested" message(s) in the terminal and then the GUI just closes.
I also get those messages with python mslib/utils/mssautoplot.py --cpath mssautoplot.json as suggested by https://mss.readthedocs.io/en/stable/mssautoplot.html#examples, so maybe that is an issue on my end.

I'm able to get the plot from the GUI with the default configuration (by setting the cpath in the GUI and by leaving the other settings blank). The config shared by Preetam has a flight track file. Please try putting it in the root MSS directory if you haven't.

The error message "Invalid STYLE ....." is not correct as there's a correction to be made here which was pointed out by you sometime ago.

elif "404 Client Error" or "NOT FOUND for url" in e:
      print("Invalid STYLE and/or URL requested")

The 'elif' becomes true in all cases and thus the error message (#2295). I'll ask Preetam to push the corrected one with his code.

ReimarBauer · 2024-07-05T15:50:05Z

the tests get triggered to show a problem https://github.com/Open-MSS/MSS/actions/runs/9810353386/job/27090313984?pr=2405#step:10:1129


[gw0] [100%] PASSED tests/_test_msui/test_mscolab.py::Test_Mscolab::test_profile_dialog Exception ignored in: <function WMSControlWidget.__del__ at 0x7f12d67293a0>
Traceback (most recent call last):
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 544, in __del__
    self.service_cache()
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 1438, in service_cache
    files = [os.path.join(self.wms_cache, f) for f in os.listdir(self.wms_cache)]
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpg3phzpm3/msui/wms_cache/'
Exception ignored in: <function WMSControlWidget.__del__ at 0x7f12d67293a0>
Traceback (most recent call last):
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 544, in __del__
    self.service_cache()
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 1438, in service_cache
    files = [os.path.join(self.wms_cache, f) for f in os.listdir(self.wms_cache)]
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpg3phzpm3/msui/wms_cache/'
Exception ignored in: <function WMSControlWidget.__del__ at 0x7f12d67293a0>
Traceback (most recent call last):
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 544, in __del__
    self.service_cache()
  File "/__w/MSS/MSS/mslib/msui/wms_control.py", line 1438, in service_cache
    files = [os.path.join(self.wms_cache, f) for f in os.listdir(self.wms_cache)]
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpg3phzpm3/msui/wms_cache/'

https://github.com/Open-MSS/MSS/actions/runs/9810353386/job/27090313984?pr=2405#step:10:1129

ReimarBauer · 2024-07-18T17:33:34Z

mslib/msui/autoplot_dockwidget.py

+"""
+
+ mslib.utils.autoplot_gui
+ ~~~~~~~~~~~~~~~~


maybe font, but this needs samle length as the word in the line before

the name was changed, autoplot_dockwidget

mslib/msui/autoplot_dockwidget.py

ReimarBauer · 2024-07-18T17:37:00Z

mslib/msui/autoplot_dockwidget.py

+ fileName, _ = QFileDialog.getOpenFileName(
+ self, "Select .json Config File", const.MSUI_CONFIG_PATH, "JSON Files (*.json)", options=options)
+
+ if fileName:


is not None: ?

ReimarBauer · 2024-07-23T07:28:27Z

mslib/utils/config.py

@@ -74,18 +74,96 @@ class MSUIDefaultConfig:

 # Predefined map regions to be listed in the corresponding topview combobox
 predefined_map_sections = {
- "01 Europe (cyl)": {"CRS": "EPSG:4326",


This is the reason the test fails
test_kwargs_update_does_not_harm

ReimarBauer · 2024-07-23T07:30:18Z

mslib/utils/config.py

@@ -74,18 +74,96 @@ class MSUIDefaultConfig:

 # Predefined map regions to be listed in the corresponding topview combobox
 predefined_map_sections = {
- "01 Europe (cyl)": {"CRS": "EPSG:4326",


The position has changed
"07 Europe (cyl)":

changing that number also affects the tutorials https://github.com/Open-MSS/MSS/blob/develop/tutorials/tutorial_hexagoncontrol.py#L46

They look for the first element on the widget

ReimarBauer

see tutorials

ReimarBauer · 2024-07-23T19:03:59Z

tutorials/tutorial_kml.py

@@ -46,7 +46,7 @@ def automate_kml():


 def _switch_to_europe_map():
- find_and_click_picture('topviewwindow-01-europe-cyl.png', "Map change dropdown could not be located on the screen.")
+ find_and_click_picture('topviewwindow-07-europe-cyl.png', "Map change dropdown could not be located on the screen.")


this won't work

The image available is
topviewwindow-00-global-cyl.png

it can only find the first on screen visible value on a pulldown.

this can be verified by

$ export MSUI_CONFIG_PATH=/tmp/msui_tutorials $ export XDG_CACHE_HOME=$(mktemp -d) $ python >>> from mslib.msui.msui import main >>> main(tutorial_mode=True) CTRL+H CTRL+UP CTRL+F CTRL+Q $ ls /tmp/msui-tutorials/tutorial_images/*00*.png

ReimarBauer and others added 24 commits May 22, 2024 06:32

removed pyvirtualdisplay section

9aa6403

Merge pull request Open-MSS#2384 from Open-MSS/merge_develop_to_stable

3858525

Merge develop to stable == 9.0.0

Merge pull request Open-MSS#2389 from Open-MSS/merge_stable_to_develop

20bbf55

Merge stable to develop

Remove pytest-timeout (Open-MSS#2391)

a92e0a2

Fix typo in last name of author (Open-MSS#2398)

bc77fa8

Run flake8 for all .py files tracked in git (Open-MSS#2394)

8e673ad

* Run flake8 for all .py files tracked in git * Fix flake8 errors

Fix Open-MSS#2396 (Open-MSS#2397)

8645d04

renewed skyfield-data (Open-MSS#2408)

068faf1

COMMIT 1: GUI codes for the .py and .ui files

873e578

COMMIT 2: codes for the scripts file

7ce7c59

COMMIT 3:codes for automatically download the plots

8d0d285

set for Console handler info state, don't output all to stdout. (Open…

3255184

…-MSS#2411)

a user archived operation uses sys.maxsize in days (Open-MSS#2413)

cfa809a

this should keep it archived also when a server sets the threshold to a few years

Add a codespell lint job (Open-MSS#2395)

e7af5dc

* Add a codespell lint job * Fix misspellings found by codespell

updated data base migration to current release (Open-MSS#2410)

40c5113

* updated data base migration to current release * deprecation for manual migration added

add JuRSE badge (Open-MSS#2419)

64169ee

COMMIT 4: codes for the http authentication of username and password

59f7ac6

COMMIT 5: script codes for HTTP Authentication, all 3 use cases of pl…

fedfd1f

…ot downloading

Fix docs URL (Open-MSS#2422)

92ea726

`rtd.io` is on longer operational, according to [the project page](https://readthedocs.org/projects/mss/) the correct domain is either `rtfd.io` or the full `readthedocs.io`. Also switch to https.

docs updated envs and Miniforge dir (Open-MSS#2412)

7a7f11e

* doc updated * indenting level fixed * add details about the wsgi modules

prepare v9.1.0 (Open-MSS#2426)

8cede56

Merge branch 'stable' into merge_stable_to_develop

a650456

matrss reviewed Jul 4, 2024

View reviewed changes

Merge pull request Open-MSS#2427 from Open-MSS/merge_stable_to_develop

455718f

Merge stable to develop

ReimarBauer reviewed Jul 18, 2024

View reviewed changes

mslib/msui/autoplot_dockwidget.py Show resolved Hide resolved

ReimarBauer reviewed Jul 18, 2024

View reviewed changes

Preetam-Das26 added 9 commits July 20, 2024 16:45

COMMIT 17:updates based on the review

84f9005

COMMIT 18:updates based on the review

f64a606

COMMIT 19:updates based on the review

f1a0724

COMMIT 20:resolved the errors

2fffc7a

COMMIT 21: None to string

c122ab4

COMMIT 21: map sections updated

9da3f41

COMMIT 22:resolved errors

d6d7857

COMMIT 24: added the extension .ftml

862ce6b

COMMIT 25:added map sections,improvising the functionalities

a8b093e

ReimarBauer reviewed Jul 23, 2024

View reviewed changes

Preetam-Das26 added 6 commits July 23, 2024 21:09

COMMIT 26:resolved failing test cases,modified the update button

a18cc5c

COMMIT 27:flake8 resolved

e721f23

COMMIT 28:headline changed,config file changed

766b227

COMMIT 29

86c52ad

COMMIT 30

f72d1de

COMMIT 31

a0c4db5

ReimarBauer requested changes Jul 23, 2024

View reviewed changes

Preetam-Das26 and others added 3 commits July 24, 2024 19:07

COMMIT 32

96b8f15

Merge branch 'Open-MSS:develop' into gsoc_psd

1e730e9

COMMIT 33

691d464

Preetam-Das26 marked this pull request as ready for review July 24, 2024 19:36

Preetam-Das26 added 4 commits July 25, 2024 12:17

mssautoplot changed

7b7638d

mssautoplot changed

ab26c59

mssautoplot changed

1ad3c4a

mssautoplot changed

07b183f

swsrkty merged commit a6c8e09 into Open-MSS:GSOC2024-PreetamSundarDas Jul 25, 2024
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GUI for Automated plotting #2414

GUI for Automated plotting #2414

Preetam-Das26 commented Jun 25, 2024

matrss left a comment

matrss Jul 4, 2024

matrss Jul 4, 2024

matrss Jul 4, 2024

Preetam-Das26 Jul 4, 2024

ReimarBauer Jul 18, 2024

Preetam-Das26 commented Jul 4, 2024 •

edited

Loading

matrss commented Jul 5, 2024

swsrkty commented Jul 5, 2024 •

edited

Loading

ReimarBauer commented Jul 5, 2024

ReimarBauer Jul 18, 2024 •

edited

Loading

ReimarBauer Jul 18, 2024

ReimarBauer Jul 23, 2024

ReimarBauer Jul 23, 2024

ReimarBauer Jul 23, 2024

ReimarBauer left a comment

ReimarBauer Jul 23, 2024 •

edited

Loading

		from PyQt5 import QtCore, QtGui, QtWidgets


		class Ui_Form(object):

GUI for Automated plotting #2414

GUI for Automated plotting #2414

Conversation

Preetam-Das26 commented Jun 25, 2024

matrss left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Preetam-Das26 commented Jul 4, 2024 • edited Loading

matrss commented Jul 5, 2024

swsrkty commented Jul 5, 2024 • edited Loading

ReimarBauer commented Jul 5, 2024

ReimarBauer Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ReimarBauer left a comment

Choose a reason for hiding this comment

ReimarBauer Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

Preetam-Das26 commented Jul 4, 2024 •

edited

Loading

swsrkty commented Jul 5, 2024 •

edited

Loading

ReimarBauer Jul 18, 2024 •

edited

Loading

ReimarBauer Jul 23, 2024 •

edited

Loading