Replies: 4 comments 1 reply
-
Thanks for starting this discussion @jtwhite79! This is something we're very interested in, and I'm sure my colleague @martinvonk has more thoughts and ideas on this topic. But I had some thoughts and questions I wanted to ask/share, so here goes: I'm assuming the PEST run is performed by some Python script? If so, the following code snippets might be useful, if you weren't already doing it that way... Concerning the storing/loading of existing Pastas models: Pastas supports saving/loading JSON-style files. It is a bit of a pain to modify stuff in this file directly, so it is recommended to do that stuff via the Pastas Model interface. ml.to_file("model.pas") # store as JSON file
ml = ps.io.load("model.pas") # load JSON file There is something of an alternative, through the pastastore package, which allows storing Pastas Models and time series separately in some sort of database-like structure (which can be some folder on your PC). This package allows you to reading and writing models as dictionaries (which is very fast), instead of building the full Pastas Model instances each time. That would allow modifying parameters/settings quickly. However, since you need to run the Pastas Model at some point anyway, it might be more efficient to just use the Pastas Model class directly? The pastastore package also has a YAML interface for building Pastas models, explained here, which is a mini-language that allows you to build Pastas Models and define settings in a human-readable text file. I'm not sure if these methods are fully what you are looking for, but they may already contain some of the functionality we would need to better support PEST runs? With respect to the parameters, setting parameters is done using: ml # some pastas model
for param, value in csv_value_pairs.items():
ml.set_parameter(param, optimal=value)
# or maybe since PEST is case-insensitive
for i, (param, value) in enumerate(csv_value_pairs.items()):
ipar = ml.parameters.index[i]
ml.set_parameter(ipar, optimal=value) And pastas does have a strict parameter order, based on the order in which StressModels were added to the original model, so you can count on that being the case. Running a model without using the internal calibration routine can also be done in multiple ways:
Many methods in pastas support tmin/tmax arguments so that should force Pastas to compute the results for the simulation period you're interested in. Once we've settled on an approach to better supporting PEST runs, it would definitely be a good idea to document a clear and intuitive way to set parameters or tweak a model when using external calibration routines. Or maybe we can create some kind of Anyway, I'm curious if you were already using the logic above? If not, would that make it running PEST-o-PASTA (great name by the way :p ) easier? Let us know! |
Beta Was this translation helpful? Give feedback.
-
That is more or less how I was getting at the problem, just wanted to make sure that assuming that order of parameters was safe - I think saw a warning somewhere about messing with the parameter dataframe that a pastas model has attached to it...but maybe I was also using an older version of pastas. I like the idea of a helper class to as an entry point where we could automate and hide all the gory details of preparing for and running pest/pest++ would be great. There is a monster class in pyemu called (I also cant take credit for that name - I stole it from @raoulcollenteur !) |
Beta Was this translation helpful? Give feedback.
-
Alright, I've got a working example on the pastas-plugins repository: https://github.com/pastas/pastas-plugins/blob/pestas/docs/examples/pest_pastas.ipynb. All logic is in the solver.py file which contains the following:
Note: I know that the parameters are also available in the pas file but I figured this was faster and
There are many different versions of pest(++); with different output files and required handling. For each pest version we need another class with a solve method. On the pastas-plugins repository I now have two classes; I hope this makes things a bit more clear for everyone, and we can work from here. Next step is (I think?) to compare the result from the notebook to a simple P.S. For me, what helped was a small example I created to compare |
Beta Was this translation helpful? Give feedback.
-
Pest also does not like negative parameters, since it does log transformations by default. We don't have those negative parameters a lot, But one example: |
Beta Was this translation helpful? Give feedback.
-
The primary “forward run” process we need for PEST integration involves:
This is what Ive tried to do when I was hooking up pastas to pest before. With this in mind, it would sweet if we could pickle a pastas model instance (or save all the settings/definitions in a config, which I think is in the works) so that we don’t need to have all the py code to re-create the pastas model instance each time we run the model. I had some hackery to force pastas to run over the same time period every time - It would awesome if the pastas model would just run from a start to end datetime without supplying any observed series...
Also, I felt like I was living dangerously trying to hot-wire pastas to simply use parameter values from a file (a file that pest fills with the current values at runtime), rather than using the internal PASTAS calibration routine(s). So it would be nice if there was a clean and standard way to tell a pastas model to read parameter values from a plain-ole CSV file (with name-value pairs). For this to work, I had to hope that pastas was using the parameter info in the order I supplied it (and the “A” and “a” parameter naming is a problem in the pest world bc pest is case insensitive).
I think these are the primary obstacles to easily making a big ‘ole bowl of PEST-o-PASTA haha!
Maybe some of these things are already dealt with and/or I was just doing it wrong (highly likely) so please let me know if Ive missed somethings along the way...
Beta Was this translation helpful? Give feedback.
All reactions