Scenario.var_list() and .var() not covering all variables

[Jihoon]

I find that I cannot read the contents of some new variables (e.g. REN) using the two function calls.

[danielhuppmann]

Only the most important variables and equations are currently imported from the GAMS output to the ixmp.Scenario instance - these are what you'll get via Scenario.var_list() and Scenario.equ_list(). The idea is to not import auxiliary results, that aren't needed in postprocessing.

Are there specific variables in the gdx output that you would need for postprocessing?

[gidden]

In general, I agree that one may want to see also the "not most important" variables as well. In this case, though, REN was introduced (if memory serves) and not added to the white list on the java side.

We should probably be more explicit when touching the GAMS code to also include these. I would still love to see someone actually scrape the GAMS code and generate the full listing (similar to how we do the documentation). If anyone is interested in taking the lead on that, I would very much enjoy helping.

[Jihoon]

I agree with Matt. I think it is no harm to allow users to read those aux results for better debugging.

Originally I wanted to read REN.

[behnam-zakeri]

@Jihoon , I had the same question and checked with @khaeru how to tackle this. As it stands now, the workaround can be as follows:

1. The user needs to first initialize the variable that is missing in the scenario.var_list().

• This can be done in an automatic way through message_ix/models.py, by adding a couple of lines, for example, for variable STORAGE as below. This way, any time a new scenario is created or cloned it will include this variable.

MESSAGE_ITEMS = {
    'STORAGE': dict(ix_type='var', idx_sets=['node', 'technology', 'level', 'commodity', 'year', 'time']),
}

• Another way, mostly for one time use cases, is to initialize the variable directly in the workflow or script:

scenario.init_var('STORAGE',  idx_sets=['node', 'technology', 'level', 'commodity', 'year', 'time'])

2. Then, after doing step No. 1, the new variable should be passed when solving as scenario.solve(var_list=['STORAGE']). This makes sure that the new variable will be attached to the default variables and the solution will be read from output GDX file.
   Obviously, it's more elegant to get No. 2 done automatically when the user does No. 1. But it seems at the moment it's not the case. I hope it helps.

[khaeru]

Another point: as we get closer to resolving iiasa/message_ix#254, this will get fixed as a byproduct.

• Once the structure of MESSAGE (items and dimensions) is fully described in the (open source) Python code and not in the (closed source) Java code, then:
• Those variables/equations will always exist in every Scenario, and
• The list of variables/equations to read with the solution can then be introspected from the contents of the Scenario.