Elementary effects for models that have arrays as outputs

[lbteixeira]

Hello,

I have an application where my model output isn't a scalar, but an array (the pixels values of an image).

The analyze function of Morris method currently calculates the elementary effects by simply subtracting the model outputs and dividing by delta.

In my case, I need to subtract the model outputs, calculate their L2 norm and calculate an internal product between two vectors.

I would like to include an extra parameter to the function analyze. This parameter would be a function defined by the user, that would allow any kind of customization on how to calculate the elementary effects.

[willu47]

This sounds like feature that is more general than just for the elementary effects method. Inserting a results pre-processing function into the analyse function could be applied to any of the methods.

But can't you just do this upstream of the analyze call anyway? I'm not sure adding this really provides much value to the wider user group of SALib. Anyone else have an opinion here?

[ConnectedSystems]

Hi @lbteixeira

I'm in agreement with Will here that generalising pre/post-processing of results to be part of the package will add more complexity (to the package) than is worthwhile.

If I understand what you need correctly, this could be accomplished by wrapping the "real" model function:

def actual_model(X):
    # Does actual calculation
    return Y

def wrapped_model(X):
    Y_actual = actual_model(X)

    Y_mod = # ...process model results...

    return Y_mod

X = latin.sample(problem, 1000)
Y = wrapped_model(X)
Si = morris.analyze(problem, X, Y)

That is of course if I've understood you correctly. If not, then I guess I'd need more information.

[lbteixeira]

Hello, @willu47 and @ConnectedSystems.

For the case of Morris method, I believe I can't use your suggestions. What I want to use isn't actually a pre-processing, but I an alternative way to calculate the difference between two points of a trajectory.

The module analyze/morris.py has a method called _reorganize_output_matrix (according to the new implementation I proposed in #328). As its name suggests, this method will reorganize the output matrix in a way that the difference between two points in a trajectory is correctly calculated. For example, if the parameter x2 changed its value from point 5 to point 6 in the trajectory, the elementary effect related to x2 is the difference between the output from point 6 and point 5, divided by delta. I don't know a priori what are these points, therefore I can't "pre-process" the outputs.

In this example, in the current implementation the method _calc_results_difference only does Y(point 6) - Y(point 5). Later, the method _compute_elementary_effects divides the result of this operation by delta. What I want to do is to allow the user to customize _calc_results_difference and be able to do, lets say, sqrt(Y(point 6) - Y(point 5)). This isn't actually the function I want to use, of course...

In my case this is important because my elementary effects need to be calculated by doing some operations between arrays, not a simple difference between scalars.

I hope I was clear about my intentions, it's a little difficult to explain. Let me know what you think about this.

[ConnectedSystems]

I think I understand.

The question then is: "is this something that would be useful for others?"

If yes, then we can talk about how best to support it, and whether such changes (that is, changing how indices are calculated) would be of interest to other methods. Right now I'm thinking of a functional approach where the user would provide a function to use instead, but that will get really messy quickly if other changes need to be considered.

If no, then you might be better off creating a separate branch in your repo (or forking the project entirely) with the purpose specific changes. I have colleagues who have done just this.