Generating correlated distributions of data?

[spizwhiz]

Hello,

I have sampled data with the saltelli sampler, but so far have only been able to generate several univariate distrubtions.

I am wondering if it is possible to generate a multivariate distrubtion with a correlation matrix?

Thank You!

[jdherman]

Hi @spizwhiz ,
All of the methods in SALib currently assume independent parameters.

You could write your own modification to impose a correlation on a sample, but I'm not sure if the rest of the methods would still work as intended.

There are many papers that discuss this issue. For example:
https://www.sciencedirect.com/science/article/pii/S0951832005002231
To get these methods into SALib, we would need a pull request from an expert!

[spizwhiz]

Thank you.

I am afraid I am not that expert.

Looking at the paper, it seems like their approach was just to group correlated parameters into a sort of lumped parameter, and analyzed the sensitivity of the model to the independent groups of parameters. This makes sense for their application, but I am not sure it will apply to my model (richards equation based), as I am testing sensitivity to 4-6 parameters, with correlation coefficients ranging from +- .1 to +- .9, most of them absolute value greater than .5.

I'd be grateful if you had suggestions (or even relevant citations) for using SALib or other methods for sensitivity analysis on a non-linear model such as mine with correlated inputs.

I have developed scripts for generating the model inputs and processing the outputs, and am now looking to see if there the methods apply...maybe I got that backwards...

[spizwhiz]

I just wanted to briefly revisit this in case anyone is looking for this information.

According to Borgonovo (2007) the delta moment independent measure is valid even when parameters are correlated. It seems to me then that the delta methods implemented in SALib (from Plischke et al. (2013)) should also work for correlated parameter sets (and can even work with real data sets that are not sampled conditionally). For an example, see Borgonovo and Tarantola (2008).

Feel free to let me know if you think I've mis-read something.

Plischke E, Borgonovo E, Smith CL (2013) Global sensitivity measures from given data. European Journal of Operational Research 226:536–550. doi: 10.1016/j.ejor.2012.11.047

Borgonovo E, Tarantola S (2008) Moment independent and variance-based sensitivity analysis with correlations: An application to the stability of a chemical reactor. International Journal of Chemical Kinetics 40:687–698. doi: 10.1002/kin.20368

Borgonovo E (2007) A new uncertainty importance measure. Reliability Engineering & System Safety 92:771–784. doi: 10.1016/j.ress.2006.04.015

[willu47]

Thanks for the update @spizwhiz, that's very useful to know and record here.

[lukas1994]

Any updates on this? Would any of the implemented methods work if I provide my own sample (sample directly from the joint distribution)?

[willu47]

I just wanted to briefly revisit this in case anyone is looking for this information.

According to Borgonovo (2007) the delta moment independent measure is valid even when parameters are correlated. It seems to me then that the delta methods implemented in SALib (from Plischke et al. (2013)) should also work for correlated parameter sets (and can even work with real data sets that are not sampled conditionally). For an example, see Borgonovo and Tarantola (2008).

@lukas1994 - Plischke is implemented in SALib as DMIF, so give that a try.

[lukas1994]

If I understand correctly, LHS (SALib.sample.latin.sample) is just used to reduce the variance of the sensitivity measures. So I can provide my own (correlated) sample instead.
Is this correct?

[jdherman]

@lukas1994 yes, you should be able to swap any sampling method in place of LHS here

[sahin-abdullah]

A method that accounts for correlated inputs have been added tonight. Once it is merged with master, you can see the implementation on case2, case3, and case5. Please note that these cases are ordered same as in the original paper.

[jdherman]

Thanks! for reference this is the HDMR method in #275 , we will start reviewing.