Arrays and Product Measures

[cscherrer]

In Soss, I use the iid and For combinators for array distributions with conditionally independent components. I'd like to clean things up a bit, so I've been thinking about how this might look.

In MeasureTheory.jl, we currently have a ProductMeasure struct with one field that's usually a tuple of measures representing the components. And today it hit me, this really could cover all of these cases.

In terms of the data representation, For is really just a MappedArray, and iid is just a FillArray. So I think it's natural to allow ProductMeasure to wrap an AbstractArray. We'll need some constructors for convenience in these cases, and maybe some type synonyms.

Any thoughts?

[devmotion]

This is one of the ideas behind Turings arraydist (= For) and filldist (= iid): it just is a product measure with an AbstractArray and a FillArray of distributions, respectively 🙂 There's one additional optimization: we don't use Distributions.Product directly but rather Distributions.product_distributions which allows to optimize the returned type in case the product distribution is e.g. a common and more optimized multivariate distribution. However, Distributions.Product is restricted to multivariate distributions, and hence additional types were introduced in DistributionsAD - I suggested to fix this upstream (I still think this should be done) but in the end it was decided to add custom types without going through this process.

[cscherrer]

Thanks @devmotion . I didn't know there were issues with Distributions.Product, but we won't be using that anyway. I think the it may only be the rand methods worth hanging on to.

[sethaxen]

Are these really examples of "Power Measures"? That is, a Product Measure where each component measure is identical to the others? Or is For more flexible than that? We had similar discussions on Manifolds.jl and ended up with two different representations for points on PowerManifold and two for points on ProductManifold. I'll write up a summary soon.

[cscherrer]

Power? No, but they are still product measures. Power measures are a special case of that.

[phipsgabler]

This kinda hits the quesion of "normalization", though. I.e., how (and whether?) to make sure that

Product([A, A, B]) == (A ⊗ A) ⊗ B == A ⊗ (A ⊗ B) == A² ⊗ B

They probably will have come up with some thoughts regarding this in Catlab.jl.

[mschauer]

We kind of have figured out this for tuple-types

[cscherrer]

I don't think they'll be ==, just isomorphic