Why are SArrays extremely memory inefficient?

[Volker-Weissmann]

Hello,

This should allocate 8 MB,

zeros(SArray{Tuple{1024,1024},Float64})

but it prints

ERROR: syntax: expression too large

I did some smaller allocations, checked the output of htop and found that zeros(SArray{Tuple{128,N},Float64}) needs wayyyy more memory than the N kB expected.

[mschauer]

Note that in the current implementation, working with large StaticArrays puts a lot of stress on the compiler, and becomes slower than Base.Array as the size increases. A very rough rule of thumb is that you should consider using a normal Array for arrays larger than 100 elements.

From the ReadMe.md

[andyferris]

Yes.

Basically most functions in StaticArrays are generated dynamically (via @generated) and consist of a small number of lines of code per array element. So the example above has millions of lines of code that needs to be processed by the compiler and LLVM.

The compiler not only temporarily uses these expressions while compiling but caches much of it. So the first time you call zeros(SArray{Tuple{128,N},Float64}) for some small N you will see htop memory usage bump quite a lot (also - these compiler cached things are stored until you quite Julia and are never freed).

For each instance of that type that you keep, you'll store an additional N kB of program data on top of all that compiled code. However - even then the compiler is free to choose to box such a large instance, using slightly more allocations and RAM.

While the documentation mentions arrays up to 100 elements I tend to recommend it for e.g. geometry with 3-vectors and rotation matrices, and similar.

[Volker-Weissmann]

Ok. That's sad, but Thank you.
I think I need to stick to Base.Array then.