StaticArrays load time, possibility for abstract interface package?

[oschulz]

StaticArrays (together with ArraysInterface) seems to be a driving factor in package load times nowadays. For example, the load time of FiniteDiff, FiniteDifferences and ForwardDiff (common direct/indirect dependencies themselves) themselves is mainly due to StaticArrays (and ArrayInterface):

julia> @time_imports using FiniteDiff
      3.3 ms  ┌ Compat
      0.5 ms  ┌ Requires
    548.3 ms  ┌ StaticArrays
      0.2 ms  ┌ IfElse
     33.7 ms    ┌ Static
    737.6 ms  ┌ ArrayInterface
   1312.8 ms  FiniteDiff

julia> @time_imports using FiniteDifferences
      3.6 ms  ┌ Compat
      0.4 ms  ┌ Richardson
    539.9 ms  ┌ StaticArrays
     62.7 ms  ┌ ChainRulesCore
    617.1 ms  FiniteDifferences

julia> @time_imports using ForwardDiff
    538.5 ms    ┌ StaticArrays
    548.4 ms  ┌ DiffResults
      4.1 ms  ┌ Compat
      0.4 ms  ┌ NaNMath
     60.8 ms    ┌ ChainRulesCore
     62.0 ms  ┌ ChangesOfVariables
      0.7 ms  ┌ OpenLibm_jll
      3.0 ms  ┌ DocStringExtensions
      4.6 ms  ┌ IrrationalConstants
      0.6 ms  ┌ CompilerSupportLibraries_jll
      1.4 ms    ┌ LogExpFunctions
     18.9 ms        ┌ Preferences
     19.8 ms      ┌ JLLWrappers
     23.9 ms    ┌ OpenSpecFun_jll
     37.6 ms  ┌ SpecialFunctions
     10.3 ms    ┌ MacroTools
     11.0 ms  ┌ CommonSubexpressions
      0.8 ms  ┌ DiffRules
    845.2 ms  ForwardDiff

Even the load time of a "large" package like Optim is completely dominated by StaticArrays and ArrayInterface (indirectly, via FiniteDiff and ForwardDiff):

julia> @time using StaticArrays
  0.565813 seconds (2.21 M allocations: 175.407 MiB, 0.51% compilation time)

julia> @time using ArrayInterface
  0.786209 seconds (697.28 k allocations: 38.650 MiB, 4.49% gc time, 89.27% compilation time)

julia> @time using FiniteDiff
  0.037383 seconds (46.28 k allocations: 2.584 MiB, 77.77% compilation time)

julia> @time using ForwardDiff
  0.199031 seconds (543.28 k allocations: 35.849 MiB, 2.18% compilation time)

julia> @time using Optim
  0.302545 seconds (730.67 k allocations: 53.416 MiB)

There's currently and effort to make ArrayInterface more lightweight by splitting it up (JuliaArrays/ArrayInterface.jl#211) - I wonder if the same would be possible for StaticArrays? Would something like an AbstractStaticArrays be enough for packages like the above? I think at least some of them only "support" static arrays but don't use them for their core functionality (I may be wrong).

[mateuszbaran]

As far as I know the time spent in loading StaticArrays is dominated insertion of methods with complicated signatures (for example five-argument static structured matrix multiplication). I would totally support making something like StaticArraysCore.jl that is just a copy of StaticArrays.jl with some of these complicated methods removed. Then StaticArrays.jl would just import and extend StaticArraysCore.jl. People using StaticArrays.jl wouldn't see any difference but they would have the option to just depend on StaticArraysCore.jl if they don't need that additional functionality.

The most complicated part of work here is determining which methods shouldn't be moved to StaticArraysCore.jl which requires some cooperation from maintainers of major packages that use StaticArrays.jl.

[oschulz]

but they would have the option to just depend on StaticArraysCore.jl

Yes - there are some packages that really utilize StaticArrays for what they do - they're little that can be done there, of course. But then there's packages that only need StaticArrays to add some compatibility with it (e.g. this code I have in ArraysOfArrays, via @require to not depend on StaticArrays, but that's kinda ugly). For those cases a package StaticArraysCore should help a lot.

I'm not quite sure in which of these categories ForwardDiff falls (StaticArrays is a big part of it's load time).

[mateuszbaran]

I've just made a quick test and just removing advanced linear algebra gets us down to about 0.17-0.20 s (depending on the details). Would that be fine? Or do you think it's still too much?

[oschulz]

I guess it depends on how much it is relative to the package that depends on StaticArrays. :-)

I worry a bit about having a "stripped down" version of StaticArrays that contains concrete types (not just and interface) without full functionality. In applications that use a lot of generic code, the "full" StaticArrays might not get loaded automatically, leading to non-performant behavior (or maybe even errors due to missing functionality).

[oschulz]

Maybe having abstract static arrays would be enough to satisfy the needs of many packages now depending on StaticArrays, esp. if they use similar() and the like to create new arrays?

[mateuszbaran]

If a package only had the abstract type and some functions, it wouldn't have SVector so it would be insufficient even for your example from ArraysOfArrays. I personally doubt just dispatching on StaticArray is particularly useful but feel free to prove me wrong :).

[oschulz]

it wouldn't have SVector so it would be insufficient even for your example from ArraysOfArrays.

I have to admit that example wasn't ideal. flatview part could be done, though, as something like

@inline flatview(A::AbstractArray{SA,N}) where {S,T,M,N,SA<:AbstractStaticArray{S,T,M}} =
    reshape(reinterpret(T, A), size(SA)..., size(A)...)

And I could actually drop nestedview as currently designed.

But you're right, an abstract interface would not be sufficient. It would be better to move these definitions from ArraysOfArrays into StaticArrays itself. I'm currently in the process of moving the AbstractArrays interface into AbstractArraysOfArrays (currently under design). AbstractArraysOfArrays will be super-lightweight, so it should be an acceptable dependency for StaticArrays.

[ChrisRackauckas]

I did an audit of the SciML codebase. We recently removed all usage of Requires.jl. Our measurements show that StaticArrays.jl and GPUArrays.jl are by far and away the majority of the startup time. Period. No questions there anymore, thanks for @time_imports.

Codebase Audit

And from the codebase:

https://github.com/SciML/RecursiveArrayTools.jl/blob/master/src/utils.jl#L4
https://github.com/SciML/SciMLBase.jl/blob/bfc028d395bbc846f9a75a370a74bad41f72a007/src/operators/basic_operators.jl#L108
https://github.com/SciML/DiffEqBase.jl/blob/8e38359319ed5cb9b8bc469ec7975b36db0fee0a/src/solve.jl#L489
https://github.com/SciML/DiffEqBase.jl/blob/b533f1255da9a1461f8c25787cec77ffc6037ff0/src/common_defaults.jl#L12-L26
https://github.com/SciML/ResettableStacks.jl/blob/master/src/core.jl#L28
https://github.com/SciML/DEDataArrays.jl/blob/9f7e853d359aeb14be515943eeb4b98ddb2a2c17/src/DEDataArrays.jl#L51
https://github.com/SciML/DiffEqPhysics.jl/blob/7bec56f72c0c6e71b50e498dfdcb691e92f91d52/src/hamiltonian.jl#L29
https://github.com/SciML/NonlinearSolve.jl/blob/51c443eae5ca5e4d54067c4ca5db14792c3240c2/src/utils.jl#L189
https://github.com/SciML/StochasticDelayDiffEq.jl/blob/233e46dff6cafb938feec059796179b29d092a3f/src/solve.jl#L243
https://github.com/SciML/SimpleDiffEq.jl/blob/ef0cece5781686dbaaebc019de40d927e61fa6f1/src/SimpleDiffEq.jl#L10
https://github.com/SciML/SimpleDiffEq.jl/blob/ef0cece5781686dbaaebc019de40d927e61fa6f1/src/rk4/gpurk4.jl
https://github.com/SciML/SimpleDiffEq.jl/blob/ef0cece5781686dbaaebc019de40d927e61fa6f1/src/tsit5/gpuatsit5.jl
https://github.com/SciML/SimpleDiffEq.jl/blob/ef0cece5781686dbaaebc019de40d927e61fa6f1/src/tsit5/tsit5.jl
https://github.com/SciML/SimpleDiffEq.jl/blob/11c7f9e726f4d4d66cf2903a02f650be72c637e9/src/tsit5/atsit5.jl
https://github.com/SciML/ModelingToolkit.jl/blob/1b98b86ba716a06150fb6a7bf03ca2d52bc5ce9f/src/structural_transformation/codegen.jl#L184
https://github.com/JuliaSymbolics/SymbolicUtils.jl/blob/7f8cfbd026972a61a727a904e9b182367921093f/src/code.jl#L529-L564

If an interface package just had StaticArray, SArray, and MArray (and StaticArrays.StaticArrayStyle, though honestly that usage may be able to be deleted), then one of the libraries everyone points to as the big case of large load times will almost completely eliminate the load times.

There's two other cases in the SciML code base to look at.

With one outlier using a bit more:

https://github.com/SciML/LabelledArrays.jl/blob/edfffa99a85054aa7f184c9b90b4ba8fc6f8858b/src/slarray.jl#L31-L197

LabelledArrays.jl is a bit weird because it's defining a StaticArray kind of type. Even then, it only needs StaticArrays.similar_type and StaticArrays.LU.

The other case could just be tuples:

https://github.com/SciML/OrdinaryDiffEq.jl/blob/3a364726f8a464470a7fa03f1dae3168741124c4/src/caches/bdf_caches.jl#L292
https://github.com/SciML/OrdinaryDiffEq.jl/blob/3076b4ef7015cc3d683a82087761d56f20ac43d9/src/bdf_utils.jl#L80

so ignore those (@YingboMa we might as well just convert them now to clean the code base.

Conclusion

It's clear from the data: if there was a core library with StaticArray, SArray, MArray, StaticArrays.StaticArrayStyle, StaticArrays.similar_type, and StaticArrays.LU, then the majority of the SciML load times would be gone. This is because we only need to check if someone gives us static arrays, there are no static arrays used internally.

To me, that's a very strong indicator that there should be such an interface package.

[oschulz]

if there was a core library with [...] This is because we only need to check if someone gives us static arrays

Does that mean that even an AbstractStaticArray and so on would be sufficient?

[ChrisRackauckas]

Well, we need to know the difference between MArray and SArray since MArray is mutable, so we want a higher level type for immutable static arrays, but yes if there was an abstract type for those then we'd have all we need because we could just do parameterless_type(x)(y) as a constructor and such.

[oschulz]

Well, we need to know the difference between MArray and SArray since MArray is mutable

So maybe an AbstractStaticArray with subtypes AbstractImmutableStaticArray and AbstractMutableStaticArray?

because we could just do parameterless_type(x)(y) as a constructor and such.

Wouldn't ConstructionBase.constructorof be a clean solution here?