Test failure in `LinearAlgebra/lapack`

[giordano]

Observed in Julia CI for i686-linux-gnu: https://buildkite.com/julialang/julia-master/builds/47172#01969e04-8a10-4791-ac12-153b4fc900e3/905-1708, this is also reproducible locally for me:

julia> Base.runtests("LinearAlgebra/lapack"; seed=0x4892b45f4810fc1b1f64f21e515d4024)
Running parallel tests with:
  getpid() = 4015
  nworkers() = 1
  nthreads(:interactive) = 1
  nthreads(:default) = 1
  Sys.CPU_THREADS = 2
  Sys.total_memory() = 3.872 GiB
  Sys.free_memory() = 3.139 GiB

Test             (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
LinearAlgebra/lapack  (1) |        started at 2025-05-05T19:02:25.247
LinearAlgebra/lapack  (1) |         failed at 2025-05-05T19:03:40.848
Test Failed at /home/mose/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/LinearAlgebra/test/lapack.jl:289
  Expression: b ≈ c
   Evaluated: Float32[159.31934, -64.649155, -2131.8872, 6787.0464, -12266.052, 295.2029, 2174.87, -1643.5519, -3781.9094, 2164.3667] ≈ Float32[159.25502, -64.62289, -2131.0276, 6784.31, -12261.106, 295.0839, 2173.9937, -1642.889, -3780.3843, 2
163.494]
Test Failed at /home/mose/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/LinearAlgebra/test/lapack.jl:316
  Expression: A \ b ≈ c
   Evaluated: Float32[-593.7424 -13.895346 … -225.10986 -702.97534; 243.55264 6.9179006 … 92.377785 287.7254; … ; 14076.482 338.5684 … 5353.0425 16684.324; -8053.3696 -192.93784 … -3062.0056 -9545.351] ≈ Float32[-593.98193 -13.901099 … -225.200
93 -703.25916; 243.65048 6.9202504 … 92.414986 287.84134; … ; 14082.161 338.70483 … 5355.2017 16691.057; -8056.6187 -193.0159 … -3063.2412 -9549.203]


Test Summary:            | Pass  Fail  Total     Time
  Overall                | 1181     2   1183  1m19.4s
    LinearAlgebra/lapack | 1181     2   1183  1m18.1s
    FAILURE

The global RNG seed was 0x4892b45f4810fc1b1f64f21e515d4024.

Error in testset LinearAlgebra/lapack:
Test Failed at /home/mose/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/LinearAlgebra/test/lapack.jl:289
  Expression: b ≈ c
   Evaluated: Float32[159.31934, -64.649155, -2131.8872, 6787.0464, -12266.052, 295.2029, 2174.87, -1643.5519, -3781.9094, 2164.3667] ≈ Float32[159.25502, -64.62289, -2131.0276, 6784.31, -12261.106, 295.0839, 2173.9937, -1642.889, -3780.3843, 2163.494]
Error in testset LinearAlgebra/lapack:
Test Failed at /home/mose/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/LinearAlgebra/test/lapack.jl:316
  Expression: A \ b ≈ c
   Evaluated: Float32[-593.7424 -13.895346 … -225.10986 -702.97534; 243.55264 6.9179006 … 92.377785 287.7254; … ; 14076.482 338.5684 … 5353.0425 16684.324; -8053.3696 -192.93784 … -3062.0056 -9545.351] ≈ Float32[-593.98193 -13.901099 … -225.20093 -703.25916; 243.65048 6.9202504 … 92.414986 287.84134; … ; 14082.161 338.70483 … 5355.2017 16691.057; -8056.6187 -193.0159 … -3063.2412 -9549.203]
ERROR: LoadError: Test run finished with errors
in expression starting at /home/mose/.julia/juliaup/julia-nightly/share/julia/test/runtests.jl:87
ERROR: A test has failed. Please submit a bug report (https://github.com/JuliaLang/julia/issues)
including error messages above and the output of versioninfo():
Julia Version 1.13.0-DEV.456
Commit 175ef3eb02c (2025-04-28 13:13 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (i686-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz
  WORD_SIZE: 32
  LLVM: libLLVM-19.1.7 (ORCJIT, haswell)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)

The offending tests are

LinearAlgebra.jl/test/lapack.jl

Lines 283 to 289 in 07725da

	du = rand(elty,9)
	d = rand(elty,10)
	dl = rand(elty,9)
	b = rand(elty,10)
	c = Tridiagonal(dl,d,du) \ b
	b = LAPACK.gtsv!(dl,d,du,b)
	@test b ≈ c

and

LinearAlgebra.jl/test/lapack.jl

Lines 299 to 316 in 07725da

	du = rand(elty,9)
	d = rand(elty,10)
	dl = rand(elty,9)
	b = rand(elty,10)
	y10 = Vector{BlasInt}(undef, 10)
	x9, x11 = Vector{elty}.(undef, (9, 11))
	@test_throws DimensionMismatch LAPACK.gttrf!(x11, d, du)
	@test_throws DimensionMismatch LAPACK.gttrf!(dl, d, x11)
	@test_throws DimensionMismatch LAPACK.gttrs!('N', x11, d, du, x9, y10, b)
	@test_throws DimensionMismatch LAPACK.gttrs!('N', dl, d, x11, x9, y10, b)
	@test_throws DimensionMismatch LAPACK.gttrs!('N', dl, d, du, x9, y10, x11)
	@test_throws ArgumentError LAPACK.gttrs!('X', dl, d, du, x9, y10, x11)
	A = lu(Tridiagonal(dl,d,du))
	b = rand(elty,10,5)
	c = copy(b)
	dl,d,du,du2,ipiv = LAPACK.gttrf!(dl,d,du)
	c = LAPACK.gttrs!('N',dl,d,du,du2,ipiv,c)
	@test A\b ≈ c

both for elty = Float32

[ViralBShah]

Can you locally reproduce on a 64-bit machine?

[giordano]

No, only 32-bit so far, but I could reproduce it exactly with the fixed seed.

[ViralBShah]

Where do you test it? Do you have a 32-bit linux machine, or can one run a whole 32-bit setup on 64-bit?

[giordano]

I got access to a 32-bit machine. I presume a VM/Docker would work as well.

[aravindh-krishnamoorthy]

No, only 32-bit so far, but I could reproduce it exactly with the fixed seed.

Is this still relevant?

If so, which output (\ or LAPACK.gxxxx) is wrong, given the inputs? If you could dump the input matrices for both test cases in binary format here, I could also take a look.