vcat / hcat is broken on julia 1.10

[takbal]

hcat and vcat seems to be in conflict with SparseArrays in julia-1.10, that is also pirating these Base calls now.

Works on 1.9.

julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39 (2023-12-25 18:01 UTC)
Build Info:

    Note: This is an unofficial build, please report bugs to the project
    responsible for this build and not to the Julia project unless you can
    reproduce the issue using official builds available at https://julialang.org/downloads

Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × AMD Ryzen 7 1700 Eight-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver1)
  Threads: 24 on 16 virtual cores
Environment:
  JULIA_EDITOR = code

(@v1.10) pkg> add AxisKeys
julia> using AxisKeys

julia> a = wrapdims(zeros(2,2), foo = [1,2], bar = [1,2])
julia> b = wrapdims(zeros(2,2), foo = [1,2], bar = [1,2])
julia> [a ; b]

ERROR: MethodError: vcat(::KeyedArray{Float64, 2, NamedDimsArray{(:foo, :bar), Float64, 2, Matrix{Float64}}, Tuple{Vector{Int64}, Vector{Int64}}}, ::KeyedArray{Float64, 2, NamedDimsArray{(:foo, :bar), Float64, 2, Matrix{Float64}}, Tuple{Vector{Int64}, Vector{Int64}}}) is ambiguous.

Candidates:
  vcat(A::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, B::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:147
  vcat(X1::Union{Number, AbstractVecOrMat{<:Number}}, X::Union{Number, AbstractVecOrMat{<:Number}}...)
    @ SparseArrays ~/software/julia-1.10.0/share/julia/stdlib/v1.10/SparseArrays/src/sparsevector.jl:1235
  vcat(A::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, B::AbstractVecOrMat, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:147
  vcat(A::AbstractVecOrMat, B::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:147

Possible fix, define
  vcat(::Union{KeyedArray{…}, KeyedArray{…}}, ::Union{KeyedArray{…}, KeyedArray{…}}, ::Vararg{AbstractVecOrMat{…}})

Stacktrace:
 [1] top-level scope
   @ REPL[7]:1
Some type information was truncated. Use `show(err)` to see complete types.

julia> [a b]

ERROR: MethodError: hcat(::KeyedArray{Float64, 2, NamedDimsArray{(:foo, :bar), Float64, 2, Matrix{Float64}}, Tuple{Vector{Int64}, Vector{Int64}}}, ::KeyedArray{Float64, 2, NamedDimsArray{(:foo, :bar), Float64, 2, Matrix{Float64}}, Tuple{Vector{Int64}, Vector{Int64}}}) is ambiguous.

Candidates:
  hcat(A::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, B::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:155
  hcat(X1::Union{Number, AbstractVecOrMat{<:Number}}, X::Union{Number, AbstractVecOrMat{<:Number}}...)
    @ SparseArrays ~/software/julia-1.10.0/share/julia/stdlib/v1.10/SparseArrays/src/sparsevector.jl:1229
  hcat(A::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, B::AbstractVecOrMat, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:155
  hcat(A::AbstractVecOrMat, B::Union{KeyedArray{T, 1, AT, RT}, KeyedArray{T, 2, AT, RT}} where {T, AT, RT}, Cs::AbstractVecOrMat...)
    @ AxisKeys ~/.julia/packages/AxisKeys/QsNqy/src/functions.jl:155

Possible fix, define
  hcat(::Union{KeyedArray{…}, KeyedArray{…}}, ::Union{KeyedArray{…}, KeyedArray{…}}, ::Vararg{AbstractVecOrMat{…}})

Stacktrace:
 [1] top-level scope
   @ REPL[8]:1
Some type information was truncated. Use `show(err)` to see complete types.

[aplavin]

Yeah, that's a known breaking change in Julia 1.10 unfortunately.
JuliaLang/julia#52386 JuliaSparse/SparseArrays.jl#431

[jtackm]

I'm having the same issue unfortunately. While this is still pending, Is there any quick work around without downgrading julia?

[mcabbott]

The new thing on 1.10 is methods with AbstractVecOrMat{<:Number} defined by SparseArrays. These amount to piracy of Base as it does not own any of those types. The nicest solution would be for those methods to be removed, but from JuliaSparse/SparseArrays.jl#431 it looks like nobody has tried.

The alternative is to add ever more specific method here. There's already quite a list, in order for this package and NamedDims.jl to be able to be used together. Adding some versions with {<:Number} may solve this:

https://github.com/mcabbott/AxisKeys.jl/blob/master/src/functions.jl#L108-L113

Unfortunately I think the problem of many packages extending the same functions doesn't really have a satisfying solution, perhaps it's a flaw of multiple dispatch. The best I can think of would be first that Base uses similar(..., axes(x,1)) etc instead of size (which would also allow OffsetArrays, something like JuliaLang/julia#37629, and JuliaLang/julia#43552) and then this package return special axes (as in #6). Then perhaps no overload of vcat (nor *) would be needed, and the only tricky function would be similar.

[takbal]

@jtackm in case you can modify the failing code, replacing [a ; b] with cat(a, b; dims=1) and [a b] with cat(a, b; dims=2) works for now.

[aplavin]

A workaround I found working reliably is:

Base.delete_method.(filter(
    m -> nameof(m.module) == :SparseArrays && m.sig == Tuple{typeof(vcat), Union{Number,AbstractVecOrMat{<:Number}}, Vararg{Union{Number,AbstractVecOrMat{<:Number}}}},
    methods(vcat)
))
Base.delete_method.(filter(
    m -> nameof(m.module) == :SparseArrays && m.sig == Tuple{typeof(hcat), Union{Number,AbstractVecOrMat{<:Number}}, Vararg{Union{Number,AbstractVecOrMat{<:Number}}}},
    methods(hcat)
))

Put this code into your script/code anywhere after package imports. It deletes those pirating methods defined in SparseArrays, getting rid of such ambiguities. At the same time, it doesn't interfere with vcat/hcat on actual sparse arrays – they continue returning sparse arrays.

[NittanyLion]

fwiw: This also arises when one loads the Optim package, which presumably loads SparseArrays.

[aplavin]

#169 fixed