Improve ImmutableArrayExtensions.SequenceEqual
#118932
Conversation
github-actions
bot
added
the
needs-area-label
dotnet-policy-service
bot
added
the
community-contribution
|
@dotnet-policy-service agree |
src/libraries/System.Collections.Immutable/src/System/Linq/ImmutableArrayExtensions.cs
Outdated
Show resolved
Hide resolved
jkotas
added
area-System.Collections
and removed
needs-area-label
|
Tagging subscribers to this area: @dotnet/area-system-collections |
src/libraries/System.Collections.Immutable/src/System/Linq/ImmutableArrayExtensions.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Collections.Immutable/src/System/Linq/ImmutableArrayExtensions.cs
Outdated
Show resolved
Hide resolved
|
Perhaps a simpler (and faster) way to do this is to just call |
…utableArrayExtensions.cs Co-authored-by: Pranav Senthilnathan <[email protected]>
|
I'm not sure if it can be applied to |
This should work: Requires.NotNull(items, nameof(items));
immutableArray.ThrowNullRefIfNotInitialized();
return immutableArray.array.SequenceEqual((IEnumerable<TBase>)items, comparer);If it is acceptable to delegate the argument validation to return immutableArray.array.SequenceEqual((IEnumerable<TBase>)items, comparer); |
|
@xtqqczze Full benchmark results |
@prozolic I see a few regressions. Could you collect benchmark data for https://github.com/xtqqczze/dotnet-runtime/tree/ImmutableArrayExtensions.SequenceEqual?. This approach will not be worth the additional complexity, however. |
|
Sorry. Indeed, there were memory allocation issues and regressions in some cases. I conducted performance measurements comparing different implementations of SequenceEqual.
Full benchmark results#if NET6_0_OR_GREATER
public static bool SequenceEqual_2<TDerived, TBase>(this ImmutableArray<TBase> immutableArray, IEnumerable<TDerived> items, IEqualityComparer<TBase>? comparer = null) where TDerived : TBase
{
immutableArray.ThrowNullRefIfNotInitialized();
Requires.NotNull(items, nameof(items));
int i = 0;
if (items is ICollection<TBase> itemsCol)
{
if (itemsCol.TryGetSpan(out ReadOnlySpan<TBase> itemsSpan))
{
return immutableArray.array!.SequenceEqual(itemsSpan, comparer);
}
if (immutableArray.array!.Length != itemsCol.Count)
{
return false;
}
if (itemsCol is IList<TDerived> itemsList)
{
comparer ??= EqualityComparer<TBase>.Default;
int count = immutableArray.array!.Length;
for (i = 0; i < count; i++)
{
if (!comparer.Equals(immutableArray.array![i], itemsList[i]))
{
return false;
}
}
return true;
}
}
comparer ??= EqualityComparer<TBase>.Default;
int n = immutableArray.array!.Length;
foreach (TDerived item in items)
{
if (i == n)
{
return false;
}
if (!comparer.Equals(immutableArray.array![i], item))
{
return false;
}
i++;
}
return i == n;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)] // fast type checks that don't add a lot of overhead
private static bool TryGetSpan<TSource>(this IEnumerable<TSource> source, out ReadOnlySpan<TSource> span)
{
// Use `GetType() == typeof(...)` rather than `is` to avoid cast helpers. This is measurably cheaper
// but does mean we could end up missing some rare cases where we could get a span but don't (e.g. a uint[]
// masquerading as an int[]). That's an acceptable tradeoff. The Unsafe usage is only after we've
// validated the exact type; this could be changed to a cast in the future if the JIT starts to recognize it.
// We only pay the comparison/branching costs here for super common types we expect to be used frequently
// with LINQ methods.
bool result = true;
if (source.GetType() == typeof(TSource[]))
{
span = Unsafe.As<TSource[]>(source);
}
else if (source.GetType() == typeof(List<TSource>))
{
span = CollectionsMarshal.AsSpan(Unsafe.As<List<TSource>>(source));
}
else
{
span = default;
result = false;
}
return result;
}
#endif |
OK, so using I think the best balance of complexity to performance would be to delegate to |
|
Something like: {
if (items is ICollection<TBase> itemsCol)
{
immutableArray.ThrowNullRefIfNotInitialized();
return immutableArray.array!.SequenceEqual(itemsCol, comparer);
}
return Impl(immutableArray, items, comparer);
}and then put argument validation and existing implementation in static local function |
|
Thank you for the advice! public static bool SequenceEqual<TDerived, TBase>(this ImmutableArray<TBase> immutableArray, IEnumerable<TDerived> items, IEqualityComparer<TBase>? comparer = null) where TDerived : TBase
{
if (items is ICollection<TBase> itemsCol)
{
immutableArray.ThrowNullRefIfNotInitialized();
return immutableArray.array!.SequenceEqual(itemsCol, comparer);
}
return Impl(immutableArray, items, comparer);
static bool Impl(ImmutableArray<TBase> immutableArray, IEnumerable<TDerived> items, IEqualityComparer<TBase>? comparer)
{
Requires.NotNull(items, nameof(items));
comparer ??= EqualityComparer<TBase>.Default;
int i = 0;
int n = immutableArray.Length;
foreach (TDerived item in items)
{
if (i == n)
{
return false;
}
if (!comparer.Equals(immutableArray[i], item))
{
return false;
}
i++;
}
return i == n;
}
} |
|
@prozolic You don't need the Could you also provide benchmark data again? |
|
@xtqqczze Here are the benchmark results comparing with the current existing implementation (SequenceEqual_Original): |
|
@prozolic The results look excellent, very close to |
|
It doesn’t look like we currently have performance coverage for this method in https://github.com/dotnet/performance. |
|
You're right, there's no official performance test coverage for this yet. From what I can see, there isn't much performance test coverage for ImmutableArray itself in dotnet/performance either. |
|
Regarding the benchmark, I submitted a pull request at dotnet/performance#4915. Original: Modified: |
|
@prozolic Does PR description need updating with new performance numbers? |
|
@xtqqczze Therefore, I have updated the Summary description to match the current implementation and changed the benchmark results to those from dotnet/performance#4915. |
We cannot make this conclusion from the data, as there is high Error/StdDev for the ICollection benchmark. Were other applications running at the same time as the benchmark?
We didn't delegate to |
You're right about the ICollection claim. I've updated it to "No statistically significant performance difference" (please let me know if there's a better wording). Regarding benchmark results, other applications may have been running and affecting the measurements, so I re-ran the benchmarks in a clean environment and updated the results.
I've updated it to "Uses existing implementation for this case because the existing implementation had better performance and didn't allocate a second enumerator" |
| int i = 0; | ||
| int n = immutableArray.Length; | ||
| foreach (TDerived item in items) | ||
| static bool Impl(ImmutableArray<TBase> immutableArray, IEnumerable<TDerived> items, IEqualityComparer<TBase>? comparer) |
There was a problem hiding this comment.
I think you can change the type of the items parameter here to IEnumerable<TBase> and cast at the call site. Let's see what other reviewers think.
There was a problem hiding this comment.
I don't think there would be much difference in complexity or performance compared to the current implementation, so I personally think we don't particularly need to make the change.
There was a problem hiding this comment.
There may be a difference on R2R due to boxing, I'm not sure.
There was a problem hiding this comment.
I'm also not very familiar with R2R (ReadyToRun), so I'm sorry but I don't know the impact of this change either.
src/libraries/System.Collections.Immutable/src/System/Linq/ImmutableArrayExtensions.cs
Outdated
Show resolved
Hide resolved
…utableArrayExtensions.cs Co-authored-by: xtqqczze <[email protected]>
Summary
This PR improves the performance of ImmutableArrayExtensions.SequenceEqual method through optimized implementation strategies:
Key Performance Improvements:
Implementation Approach
The implementation was refined based on review feedback from @neon-sunset and @xtqqczze to achieve optimal performance while balancing complexity:
ICollection<T>and delegates to the highly optimizedEnumerable.SequenceEqualusing the underlying arrayIEnumerable<T>types to maintain behavioral consistencyBenchmark (dotnet/performance#4915)
Original:
Modified: