Skip to content

Fix: Cosmos ToList on structural collection is treated as scalar #38122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JoasE wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix: Cosmos ToList on structural collection is treated as scalar #38122

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions src/EFCore.Cosmos/Query/Internal/CosmosProjectionBindingExpressionVisitor.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -619,6 +619,8 @@ UnaryExpression unaryExpression
EnumerableMethods.Select.MakeGenericMethod(method.GetGenericArguments()),
shaper,
lambda);
default:
throw new InvalidOperationException(CoreStrings.TranslationFailed(methodCallExpression.Print()));
}
}
else if (method is { Name: nameof(Enumerable.ToList), IsGenericMethod: true }
Expand All @@ -632,14 +634,19 @@ UnaryExpression unaryExpression
throw new InvalidOperationException(CoreStrings.TranslationFailed(methodCallExpression.Print()));
}

if (array is not SqlExpression scalarArray)
{
// #34067
return Visit(argument);
}

// If ToList() was composed over a subquery with operators, the result here is an ArrayExpression (ARRAY(SELECT ...)), whose
// CLR Type is IEnumerable<T>. This can be directly used in the resulting ProjectingBindingExpression - the shaper will
// simply read the JSON results out successfully.
// But if ToList() is composed directly over an array property, that property could have type e.g. T[], which will be read
// in the shaper, and then the cast from T[] to List<T> will fail. As a result, wrap the array in an additional
// "reprojection" subquery, effectively to change the CLR type.
if (array is SqlExpression scalarArray
&& !(array.Type.IsGenericType && array.Type.GetGenericTypeDefinition() == typeof(IEnumerable<>)))
if (!(array.Type.IsGenericType && array.Type.GetGenericTypeDefinition() == typeof(IEnumerable<>)))
{
Check.DebugAssert(
array is not ScalarArrayExpression and not ObjectArrayExpression, "ArrayExpression should be IEnumerable");
Expand Down
15 changes: 2 additions & 13 deletions ...tionalTests/Query/Associations/ComplexProperties/ComplexPropertiesCollectionCosmosTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,19 +110,8 @@ FROM a IN c["AssociateCollection"]
public override Task Distinct()
=> AssertTranslationFailed(base.Distinct);

public override async Task Distinct_projected(QueryTrackingBehavior queryTrackingBehavior)
Copy link
Copy Markdown
Contributor Author

@JoasE JoasE Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This used to generate a JsonConvert.Deserialize call for materialization. I think we can't translate this because the Distinct would have to be ran in the materializer since we don't bind ["AssociateCollection"], and 34067 should solve that

{
await base.Distinct_projected(queryTrackingBehavior);

AssertSql(
"""
SELECT VALUE ARRAY(
SELECT DISTINCT VALUE a
FROM a IN c["AssociateCollection"])
FROM root c
ORDER BY c["Id"]
""");
}
public override Task Distinct_projected(QueryTrackingBehavior queryTrackingBehavior)
=> AssertTranslationFailed(() => base.Distinct_projected(queryTrackingBehavior));

public override Task Distinct_over_projected_nested_collection()
=> AssertTranslationFailed(base.Distinct_over_projected_nested_collection);
Expand Down
17 changes: 1 addition & 16 deletions test/EFCore.Cosmos.FunctionalTests/Query/JsonQueryCosmosTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -881,22 +881,7 @@ public override Task Json_collection_in_projection_with_composition_where_and_an
=> base.Json_collection_in_projection_with_composition_where_and_anonymous_projection_of_scalars(async);

public override Task Json_collection_leaf_filter_in_projection(bool async)
=> Fixture.NoSyncTest(
async, async a =>
{
Copy link
Copy Markdown
Contributor Author

@JoasE JoasE Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This used to generate a JsonConvert.Deserialize call for materialization. I think we can't translate this because the Where would have to be ran in the materializer since we don't bind ["OwnedReferenceRoot"]["OwnedReferenceBranch"]["OwnedCollectionLeaf"], and 34067 should solve that

await base.Json_collection_leaf_filter_in_projection(a);

AssertSql(
"""
SELECT VALUE ARRAY(
SELECT VALUE o
FROM o IN c["OwnedReferenceRoot"]["OwnedReferenceBranch"]["OwnedCollectionLeaf"]
WHERE (o["SomethingSomething"] != "Baz"))
FROM root c
WHERE (c["Discriminator"] = "Basic")
ORDER BY c["Id"]
""");
});
=> AssertTranslationFailed(() => base.Json_collection_leaf_filter_in_projection(async));

public override Task Json_collection_of_primitives_contains_in_predicate(bool async)
=> Fixture.NoSyncTest(
Expand Down
4 changes: 2 additions & 2 deletions test/EFCore.Cosmos.FunctionalTests/Query/OwnedQueryCosmosTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ await AssertQuery(

AssertSql(
"""
SELECT VALUE o["Details"]
SELECT VALUE o
FROM root c
Copy link
Copy Markdown
Contributor Author

@JoasE JoasE Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This projection now happens on the client side in the materializer. This used to generate a JsonConvert.Deserialize call for materialization

Copy link
Copy Markdown
Contributor Author

@JoasE JoasE Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is a SelectMany, we do bind the c["Orders"], but the "Details" projection part happens in the materializer

JOIN o IN c["Orders"]
WHERE (c["Terminator"] IN ("OwnedPerson", "Branch", "LeafB", "LeafA") AND (ARRAY_LENGTH(o["Details"]) = 1))
Expand Down Expand Up @@ -546,7 +546,7 @@ await AssertQuery(

AssertSql(
"""
SELECT VALUE o["Details"]
SELECT VALUE o
FROM root c
JOIN o IN c["Orders"]
WHERE (c["Terminator"] IN ("OwnedPerson", "Branch", "LeafB", "LeafA") AND (ARRAY_LENGTH(o["Details"]) = 1))
Expand Down
Loading