Skip to content

Be able to cast struct to have non-nullable inner field #22709

@Jefffrey

Description

@Jefffrey

Is your feature request related to a problem or challenge?

these work:

> select arrow_cast(struct(10), 'Struct("c0": nullable Int64)');
+--------------------------------------------------------------------+
| arrow_cast(struct(Int64(10)),Utf8("Struct("c0": nullable Int64)")) |
+--------------------------------------------------------------------+
| {c0: 10}                                                           |
+--------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.065 seconds.

> select arrow_cast(10, 'List(non-null Int64)');
+----------------------------------------------------+
| arrow_cast(Int64(10),Utf8("List(non-null Int64)")) |
+----------------------------------------------------+
| [10]                                               |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.010 seconds.

this doesn't work, when field is specified to be non-nullable inside a struct:

> select arrow_cast(struct(10), 'Struct("c0": non-null Int64)');
Optimizer rule 'simplify_expressions' failed
caused by
This feature is not implemented: Unsupported CAST from Struct("c0": Int64) to Struct("c0": non-null Int64)

Describe the solution you'd like

see if its possible to implement this support? it seems to work for lists but not structs, so not sure if this is a limitation or theres a valid reason to not allow this

Describe alternatives you've considered

No response

Additional context

i think somewhere around here in the codebase?

pub fn cast_with_target_field(
expr: Arc<dyn PhysicalExpr>,
input_schema: &Schema,
target_field: FieldRef,
cast_options: Option<CastOptions<'static>>,
) -> Result<Arc<dyn PhysicalExpr>> {
let expr_type = expr.data_type(input_schema)?;
let cast_type = target_field.data_type();
if expr_type == *cast_type && is_default_target_field(&target_field) {
return Ok(Arc::clone(&expr));
}
let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) {
// Allow casts involving structs (including nested inside Lists, Dictionaries,
// etc.) that pass name-based compatibility validation. This validation is
// applied at planning time (now) to fail fast, rather than deferring errors
// to execution time. The name-based casting logic will be executed at runtime
// via ColumnarValue::cast_to.
can_cast_named_struct_types(&expr_type, cast_type)
} else {
can_cast_types(&expr_type, cast_type)
};
if !can_build_cast {
return not_impl_err!("Unsupported CAST from {expr_type} to {cast_type}");
}
Ok(Arc::new(CastExpr::new_with_target_field(
expr,
target_field,
cast_options,
)))
}

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request
No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions