Skip to content

Remove usage of Accumulator from window functions #5948

Description

@Dandandan

Is your feature request related to a problem or challenge?

Currently the window expressions use Accumulator to generate the output values.
This should be relatively inefficient as each individual value will generate a ScalarValue, leading to many allocations, high memory usage and limited vectorization.

Describe the solution you'd like

Write more efficient code moving away from Accumulator and producing the values more directly.

Describe alternatives you've considered

No response

Additional context

If we don't use Accumulator in window functions anymore, we can remove count from SumAccumulator, making conventional sums slightly more efficient.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceMake DataFusion faster

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions