Skip to content

[export-csv] Ensure reasonable memory usage of the export CSV connector #2284

@sbocahu

Description

@sbocahu

Description

We've witnessed very high memory usage of the export CSV connector during exports, up to more than 30GB (which led to container being killed by Linux' OOM killer).

Even if the export is very demanding, we expect the connector to keep a reasonable maximum memory usage.
Could you please check that the data is not loaded and exported to a file in a single shot, but written chunk by chunk on a temporary file ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    buguse for describing something not working as expectedexport csvfiligran support[optional] use to identify an issue related to feature developed & maintained by Filigran.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions