Skip to content

[FEATURE] Add "delete searchable_snapshot" in ISM policies #1476

@rpasche

Description

@rpasche

Is your feature request related to a problem?
The goal is to also cleanup the data from remote snapshot repository, if the data has aged out.

Currently, we can "only" delete indices which might still be on local disks or mounted via a searchable_snapshot (currently, index names following $1_remote pattern).

But deleting a mounted index does not also delete the corresponding snapshot on the remote repository. This means, the remote repository will never be cleaned up, even if the ISM policy states, that the data is too old to be kept.

Or is this wrong?

We want to push older data early into the warm tier (using searchable_snapshots) and then, having those snapshots queryable for a longer amount of time.

What solution would you like?
Adding an additional option, like delete_snapshot to the delete action to support to also delete the snapshot, that was used to mount the remote index

actions": [
    {
        "retry": {
            "count": 3,
            "backoff": "exponential",
            "delay": "1m"
        },
        "delete": {
          "delete_snapshot": true|false
        }

What alternatives have you considered?
Currently, none. Could only think of some kind of cronjob, checking the age of all *_remote indices and if the index is too old, to delete the index + delete the corresponding snapshot. But this would also mean, we would need to have some kind of map in case you want to use different kind of retentions for different indices (what we definitely do :-/ )

Do you have any additional context?
It is basically this

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions