ozone cli: creating bucket with SSD storage policy

[greenwich]

Hi,

I found that the ozone sh bucket create command (v2.0.0) lacks an option to specify the storage type, resulting in a bucket being created with the default storage type (DISK). Looking at the code, I found that the following storage types are supported:

public enum StorageType {
  RAM_DISK,
  SSD,
  DISK,
  ARCHIVE;

However, it's hardcoded in the client code always to set .setStorageType(StorageType.DEFAULT).

Therefore, I have the following questions:

1. Is there a way to create a bucket via CLI with a non-DEFAULT storage type? And if not, what is the reason for that? Is bucket creation with SSD storage type available through the API?
2. Does Ozone practically support multiple storage types? Does Ozone support storage policies, and how do they work in that case? What about an analogue of hdfs mover?
3. We have a cluster with datanodes using SSDs, and hdds.datanode.dir set as a list of [SSD] and [DISK] directories, but buckets have only DISK storage types available. How can we configure the storage policies in that case?

[errose28]

Hi, storage types/storage tiering has not been implemented in Ozone, that is why no option is present in the CLI. The code internally contains some artifacts left over from early development but nothing acts on them. So to answer your questions:

1. Buckets are always created with the default storage type internally because the value is currently ignored.
2. Ozone currently does not support multiple storage types, storage policies, or storage tiering, but it may in the future.
3. This use case is currently not possible. SSDs and HDDs on the datanodes configured as hdds.datanode.dir entries will be treated equally.

[ivandika3]

There is an ongoing effort to introduce Storage Policy and Storage Tiering in https://issues.apache.org/jira/browse/HDDS-11233 (#6989). However, it's still under development and testing internally and probably will not be available in the near future.

[greenwich]

@ivandika3, I know it's hard to estimate, but do we know which ozone version may have it included? (For example, next minor, next major or in two major versions or maybe in two years) What is required to push it forward?

[ivandika3]

Rough estimate for enabling storage policy and storage types would be 2-3 major versions in the future, the timeline would be around 1-2 years, depending on a few factors:

• Internal testing and deployments
• Community design and implementation receptions and reviews
• Availability and ease to backport to the community
• Release cycle (as discussed in https://lists.apache.org/thread/mf9f1plo1o5wj0483pyqsls5ryk6xd4p we try to have some a regular release cycle).
• Other factors

I have attached some diffs on https://issues.apache.org/jira/browse/HDDS-11233 to give the rough picture for supporting storage policy in Ozone. You can refer to it and maybe use it as a reference to roll out your storage policy implementation.

Regarding storage policy migration, it will take even longer (another 2-3 major versions) since currently our implementation requires a separate job workers subsystem (with OM as master) to migrate the keys from one storage policy to another, which requires another set of design cycle.

[greenwich]

Thanks for your reply. @ivandika3, @errose28, any plans on implementing bucket versioning? Thanks.

[errose28]

By bucket versioning are you referring to object versioning? It's currently not supported since we have snapshots. Similar to storage policies it may be added in the future but I don't think we have a timeline for it.

[greenwich]

Thanks for your reply. Yes, I meant the object versioning.

The use case of the snapshots differs significantly from versioning. Snapshots are meant to be executed manually or by schedule by the user, versioning in turn happens on the fly under the hood.

[devabhishekpal]

@greenwich I see you had an open question for plans on implementation of object versioning.
I am closing this discussion as the root question has been answered and it has been inactive for sometime, but feel free to open a new discussion in case you want to discuss this further.