Option to define a DataVolume-specific `scratchSpaceStorageClass`

[lennartmoeller]

We installed Kubevirt and CDI in a namespace called kubevirt.

We created a DataVolume in a separate namespace restricted-namespace that uses a namespace-specific storage class and has no permission to use the default storage class default:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: my-image-dv
spec:
  source:
    registry:
      url: docker://myrepo.azurecr.io/my/image:latest
  storage:
    accessModes:
      - ReadWriteMany
    storageClassName: namespace-specific-storage-class
    volumeMode: Filesystem
    resources:
      requests:
        storage: 7Gi

Now the pod importer-prime-4faba107-5228-4956-8378-cd9a34b89e98 creates two PVCs: prime-4faba107-5228-4956-8378-cd9a34b89e98 that uses the storage class namespace-specific-storage-class and prime-4faba107-5228-4956-8378-cd9a34b89e98-scratch that uses the default storage class. But as the namespace does not allow PVCs to use the default storage class, the importing process fails.

We can set the scratchSpaceStorageClass in the CDI CR globally (API reference), but as we have multiple restricted namespaces with their individual storage classes, this is not an option. So it would be great to have an option to set the scratchSpaceStorageClass for each DataVolume individually and override the global value. An alternative would be to have an option to always set the scratch PVC storage class to the one specified in the DataVolume.

Or is there a workaround we're missing?

[awels]

If you don't set the scratchSpacestorageClass and you don't have a default storage class it should default to using the same storage class as the target PVC.

[lennartmoeller]

If you don't set the scratchSpacestorageClass and you don't have a default storage class it should default to using the same storage class as the target PVC.

Thanks for your reply!
Unfortunately we have a default storage class that is used in other places. So not having a default storage class is not an option.

[akalenyu]

Are namespace restricted storage classes a thing? I haven't encountered those before

[lennartmoeller]

Are namespace restricted storage classes a thing? I haven't encountered those before

We use namespace-specific storage classes to separate the data of our clients and restrict the use of any other to avoid mistakes. I don't know if it's widely used.

In my opinion it's a good idea to let the DataVolume decide which storage class is used, even for a scratch PVC to have full control over it. A global setting is enough for most use cases, but sometimes you want to have the full control and this is the case here.

Is it possible to add a configuration option to the DataVolume?

[akalenyu]

We will probably discuss this tomorrow in the sig-storage meeting, here's a link to the calendar https://calendar.google.com/calendar/u/0/embed?src=kubevirt@cncf.io

[skazi0]

We have exactly the same problem.
An alternative (breaking) change would be to change the logic of selecting storage class: CDI config, DV, default instead of current CDI config, default, DV. I think this would make more sense as default should be... well default (i.e. last resort).
One way or another, we vote +1 for adding an option to handle cases with multiple storage classes and default class in cluster.

[aglitke]

During the SIG-Storage meeting we discussed this issue and the general consensus is that it makes sense to switch the order of scratch space storage class collection to be the following: 1. Use the CDI configured storage class, 2. Use the same storage class that is being used to provision the DV itself.

Would @skazi0 or @lennartmoeller be willing to take a stab at such a PR?

[skazi0]

@lennartmoeller I see you started working on fixing this. Will you submit it as a PR?

[ahmed-kamal2004]

@skazi0 I will be happy to work on this, if no one is currently is working on