SMB/NFS backend

[mateuszdrab]

Hi guys

I currently run Versity in Kubernetes where the PV is a block device provided by native iSCSI driver.

Considering Versity's ability to run in a load balanced setup and its stateless nature, I have been wondering for a while how I can achieve replication so that I can phase out Minio.

I am considering relocating the backing storage onto Windows so that I can utilize DFS-R to replicate the files between sites - this would provide an eventually consistent availability model which is fine with S3 I believe.

To achieve this, either NFS or SMB CSI driver would be used to mount the share into the pod from Windows.

I'm aware the NFS and SMB from Windows are somewhat POSIX compliant but not it's not 100% compliant.

I am wondering if anyone had attempted to run Versity with a Windows based file share backend?

Thanks

[benmcclelland]

I dont have any experience with the majority of what you are asking about. But generally speaking s3 is not ok with eventually consistent. And the ability to deal with eventually consistent storage models is more up to the assumptions of the client. Most clients these days assume atomic consistency across load balanced servers.

NFS and SMB should generally work fine for backend filesystems. It's best if you can pick an NFS server/version that supports xattrs, but you can use sidecar metadata to get these going if not. There is a powershell test script in the top level of the repo that github actions uses to test on windows. Maybe this will be of help to see whats currently tested.