WebDataset

[usuyama]

Is anyone using WebDataset with mmselfsup?

I suppose I can define a custom dataset following the doc, but just wondering if anyone using wds already.

I've been using webdataset for a few projects, and so far happy with the format and scalability. For example, open_clip uses webdataset for image-text datasets, but wds can be used for other setups like image-only or other annotations in json files.

[fangyixiao18]

Hi, if you implemented a custom dataset of WebDataset, welcome to create a PR, currently we don't have such feature.
Thanks~