Merge pull request #425 from scverse/Ori-custom-dataloaders-tutos

added custom dataloaders initial tutos

Author: ori-kron-wis

dev/data_tutorial.ipynb

@@ -1918,6 +1918,38 @@
    "source": [
     "The `setup_anndata()` function itself is quite simple since any complexity in preprocessing is contained within the `AnnDataField` functions. By factorizing the preprocessing steps into each subclass, model developers can easily extend and reuse logic across models and fields."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "872b95f9-f17d-49c7-956e-d7b9005a4f3a",
+   "metadata": {},
+   "source": [
+    "## Custom DataLoaders\n",
+    "\n",
+    "In SCVI, custom dataloaders allow you to create a tailored data pipeline that can handle unique formats or complex datasets not covered by the default loaders. A custom dataloader can be useful when you have a specific structure for your data or need to preprocess it in a particular way before feeding it into the model, in order to gain some advantage.\n",
+    "See more [here](\"https://docs.scvi-tools.org/en/stable/user_guide/use_case/custom_dataloaders.html\")\n",
+    "\n",
+    "In SCVI-tools a custom dataloader class is a LightningDataModule inherited class which should create batches of data from an external source and feed them into a scvi pytorch model during training and inference.\n",
+    "\n",
+    "Beucase it is tailored made for a specific data source, custom dataloders differ from each other. \n",
+    "Nevertheless, there are some common bulding blocks that are required in order to create it:\n",
+    "- a 'linkage' to the data source that the custom data loder need to query from.\n",
+    "- `batch_key` is the key for batch information. \n",
+    "- `labels_key` is the key for label information. \n",
+    "- `unlabeled_category` is the key for the unlabeled groyp information. \n",
+    "- `train_dataloader` a function to create a training set pytorch Dataloder\n",
+    "- `val_dataloader` a function to create a validation set pytorch Dataloder\n",
+    "- `registry` its the manual implementaion of the scvi tools registry as a dict filled with information taken from the datamodule itself.\n",
+    "  Note that each datamodule will have its own registry implementation and also it should be extended to work with other models (currently only SCVI and SCANVI are supported, but it should be generic enough to work with any model)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1f31df4c-58d2-403b-8f87-d39f9e9cd5d8",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -1926,7 +1958,7 @@
    "provenance": []
   },
   "kernelspec": {
-   "display_name": "scvi-tools-dev",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -1940,7 +1972,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.12.6"
+   "version": "3.12.8"
   },
   "vscode": {
    "interpreter": {