-{{< how-to-expand "Capture data" "3" "BEGINNER-FRIENDLY" "" "data-platform-capture" >}}
+{{< how-to-expand "Capture data" "4" "BEGINNER-FRIENDLY" "" "data-platform-capture" >}}
{{< cards >}}
{{% card link="/data-ai/capture-data/capture-sync/" noimage="true" %}}
{{% card link="/data-ai/capture-data/filter-before-sync/" noimage="true" %}}
{{% card link="/data-ai/capture-data/conditional-sync/" noimage="true" %}}
+{{% card link="/data-ai/capture-data/lorawan/" noimage="true" %}}
{{< /cards >}}
{{< /how-to-expand >}}
@@ -42,11 +43,18 @@ You can also monitor your machines through teleop, power your application logic,
{{< /cards >}}
{{< /how-to-expand >}}
-{{< how-to-expand "Leverage AI" "8" "INTERMEDIATE" "" "data-platform-ai" >}}
+{{< how-to-expand "Train an ML model" "5" "INTERMEDIATE" "" "data-platform-ai" >}}
+{{< cards >}}
+{{% card link="/data-ai/train/create-dataset/" noimage="true" %}}
+{{% card link="/data-ai/train/capture-annotate-images/" noimage="true" %}}
+{{% card link="/data-ai/train/train-tflite/" noimage="true" %}}
+{{% card link="/data-ai/train/train/" noimage="true" %}}
+{{% card link="/data-ai/train/upload-external-data/" noimage="true" %}}
+{{< /cards >}}
+{{< /how-to-expand >}}
+
+{{< how-to-expand "Infer with ML models" "4" "INTERMEDIATE" "" "data-platform-ai" >}}
{{< cards >}}
-{{% card link="/data-ai/ai/create-dataset/" noimage="true" %}}
-{{% card link="/data-ai/ai/train-tflite/" noimage="true" %}}
-{{% card link="/data-ai/ai/train/" noimage="true" %}}
{{% card link="/data-ai/ai/deploy/" noimage="true" %}}
{{% card link="/data-ai/ai/run-inference/" noimage="true" %}}
{{% card link="/data-ai/ai/alert/" noimage="true" %}}
diff --git a/docs/data-ai/ai/_index.md b/docs/data-ai/ai/_index.md
index a7a53e72e5..e3a9a13527 100644
--- a/docs/data-ai/ai/_index.md
+++ b/docs/data-ai/ai/_index.md
@@ -1,6 +1,6 @@
---
-linkTitle: "Leverage AI"
-title: "Leverage AI"
+linkTitle: "Infer with ML models"
+title: "Infer with ML models"
weight: 300
layout: "empty"
type: "docs"
diff --git a/docs/data-ai/ai/act.md b/docs/data-ai/ai/act.md
index 8fbd1ca309..b66b36a3a7 100644
--- a/docs/data-ai/ai/act.md
+++ b/docs/data-ai/ai/act.md
@@ -5,7 +5,7 @@ weight: 70
layout: "docs"
type: "docs"
description: "Use the vision service API to act based on inferences."
-next: "/data-ai/ai/advanced/upload-external-data/"
+next: "/data-ai/train/upload-external-data/"
---
You can use the [vision service API](/dev/reference/apis/services/vision/) to get information about your machine's inferences and program behavior based on that.
diff --git a/docs/data-ai/ai/advanced/_index.md b/docs/data-ai/ai/advanced/_index.md
deleted file mode 100644
index f004c032cf..0000000000
--- a/docs/data-ai/ai/advanced/_index.md
+++ /dev/null
@@ -1,8 +0,0 @@
----
-linkTitle: "Advanced"
-title: "Advanced"
-weight: 200
-layout: "empty"
-type: "docs"
-empty_node: true
----
diff --git a/docs/data-ai/ai/create-dataset.md b/docs/data-ai/ai/create-dataset.md
deleted file mode 100644
index b08766a59f..0000000000
--- a/docs/data-ai/ai/create-dataset.md
+++ /dev/null
@@ -1,341 +0,0 @@
----
-linkTitle: "Create a dataset"
-title: "Create a dataset"
-weight: 10
-layout: "docs"
-type: "docs"
-description: "Create a dataset to train a machine learning model."
-aliases:
- - /fleet/dataset/
- - /manage/data/label/
- - /manage/data/dataset/
- - /data/dataset/
----
-
-To ensure a machine learning model you create performs well, you need to train it on a variety of images that cover the range of things your machine should be able to recognize.
-
-To train a model, you need a dataset that meets the following criteria:
-
-- the dataset contains at least 15 images
-- at least 80% of the images have labels
-- for each selected label, at least 10 bounding boxes exist
-
-This page explains how to create a dataset that meets these criteria for your training purposes.
-
-## Prerequisites
-
-{{% expand "a machine connected to Viam" %}}
-
-{{% snippet "setup.md" %}}
-
-{{% /expand %}}
-
-{{% expand "a camera, connected to your machine, to capture images" %}}
-
-Follow the guide to configure a [webcam](/operate/reference/components/camera/webcam/) or similar [camera component](/operate/reference/components/camera/).
-
-{{% /expand%}}
-
-## Create a dataset
-
-To create a dataset, use the CLI or the web UI:
-
-{{< tabs >}}
-{{% tab name="Web UI" %}}
-
-1. Open the [**DATASETS** tab on the **DATA** page](https://app.viam.com/data/datasets).
-
-1. Click the **+ Create dataset** button.
-
- {{< imgproc src="/services/data/create-dataset.png" alt="The **DATASET** tab of the **DATA** page, showing the **+ Create dataset** button." resize="800x" style="width:500px" class="imgzoom" >}}
-
-1. Enter a unique name for the dataset.
-
-1. Click the **Create dataset** button to create the dataset.
-
-{{% /tab %}}
-{{% tab name="CLI" %}}
-
-1. First, install the Viam CLI and authenticate:
-
- {{< readfile "/static/include/how-to/install-cli.md" >}}
-
-1. [Log in to the CLI](/dev/tools/cli/#authenticate).
-
-1. Run the following command to create a dataset, replacing the `
` and `` placeholders with your organization ID and a unique name for the dataset:
-
- ```sh {class="command-line" data-prompt="$"}
- viam dataset create --org-id= --name=
- ```
-
-{{% /tab %}}
-{{< /tabs >}}
-
-## Capture images
-
-{{< tabs >}}
-{{% tab name="One image" %}}
-
-You can add images to a dataset directly from a camera or vision component feed in the machine's **CONTROL** or **CONFIGURATION** tabs.
-
-To add an image directly to a dataset from a visual feed, complete the following steps:
-
-1. Open the **TEST** panel of any camera or vision service component to view a feed of images from the camera.
-1. Click the button marked with the camera icon to save the currently displayed image to a dataset:
- {{< imgproc src="/components/camera/add_image_to_dataset_button.png" alt="A button marked with the outline of a camera, emphasized in red" resize="800x" style="width:500px" class="imgzoom" >}}
-1. Select an existing dataset.
-1. Click **Add** to add the image to the selected dataset.
-1. When you see a success notification that reads "Saved image to dataset", you have successfully added the image to the dataset.
-
-To view images added to your dataset, go to the **DATA** page's [**DATASETS** tab](https://app.viam.com/data/datasets) and select your dataset.
-
-{{% /tab %}}
-{{% tab name="Many images" %}}
-
-To capture a large number of images for training an ML model, [Capture and sync image data](/data-ai/capture-data/capture-sync/) using the data management service with your camera.
-
-Viam stores the images saved by capture and sync on the [**DATA** page](https://app.viam.com/data/), but does not add the images to a dataset.
-We recommend you tag the images first and then use the CLI to [add the tagged images to a dataset](/data-ai/ai/create-dataset/#add-tagged-images-to-a-dataset).
-
-{{< alert title="Tip" color="tip" >}}
-
-Once you have enough images, consider disabling data capture to [avoid incurring fees](https://www.viam.com/product/pricing) for capturing large amounts of training data.
-
-{{< /alert >}}
-{{% /tab %}}
-{{< /tabs >}}
-
-Once you've captured enough images for training, you must annotate them to train a model.
-
-## Annotate images
-
-Use the interface on the [**DATA** page](https://app.viam.com/data/view) to annotate your images.
-Always follow best practices when you label your images:
-
-More data means better models
-
-: Incorporate as much data as you practically can to improve your model's overall performance.
-
-Include counterexamples
-
-: Include images with and without the object you’re looking to classify.
-This helps the model distinguish the target object from the background and reduces the chances of false positives by teaching the model what the object is _not_.
-
-Avoid class imbalance
-
-: Don't train excessively on one specific type or class, make sure each category has a roughly equal number of images.
-For instance, if you're training a dog detector, include images of various dog breeds to avoid bias towards one breed.
-An imbalanced dataset can lead the model to favor one class over others, reducing its overall accuracy.
-
-Match training images to intended use case
-
-: Use images that reflect the quality and conditions of your production environment.
-For example, if you plan to use a low-quality camera in production, train with low-quality images.
-Similarly, if your model will run all day, capture images in daylight, nighttime, dusk, and dawn conditions.
-
-Vary angles and distances
-
-: Include image examples from every angle and distance that you expect the model to handle.
-
-Viam enables you to annotate images for the following machine learning methods:
-
-{{< tabs >}}
-{{% tab name="Classification" %}}
-
-Classification determines a descriptive tag or set of tags for an image.
-For example, classification could help you identify:
-
-- whether an image of a food display appears `full`, `empty`, or `average`
-- the quality of manufacturing output: `good` or `bad`
-- what combination of toppings exists on a pizza: `pepperoni`, `sausage` and `pepper`, or `pineapple` and `ham` and `mushroom`
-
-Viam supports single and multiple label classification.
-To create a training set for classification, annotate tags to describe your images.
-
-To tag an image:
-
-1. Click on an image, then click the **+** next to the **Tags** option.
-1. Add one or more tags to your image.
-
- {{}}
-
-Repeat these steps for all images in the dataset.
-
-{{% /tab %}}
-{{% tab name="Object detection" %}}
-
-Object detection identifies and determines the location of certain objects in an image.
-For example, object detection could help you identify:
-
-- how many `pizza` objects appear on a counter
-- the number of `bicycle` and `pedestrian` objects on a greenway
-- which `plant` objects are popular with `deer` in your garden
-
-To create a training set for object detection, annotate bounding boxes to teach your model to identify objects that you want to detect in future images.
-
-To label an object with a bounding box:
-
-1. Click on an image, then click the **Annotate** button in right side menu.
-1. Choose an existing label or create a new label.
-1. Holding the command key (on macOS), or the control key (on Linux and Windows), click and drag on the image to create the bounding box:
-
- {{}}
-
-{{< alert title="Tip" color="tip" >}}
-
-Once created, you can move, resize, or delete the bounding box.
-{{< /alert >}}
-
-Repeat these steps for all images in the dataset.
-
-{{% /tab %}}
-{{< /tabs >}}
-
-## Add tagged images to a dataset
-
-{{< tabs >}}
-{{% tab name="Web UI" %}}
-
-1. Open the [**DATA** page](https://app.viam.com/data/view).
-
-1. Navigate to the **ALL DATA** tab.
-
-1. Use the checkbox in the upper left of each image to select labeled images.
-
-1. Click the **Add to dataset** button, select a dataset, and click the **Add ... images** button to add the selected images to the dataset.
-
-{{% /tab %}}
-{{% tab name="CLI" %}}
-
-Use the Viam CLI to filter images by label and add the filtered images to a dataset:
-
-1. First, [create a dataset](#create-a-dataset), if you haven't already.
-
-1. If you just created a dataset, use the dataset ID output by the creation command.
- If your dataset already exists, run the following command to get a list of dataset names and corresponding IDs:
-
- ```sh {class="command-line" data-prompt="$"}
- viam dataset list
- ```
-
-1. Run the following [command](/dev/tools/cli/#dataset) to add all images labeled with a subset of tags to the dataset, replacing the `` placeholder with the dataset ID output by the command in the previous step:
-
- ```sh {class="command-line" data-prompt="$"}
- viam dataset data add filter --dataset-id= --tags=red_star,blue_square
- ```
-
-{{% /tab %}}
-{{% tab name="Data Client API" %}}
-
-The following script adds all images captured from a certain machine to a new dataset. Complete the following steps to use the script:
-
-1. Copy and paste the following code into a file named add_images_from_machine_to_dataset.py on your machine.
-
- ```python {class="line-numbers linkable-line-numbers" data-line="9-13" }
- import asyncio
- from typing import List, Optional
-
- from viam.rpc.dial import DialOptions, Credentials
- from viam.app.viam_client import ViamClient
- from viam.utils import create_filter
-
- # Configuration constants – replace with your actual values
- DATASET_NAME = "" # a unique, new name for the dataset you want to create
- ORG_ID = "" # your organization ID, find in your organization settings
- PART_ID = "" # id of machine that captured target images, find in machine config
- API_KEY = "" # API key, find or create in your organization settings
- API_KEY_ID = "" # API key ID, find or create in your organization settings
-
- # Adjust the maximum number of images to add to the dataset
- MAX_MATCHES = 500
-
- async def connect() -> ViamClient:
- """Establish a connection to the Viam client using API credentials."""
- dial_options = DialOptions(
- credentials=Credentials(
- type="api-key",
- payload=API_KEY,
- ),
- auth_entity=API_KEY_ID,
- )
- return await ViamClient.create_from_dial_options(dial_options)
-
-
- async def fetch_binary_data_ids(data_client, part_id: str) -> List[str]:
- """Fetch binary data metadata and return a list of BinaryData objects."""
- data_filter = create_filter(part_id=part_id)
- all_matches = []
- last: Optional[str] = None
-
- print("Getting data for part...")
-
- while len(all_matches) < MAX_MATCHES:
- print("Fetching more data...")
- data, _, last = await data_client.binary_data_by_filter(
- data_filter,
- limit=50,
- last=last,
- include_binary_data=False,
- )
- if not data:
- break
- all_matches.extend(data)
-
- return all_matches
-
-
- async def main() -> int:
- """Main execution function."""
- viam_client = await connect()
- data_client = viam_client.data_client
-
- matching_data = await fetch_binary_data_ids(data_client, PART_ID)
-
- print("Creating dataset...")
-
- try:
- dataset_id = await data_client.create_dataset(
- name=DATASET_NAME,
- organization_id=ORG_ID,
- )
- print(f"Created dataset: {dataset_id}")
- except Exception as e:
- print("Error creating dataset. It may already exist.")
- print("See: https://app.viam.com/data/datasets")
- print(f"Exception: {e}")
- return 1
-
- print("Adding data to dataset...")
-
- await data_client.add_binary_data_to_dataset_by_ids(
- binary_ids=[obj.metadata.binary_data_id for obj in matching_data],
- dataset_id=dataset_id
- )
-
- print("Added files to dataset.")
- print(f"See dataset: https://app.viam.com/data/datasets?id={dataset_id}")
-
- viam_client.close()
- return 0
-
-
- if __name__ == "__main__":
- asyncio.run(main())
- ```
-
-1. Fill in the placeholders with values for your own organization, API key, machine, and dataset.
-
-1. Install the [Viam Python SDK](https://python.viam.dev/) by running the following command:
-
- ```sh {class="command-line" data-prompt="$"}
- pip install viam-sdk
- ```
-
-1. Finally, run the following command to add the images to the dataset:
-
- ```sh {class="command-line" data-prompt="$"}
- python add_images_from_machine_to_dataset.py
- ```
-
-{{% /tab %}}
-{{< /tabs >}}
diff --git a/docs/data-ai/ai/deploy.md b/docs/data-ai/ai/deploy.md
index 5a7dc28b0f..251949cb6a 100644
--- a/docs/data-ai/ai/deploy.md
+++ b/docs/data-ai/ai/deploy.md
@@ -75,7 +75,7 @@ Save your config to use your specified version of the ML model.
The service works with models trained on Viam or elsewhere:
-- You can [train TFlite](/data-ai/ai/train-tflite/) or [other model frameworks](/data-ai/ai/train/) on data from your machines.
+- You can [train TFlite](/data-ai/train/train-tflite/) or [other model frameworks](/data-ai/train/train/) on data from your machines.
- You can use [ML models](https://app.viam.com/registry?type=ML+Model) from the [registry](https://app.viam.com/registry).
- You can upload externally trained models from a model file on the [**MODELS** tab](https://app.viam.com/models).
- You can use a [model](/data-ai/ai/deploy/#deploy-your-ml-model-on-an-ml-model-service) trained outside the Viam platform whose files are on your machine. See the documentation of the model of ML model service you're using (pick one that supports your model framework) for instructions on this.
diff --git a/docs/data-ai/capture-data/capture-sync.md b/docs/data-ai/capture-data/capture-sync.md
index e49114f86b..68b782f233 100644
--- a/docs/data-ai/capture-data/capture-sync.md
+++ b/docs/data-ai/capture-data/capture-sync.md
@@ -139,4 +139,4 @@ For other ways to control data synchronization, see:
## Next steps
For more information on available configuration attributes and options like capturing directly to MongoDB or conditional sync, see [Advanced data capture and sync configurations](/data-ai/capture-data/advanced/advanced-data-capture-sync/).
-To leverage AI, you can now [create a dataset](/data-ai/ai/create-dataset/) with the data you've captured.
+To leverage AI, you can now [create a dataset](/data-ai/train/create-dataset/) with the data you've captured.
diff --git a/docs/data-ai/data/_index.md b/docs/data-ai/data/_index.md
index 422b01922b..ed4ebdbf3e 100644
--- a/docs/data-ai/data/_index.md
+++ b/docs/data-ai/data/_index.md
@@ -1,7 +1,7 @@
---
linkTitle: "Work with data"
title: "Work with data"
-weight: 200
+weight: 150
layout: "empty"
type: "docs"
empty_node: true
diff --git a/docs/data-ai/train/_index.md b/docs/data-ai/train/_index.md
new file mode 100644
index 0000000000..d0376f093a
--- /dev/null
+++ b/docs/data-ai/train/_index.md
@@ -0,0 +1,11 @@
+---
+linkTitle: "Train an ML model"
+title: "Train an ML model"
+weight: 200
+layout: "empty"
+type: "docs"
+empty_node: true
+open_on_desktop: true
+header_only: true
+noedit: true
+---
diff --git a/docs/data-ai/train/capture-annotate-images.md b/docs/data-ai/train/capture-annotate-images.md
new file mode 100644
index 0000000000..4e9a078a7a
--- /dev/null
+++ b/docs/data-ai/train/capture-annotate-images.md
@@ -0,0 +1,1277 @@
+---
+linkTitle: "Capture and annotate images"
+title: "Capture and annotate images for training"
+weight: 20
+layout: "docs"
+type: "docs"
+description: "Capture images that you can use to train a machine learning model."
+---
+
+## Prerequisites
+
+{{% expand "A machine connected to Viam" %}}
+
+{{% snippet "setup.md" %}}
+
+{{% /expand %}}
+
+{{% expand "A camera, connected to your machine, to capture images" %}}
+
+Follow the guide to configure a [webcam](/operate/reference/components/camera/webcam/) or similar [camera component](/operate/reference/components/camera/).
+
+{{% /expand%}}
+
+{{< alert title="Tip" color="tip" >}}
+
+For the best results, use the same camera for both training data capture and production deployment.
+
+{{< /alert >}}
+
+## Capture images
+
+### Capture individual images
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+You can add images to a dataset directly from a camera or vision component feed in the machine's **CONTROL** or **CONFIGURATION** tabs.
+
+To add an image directly to a dataset from a visual feed, complete the following steps:
+
+1. Open the **TEST** panel of any camera or vision service component to view a feed of images from the camera.
+1. Click the button marked with the camera icon to save the currently displayed image to a dataset:
+ {{< imgproc src="/components/camera/add_image_to_dataset_button.png" alt="A button marked with the outline of a camera, emphasized in red" resize="800x" style="width:500px" class="imgzoom" >}}
+1. Select an existing dataset.
+1. Click **Add** to add the image to the selected dataset.
+1. When you see a success notification that reads "Saved image to dataset", you have successfully added the image to the dataset.
+
+To view images added to your dataset, go to the **DATA** page, open the [**DATASETS** tab](https://app.viam.com/data/datasets), then select your dataset.
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+To capture an image and add it to your **DATA** page, fetch an image from your camera through your machine.
+Pass that image and an appropriate set of metadata to [`data_client.binary_data_capture_upload`](/dev/reference/apis/data-client/#binarydatacaptureupload):
+
+```python
+CAMERA_NAME = ""
+MACHINE_ADDRESS = ""
+
+dial_options = DialOptions(
+ credentials=Credentials(
+ type="api-key",
+ payload=API_KEY,
+ ),
+ auth_entity=API_KEY_ID,
+)
+
+robot_opts = RobotClient.Options.with_api_key(
+ api_key=API_KEY,
+ api_key_id=API_KEY_ID
+)
+
+viam_client = await ViamClient.create_from_dial_options(dial_options)
+data_client = viam_client.data_client
+
+robot_client = await RobotClient.at_address(ROBOT_ADDRESS, robot_opts)
+camera = Camera.from_robot(robot_client, CAMERA_NAME)
+
+# Capture image
+image_frame = await camera.get_image()
+
+# Upload data
+file_id = await data_client.binary_data_capture_upload(
+ part_id=PART_ID,
+ component_type="camera",
+ component_name=CAMERA_NAME,
+ method_name="GetImage",
+ data_request_times=[datetime.utcnow(), datetime.utcnow()],
+ file_extension=".jpg",
+ binary_data=image_frame
+)
+
+# Cleanup
+await robot_client.close()
+viam_client.close()
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+To capture an image and add it to your **DATA** page, fetch an image from your camera through your machine.
+Pass that image and an appropriate set of metadata to [`DataClient.BinaryDataCaptureUpload`](/dev/reference/apis/data-client/#binarydatacaptureupload):
+
+```go
+const (
+ CAMERA_NAME = ""
+ MACHINE_ADDRESS = ""
+ API_KEY = ""
+ API_KEY_ID = ""
+ PART_ID = ""
+)
+
+ctx := context.Background()
+machine, err := client.New(
+ ctx,
+ MACHINE_ADDRESS,
+ logger,
+ client.WithDialOptions(rpc.WithEntityCredentials(
+ API_KEY_ID,
+ rpc.Credentials{
+ Type: rpc.CredentialsTypeAPIKey,
+ Payload: API_KEY,
+ },
+ )),
+)
+if err != nil {
+ return "", err
+}
+defer machine.Close(ctx)
+
+viamClient, err := client.New(ctx, MACHINE_ADDRESS, logger)
+if err != nil {
+ log.Fatal(err)
+}
+defer viamClient.Close(ctx)
+
+dataClient := viamClient.DataClient()
+
+camera, err := camera.FromRobot(machine, CAMERA_NAME)
+if err != nil {
+ return "", err
+}
+
+// Capture image
+img, _, err := camera.GetImage(ctx)
+if err != nil {
+ return "", err
+}
+
+// Upload binary data
+now := time.Now().UTC()
+fileID, err := dataClient.BinaryDataCaptureUpload(ctx, app.BinaryDataCaptureUploadOptions{
+ PartID: PART_ID,
+ ComponentType: "camera",
+ ComponentName: CAMERA_NAME,
+ MethodName: "GetImage",
+ DataRequestTimes: []time.Time{now, now},
+ FileExtension: ".jpg",
+ BinaryData: img,
+})
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+To capture an image and add it to your **DATA** page, fetch an image from your camera through your machine.
+Pass that image and an appropriate set of metadata to [`dataClient.binaryDataCaptureUpload`](/dev/reference/apis/data-client/#binarydatacaptureupload):
+
+```typescript
+const CAMERA_NAME = "";
+const MACHINE_ADDRESS = "";
+const API_KEY = "";
+const API_KEY_ID = "";
+const PART_ID = "";
+
+const machine = await Viam.createRobotClient({
+ host: MACHINE_ADDRESS,
+ credential: {
+ type: "api-key",
+ payload: API_KEY,
+ },
+ authEntity: API_KEY_ID,
+});
+
+const client: ViamClient = await createViamClient({
+ credential: {
+ type: "api-key",
+ payload: API_KEY,
+ },
+ authEntity: API_KEY_ID,
+});
+
+const dataClient = client.dataClient;
+
+const camera = new Viam.CameraClient(machine, CAMERA_NAME);
+
+// Capture image
+const imageFrame = await camera.getImage();
+
+// Upload binary data
+const now = new Date();
+const fileId = await dataClient.binaryDataCaptureUpload({
+ partId: PART_ID,
+ componentType: "camera",
+ componentName: CAMERA_NAME,
+ methodName: "GetImage",
+ dataRequestTimes: [now, now],
+ fileExtension: ".jpg",
+ binaryData: imageFrame,
+});
+
+// Cleanup
+await machine.disconnect();
+dataClient.close();
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+To capture an image and add it to your **DATA** page, fetch an image from your camera through your machine.
+Pass that image and an appropriate set of metadata to [`dataClient.binaryDataCaptureUpload`](/dev/reference/apis/data-client/#binarydatacaptureupload):
+
+```dart
+const String CAMERA_NAME = '';
+const String MACHINE_ADDRESS = '';
+const String API_KEY = '';
+const String API_KEY_ID = '';
+const String PART_ID = '';
+
+final machine = await RobotClient.atAddress(
+ MACHINE_ADDRESS,
+ RobotClientOptions.withApiKey(
+ apiKey: API_KEY,
+ apiKeyId: API_KEY_ID,
+ ),
+);
+
+final client = await ViamClient.withApiKey(
+ apiKeyId: API_KEY_ID,
+ apiKey: API_KEY,
+);
+
+final dataClient = client.dataClient;
+
+final camera = Camera.fromRobot(machine, CAMERA_NAME);
+
+// Capture image
+final imageFrame = await camera.getImage();
+
+// Upload binary data
+final now = DateTime.now().toUtc();
+final fileId = await dataClient.binaryDataCaptureUpload(
+ partId: PART_ID,
+ componentType: 'camera',
+ componentName: CAMERA_NAME,
+ methodName: 'GetImage',
+ dataRequestTimes: [now, now],
+ fileExtension: '.jpg',
+ binaryData: imageFrame,
+);
+
+// Cleanup
+await robotClient.close();
+dataClient.close();
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+Once you've captured [enough images for training](/data-ai/train/train-tflite/), you must [annotate](#annotate-images) the images before you can use them to train a model.
+
+### Capture images over time
+
+To capture a large number of images for training an ML model, use the data management service to [capture and sync image data](/data-ai/capture-data/capture-sync/) from your camera.
+
+When you sync with data management, Viam stores the images saved by capture and sync on the [**DATA** page](https://app.viam.com/data/), but does not add the images to a dataset.
+To use your captured images for training, [add the images to a dataset](/data-ai/train/create-dataset/#add-to-a-dataset) and [annotate them](#annotate-images), so you can use them to train a model.
+
+{{< alert title="Tip" color="tip" >}}
+
+Once you have enough images, consider disabling data capture to [avoid incurring fees](https://www.viam.com/product/pricing) for capturing large amounts of training data.
+
+{{< /alert >}}
+
+You can either manually add annotations through the Viam web UI, or add annotations with an existing ML model.
+
+## Annotate images
+
+### Classify images with tags
+
+Classification determines a descriptive tag or set of tags for an image.
+For example, you could use classification to answer the following questions:
+
+- does an image of a food display appear `full`, `empty`, or `average`?
+- the quality of manufacturing output `good` or `bad`?
+- what combination of toppings exists on a pizza: `pepperoni`, `sausage`, and `pepper`? or `pineapple`, `ham`, and `mushroom`?
+
+Viam supports single and multiple classification.
+To create a training dataset for classification, annotate tags to describe your images.
+
+{{< alert title="Tip" color="tip" >}}
+
+If you have an ML model, use code to speed up annotating your data, otherwise use the Web UI.
+
+{{< /alert >}}
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+The [**DATA** page](https://app.viam.com/data/view) provides an interface for annotating images.
+
+To tag an image:
+
+1. Click on an image, then click the **+** next to the **Tags** option.
+1. Add one or more tags to your image.
+
+ {{}}
+
+Repeat these steps for all images in the dataset.
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+Use an ML model to generate tags for an image or set of images.
+Then, pass the tags and image IDs to [`data_client.add_tags_to_binary_data_by_ids`](/dev/reference/apis/data-client/#addtagstobinarydatabyids):
+
+```python
+detector = VisionClient.from_robot(machine, "")
+
+# Get the captured data for a camera
+result = await detector.capture_all_from_camera(
+ "",
+ return_image=True,
+ return_detections=True,
+)
+image = result.image
+detections = result.detections
+
+tags = ["tag1", "tag2"]
+
+my_filter = create_filter(
+ component_name="camera-1", organization_ids=[""]
+)
+binary_metadata, count, last = await data_client.binary_data_by_filter(
+ filter=my_filter,
+ limit=20,
+ include_binary_data=False
+)
+
+my_ids = []
+
+for obj in binary_metadata:
+ my_ids.append(
+ obj.metadata.binary_data_id
+ )
+
+binary_data = await data_client.add_tags_to_binary_data_by_ids(tags, my_ids)
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+Use an ML model to generate tags for an image or set of images.
+Then, pass the tags and image IDs to [`DataClient.AddTagsToBinaryDataByIDs`](/dev/reference/apis/data-client/#addtagstobinarydatabyids):
+
+```go
+ctx := context.Background()
+
+viamClient, err := client.New(ctx, "", logger)
+if err != nil {
+ log.Fatal(err)
+}
+defer viamClient.Close(ctx)
+
+myDetector, err := vision.FromRobot(viamClient, "")
+if err != nil {
+ log.Fatal(err)
+}
+
+dataClient := viamClient.DataClient()
+
+// Get the captured data for a camera
+result, err := myDetector.CaptureAllFromCamera(ctx, "", &vision.CaptureAllFromCameraRequest{
+ ReturnImage: true,
+ ReturnDetections: true,
+})
+if err != nil {
+ log.Fatal(err)
+}
+
+image := result.Image
+detections := result.Detections
+
+tags := []string{"tag1", "tag2"}
+
+myFilter := &datamanager.Filter{
+ ComponentName: "camera-1",
+ OrganizationIDs: []string{""},
+}
+
+binaryResult, err := dataClient.BinaryDataByFilter(ctx, &datamanager.BinaryDataByFilterRequest{
+ Filter: myFilter,
+ Limit: 20,
+ IncludeBinaryData: false,
+})
+if err != nil {
+ log.Fatal(err)
+}
+
+var myIDs []string
+for _, obj := range binaryResult.BinaryMetadata {
+ myIDs = append(myIDs, obj.Metadata.BinaryDataID)
+}
+
+_, err = dataClient.AddTagsToBinaryDataByIDs(ctx, &datamanager.AddTagsRequest{
+ Tags: tags,
+ BinaryDataIDs: myIDs,
+})
+if err != nil {
+ log.Fatal(err)
+}
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+Use an ML model to generate tags for an image or set of images.
+Then, pass the tags and image IDs to [`dataClient.addTagsToBinaryDataByIds`](/dev/reference/apis/data-client/#addtagstobinarydatabyids):
+
+```typescript
+const client = await createViamClient();
+const myDetector = new VisionClient(client, "");
+const dataClient = client.dataClient;
+
+// Get the captured data for a camera
+const result = await myDetector.captureAllFromCamera("", {
+ returnImage: true,
+ returnDetections: true,
+});
+const image = result.image;
+const detections = result.detections;
+
+const tags = ["tag1", "tag2"];
+
+const myFilter = createFilter({
+ componentName: "camera-1",
+ organizationIds: [""],
+});
+
+const binaryResult = await dataClient.binaryDataByFilter({
+ filter: myFilter,
+ limit: 20,
+ includeBinaryData: false,
+});
+
+const myIds: string[] = [];
+
+for (const obj of binaryResult.binaryMetadata) {
+ myIds.push(obj.metadata.binaryDataId);
+}
+
+await dataClient.addTagsToBinaryDataByIds(tags, myIds);
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+Use an ML model to generate tags for an image or set of images.
+Then, pass the tags and image IDs to [`dataClient.addTagsToBinaryDataByIds`](/dev/reference/apis/data-client/#addtagstobinarydatabyids):
+
+```dart
+final viamClient = await ViamClient.connect();
+final myDetector = VisionClient.fromRobot(viamClient, "");
+final dataClient = viamClient.dataClient;
+
+// Get the captured data for a camera
+final result = await myDetector.captureAllFromCamera(
+ "",
+ returnImage: true,
+ returnDetections: true,
+);
+final image = result.image;
+final detections = result.detections;
+
+final tags = ["tag1", "tag2"];
+
+final myFilter = createFilter(
+ componentName: "camera-1",
+ organizationIds: [""],
+);
+
+final binaryResult = await dataClient.binaryDataByFilter(
+ filter: myFilter,
+ limit: 20,
+ includeBinaryData: false,
+);
+
+final myIds = [];
+
+for (final obj in binaryResult.binaryMetadata) {
+ myIds.add(obj.metadata.binaryDataId);
+}
+
+await dataClient.addTagsToBinaryDataByIds(tags, myIds);
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+Once you've annotated your dataset, you can [train](/data-ai/train/train-tflite/) an ML model to make inferences.
+
+### Detect objects with bounding boxes
+
+Object detection identifies and determines the location of certain objects in an image.
+For example, object detection could help you identify:
+
+- how many `pizza` objects appear on a counter
+- the number of `bicycle` and `pedestrian` objects on a greenway
+- which `plant` objects are popular with `deer` in your garden
+
+To create a training set for object detection, annotate bounding boxes to teach your model to identify objects that you want to detect in future images.
+
+{{< alert title="Tip" color="tip" >}}
+
+If you have an ML model, use code to speed up annotating your data, otherwise use the Web UI.
+
+{{< /alert >}}
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+The [**DATA** page](https://app.viam.com/data/view) provides an interface for annotating images.
+
+To label an object with a bounding box:
+
+1. Click on an image, then click the **Annotate** button in right side menu.
+1. Choose an existing label or create a new label.
+1. Holding the command key (on macOS), or the control key (on Linux and Windows), click and drag on the image to create the bounding box:
+
+ {{}}
+
+{{< alert title="Tip" color="tip" >}}
+
+Once created, you can move, resize, or delete the bounding box.
+
+{{< /alert >}}
+
+Repeat these steps for all images in the dataset.
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+Use an ML model to generate bounding boxes for an image.
+Then, separately pass each bounding box and the image ID to [`data_client.add_bounding_box_to_image_by_id`](/dev/reference/apis/data-client/#addboundingboxtoimagebyid):
+
+```python
+detector = VisionClient.from_robot(machine, "")
+
+# Get the captured data for a camera
+
+# Initialize data client
+data_client = DataClient.create_from_dial_options(
+ dial_options=DialOptions(
+ credentials=Credentials(
+ type="api-key",
+ payload=""
+ )
+ ),
+ organization_id="",
+ location_id=""
+)
+
+# Initialize vision service
+detector = VisionClient.from_robot(machine, "")
+
+# Capture data with image and detections
+result = await detector.capture_all_from_camera(
+ "",
+ return_image=True,
+ return_detections=True,
+)
+
+image = result.image
+detections = result.detections
+
+# Upload image to obtain binary ID
+binary_id = await data_client.binary_data_capture_upload(
+ binary_data=image.data,
+ part_id="",
+ component_type="camera",
+ component_name="",
+ method_name="get_image",
+ file_extension=".jpg"
+)
+
+# Process each detection and create bounding boxes
+for detection in detections:
+ bbox = detection.bounding_box
+
+ # Create bounding box annotation
+ bbox_id = await data_client.add_bounding_box_to_image_by_id(
+ binary_id=binary_id.file_id,
+ label=detection.class_name,
+ x_min_normalized=bbox.x_min_normalized,
+ y_min_normalized=bbox.y_min_normalized,
+ x_max_normalized=bbox.x_max_normalized,
+ y_max_normalized=bbox.y_max_normalized
+ )
+
+ print(f"Bounding box created: {bbox_id} for class: {detection.class_name}")
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+Use an ML model to generate bounding boxes for an image.
+Then, separately pass each bounding box and the image ID to [`DataClient.AddBoundingBoxToImageByID`](/dev/reference/apis/data-client/#addboundingboxtoimagebyid):
+
+```go
+ctx := context.Background()
+
+viamClient, err := client.New(ctx, "", logger)
+if err != nil {
+ log.Fatal(err)
+}
+defer viamClient.Close(ctx)
+
+myDetector, err := vision.FromRobot(viamClient, "")
+if err != nil {
+ log.Fatal(err)
+}
+
+dataClient := viamClient.DataClient()
+
+// Get the captured data for a camera
+result, err := myDetector.CaptureAllFromCamera(ctx, "", &vision.CaptureAllFromCameraRequest{
+ ReturnImage: true,
+ ReturnDetections: true,
+})
+if err != nil {
+ log.Fatal(err)
+}
+
+image := result.Image
+detections := result.Detections
+
+for _, detection := range detections {
+ bboxID, err := dataClient.AddBoundingBoxToImageByID(ctx, &datamanager.AddBoundingBoxRequest{
+ BinaryID: "",
+ Label: detection.Label,
+ XMinNormalized: detection.BoundingBox.Min.X,
+ YMinNormalized: detection.BoundingBox.Min.Y,
+ XMaxNormalized: detection.BoundingBox.Max.X,
+ YMaxNormalized: detection.BoundingBox.Max.Y,
+ })
+
+ fmt.Printf("Added bounding box ID: %s for detection: %s\n", bboxID, detection.ClassName)
+}
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+Use an ML model to generate bounding boxes for an image.
+Then, separately pass each bounding box and the image ID to [`dataClient.addBoundingBoxToImageById`](/dev/reference/apis/data-client/#addboundingboxtoimagebyid):
+
+```typescript
+const client = await createViamClient();
+const myDetector = new VisionClient(client, "");
+const dataClient = client.dataClient;
+
+// Get the captured data for a camera
+const result = await myDetector.captureAllFromCamera("", {
+ returnImage: true,
+ returnDetections: true,
+});
+const image = result.image;
+const detections = result.detections;
+
+// Process each detection and add bounding boxes
+for (const detection of detections) {
+ const bboxId = await dataClient.addBoundingBoxToImageById({
+ binaryId: "",
+ label: detection.className,
+ xMinNormalized: detection.boundingBox.xMin,
+ yMinNormalized: detection.boundingBox.yMin,
+ xMaxNormalized: detection.boundingBox.xMax,
+ yMaxNormalized: detection.boundingBox.yMax,
+ });
+
+ console.log(
+ `Added bounding box ID: ${bboxId} for detection: ${detection.className}`,
+ );
+}
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+Use an ML model to generate bounding boxes for an image.
+Then, separately pass each bounding box and the image ID to [`dataClient.addBoundingBoxToImageById`](/dev/reference/apis/data-client/#addboundingboxtoimagebyid):
+
+```dart
+final viamClient = await ViamClient.connect();
+final myDetector = VisionClient.fromRobot(viamClient, "");
+final dataClient = viamClient.dataClient;
+
+// Get the captured data for a camera
+final result = await myDetector.captureAllFromCamera(
+ "",
+ returnImage: true,
+ returnDetections: true,
+);
+final image = result.image;
+final detections = result.detections;
+
+// Process each detection and add bounding boxes
+for (final detection in detections) {
+ final bboxId = await dataClient.addBoundingBoxToImageById(
+ binaryId: "",
+ label: detection.className,
+ xMinNormalized: detection.boundingBox.xMin,
+ yMinNormalized: detection.boundingBox.yMin,
+ xMaxNormalized: detection.boundingBox.xMax,
+ yMaxNormalized: detection.boundingBox.yMax,
+ );
+
+ print('Added bounding box ID: $bboxId for detection: ${detection.className}');
+}
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+Once you've annotated your dataset, you can [train](/data-ai/train/train-tflite/) an ML model to make inferences.
+
+## Combine these steps: capture, annotate, and add images to a dataset in a single script
+
+The following example demonstrates how you can capture an image, use an ML model to generate annotations, and then add the image to a dataset.
+You can use this logic to expand and improve your datasets continuously over time.
+Check the annotation accuracy in the **DATA** tab, then re-train your ML model on the improved dataset to improve the ML model.
+
+{{< tabs >}}
+{{% tab name="Python" %}}
+
+```python
+import asyncio
+import os
+import time
+from typing import Optional
+from io import BytesIO
+
+from PIL import Image
+from viam.app.app_client import AppClient
+from viam.logging import getLogger
+
+# Global machine configuration
+MACHINE_PART_ID = "your-machine-part-id-here"
+
+
+class DataCollector:
+ def __init__(self,
+ component_name: str, dataset_id: str,
+ api_key_id: str, api_key: str):
+
+ self.logger = getLogger(__name__)
+ self.component_name = component_name
+ self.dataset_id = dataset_id
+ self.api_key_id = api_key_id
+ self.api_key = api_key
+
+ async def capture_and_store_image(self,
+ processed_image: Image.Image,
+ classification: str) -> None:
+
+ if not MACHINE_PART_ID:
+ raise ValueError("machine part ID not configured")
+
+ # Create fresh data client connection
+ async with await self._create_data_client() as data_client:
+ image_data = self._encode_image_to_png(processed_image)
+
+ # Generate unique filename with timestamp
+ timestamp = int(time.time())
+ filename = f"{classification}-sample-{timestamp}.png"
+
+ component_type = "rdk:component:camera"
+
+ upload_metadata = {
+ "component_type": component_type,
+ "component_name": self.component_name,
+ "file_name": filename,
+ "file_extension": "png"
+ }
+
+ try:
+ file_id = await data_client.file_upload_from_bytes(
+ part_id=MACHINE_PART_ID,
+ data=image_data,
+ **upload_metadata
+ )
+
+ # Associate file with dataset immediately
+ await data_client.add_binary_data_to_dataset_by_ids(
+ binary_ids=[file_id],
+ dataset_id=self.dataset_id
+ )
+
+ self.logger.info(
+ f"successfully added {classification} image to dataset "
+ f"{self.dataset_id} (file ID: {file_id}, "
+ f"machine: {MACHINE_PART_ID})"
+ )
+
+ except Exception as e:
+ self.logger.error(f"failed to upload and associate image: {e}")
+ raise
+
+ async def _create_data_client(self) -> AppClient:
+ if not self.api_key_id or not self.api_key:
+ raise ValueError("API credentials not configured")
+
+ try:
+ client = AppClient.create_from_dial_options(
+ dial_options={
+ "auth_entity": self.api_key_id,
+ "credentials": {
+ "type": "api-key",
+ "payload": self.api_key
+ }
+ }
+ )
+ return client
+
+ except Exception as e:
+ raise ValueError(f"failed to create app client: {e}")
+
+ def _encode_image_to_png(self, img: Image.Image) -> bytes:
+ buffer = BytesIO()
+ img.save(buffer, format='PNG', optimize=True)
+ return buffer.getvalue()
+
+
+def create_data_collector(
+ component_name: str,
+ dataset_id: str,
+ api_key_id: str,
+ api_key: str
+ ) -> DataCollector:
+ if not component_name:
+ raise ValueError("component name is required")
+ if not dataset_id:
+ raise ValueError("dataset ID is required")
+ if not api_key_id or not api_key:
+ raise ValueError("API credentials are required")
+
+ return DataCollector(
+ component_name=component_name,
+ dataset_id=dataset_id,
+ api_key_id=api_key_id,
+ api_key=api_key
+ )
+
+
+# Example usage
+async def main():
+ collector = create_data_collector(
+ component_name="main_camera",
+ dataset_id="your-dataset-id",
+ api_key_id="your-api-key-id",
+ api_key="your-api-key"
+ )
+
+ # Example with PIL Image
+ sample_image = Image.new('RGB', (640, 480), color='red')
+ await collector.capture_and_store_image(sample_image, "positive_sample")
+
+if __name__ == "__main__":
+ asyncio.run(main())
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+```go
+package datasetcreation
+
+import (
+ "context"
+ "fmt"
+ "image"
+ "time"
+
+ "go.viam.com/rdk/logging"
+ "go.viam.com/rdk/services/datamanager/app"
+ "go.viam.com/rdk/app/appclient"
+)
+
+var MachinePartID string = "your-machine-part-id-here"
+
+type DataCollector struct {
+ logger logging.Logger
+ componentName string
+ datasetID string
+ apiKeyID string
+ apiKey string
+}
+
+func (dc *DataCollector) CaptureAndStoreImage(ctx context.Context, processedImage image.Image, classification string) error {
+ if MachinePartID == "" {
+ return fmt.Errorf("machine part ID not configured")
+ }
+
+ // Create fresh data client connection
+ dataClient, err := dc.createDataClient(ctx)
+ if err != nil {
+ dc.logger.Errorf("failed to create data client: %v", err)
+ return fmt.Errorf("data client initialization failed: %w", err)
+ }
+ defer dataClient.Close()
+
+ imageData, err := dc.encodeImageToPNG(processedImage)
+ if err != nil {
+ dc.logger.Errorf("image encoding failed: %v", err)
+ return fmt.Errorf("failed to encode image: %w", err)
+ }
+
+ // Generate unique filename with timestamp
+ timestamp := time.Now().Unix()
+ filename := fmt.Sprintf("%s-sample-%d.png", classification, timestamp)
+
+ componentType := "rdk:component:camera"
+ fileExtension := "png"
+
+ uploadOptions := app.FileUploadOptions{
+ ComponentType: &componentType,
+ ComponentName: &dc.componentName,
+ FileName: &filename,
+ FileExtension: &fileExtension,
+ }
+
+ fileID, err := dataClient.FileUploadFromBytes(ctx, MachinePartID, imageData, &uploadOptions)
+ if err != nil {
+ dc.logger.Errorf("file upload failed for %s: %v", filename, err)
+ return fmt.Errorf("failed to upload image: %w", err)
+ }
+
+ // Associate file with dataset immediately
+ err = dataClient.AddBinaryDataToDatasetByIDs(ctx, []string{fileID}, dc.datasetID)
+ if err != nil {
+ dc.logger.Errorf("dataset association failed for file %s: %v", fileID, err)
+ return fmt.Errorf("failed to add image to dataset: %w", err)
+ }
+
+ dc.logger.Infof("successfully added %s image to dataset %s (file ID: %s, machine: %s)",
+ classification, dc.datasetID, fileID, MachinePartID)
+
+ return nil
+}
+
+func (dc *DataCollector) createDataClient(ctx context.Context) (app.AppServiceClient, error) {
+ if dc.apiKeyID == "" || dc.apiKey == "" {
+ return nil, fmt.Errorf("API credentials not configured")
+ }
+
+ client, err := appclient.New(ctx, appclient.Config{
+ KeyID: dc.apiKeyID,
+ Key: dc.apiKey,
+ })
+ if err != nil {
+ return nil, fmt.Errorf("failed to create app client: %w", err)
+ }
+
+ return client.DataClient, nil
+}
+
+func (dc *DataCollector) encodeImageToPNG(img image.Image) ([]byte, error) {
+ // PNG encoding implementation
+ return nil, nil // Placeholder
+}
+
+func NewDataCollector(logger logging.Logger, componentName, datasetID, apiKeyID, apiKey string) (*DataCollector, error) {
+ if logger == nil {
+ return nil, fmt.Errorf("logger is required")
+ }
+ if componentName == "" {
+ return nil, fmt.Errorf("component name is required")
+ }
+ if datasetID == "" {
+ return nil, fmt.Errorf("dataset ID is required")
+ }
+ if apiKeyID == "" || apiKey == "" {
+ return nil, fmt.Errorf("API credentials are required")
+ }
+
+ return &DataCollector{
+ logger: logger,
+ componentName: componentName,
+ datasetID: datasetID,
+ apiKeyID: apiKeyID,
+ apiKey: apiKey,
+ }, nil
+}
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+```typescript
+import { createRobotClient, RobotClient } from "@viamrobotics/sdk";
+import { AppClient, DataClient } from "@viamrobotics/app-client";
+import { Logger } from "@viamrobotics/utils";
+
+const MACHINE_PART_ID: string = "your-machine-part-id-here";
+
+interface FileUploadOptions {
+ componentType?: string;
+ componentName?: string;
+ fileName?: string;
+ fileExtension?: string;
+}
+
+export class DataCollector {
+ private logger: Logger;
+ private componentName: string;
+ private datasetId: string;
+ private apiKeyId: string;
+ private apiKey: string;
+
+ constructor(
+ logger: Logger,
+ componentName: string,
+ datasetId: string,
+ apiKeyId: string,
+ apiKey: string,
+ ) {
+ this.logger = logger;
+ this.componentName = componentName;
+ this.datasetId = datasetId;
+ this.apiKeyId = apiKeyId;
+ this.apiKey = apiKey;
+ }
+
+ async captureAndStoreImage(
+ processedImage: ArrayBuffer,
+ classification: string,
+ ): Promise {
+ if (!MACHINE_PART_ID) {
+ throw new Error("Machine part ID not configured");
+ }
+
+ let dataClient: DataClient | null = null;
+
+ try {
+ dataClient = await this.createDataClient();
+
+ const imageData = await this.encodeImageToPng(processedImage);
+ const timestamp = Math.floor(Date.now() / 1000);
+ const filename = `${classification}-sample-${timestamp}.png`;
+
+ const componentType = "rdk:component:camera";
+ const fileExtension = "png";
+
+ const uploadOptions: FileUploadOptions = {
+ componentType,
+ componentName: this.componentName,
+ fileName: filename,
+ fileExtension,
+ };
+
+ const fileId = await dataClient.fileUploadFromBytes(
+ MACHINE_PART_ID,
+ new Uint8Array(imageData),
+ uploadOptions,
+ );
+
+ await dataClient.addBinaryDataToDatasetByIds([fileId], this.datasetId);
+
+ this.logger.info(
+ `Successfully added ${classification} image to dataset ${this.datasetId} ` +
+ `(file ID: ${fileId}, machine: ${MACHINE_PART_ID})`,
+ );
+ } catch (error) {
+ this.logger.error(`File upload failed for ${classification}: ${error}`);
+ throw new Error(`Failed to upload image: ${error}`);
+ } finally {
+ if (dataClient) {
+ await dataClient.close();
+ }
+ }
+ }
+
+ private async createDataClient(): Promise {
+ if (!this.apiKeyId || !this.apiKey) {
+ throw new Error("API credentials not configured");
+ }
+
+ const appClient = new AppClient({
+ apiKeyId: this.apiKeyId,
+ apiKey: this.apiKey,
+ });
+
+ return appClient.dataClient();
+ }
+
+ private async encodeImageToPng(image: ArrayBuffer): Promise {
+ try {
+ // PNG encoding implementation would depend on your image processing library
+ // This is a placeholder - you would use a library like 'pngjs' or 'canvas'
+ return image; // Assuming image is already PNG encoded
+ } catch (error) {
+ this.logger.error(`Image encoding failed: ${error}`);
+ throw new Error(`Failed to encode image: ${error}`);
+ }
+ }
+
+ static create(
+ logger: Logger,
+ componentName: string,
+ datasetId: string,
+ apiKeyId: string,
+ apiKey: string,
+ ): DataCollector {
+ if (!logger) {
+ throw new Error("Logger is required");
+ }
+ if (!componentName) {
+ throw new Error("Component name is required");
+ }
+ if (!datasetId) {
+ throw new Error("Dataset ID is required");
+ }
+ if (!apiKeyId || !apiKey) {
+ throw new Error("API credentials are required");
+ }
+
+ return new DataCollector(
+ logger,
+ componentName,
+ datasetId,
+ apiKeyId,
+ apiKey,
+ );
+ }
+}
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+```dart
+import 'dart:typed_data';
+import 'dart:io';
+import 'package:viam_sdk/viam_sdk.dart';
+import 'package:viam_sdk/src/app/app_client.dart';
+import 'package:viam_sdk/src/app/data_client.dart';
+import 'package:image/image.dart' as img;
+
+const String machinePartId = 'your-machine-part-id-here';
+
+class DataCollector {
+ final Logging logger;
+ final String componentName;
+ final String datasetId;
+ final String apiKeyId;
+ final String apiKey;
+
+ DataCollector({
+ required this.logger,
+ required this.componentName,
+ required this.datasetId,
+ required this.apiKeyId,
+ required this.apiKey,
+ });
+
+ Future captureAndStoreImage(
+ Image processedImage,
+ String classification,
+ ) async {
+ if (machinePartId.isEmpty) {
+ throw Exception('Machine part ID not configured');
+ }
+
+ DataClient? dataClient;
+ try {
+ dataClient = await _createDataClient();
+
+ final imageData = await _encodeImageToPng(processedImage);
+ final timestamp = DateTime.now().millisecondsSinceEpoch ~/ 1000;
+ final filename = '$classification-sample-$timestamp.png';
+
+ const componentType = 'rdk:component:camera';
+ const fileExtension = 'png';
+
+ final uploadOptions = FileUploadOptions(
+ componentType: componentType,
+ componentName: componentName,
+ fileName: filename,
+ fileExtension: fileExtension,
+ );
+
+ final fileId = await dataClient.fileUploadFromBytes(
+ machinePartId,
+ imageData,
+ uploadOptions,
+ );
+
+ await dataClient.addBinaryDataToDatasetByIds(
+ [fileId],
+ datasetId,
+ );
+
+ logger.info(
+ 'Successfully added $classification image to dataset $datasetId '
+ '(file ID: $fileId, machine: $machinePartId)',
+ );
+ } catch (error) {
+ logger.error('File upload failed for $classification: $error');
+ rethrow;
+ } finally {
+ await dataClient?.close();
+ }
+ }
+
+ Future _createDataClient() async {
+ if (apiKeyId.isEmpty || apiKey.isEmpty) {
+ throw Exception('API credentials not configured');
+ }
+
+ final appClient = await AppClient.withApiKey(
+ apiKeyId: apiKeyId,
+ apiKey: apiKey,
+ );
+
+ return appClient.dataClient;
+ }
+
+ Future _encodeImageToPng(Image image) async {
+ try {
+ final pngBytes = img.encodePng(image);
+ return Uint8List.fromList(pngBytes);
+ } catch (error) {
+ logger.error('Image encoding failed: $error');
+ throw Exception('Failed to encode image: $error');
+ }
+ }
+
+ static DataCollector create({
+ required Logging logger,
+ required String componentName,
+ required String datasetId,
+ required String apiKeyId,
+ required String apiKey,
+ }) {
+ if (componentName.isEmpty) {
+ throw ArgumentError('Component name is required');
+ }
+ if (datasetId.isEmpty) {
+ throw ArgumentError('Dataset ID is required');
+ }
+ if (apiKeyId.isEmpty || apiKey.isEmpty) {
+ throw ArgumentError('API credentials are required');
+ }
+
+ return DataCollector(
+ logger: logger,
+ componentName: componentName,
+ datasetId: datasetId,
+ apiKeyId: apiKeyId,
+ apiKey: apiKey,
+ );
+ }
+}
+```
+
+{{% /tab %}}
+{{< /tabs >}}
diff --git a/docs/data-ai/train/create-dataset.md b/docs/data-ai/train/create-dataset.md
new file mode 100644
index 0000000000..f6143e68ca
--- /dev/null
+++ b/docs/data-ai/train/create-dataset.md
@@ -0,0 +1,777 @@
+---
+linkTitle: "Create a training dataset"
+title: "Create a training dataset"
+weight: 10
+layout: "docs"
+type: "docs"
+description: "Create a dataset to train a machine learning model."
+aliases:
+ - /fleet/dataset/
+ - /manage/data/label/
+ - /manage/data/dataset/
+ - /data/dataset/
+ - /data-ai/ai/create-dataset/
+---
+
+To train a machine learning model, you will need a dataset.
+
+## Create a dataset
+
+You can create a dataset using the web UI, the CLI, or one of the SDKs:
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+1. Navigate to the **DATA** page and open the [**DATASETS** tab](https://app.viam.com/data/datasets).
+
+1. Click the **+ Create dataset** button.
+
+ {{< imgproc src="/services/data/create-dataset.png" alt="The **DATASET** tab of the **DATA** page, showing the **+ Create dataset** button." resize="800x" style="width:500px" class="imgzoom" >}}
+
+1. Enter a unique name for the dataset.
+
+1. Click **Create dataset**.
+
+{{% /tab %}}
+{{% tab name="CLI" %}}
+
+Run the following [Viam CLI](/dev/tools/cli/) command to create a dataset, replacing the `` and `` placeholders with your organization ID and a unique name for the dataset:
+
+```sh {class="command-line" data-prompt="$"}
+viam dataset create --org-id= --name=
+```
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+To create a dataset, pass a unique dataset name and organization ID to [`data_client.create_dataset`](/dev/reference/apis/data-client/#createdataset):
+
+```python
+viam_client = await connect()
+data_client = viam_client.data_client
+
+print("Creating dataset...")
+
+try:
+ dataset_id = await data_client.create_dataset(
+ name="",
+ organization_id="",
+ )
+ print(f"Created dataset: {dataset_id}")
+except Exception as e:
+ print("Error creating dataset. It may already exist.")
+ print(f"Exception: {e}")
+ raise
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+To create a dataset, pass a unique dataset name and organization ID to [`DataClient.CreateDataset`](/dev/reference/apis/data-client/#createdataset):
+
+```go
+ctx := context.Background()
+viamClient, err := client.New(ctx, "", logger)
+if err != nil {
+ log.Fatal(err)
+}
+defer viamClient.Close(ctx)
+
+dataClient := viamClient.DataClient()
+
+fmt.Println("Creating dataset...")
+
+datasetID, err := dataClient.CreateDataset(ctx, &datamanager.CreateDatasetRequest{
+ Name: "",
+ OrganizationID: "",
+})
+
+if err != nil {
+ fmt.Println("Error creating dataset. It may already exist.")
+ fmt.Printf("Exception: %v\n", err)
+ return
+}
+
+fmt.Printf("Created dataset: %s\n", datasetID)
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+To create a dataset, pass a unique dataset name and organization ID to [`dataClient.createDataset`](/dev/reference/apis/data-client/#createdataset):
+
+```typescript
+const client = await createViamClient();
+const dataClient = client.dataClient;
+
+console.log("Creating dataset...");
+
+try {
+ const datasetId = await dataClient.createDataset({
+ name: "",
+ organizationId: "",
+ });
+ console.log(`Created dataset: ${datasetId}`);
+} catch (error) {
+ console.log("Error creating dataset. It may already exist.");
+ console.log(`Exception: ${error}`);
+ process.exit(1);
+}
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+To create a dataset, pass a unique dataset name and organization ID to [`dataClient.createDataset`](/dev/reference/apis/data-client/#createdataset):
+
+```dart
+final viamClient = await ViamClient.connect();
+final dataClient = viamClient.dataClient;
+
+print("Creating dataset...");
+
+try {
+ final datasetId = await dataClient.createDataset(
+ name: "",
+ organizationId: "",
+ );
+ print("Created dataset: $datasetId");
+} catch (e) {
+ print("Error creating dataset. It may already exist.");
+ print("Exception: $e");
+ return;
+}
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+Finish creating a dataset by adding annotated images to it.
+You can capture new images or add existing images:
+
+## Add to a dataset
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+You can add images to a dataset from the **Images** tab of the [**DATA** page](https://app.viam.com/data/view):
+
+1. Click to select the images you would like to add to your dataset.
+
+1. Click the **Add to dataset** button in the top right.
+
+1. From the **Dataset** dropdown, select the name of your dataset.
+
+1. Click **Add \ images** to add the selected images to the dataset.
+
+{{< alert title="Tip" color="tip" >}}
+
+To select a range of images, select one image, then hold **Ctrl/Cmd** while clicking another image.
+This will select both images as well as the entire range of images between those images.
+
+{{< /alert >}}
+
+{{% /tab %}}
+{{% tab name="CLI" %}}
+
+Use the Viam CLI to filter images by label and add the filtered images to a dataset:
+
+1. First, [create a dataset](/data-ai/train/create-dataset/), if you haven't already.
+
+1. If you just created a dataset, use the dataset ID output by the creation command.
+ If your dataset already exists, run the following command to get a list of dataset names and corresponding IDs:
+
+ ```sh {class="command-line" data-prompt="$"}
+ viam dataset list
+ ```
+
+1. Run the following [command](/dev/tools/cli/#dataset) to add all images labeled with a subset of tags to the dataset, replacing the `` placeholder with the dataset ID output by the command in the previous step:
+
+ ```sh {class="command-line" data-prompt="$"}
+ viam dataset data add filter --dataset-id= --tags=red_star,blue_square
+ ```
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+To add an image to a dataset, find the binary data ID for the image and the dataset ID.
+Pass both IDs to [`data_client.add_binary_data_to_dataset_by_ids`](/dev/reference/apis/data-client/#addbinarydatatodatasetbyids):
+
+```python
+# Connect to Viam client
+dial_options = DialOptions(
+ credentials=Credentials(
+ type="api-key",
+ payload=API_KEY,
+ ),
+ auth_entity=API_KEY_ID,
+)
+
+viam_client = await ViamClient.create_from_dial_options(dial_options)
+data_client = viam_client.data_client
+
+# Add image to dataset
+await data_client.add_binary_data_to_dataset_by_ids(
+ binary_ids=[EXISTING_IMAGE_ID],
+ dataset_id=EXISTING_DATASET_ID
+)
+
+print("Image added to dataset successfully")
+viam_client.close()
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+To add an image to a dataset, find the binary data ID for the image and the dataset ID.
+Pass both IDs to [`DataClient.AddBinaryDataToDatasetByIDs`](/dev/reference/apis/data-client/#addbinarydatatodatasetbyids):
+
+```go
+ctx := context.Background()
+viamClient, err := client.New(ctx, "", logger)
+if err != nil {
+ log.Fatal(err)
+}
+defer viamClient.Close(ctx)
+
+dataClient := viamClient.DataClient()
+
+// Add image to dataset
+err = dataClient.AddBinaryDataToDatasetByIDs(
+ context.Background(),
+ []string{ExistingImageID},
+ ExistingDatasetID,
+)
+if err != nil {
+ return fmt.Errorf("failed to add image to dataset: %w", err)
+}
+
+fmt.Println("Image added to dataset successfully")
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+To add an image to a dataset, find the binary data ID for the image and the dataset ID.
+Pass both IDs to [`dataClient.addBinaryDataToDatasetByIDs`](/dev/reference/apis/data-client/#addbinarydatatodatasetbyids):
+
+```typescript
+const client: ViamClient = await createViamClient({
+ credential: {
+ type: "api-key",
+ payload: API_KEY,
+ },
+ authEntity: API_KEY_ID,
+});
+
+const dataClient = client.dataClient;
+
+// Add image to dataset
+await dataClient.addBinaryDataToDatasetByIds({
+ binaryIds: [EXISTING_IMAGE_ID],
+ datasetId: EXISTING_DATASET_ID,
+});
+
+console.log("Image added to dataset successfully");
+client.disconnect();
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+To add an image to a dataset, find the binary data ID for the image and the dataset ID.
+Pass both IDs to [`dataClient.addBinaryDataToDatasetByIDs`](/dev/reference/apis/data-client/#addbinarydatatodatasetbyids):
+
+```dart
+final client = await ViamClient.withApiKey(
+ apiKeyId: apiKeyId,
+ apiKey: apiKey,
+);
+
+final dataClient = client.dataClient;
+
+try {
+ // Add image to dataset
+ await dataClient.addBinaryDataToDatasetByIds(
+ binaryIds: [existingImageId],
+ datasetId: existingDatasetId,
+ );
+
+ print('Image added to dataset successfully');
+} finally {
+ await client.close();
+}
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+## Add all images captured by a specific machine to a dataset
+
+{{< tabs >}}
+{{% tab name="Web UI" %}}
+
+You can add images to a dataset from the **Images** tab of the [**DATA** page](https://app.viam.com/data/view):
+
+1. From the **Machine name** dropdown, select the name of a machine.
+1. Click the **Apply** button at the bottom of the left sidebar.
+1. Click to select the images you would like to add to your dataset.
+1. Click the **Add to dataset** button in the top right.
+1. From the **Dataset** dropdown, select the name of your dataset.
+1. Click **Add \ images** to add the selected images to the dataset.
+
+{{< alert title="Tip" color="tip" >}}
+
+To select a range of images, select one image, then hold **Ctrl/Cmd** while clicking another image.
+This will select both images as well as the entire range of images between those images.
+
+{{< /alert >}}
+
+{{% /tab %}}
+{{% tab name="Python" %}}
+
+The following script adds all images captured from a certain machine to a new dataset:
+
+```python
+import asyncio
+from typing import List, Optional
+
+from viam.rpc.dial import DialOptions, Credentials
+from viam.app.viam_client import ViamClient
+from viam.utils import create_filter
+
+# Configuration constants – replace with your actual values
+DATASET_NAME = "" # a unique, new name for the dataset you want to create
+ORG_ID = "" # your organization ID, find in your organization settings
+PART_ID = "" # ID of machine that captured images, find in machine config
+API_KEY = "" # API key, find or create in your organization settings
+API_KEY_ID = "" # API key ID, find or create in your organization settings
+
+# Adjust the maximum number of images to add to the dataset
+MAX_MATCHES = 500
+
+
+async def connect() -> ViamClient:
+ """Establish a connection to the Viam client using API credentials."""
+ dial_options = DialOptions(
+ credentials=Credentials(
+ type="api-key",
+ payload=API_KEY,
+ ),
+ auth_entity=API_KEY_ID,
+ )
+ return await ViamClient.create_from_dial_options(dial_options)
+
+
+async def fetch_binary_data_ids(data_client, part_id: str) -> List[str]:
+ """Fetch binary data metadata and return a list of BinaryData objects."""
+ data_filter = create_filter(part_id=part_id)
+ all_matches = []
+ last: Optional[str] = None
+
+ print("Getting data for part...")
+
+ while len(all_matches) < MAX_MATCHES:
+ print("Fetching more data...")
+ data, _, last = await data_client.binary_data_by_filter(
+ data_filter,
+ limit=50,
+ last=last,
+ include_binary_data=False,
+ )
+ if not data:
+ break
+ all_matches.extend(data)
+
+ return all_matches
+
+
+async def main() -> int:
+ """Main execution function."""
+ viam_client = await connect()
+ data_client = viam_client.data_client
+
+ matching_data = await fetch_binary_data_ids(data_client, PART_ID)
+
+ print("Creating dataset...")
+
+ try:
+ dataset_id = await data_client.create_dataset(
+ name=DATASET_NAME,
+ organization_id=ORG_ID,
+ )
+ print(f"Created dataset: {dataset_id}")
+ except Exception as e:
+ print("Error creating dataset. It may already exist.")
+ print("See: https://app.viam.com/data/datasets")
+ print(f"Exception: {e}")
+ return 1
+
+ print("Adding data to dataset...")
+
+ await data_client.add_binary_data_to_dataset_by_ids(
+ binary_ids=[
+ obj.metadata.binary_data_id for obj in matching_data
+ ],
+ dataset_id=dataset_id
+ )
+
+ print("Added files to dataset.")
+ print(
+ f"See dataset: https://app.viam.com/data/datasets?id={dataset_id}"
+ )
+
+ viam_client.close()
+ return 0
+
+if __name__ == "__main__":
+ asyncio.run(main())
+```
+
+{{% /tab %}}
+{{% tab name="Go" %}}
+
+The following script adds all images captured from a certain machine to a new dataset:
+
+```go
+package main
+
+import (
+ "context"
+ "fmt"
+ "log"
+
+ "go.viam.com/rdk/rpc"
+ "go.viam.com/utils"
+ "go.viam.com/rdk/app"
+ "go.viam.com/rdk/app/data"
+)
+
+// Configuration constants - replace with your actual values
+const (
+ DATASET_NAME = "" // a unique, new name for the dataset you want to create
+ ORG_ID = "" // your organization ID, find in your organization settings
+ PART_ID = "" // ID of machine that captured target images, find in machine config
+ API_KEY = "" // API key, find or create in your organization settings
+ API_KEY_ID = "" // API key ID, find or create in your organization settings
+ MAX_MATCHES = 500
+)
+
+func connect(ctx context.Context) (*app.ViamClient, error) {
+ client, err := app.NewViamClientWithAPIKey(ctx, API_KEY, API_KEY_ID)
+ if err != nil {
+ return nil, fmt.Errorf("failed to create client: %w", err)
+ }
+ return client, nil
+}
+
+func fetchBinaryDataIDs(
+ ctx context.Context,
+ dataClient data.DataServiceClient,
+ partID string) ([]*data.BinaryData, error) {
+ filter := &data.Filter{
+ PartId: partID,
+ }
+
+ var allMatches []*data.BinaryData
+ var last string
+
+ fmt.Println("Getting data for part...")
+
+ for len(allMatches) < MAX_MATCHES {
+ fmt.Println("Fetching more data...")
+
+ resp, err := dataClient.BinaryDataByFilter(ctx, &data.BinaryDataByFilterRequest{
+ DataRequest: &data.DataRequest{
+ Filter: filter,
+ Limit: 50,
+ Last: last,
+ IncludeBinaryData: false,
+ },
+ })
+ if err != nil {
+ return nil, fmt.Errorf("failed to fetch binary data: %w", err)
+ }
+
+ if len(resp.Data) == 0 {
+ break
+ }
+
+ allMatches = append(allMatches, resp.Data...)
+ last = resp.Last
+ }
+
+ return allMatches, nil
+}
+
+func main() {
+ ctx := context.Background()
+
+ viamClient, err := connect(ctx)
+ if err != nil {
+ log.Fatalf("Connection failed: %v", err)
+ }
+ defer viamClient.Close()
+
+ dataClient := viamClient.DataClient
+
+ matchingData, err := fetchBinaryDataIDs(ctx, dataClient, PART_ID)
+ if err != nil {
+ log.Fatalf("Failed to fetch binary data: %v", err)
+ }
+
+ fmt.Println("Creating dataset...")
+
+ datasetResp, err := dataClient.CreateDataset(ctx, &data.CreateDatasetRequest{
+ Name: DATASET_NAME,
+ OrganizationId: ORG_ID,
+ })
+ if err != nil {
+ fmt.Println("Error creating dataset. It may already exist.")
+ fmt.Println("See: https://app.viam.com/data/datasets")
+ fmt.Printf("Exception: %v\n", err)
+ return
+ }
+
+ datasetID := datasetResp.Id
+ fmt.Printf("Created dataset: %s\n", datasetID)
+
+ fmt.Println("Adding data to dataset...")
+
+ var binaryIDs []string
+ for _, obj := range matchingData {
+ binaryIDs = append(binaryIDs, obj.Metadata.Id)
+ }
+
+ _, err = dataClient.AddBinaryDataToDatasetByIds(ctx, &data.AddBinaryDataToDatasetByIdsRequest{
+ BinaryIds: binaryIDs,
+ DatasetId: datasetID,
+ })
+ if err != nil {
+ log.Fatalf("Failed to add binary data to dataset: %v", err)
+ }
+
+ fmt.Println("Added files to dataset.")
+ fmt.Printf("See dataset: https://app.viam.com/data/datasets?id=%s\n", datasetID)
+}
+
+```
+
+{{% /tab %}}
+{{% tab name="TypeScript" %}}
+
+The following script adds all images captured from a certain machine to a new dataset:
+
+```typescript
+import { ViamClient, createViamClient } from "@viamrobotics/sdk";
+import { DataServiceClient } from "@viamrobotics/sdk/dist/gen/app/data/v1/data_pb_service";
+import {
+ BinaryDataByFilterRequest,
+ CreateDatasetRequest,
+ AddBinaryDataToDatasetByIdsRequest,
+ Filter,
+ DataRequest,
+} from "@viamrobotics/sdk/dist/gen/app/data/v1/data_pb";
+
+// Configuration constants - replace with your actual values
+const DATASET_NAME = ""; // a unique, new name for the dataset you want to create
+const ORG_ID = ""; // your organization ID, find in your organization settings
+const PART_ID = ""; // ID of machine that captured target images, find in machine config
+const API_KEY = ""; // API key, find or create in your organization settings
+const API_KEY_ID = ""; // API key ID, find or create in your organization settings
+const MAX_MATCHES = 500;
+
+async function connect(): Promise {
+ return await createViamClient({
+ credential: {
+ type: "api-key",
+ authEntity: API_KEY_ID,
+ payload: API_KEY,
+ },
+ });
+}
+
+async function fetchBinaryDataIds(
+ dataClient: DataServiceClient,
+ partId: string,
+): Promise {
+ const filter = new Filter();
+ filter.setPartId(partId);
+
+ const allMatches: any[] = [];
+ let last = "";
+
+ console.log("Getting data for part...");
+
+ while (allMatches.length < MAX_MATCHES) {
+ console.log("Fetching more data...");
+
+ const dataRequest = new DataRequest();
+ dataRequest.setFilter(filter);
+ dataRequest.setLimit(50);
+ dataRequest.setLast(last);
+ dataRequest.setIncludeBinaryData(false);
+
+ const request = new BinaryDataByFilterRequest();
+ request.setDataRequest(dataRequest);
+
+ const response = await dataClient.binaryDataByFilter(request);
+ const data = response.getDataList();
+
+ if (data.length === 0) {
+ break;
+ }
+
+ allMatches.push(...data);
+ last = response.getLast();
+ }
+
+ return allMatches;
+}
+
+async function main(): Promise {
+ const viamClient = await connect();
+ const dataClient = viamClient.dataClient;
+
+ const matchingData = await fetchBinaryDataIds(dataClient, PART_ID);
+
+ console.log("Creating dataset...");
+
+ try {
+ const createRequest = new CreateDatasetRequest();
+ createRequest.setName(DATASET_NAME);
+ createRequest.setOrganizationId(ORG_ID);
+
+ const datasetResponse = await dataClient.createDataset(createRequest);
+ const datasetId = datasetResponse.getId();
+ console.log(`Created dataset: ${datasetId}`);
+
+ console.log("Adding data to dataset...");
+
+ const binaryIds = matchingData.map(
+ (obj) => obj.getMetadata()?.getId() || "",
+ );
+
+ const addRequest = new AddBinaryDataToDatasetByIdsRequest();
+ addRequest.setBinaryIdsList(binaryIds);
+ addRequest.setDatasetId(datasetId);
+
+ await dataClient.addBinaryDataToDatasetByIds(addRequest);
+
+ console.log("Added files to dataset.");
+ console.log(
+ `See dataset: https://app.viam.com/data/datasets?id=${datasetId}`,
+ );
+
+ viamClient.disconnect();
+ return 0;
+ } catch (error) {
+ console.log("Error creating dataset. It may already exist.");
+ console.log("See: https://app.viam.com/data/datasets");
+ console.log(`Exception: ${error}`);
+ viamClient.disconnect();
+ return 1;
+ }
+}
+
+if (require.main === module) {
+ main().then((code) => process.exit(code));
+}
+```
+
+{{% /tab %}}
+{{% tab name="Flutter" %}}
+
+The following script adds all images captured from a certain machine to a new dataset:
+
+```dart
+import 'package:viam_sdk/viam_sdk.dart';
+
+// Configuration constants - replace with your actual values
+const String datasetName = ""; // a unique, new name for the dataset you want to create
+const String orgId = ""; // your organization ID, find in your organization settings
+const String partId = ""; // ID of machine that captured target images, find in machine config
+const String apiKey = ""; // API key, find or create in your organization settings
+const String apiKeyId = ""; // API key ID, find or create in your organization settings
+const int maxMatches = 500;
+
+Future connect() async {
+ return await ViamClient.withApiKey(
+ apiKey: apiKey,
+ apiKeyId: apiKeyId,
+ );
+}
+
+Future> fetchBinaryDataIds(
+ DataClient dataClient, String partId) async {
+ final filter = Filter(partId: partId);
+ final List allMatches = [];
+ String? last;
+
+ print("Getting data for part...");
+
+ while (allMatches.length < maxMatches) {
+ print("Fetching more data...");
+
+ final response = await dataClient.binaryDataByFilter(
+ filter: filter,
+ limit: 50,
+ last: last,
+ includeBinaryData: false,
+ );
+
+ if (response.data.isEmpty) {
+ break;
+ }
+
+ allMatches.addAll(response.data);
+ last = response.last;
+ }
+
+ return allMatches;
+}
+
+Future main() async {
+ final viamClient = await connect();
+ final dataClient = viamClient.dataClient;
+
+ final matchingData = await fetchBinaryDataIds(dataClient, partId);
+
+ print("Creating dataset...");
+
+ try {
+ final datasetId = await dataClient.createDataset(
+ name: datasetName,
+ organizationId: orgId,
+ );
+ print("Created dataset: $datasetId");
+
+ print("Adding data to dataset...");
+
+ final binaryIds = matchingData
+ .map((obj) => obj.metadata.binaryDataId)
+ .toList();
+
+ await dataClient.addBinaryDataToDatasetByIds(
+ binaryIds: binaryIds,
+ datasetId: datasetId,
+ );
+
+ print("Added files to dataset.");
+ print("See dataset: https://app.viam.com/data/datasets?id=$datasetId");
+
+ viamClient.close();
+ return 0;
+
+ } catch (error) {
+ print("Error creating dataset. It may already exist.");
+ print("See: https://app.viam.com/data/datasets");
+ print("Exception: $error");
+ viamClient.close();
+ return 1;
+ }
+}
+```
+
+{{% /tab %}}
+{{< /tabs >}}
diff --git a/docs/data-ai/ai/train-tflite.md b/docs/data-ai/train/train-tflite.md
similarity index 86%
rename from docs/data-ai/ai/train-tflite.md
rename to docs/data-ai/train/train-tflite.md
index e51f87652b..30045ec4d6 100644
--- a/docs/data-ai/ai/train-tflite.md
+++ b/docs/data-ai/train/train-tflite.md
@@ -1,7 +1,7 @@
---
-linkTitle: "Train TFlite model"
+linkTitle: "Train TFLite model"
title: "Train a TFlite model"
-weight: 20
+weight: 50
type: "docs"
tags: ["vision", "data", "services"]
images: ["/services/ml/train.svg"]
@@ -19,6 +19,7 @@ aliases:
- /extend/modular-resources/examples/tflite-module/
- /modular-resources/examples/tflite-module/
- /registry/examples/tflite-module/
+ - /data-ai/ai/train-tflite/
languages: []
viamresources: ["data_manager", "mlmodel", "vision"]
platformarea: ["ml"]
@@ -31,17 +32,30 @@ Follow this guide to use your image data to train an ML model, so that your mach
## Prerequisites
-{{% expand "a machine connected to Viam" %}}
+{{% expand "A machine connected to Viam" %}}
{{% snippet "setup.md" %}}
{{% /expand %}}
-{{% expand "a dataset with labels" %}}
+{{% expand "A dataset that meets training requirements" %}}
-Follow the guide to [create a dataset](/data-ai/ai/create-dataset/).
+To train a model, your dataset must contain the following:
-{{% /expand%}}
+- At least 15 images
+- At least 80% of the images have labels
+- For each training label, at least 10 examples
+
+When you label your dataset, include:
+
+- images with and _without_ the categories you're looking to identify
+- a roughly equal number of images for each category
+- images from your production environment, including lighting and camera quality
+- examples from every angle and distance that you expect the model to handle
+
+Follow the guide to [create a dataset](/data-ai/train/create-dataset/).
+
+{{% /expand %}}
## Train a machine learning model
@@ -145,11 +159,11 @@ Using this approach, each subsequent model version becomes more accurate than th
To capture images of edge cases and re-train your model using those images, complete the following steps:
-1. Add edge case images to your training dataset. You can find edge cases in your existing data on the [**DATA** page](https://app.viam.com/data/) or [capture new images and add them to your training dataset](/data-ai/ai/create-dataset/#capture-images).
+1. Add edge case images to your training dataset. You can find edge cases in your existing data on the [**DATA** page](https://app.viam.com/data/) or [capture new images and add them to your training dataset](/data-ai/train/capture-annotate-images/).
1. Visit the **DATASET** tab of the **DATA** page and annotate the image.
-1. Repeat the [steps above](/data-ai/ai/train-tflite/#train-a-machine-learning-model) to train and release a new version of your ML model. Your machines will automatically update to the new version of the model soon after release.
+1. Repeat the [steps above](/data-ai/train/train-tflite/#train-a-machine-learning-model) to train and release a new version of your ML model. Your machines will automatically update to the new version of the model soon after release.
## Next steps
diff --git a/docs/data-ai/ai/train.md b/docs/data-ai/train/train.md
similarity index 99%
rename from docs/data-ai/ai/train.md
rename to docs/data-ai/train/train.md
index 71caae8cda..974447709e 100644
--- a/docs/data-ai/ai/train.md
+++ b/docs/data-ai/train/train.md
@@ -2,7 +2,7 @@
linkTitle: "Train other models"
title: "Train other models"
tags: ["data management", "ml", "model training"]
-weight: 30
+weight: 51
layout: "docs"
type: "docs"
aliases:
@@ -10,6 +10,7 @@ aliases:
- /how-tos/create-custom-training-scripts/
- /services/ml/training-scripts/
- /registry/training-scripts/
+ - /data-ai/ai/train/
languages: ["python"]
viamresources: ["mlmodel", "data_manager"]
platformarea: ["ml"]
@@ -25,15 +26,15 @@ If you wish to do this, skip to [Submit a training job](#submit-a-training-job).
## Prerequisites
-{{% expand "A dataset with data you can train an ML model on. Click to see instructions." %}}
+{{% expand "A dataset that contains training data" %}}
-For images, follow the instructions to [Create a dataset](/data-ai/ai/create-dataset/) to create a dataset and label data.
+For images, follow the instructions to [Create a dataset](/data-ai/train/create-dataset/) to create a dataset and label data.
For other data, use the [Data Client API](/dev/reference/apis/data-client/) from within the training script to store data in the Viam Cloud.
{{% /expand%}}
-{{% expand "The Viam CLI. Click to see instructions." %}}
+{{% expand "The Viam CLI" %}}
You must have the Viam CLI installed to upload training scripts to the registry.
diff --git a/docs/data-ai/ai/advanced/upload-external-data.md b/docs/data-ai/train/upload-external-data.md
similarity index 95%
rename from docs/data-ai/ai/advanced/upload-external-data.md
rename to docs/data-ai/train/upload-external-data.md
index 56ac6d7307..457ef41077 100644
--- a/docs/data-ai/ai/advanced/upload-external-data.md
+++ b/docs/data-ai/train/upload-external-data.md
@@ -2,7 +2,7 @@
linkTitle: "Upload external data"
title: "Upload external data for training"
images: ["/services/icons/data-folder.svg"]
-weight: 20
+weight: 60
layout: "docs"
type: "docs"
languages: ["python"]
@@ -11,6 +11,8 @@ aliases:
- /data/upload/
- /services/data/upload/
- /how-tos/upload-data/
+ - /data-ai/ai/advanced/upload-external-data/
+ - /data-ai/ai/advanced/
date: "2024-12-04"
description: "Upload data to Viam from your local computer or mobile device using the data client API, Viam CLI, or Viam mobile app."
prev: "/data-ai/ai/act/"
@@ -30,13 +32,13 @@ However, if you already have a cache of data you'd like to use with Viam, you ca
### Prerequisites
-{{% expand "A running machine connected to Viam. Click to see instructions." %}}
+{{% expand "A running machine connected to Viam" %}}
{{% snippet "setup-both.md" %}}
{{% /expand%}}
-{{< expand "Enable data capture and sync on your machine." >}}
+{{< expand "Enable data capture and sync on your machine" >}}
Add the [data management service](/data-ai/capture-data/capture-sync/#configure-data-capture-and-sync-for-individual-resources):
@@ -135,9 +137,7 @@ Create a Python script and use the `file_upload_from_path` method to upload your
{{< tabs >}}
{{< tab name="Upload a single file" >}}
-To upload just one file, make a call to [`file_upload_from_path`](/dev/reference/apis/data-client/#fileuploadfrompath).
-
-{{< expand "Click this to see example code" >}}
+To upload just one file, make a call to [`file_upload_from_path`](/dev/reference/apis/data-client/#fileuploadfrompath):
```python {class="line-numbers linkable-line-numbers"}
import asyncio
@@ -180,15 +180,11 @@ if __name__ == "__main__":
asyncio.run(main())
```
-{{< /expand >}}
-
{{% /tab %}}
{{< tab name="Upload all files in a directory" >}}
To upload all the files in a directory, you can use the [`file_upload_from_path`](/dev/reference/apis/data-client/#fileuploadfrompath) method inside a `for` loop.
-{{< expand "Click this to see example code" >}}
-
```python {class="line-numbers linkable-line-numbers"}
import asyncio
import os
@@ -232,8 +228,6 @@ if __name__ == "__main__":
asyncio.run(main())
```
-{{< /expand >}}
-
{{% /tab %}}
{{< /tabs >}}
@@ -296,5 +290,5 @@ However, the uploaded images will not be associated with a component or method.
## Next steps
-Now that you have a batch of data uploaded, you can [train an ML model](/data-ai/ai/train-tflite/) on it.
-Or, if you want to collect and upload data _not_ in a batch, see [Create a dataset](/data-ai/ai/create-dataset/).
+Now that you have a batch of data uploaded, you can [train an ML model](/data-ai/train/train-tflite/) on it.
+Or, if you want to collect and upload data _not_ in a batch, see [Create a dataset](/data-ai/train/create-dataset/).
diff --git a/docs/dev/_index.md b/docs/dev/_index.md
index 5e7a3bff4c..96d513ab79 100644
--- a/docs/dev/_index.md
+++ b/docs/dev/_index.md
@@ -753,7 +753,7 @@ job_metadata = await ml_training_client.get_training_job(
Build machine learning models based on your machines' data any time using the ML training client API
-[Train and deploy ML models →](/data-ai/ai/train-tflite/)
+[Train and deploy ML models →](/data-ai/train/train-tflite/)