---
updatedAt: 2026-05-12T09:56:03.000Z
agentTools:
  projectIndex: https://vi.developers.datature.com/llms.txt
---

# Sync From MinIO

Connect a Datature Vi dataset to a self-hosted MinIO deployment using access keys and a scoped read policy. Designed for on-premise and private-cloud storage.

<WorkflowStages current="prepare-your-data" />

MinIO is a self-hosted, S3-compatible object store that many teams run on-premise or inside a private cloud. Datature Vi connects to MinIO the same way it connects to a cloud bucket, using an access key, a secret key, and an HTTPS endpoint. The integration is read-only, so the platform reads object metadata in place without copying files into Vi infrastructure.

<ViCallout type="info" title="Before You Start">
  * A paid Datature Vi account. External bucket sync is not available on the free tier.
  * A running MinIO deployment reachable from the public internet over HTTPS, with a valid TLS certificate.
  * A bucket with the assets you want to sync.
  * Permission to create users, access keys, and policies in MinIO.
</ViCallout>

<StepCarousel
  steps={[
    {
      title: "Open the Explorer tab",
      image: "https://files.readme.io/25d607264bda9f61746d57d93eeeb709416b0c73aceebdc2e912cb4f67a0b6ce-Explorer-05-12-2026_03_03_PM.png",
      description: "In the left sidebar, click the Explorer tab on your dataset. This is where the synced assets will appear after the connection is set up."
    },
    {
      title: "Click Connect to External Buckets",
      image: "https://files.readme.io/ffc4e071a1359efcd4143bbec3d878436c5c2cf554459eaaabfdf0bf60467642-Explorer-05-12-2026_04_43_PM_1.png",
      description: "Click the Connect to External Buckets button in the top-right of the Explorer, then choose MinIO from the storage options."
    },
    {
      title: "Bucket Details",
      image: "https://files.readme.io/5b1b9146157b3f3a93ee3fe1a5485f97c8ca7c9673914c4c394b0651797f2df2-Explorer-05-12-2026_05_10_PM.png",
      description: "Enter the connection name, bucket name, and HTTPS endpoint. Open Advanced Options to set a folder prefix or force path-style URLs."
    },
    {
      title: "Access Credentials",
      image: "https://files.readme.io/ba5f838ba275e02661fddf7b0a1f59ad69506cf8372a8b01a72fdb310da3e0a3-Explorer-05-12-2026_05_10_PM_1.png",
      description: "Paste the access key ID and secret access key for the MinIO user you created in Step 2. Both values are encrypted at rest inside Vi."
    },
        {
      title: "Verify Connection Status",
      image: "https://files.readme.io/6f0ca4fc649eea6262fe528085b859c2043a88941e98b8b1db561aa408d8d026-Explorer-05-12-2026_03_06_PM.png",
      description: "Vi tests the connection by listing one object in the bucket. A green status means the credentials and endpoint are correct. A red status surfaces the specific failure reason. You can choose to sync assets now or later."
    },
    {
      title: "Preview Files to Sync",
      image: "https://files.readme.io/28fb2d1b2c33f74e019a39e899f3d9b3f5cbb58fc1ece80409e2fcd57931fbeb-Explorer-05-12-2026_03_06_PM_1.png",
      description: "If you choose to sync now, Vi scans the bucket prefix and lists the objects it found. Confirm the file count and sample paths look right, then click Sync to start ingestion."
    },
    {
      title: "Preview Files to Sync",
      image: "https://files.readme.io/28fb2d1b2c33f74e019a39e899f3d9b3f5cbb58fc1ece80409e2fcd57931fbeb-Explorer-05-12-2026_03_06_PM_1.png",
      description: "Vi scans the bucket prefix and lists the objects it found. Confirm the file count and sample paths look right, then click Sync to start ingestion."
    },
    {
      title: "Sync Started",
      image: "https://files.readme.io/5d431094f67ec474ff3b21a88a8817f3f686311c3bd0764ca478b63a7788748f-Explorer-05-12-2026_03_06_PM_2.png",
      description: "Vi confirms the sync is running in the background. Click I Understand to close the wizard; the sync continues without an open browser tab."
    },
    {
      title: "Track Sync Progress",
      image: "https://files.readme.io/6dc88ababa705ae129a33267f098ecbf50a038a33691ef1c09173552a9cf5085-Explorer-05-12-2026_03_06_PM_3.png",
      description: "Open the Connected Bucket dropdown in the top-right of the Explorer to see the connection status, bucket, prefix, asset count, and a progress bar while assets are retrieved."
    }
  ]}
/>

<UIResultPreview variant="verify" screenshot="https://files.readme.io/5b03c3880ab1ba83f05effb270032011269b2bd890949543ce90f0c369836f0d-Explorer-05-12-2026_04_37_PM_1.png" caption="Synced images appear in the dataset Explorer. The asset count in the header reflects the MinIO objects that passed the format checks." />

## Step 1: Create a scoped read policy

The default `readonly` policy in MinIO does not include `s3:ListBucket`, which Vi needs to enumerate objects. Create a policy that grants the three permissions Vi requires, scoped to a single bucket.

<CodeGroup>
  ```json vi-read-policy.json
  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "s3:GetBucketLocation",
                  "s3:GetObject",
                  "s3:ListBucket"
              ],
              "Resource": [
                  "arn:aws:s3:::your-bucket-name",
                  "arn:aws:s3:::your-bucket-name/*"
              ]
          }
      ]
  }
  ```
</CodeGroup>

Replace `your-bucket-name` with the bucket you plan to sync. Apply the policy in the MinIO console or with `mc`:

<Terminal title="Apply the read policy">
  {`mc admin policy create myminio vi-read-policy vi-read-policy.json`}
</Terminal>

## Step 2: Create an access key for the integration

Create a dedicated MinIO user for the Vi integration so you can rotate or revoke its credentials without touching other services.

<Steps>
  <Step stepNumber="1" title="Create the user">
    In the MinIO console, open **Identity** > **Users**, click **Create User**, and choose a username such as `vi-sync`. Generate a strong secret key.
  </Step>

  <Step stepNumber="2" title="Attach the policy">
    On the user's profile, attach the `vi-read-policy` policy you created in Step 1. Do not attach the default admin policies.
  </Step>

  <Step stepNumber="3" title="Generate an access key">
    Open **Access Keys** for the user and create a new key pair. Copy both the access key ID and the secret key now; the secret is not shown again.
  </Step>
</Steps>

<ViCallout type="warning" title="Security checklist">
  * Grant only the three actions in the policy above. Wider permissions break the principle of least privilege.
  * Use a dedicated access key for the Vi integration. Sharing keys with other services makes audit and rotation hard.
  * Rotate the access key every 90 days as a baseline.
  * Avoid using MinIO root credentials for sync.
</ViCallout>

## Step 3: Enter the connection details in Vi

Open your dataset, then walk through the wizard.

The Bucket Details tab asks for these fields:

<ParamTable title="MinIO connection details">
  <Param name="MinIO Connection Name" type="string" required={true} description="A label you choose for this connection. Used to identify the connection in the Connection Manager." />

  <Param name="MinIO Bucket Name" type="string" required={true} description="The name of the MinIO bucket you set in the read policy." />

  <Param name="MinIO Connection Endpoint" type="string" required={true} description="The HTTPS URL of your MinIO server, for example https://minio.your-domain.com. Must use HTTPS with a valid TLS certificate." />

  <Param name="Folder Prefix" type="string" required={false} description="Optional path prefix to scope the sync to a subset of the bucket." />

  <Param name="Path Style" type="boolean" required={false} description="Toggle on if your MinIO deployment uses path-style URLs (https://endpoint/bucket/object) rather than virtual-hosted-style (https://bucket.endpoint/object). Most MinIO deployments use path style." defaultValue="false" />
</ParamTable>

Click **Next** to advance to the credentials step.

## Step 4: Enter the access credentials

Paste the access key ID and the secret access key from Step 2.

<ParamTable title="Access credentials">
  <Param name="Access Key ID" type="string" required={true} description="The MinIO access key for the user you created. Treated as the username during S3 signature verification." />

  <Param name="Access Secret Key" type="string" required={true} description="The MinIO secret key paired with the access key ID. Encrypted at rest in Vi infrastructure." />
</ParamTable>

Click **Next**. Vi tries the connection and reports success or failure on the **Connection Status** screen.

<ViCallout type="info" title="Need to whitelist Vi's egress IP?">
  If your MinIO deployment is behind an IP allowlist, contact `support@datature.io` to request the current set of egress IPs. We rotate them periodically, so plan to refresh the allowlist when notified.
</ViCallout>

## Step 5: Sync your assets

Choose **Sync Now** to start the first sync immediately, or **Sync Later** to set up the connection without syncing. If you pick **Sync Now**, Vi walks through three more screens before the sync starts in earnest.

1. **Preview Files to Sync.** Vi scans the bucket prefix and shows the file count alongside a sample of object paths. Confirm the preview matches what you expect, then click **Sync**.
2. **Sync Started.** A confirmation appears letting you know the job is running in the background. Click **I Understand** to dismiss the dialog; the sync continues even after you close the wizard or the browser tab.
3. **Track progress.** Open the **Connected Bucket** dropdown in the top-right of the Explorer to see the connection name, status, provider, bucket, prefix, asset count, and a live progress bar while assets are retrieved.

The first sync takes 5 to 40 minutes depending on the bucket size. Progress is also visible in the **Connection Manager** tab.

## Asset requirements

MinIO accepts a wider set of MP4 major brands than the cloud providers, because the metadata sync uses the broader S3-compatible video pipeline.

<InfoTable title="MinIO asset requirements" headers={['Asset type', 'Requirement']}>
  <InfoRow cells={['Images', 'No EXIF orientation tag, or an orientation value of 1.']} />

  <InfoRow cells={['MP4 videos', 'Major brand from {`isom`, `iso2`, `mp41`, `mp42`} and pixel format `yuv420p`. Frame count must align with the declared frame rate. Sample aspect ratio of 1:1 (or equivalent).']} />

  <InfoRow cells={['Other formats', 'See [Upload Images](https://vi.developers.datature.com/docs/upload-images) and [Upload Videos](https://vi.developers.datature.com/docs/upload-videos) for the full supported list.']} />
</InfoTable>

Use `ffprobe your-video.mp4` on Linux or macOS to read the major brand and pixel format before troubleshooting a video that fails to sync.

Annotations are not part of the bucket sync. Vi reads only image and video metadata from MinIO. If you have existing labels in COCO, YOLO, Pascal VOC, CSV, or Vi JSONL, [upload them directly to Vi](https://vi.developers.datature.com/docs/upload-annotations) once the assets finish syncing.

## Multiple buckets, one dataset

You can connect more than one MinIO bucket to the same dataset. Vi merges objects from every connection into a single asset list. If two buckets contain a file with the same name, the latest sync overwrites the earlier reference. Use folder prefixes to keep collections separate, or rename files at the source.

## Troubleshooting

<ViAccordionGroup>
  <ViAccordionItem title="Connection fails with a TLS or certificate error">
    Vi only connects to MinIO over HTTPS with a publicly trusted certificate. Self-signed certificates and certificates issued by an internal CA are rejected. Issue a certificate from a public authority (such as Let's Encrypt) for the MinIO endpoint.
  </ViAccordionItem>

  <ViAccordionItem title="Connection fails with an access denied error">
    Two things to check. First, the access key has the `vi-read-policy` attached and no conflicting deny rule. Second, the policy lists the bucket you are syncing in the `Resource` ARNs. Replace `your-bucket-name` in the JSON before applying.
  </ViAccordionItem>

  <ViAccordionItem title="Connection fails with a bucket not found error">
    Try toggling **Path Style** on the Bucket Details step. Many MinIO deployments require path-style URLs because the bucket is not addressable as a subdomain.
  </ViAccordionItem>

  <ViAccordionItem title="Sync finished but some assets are missing">
    Check your remaining data row quota in [Billing](https://vi.developers.datature.com/docs/billing). Files that fail the format requirements above are skipped during sync.
  </ViAccordionItem>

  <ViAccordionItem title="Previews load but the image is rotated">
    The image has an EXIF orientation tag other than 1. You have two options.

    **Option 1: Bake the orientation into the pixels with ImageMagick.** This rotates the image data and resets the orientation tag to 1.

    <Terminal title="Auto-orient with ImageMagick">
      {`mogrify -auto-orient your-image.jpg`}
    </Terminal>

    **Option 2: Strip the orientation tag with exiftool.** Use this when the pixels are already correct and only the tag is wrong.

    <Terminal title="Remove the EXIF orientation tag">
      {`exiftool -Orientation= -overwrite_original your-image.jpg`}
    </Terminal>

    To process a whole folder, point either tool at the directory:

    <Terminal title="Batch fix every image in a folder">
      {`mogrify -auto-orient ./images/*.jpg
        exiftool -Orientation= -overwrite_original -r ./images`}
    </Terminal>

    Re-upload the fixed files to the bucket and run the sync again.
  </ViAccordionItem>

  <ViAccordionItem title="Multiple buckets, files overwriting each other">
    Vi keys assets by filename inside the dataset. If two buckets hold an `image_001.jpg`, the second sync replaces the first. Add a unique prefix in each bucket or rename the files before syncing.
  </ViAccordionItem>
</ViAccordionGroup>

## Next steps

<BentoGrid className="grid grid-cols-1 md:grid-cols-3 gap-4 my-6">
  <BentoCard title="Sync From S3-Compatible Storage" description="Use the same flow for Wasabi, Backblaze B2, Cloudflare R2, and other S3 services." href="doc:sync-from-s3-compatible" faIcon="fa-solid fa-cube" />

  <BentoCard title="Annotate Data" description="Label the synced images and videos in the visual annotator." href="doc:annotate-data" faIcon="fa-solid fa-pen-to-square" />

  <BentoCard title="Train A Model" description="Fine-tune a vision-language model on the synced dataset." href="doc:train-a-model" faIcon="fa-solid fa-brain" />
</BentoGrid>