Guides
Vi Dashboard
  1. Concepts
  2. Dataset Types

Freeform Text

Learn about freeform text datasets for custom annotation schemas and specialized vision AI applications in Datature Vi.

Freeform text is a dataset type in Datature Vi that lets you define custom annotation schemas tailored to your specific use case. Unlike phrase grounding or visual question answering, which have predefined structures, freeform text datasets give you full control over annotation format for specialized vision AI applications.

New to Datature Vi?

Datature Vi lets you train a custom VLM for freeform text tasks on your own images. Learn what Datature Vi does or follow the quickstart.

By the end of this guide

Understand how freeform text datasets let you define custom annotation schemas for specialized use cases not covered by phrase grounding or VQA.

What is freeform text?

Freeform text datasets let you create annotation formats that match your exact requirements. Rather than conforming to predefined structures like bounding boxes with labels or question-answer pairs, you define:

  • Custom data structures: Design annotation formats specific to your domain
  • Flexible schemas: Mix and match different annotation types
  • Novel vision tasks: Experiment with new computer vision approaches
  • Specialized workflows: Build annotation pipelines for unique use cases
Best for
  • Research projects exploring novel computer vision tasks
  • Specialized industries with domain-specific annotation requirements
  • Use cases that combine multiple annotation types
  • Scenarios where neither phrase grounding nor VQA fits
Not for
  • Object localization with natural language (use phrase grounding instead)
  • Question-answering about images (use visual question answering instead)
  • Projects with standard annotation requirements

Freeform text vs. other dataset types

Dataset Type
Best For
Structure
Flexibility
Phrase Grounding
Object localization with natural language
Captions + bounding boxes
Structured format
Visual Question Answering
Question-answering about images
Question-answer pairs
Structured format
Freeform Text
Custom annotation requirements
User-defined
Fully customizable

Common use cases

Research

Novel CV approaches, experimental annotation strategies, multi-modal learning, and publishing new benchmarks.

Specialized domains

Medical diagnostic annotations, microscopy analysis, astronomical data, manufacturing defect classification with custom quality metrics.

Advanced applications

Multi-task learning, hierarchical annotation relationships, and custom pipelines with domain-specific validation.

Best practices

Area
Recommendation
Schema design
Start minimal, add complexity only when needed. Version your format for backward compatibility.
Data quality
Define validation rules upfront. Document your schema with examples for annotators.
Integration
Design with training metrics in mind. Use standard data types where possible for easier export.

How freeform text works in Datature Vi

The freeform text workflow: create a freeform text dataset, upload images, annotate with your custom text schema, and train.

from vi.inference import ViModel

model = ViModel(
    run_id="your-run-id",
    secret_key=".your-secret-key.",
    organization_id="your-organization-id",
)
result, error = model(source="document.jpg", user_prompt="Extract the invoice details from this image", stream=False)
print(result.result)

For full inference options, see the Vi SDK inference docs.

You can include step-by-step reasoning in freeform text annotations by prepending <datature_think> tags to the annotation text. During training, Datature Vi converts these to the model's native <think> tags.

See Chain-of-Thought Reasoning and Annotation Guide for details.

{
  "finding": "2.3cm nodule in right upper lobe",
  "location": "right upper lobe, posterior segment",
  "severity": "moderate",
  "recommendation": "Follow-up CT in 3 months"
}
Component: Bearing assembly #4
Status: FAIL
Defects found: surface pitting, minor corrosion
Severity: High
Action: Replace before next production cycle
A busy warehouse aisle with metal shelving on both sides. The left shelf holds
stacked cardboard boxes, three of which show water damage along the bottom edge.
A yellow forklift is parked at the far end, partially blocking the aisle.

Related resources

Structured Data Extraction

Train your model to return JSON, YAML, code, or any custom structured format.

Annotate For Freeform Text

Step-by-step guide to creating freeform text annotations in Datature Vi.

Chain-Of-Thought Reasoning

Add step-by-step reasoning to freeform text annotations.

Phrase Grounding

Object localization with natural language descriptions.

Visual Question Answering

Answer questions about images with natural language.

Contact Us

Discuss your custom annotation requirements with the Datature team.

Updated about 1 month ago