Medical Imaging

For an interactive overview of this application, visit the medical imaging use case on vi.datature.com.

What medical imaging AI does

Radiologists and pathologists review hundreds of images a day. Each image needs careful attention, and fatigue leads to missed findings. AI-assisted imaging gives clinicians a second set of eyes: the model scans each image first and highlights areas that may need closer review.

Datature Vi trains custom models on your institution's own imaging data. You label a set of images with findings, train a model, and then use it to pre-screen new images or draft structured reports. The clinician stays in control of every diagnosis. The model handles the repetitive scanning so the clinician can spend more time on the cases that matter most.

This is not a replacement for clinical expertise. It is a tool that helps your team work faster and catch more.

Common applications

Anomaly detection and flagging

Train a model to identify and describe potential anomalies in radiology or clinical images for radiologist review.

Task type: VQA

Use Visual Question Answering with clinical questions:

Task type: Phrase Grounding

Use Phrase Grounding to draw bounding boxes around areas of interest. This is useful when the downstream system needs a localized region for cropping, measurement, or viewer overlay:

• Annotate with boxes and label text: "area of consolidation", "pleural effusion", "nodule"
• At inference, the model returns normalized bounding box coordinates

Structured findings extraction

Extract structured fields from imaging studies for integration with PACS, EHR, or research databases.

Setup

Use a freeform text dataset with JSON annotations and a system prompt specifying the schema. See Structured Data Extraction for full setup.

Example annotation schema:

{
  "finding": "pulmonary nodule",
  "location": "right upper lobe, posterior segment",
  "size_cm": "1.2",
  "severity": "indeterminate",
  "recommendation": "Follow-up CT in 3 months"
}

System prompt:

You are a radiology assistant. Analyze the chest X-ray and return a JSON object with these fields:
- finding (string): the primary finding, or "no significant finding" if normal
- location (string or null): anatomical location, null if no finding
- size_cm (string or null): approximate size in centimeters, null if not measurable
- severity (string): one of "normal", "mild", "moderate", "severe", "indeterminate"
- recommendation (string): suggested follow-up action

Respond with only the JSON object.

Vi SDK parsing:

import json
from vi.inference import ViModel

model = ViModel(
    run_id="your-run-id",
    secret_key=".your-secret-key.",
    organization_id="your-organization-id",
)

result, error = model(
    source="chest_xray.jpg",
    user_prompt="Analyze this chest X-ray.",
    generation_config={"temperature": 0.0, "do_sample": False}
)

if error is None:
    report = json.loads(result.result)
    print(f"Finding: {report['finding']}")
    print(f"Location: {report['location']}")
    print(f"Severity: {report['severity']}")
    print(f"Recommendation: {report['recommendation']}")

Research dataset annotation

For research teams managing large collections of clinical images, Vi can accelerate annotation by generating initial labels that human annotators then review and correct.

Workflow

1. Train a base model on a small set of manually annotated images (100–200)
2. Run inference on your unannotated collection to generate candidate labels
3. Human review: annotators verify and correct the model's outputs (faster than labeling from scratch)
4. Retrain with the verified set to improve accuracy
5. Repeat: each iteration expands your verified dataset with less annotator effort

This approach is particularly useful for:

• Large pathology slide collections
• Multi-site clinical image repositories
• Rare condition datasets where expert annotation time is limited

Training tips for medical imaging

Use domain-appropriate terminology: annotations and system prompts should use clinical language consistently. Avoid mixing lay terms ("abnormal") with clinical terms ("consolidation") across your dataset.

Include normal examples: train on both abnormal and normal images. If your dataset is only abnormal cases, the model will over-predict findings.

Annotate ambiguous cases explicitly: where expert consensus is unclear, use a consistent convention (e.g., label as "indeterminate") rather than guessing. Inconsistent labeling on ambiguous cases degrades model performance.

Validate on a held-out set: before using a model in any workflow, test it on a set of images not used in training. Measure performance against your clinical acceptance criteria.

Start with coarse tasks: binary classification ("normal vs. abnormal") is easier to train than fine-grained severity grading. Build confidence in the coarser task before adding complexity.

Next steps