---
updatedAt: 2026-04-12T19:02:59.000Z
agentTools:
  projectIndex: https://vi.developers.datature.com/llms.txt
---

# Prediction Result Schemas

The [Datature Vi](https://vi.developers.datature.com/docs/vi-sdk) SDK returns one of three response types depending on the [task type](https://vi.developers.datature.com/docs/task-types): `VQAResponse`, `PhraseGroundingResponse`, or `GenericResponse`. All three inherit from a base `PredictionResponse` class that provides common fields like `prompt`, `raw_output`, and `thinking`.

Use `isinstance()` to check which type you received before accessing task-specific fields.

## Base response fields

All response types share these fields from `PredictionResponse`:

<ParamTable title="Base response fields">
  <Param name="prompt" type="str" required={true} description="The user prompt used for this prediction" />
  <Param name="raw_output" type="str | None" required={false} description="Raw model output before parsing. Includes `<think>` and `<answer>` tags when you pass `cot=True` on `model(...)` or the model emits that structure" />
  <Param name="thinking" type="str | None" required={false} description="Extracted content from `<think>...</think>` tags when present (typical with `cot=True` at inference)" />
</ParamTable>

<CodeGroup>

```python Access base fields
from vi.inference import ViModel

model = ViModel(run_id="your-run-id")
result, error = model(source="image.jpg", user_prompt="Describe this image")

if error is None:
    print(f"Prompt: {result.prompt}")
    print(f"Raw output length: {len(result.raw_output) if result.raw_output else 0}")

    if result.thinking:
        print(f"Model reasoning: {result.thinking}")
```

</CodeGroup>

## VQA response

Returned by [Visual Question Answering](https://vi.developers.datature.com/docs/visual-question-answering) inference. Contains a text answer to the user's question.

### Schema

<CodeGroup>

```python VQA schema
class VQAResponse(PredictionResponse):
    result: VQAAnswer

class VQAAnswer:
    answer: str  # minimum 1 character
```

</CodeGroup>

### Fields

<ParamTable title="VQA response fields">
  <Param name="result" type="VQAAnswer" required={true} description="Container for the answer" />
  <Param name="result.answer" type="str" required={true} description="The answer text" />
</ParamTable>

### Example

<CodeGroup>

```python VQA result access
from vi.inference import ViModel

model = ViModel(run_id="your-run-id")

result, error = model(
    source="image.jpg",
    user_prompt="What color is the car in this image?"
)

if error is None:
    answer = result.result.answer
    print(f"Answer: {answer}")

    print(f"Question: {result.prompt}")
    if result.thinking:
        print(f"Reasoning: {result.thinking}")
```

</CodeGroup>

### Streaming mode

<CodeGroup>

```python VQA with streaming
stream = model(
    source="image.jpg",
    user_prompt="Describe this image in detail",
    stream=True
)

for token in stream:
    print(token, end="", flush=True)

result = stream.get_final_completion()
print(f"\n\nFinal answer: {result.result.answer}")
```

</CodeGroup>

## Phrase grounding response

Returned by [Phrase Grounding](https://vi.developers.datature.com/docs/phrase-grounding) inference. Contains a caption sentence and a list of detected objects with bounding boxes.

### Schema

<CodeGroup>

```python Phrase grounding schema
class PhraseGroundingResponse(PredictionResponse):
    result: PhraseGrounding

class PhraseGrounding:
    sentence: str                    # minimum 1 character
    groundings: list[GroundedPhrase] # minimum 1

class GroundedPhrase:
    phrase: str                      # text label, minimum 1 character
    grounding: list[list[int]]       # each box: [xmin, ymin, xmax, ymax]
```

</CodeGroup>

### Fields

<ParamTable title="Phrase grounding fields">
  <Param name="result" type="PhraseGrounding" required={true} description="Container for phrase grounding results" />
  <Param name="result.sentence" type="str" required={true} description="Full caption or description text" />
  <Param name="result.groundings" type="list[GroundedPhrase]" required={true} description="Detected objects with bounding boxes" />
  <Param name="result.groundings[i].phrase" type="str" required={true} description="Text label for detected object `i`" />
  <Param name="result.groundings[i].grounding" type="list[list[int]]" required={true} description="Bounding boxes for object `i`" />
</ParamTable>

### Bounding box format

<ViCallout type="info">

Bounding boxes use normalized coordinates in the range `[0, 1024]`:

* Format: `[x_min, y_min, x_max, y_max]`
* Top-left corner: `(0, 0)`
* Bottom-right corner: `(1024, 1024)`
* Independent of actual image dimensions

To convert to pixel coordinates, scale each value by `image_width / 1024` (x) or `image_height / 1024` (y).

</ViCallout>

<CodeGroup>

```python Convert bbox to pixels
from PIL import Image

image = Image.open("image.jpg")
width, height = image.size

x_min, y_min, x_max, y_max = bbox  # [0-1024] range
pixel_x_min = int(x_min / 1024 * width)
pixel_y_min = int(y_min / 1024 * height)
pixel_x_max = int(x_max / 1024 * width)
pixel_y_max = int(y_max / 1024 * height)
```

</CodeGroup>

Use the [built-in `visualize_prediction()` utility](https://vi.developers.datature.com/docs/handle-results#built-in-visualization) for automatic coordinate conversion and rendering.

### Example

<CodeGroup>

```python Phrase grounding result access
from vi.inference import ViModel

model = ViModel(run_id="your-run-id")

result, error = model(
    source="image.jpg",
    user_prompt="Describe the objects in this image"
)

if error is None:
    caption = result.result.sentence
    print(f"Caption: {caption}")

    for grounded_phrase in result.result.groundings:
        phrase = grounded_phrase.phrase
        bboxes = grounded_phrase.grounding

        print(f"\nObject: {phrase}")
        print(f"  Bounding boxes: {len(bboxes)}")
        for i, bbox in enumerate(bboxes):
            x_min, y_min, x_max, y_max = bbox
            print(f"  Box {i+1}: [{x_min}, {y_min}, {x_max}, {y_max}]")
```

</CodeGroup>

### Pixel coordinate conversion

<CodeGroup>

```python Full pixel conversion example
from PIL import Image

def convert_bbox_to_pixels(bbox, image_path):
    """Convert normalized bbox [0-1024] to pixel coordinates."""
    image = Image.open(image_path)
    width, height = image.size
    x_min, y_min, x_max, y_max = bbox

    return [
        int(x_min / 1024 * width),
        int(y_min / 1024 * height),
        int(x_max / 1024 * width),
        int(y_max / 1024 * height)
    ]

result, error = model(source="image.jpg")

if error is None:
    for grounded_phrase in result.result.groundings:
        print(f"Object: {grounded_phrase.phrase}")
        for bbox in grounded_phrase.grounding:
            pixel_bbox = convert_bbox_to_pixels(bbox, "image.jpg")
            print(f"  Pixel coordinates: {pixel_bbox}")
```

</CodeGroup>

### Filter groundings by object type

<CodeGroup>

```python Filter by object type
result, error = model(source="image.jpg")

if error is None:
    people = [
        g for g in result.result.groundings
        if "person" in g.phrase.lower()
    ]
    vehicles = [
        g for g in result.result.groundings
        if any(v in g.phrase.lower() for v in ["car", "truck", "vehicle"])
    ]

    print(f"Found {len(people)} people and {len(vehicles)} vehicles")
```

</CodeGroup>

## Generic response

Returned as a fallback when:

* The model output cannot be parsed into VQA or phrase grounding format
* JSON parsing fails
* Task type is explicitly set to `GENERIC`

Contains raw text output without structured parsing.

### Schema

<CodeGroup>

```python Generic response schema
class GenericResponse(PredictionResponse):
    result: str  # complete raw output text
```

</CodeGroup>

### Fields

<ParamTable title="Generic response fields">
  <Param name="result" type="str" required={true} description="Complete raw output text from the model" />
</ParamTable>

### Example

<CodeGroup>

```python Generic response handling
from vi.inference import ViModel

model = ViModel(run_id="your-run-id")

result, error = model(source="image.jpg", user_prompt="Analyze this image")

if error is None:
    output = result.result
    print(f"Model output: {output}")

    if result.raw_output != result.result:
        print("Note: Structured parsing failed; using raw output")
```

</CodeGroup>

<ViCallout type="warning" title="When Generic Responses Occur">

You receive a `GenericResponse` when:

* The model's JSON output is malformed or incomplete
* The output does not match the expected schema
* Task type is explicitly set to `GENERIC`

To debug: check `result.raw_output` for the full model output. Verify the model is trained for the task type you expect, and try adjusting [generation config](https://vi.developers.datature.com/docs/configure-generation) parameters.

</ViCallout>

## Type checking and handling

### Checking response type

<CodeGroup>

```python Check with isinstance
from vi.inference.task_types import GenericResponse
from vi.inference.task_types.vqa import VQAResponse
from vi.inference.task_types.phrase_grounding import PhraseGroundingResponse

result, error = model(source="image.jpg")

if error is None:
    if isinstance(result, VQAResponse):
        print(f"VQA Answer: {result.result.answer}")

    elif isinstance(result, PhraseGroundingResponse):
        print(f"Caption: {result.result.sentence}")
        print(f"Objects: {len(result.result.groundings)}")

    elif isinstance(result, GenericResponse):
        print(f"Raw output: {result.result}")
        print("Warning: structured parsing may have failed")
```

</CodeGroup>

### Safe attribute access

<CodeGroup>

```python Safe field extraction
def extract_text(result):
    """Extract text from any response type."""
    if hasattr(result, "result"):
        if hasattr(result.result, "answer"):
            return result.result.answer
        if hasattr(result.result, "sentence"):
            return result.result.sentence
        if isinstance(result.result, str):
            return result.result
    return None

result, error = model(source="image.jpg")
if error is None:
    text = extract_text(result)
    print(f"Extracted text: {text}")
```

</CodeGroup>

## Batch inference schemas

Batch inference returns a list of `(result, error)` tuples, one per image:

<CodeGroup>

```python Process batch results
from vi.inference.task_types.vqa import VQAResponse

images = ["img1.jpg", "img2.jpg", "img3.jpg"]
results = model(source=images, user_prompt="What's in this image?")

for i, (result, error) in enumerate(results):
    if error is None:
        if isinstance(result, VQAResponse):
            print(f"Image {i+1}: {result.result.answer}")
    else:
        print(f"Image {i+1} failed: {error}")
```

</CodeGroup>

<CodeGroup>

```python Separate successful and failed
successful = []
failed = []

for img, (result, error) in zip(images, results):
    if error is None:
        successful.append((img, result))
    else:
        failed.append((img, error))

print(f"Successful: {len(successful)}/{len(images)}")

for img_path, result in successful:
    if isinstance(result, VQAResponse):
        print(f"{img_path}: {result.result.answer}")
```

</CodeGroup>

## Advanced usage

### Accessing raw output for debugging

<CodeGroup>

```python Inspect raw output
result, error = model(source="image.jpg")

if error is None:
    print("=== Raw Model Output ===")
    print(result.raw_output)

    if result.thinking:
        print("\n=== Model Reasoning ===")
        print(result.thinking)

    if isinstance(result, VQAResponse):
        print("\n=== Parsed Answer ===")
        print(result.result.answer)
```

</CodeGroup>

### Chain-of-thought (CoT) responses

At inference time, pass **`cot=True`** to `model(...)` to request chain-of-thought decoding. The model emits `<think>` and `<answer>` segments; the SDK exposes reasoning in `result.thinking` (when present) alongside the parsed task result.

<CodeGroup>

```python Chain-of-thought
from vi.inference import ViModel

model = ViModel(run_id="your-run-id")

result, error = model(
    source="image.jpg",
    user_prompt="Count the number of cars",
    cot=True,
    stream=False,
)

if error is None:
    if result.thinking:
        print("Model reasoning:")
        print(result.thinking)

    print("\nFinal answer:")
    print(result.result.answer)
```

</CodeGroup>

### Export schema to JSON

<CodeGroup>

```python Serialize result to JSON
import json
from vi.inference.task_types.vqa import VQAResponse
from vi.inference.task_types.phrase_grounding import PhraseGroundingResponse

result, error = model(source="image.jpg")

if error is None:
    if isinstance(result, VQAResponse):
        output = {
            "type": "vqa",
            "prompt": result.prompt,
            "answer": result.result.answer,
            "thinking": result.thinking
        }
    elif isinstance(result, PhraseGroundingResponse):
        output = {
            "type": "phrase_grounding",
            "prompt": result.prompt,
            "sentence": result.result.sentence,
            "objects": [
                {"phrase": g.phrase, "bounding_boxes": g.grounding}
                for g in result.result.groundings
            ],
            "thinking": result.thinking
        }

    with open("result.json", "w") as f:
        json.dump(output, f, indent=2)
```

</CodeGroup>

## Related resources

<BentoGrid className="grid grid-cols-1 md:grid-cols-3 gap-6 my-10">
  <BentoCard
    title="Configure Generation"
    description="Control output length, temperature, and sampling with generation_config."
    href="http://vi.developers.datature.com/docs/configure-generation"
    faIcon="fa-solid fa-sliders"
  />
  <BentoCard
    title="Task Types"
    description="When to use VQA vs phrase grounding, prompt guidelines, and response differences."
    href="http://vi.developers.datature.com/docs/task-types"
    faIcon="fa-solid fa-tags"
  />
  <BentoCard
    title="Handle Results"
    description="Visualize predictions, convert bounding boxes, and export to JSON or CSV."
    href="http://vi.developers.datature.com/docs/handle-results"
    faIcon="fa-solid fa-chart-bar"
  />
</BentoGrid>