# inference.minimax.h3.chili.ref2vid.v0

Source: https://docs.prodia.com/job-types/inference-minimax-h3-chili-ref2vid-v0/

The `inference.minimax.h3.chili.ref2vid.v0` job generates a 24 fps video
with synchronized stereo audio using uploaded references. Images guide
subjects and style, videos guide motion, and audio guides voices and music.

Upload the references as job inputs and list their filenames in
`references`, in the order you want them used. If `references` is omitted,
all uploaded inputs are used in upload order. Include at least one image
or video; audio cannot be used on its own.

```json
{
    "type": "inference.minimax.h3.chili.ref2vid.v0",
    "config": {
        "prompt": "the subject walks through a sunlit park, birds chirping",
        "references": ["face.jpg", "profile.jpg"],
        "duration": 6
    }
}
```

## Prompting

The prompt is used exactly as written: nothing rewrites or expands it before
it reaches the model, which was trained on descriptive captions. Describe
the shot rather than making a request: the subject, setting, time of day,
weather, lighting, camera move and motion. For example, "show me what
christmas would be like in Tokyo" tends to produce a generic winter scene,
while "Shibuya crossing in Tokyo on Christmas Eve at night, light snow
falling, crowds with umbrellas, a giant illuminated Christmas tree, neon
signs reflecting in the wet street, slow cinematic dolly shot" produces the
intended scene. The references already fix who or what appears, so use the
prompt to describe the setting, action, motion and camera move you want for
them. This job type runs a distilled 8-evaluation recipe, so it follows the
prompt less closely than full-step generation; vague prompts steer it even
less, so a detailed description matters more here.

You can upload up to 15 references, subject to these limits:

| Reference | Maximum count | Supported formats           | Maximum file size | Duration                               |
| --------- | ------------- | --------------------------- | ----------------- | -------------------------------------- |
| Image     | 9             | JPEG, PNG, WEBP, HEIC, HEIF | 30 MB each        | —                                      |
| Video     | 3             | MP4 or MOV with H.264/H.265 | 50 MB each        | 2–15 seconds each; 15 seconds combined |
| Audio     | 3             | WAV or MP3                  | 15 MB each        | 2–15 seconds each; 15 seconds combined |

Images and video frames must be 256–5760 pixels per side, with a
width-to-height ratio between 0.4 and 2.5. Video frame rates must be
23.976–60 fps. The combined duration limits apply separately to video
references and audio references.

## Structured prompt format

MiniMax trains H3 on a structured prompt, not on free text. Sending that
structure yourself is the most reliable way to control the soundtrack: on a
24-prompt bank rendered on this tier, 10 of 24 free-text prompts came back
with an unrequested music bed, and 0 of 24 did when the same prompts were
sent in this format (same seeds, same duration). The format adds three
things free text usually leaves to the model's own judgement: an explicit
soundscape, an explicit music decision, and an explicit statement of whether
anyone speaks.

Rules that apply to every section below:

- A line is spoken only when it is written as dialogue:
  `she (S1) says softly: <d>[English] Don't tell me you forgot again.</d>`.
  `(S1)` numbers the speaker, `<d>…</d>` encloses the spoken words, and the
  language tag is always `[English]` (the model voices other languages inside
  the tag as written). Put dialogue last in the shot: prose after a spoken
  line tends to be voiced too.
- When nobody speaks, say so: `She does not speak.`, `He does not speak.`
  or, for a scene with no single subject, `No spoken dialogue.` Without this
  sentence the model may invent speech.
- `overall_soundscape:` is one sentence of diegetic sound, i.e. what the
  scene itself produces. For a person: `Soft room tone, her quiet breathing
  and the faint rustle of fabric as she moves.` For a scene: `Natural ambient
  sound matching the scene and its visible actions.`
- `non_diegetic_music:` is `N/A` unless you want a score, in which case
  describe it in one sentence, e.g. `Calm ambient piano plays throughout.`
- Separate sections with a blank line. The whole prompt, format included,
  must stay within the 7000-character limit.
- Write the visual description exactly as the Prompting section above
  recommends; the format wraps it, it does not replace it.

Beyond the soundtrack, the same format carries the rest of what the model
was trained to read:

- Write the description as a timeline, not a caption: what is visible and
  audible first, then the actions, reactions and state changes in the order
  they happen. Every detail should be something that can be seen or heard;
  requests like "make it cinematic" steer little.
- Give camera movement as type, amplitude and speed: "the camera pushes in
  with small amplitude at slow speed", "trucks right slowly", "holds a
  locked-off shot".
- For more than one shot, keep `[Shot 1]` without a timestamp and start each
  later shot with a strictly increasing cut time inside the duration:
  `[Shot 2] At 00:05.000, the camera cuts to a close-up of steam rising from
  the sliced bread.`
- Keep a speaker's id across shots. For narration that is not spoken on
  screen, use the exact phrase "says in an off-screen voiceover" and add
  that the on-screen character's lips remain closed. When a line crosses a
  cut, mark both parts with `<scenetrans>` and say the audio continues
  across the cut; when the clip ends mid-sentence, mark the line with
  `<cutoff>`.
- When you do want a score, describe it as instrumentation, tempo and
  dynamics ("sparse piano at a slow tempo, joined by low strings that build
  and fade") rather than mood words.
- Text that must appear on screen is spelled out verbatim in the prompt.
- An unspecified clip gets natural ambience; write `overall_soundscape: N/A`
  only when you want silence.

Reference-to-video uses the full six-section form. Uploaded images are
`<Picture 1>`, `<Picture 2>`, … and uploaded videos `<Video 1>`, `<Video 2>`,
… in upload order; the subject they define is `<Subject 1>`. Every uploaded
reference should be named in `subject_definitions:` and `retention_analysis:`.
This example has two images and one video:

```
subject_definitions:
<Subject 1> is the subject in <Video 1>, keeping its appearance and the setting, lighting and framing shown there.
<Video 1> is the reference video that provides the scene, framing and the motion; its audio is not reused.
<Picture 1> is an appearance reference for the subject described below.
<Picture 2> is an appearance reference for the subject described below.

summary:
[reference generation] The target video shows <Subject 1> as described below. The subject walks through a park and waves at the camera. No spoken dialogue.

retention_analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved - its identity and appearance are retained; everything else follows the description.
<Video 1>: reference - its scene, framing and motion guide the target video; the audio signal is not copied.

detailed_description:
The target video is a single continuous shot.
[Shot 1] <Subject 1>, from <Video 1>, the subject walks through a park and waves at the camera. No spoken dialogue.

overall_soundscape:
Natural ambient sound matching the scene and its visible actions.

non_diegetic_music:
N/A
```

With images only, drop the `<Video 1>` lines and describe `<Subject 1>` as
the subject shown in `<Picture 1>`. Uploaded audio is not named in the
prompt; it is used as the soundtrack reference directly.

The task type in `summary:` tells the model how the references are used:
`[reference generation]` when they guide appearance, motion or style;
`[video continuation]` when the clip should pick up where a reference video
ends; `[audio reuse]` when an uploaded track is copied, `[audio reference]`
when only its style, timbre or beat is imitated. Combine them with `+`
when more than one applies.

`prompt` is required and must contain 3–7000 characters. Describe both the
visual scene and the sounds you want to hear.

`duration` accepts whole seconds from 4 to 15 and defaults to 6. The actual
video may be slightly longer than requested: 4 seconds produces about
4.46 seconds, and 15 seconds produces about 15.08 seconds.

`resolution` supports only `768P`, which is also the default. Exact output
dimensions depend on the aspect ratio. `seed` accepts an integer from 0 to
2147483647 for reproducible generation; omit it to use a random seed.

`aspect_ratio` defaults to `adaptive`, which uses the supported ratio
closest to the first image or video reference. To choose the output shape
explicitly, set it to `16:9`, `4:3`, `1:1`, `3:4`, `9:16`, or `21:9`.

## Schema

```json
{
	"type": "object",
	"required": [
		"type",
		"config"
	],
	"additionalProperties": false,
	"properties": {
		"type": {
			"type": "string",
			"enum": [
				"inference.minimax.h3.base.ref2vid.v0",
				"inference.minimax.h3.fast.ref2vid.v0",
				"inference.minimax.h3.base.ref2vid.v1",
				"inference.minimax.h3.fast.ref2vid.v1",
				"inference.minimax.h3.turbo.ref2vid.v0",
				"inference.minimax.h3.hyper.ref2vid.v0",
				"inference.minimax.h3.chili.ref2vid.v0"
			]
		},
		"config": {
			"type": "object",
			"required": [
				"prompt"
			],
			"additionalProperties": false,
			"properties": {
				"references": {
					"type": "array",
					"minItems": 1,
					"maxItems": 15,
					"items": {
						"type": "string",
						"format": "filename",
						"example": "subject.png",
						"description": "Filename of a reference file."
					},
					"description": "Reference files to feature in the video, in the order the model should read them. The role of each reference is derived from its content type: images, videos, and audio act as subject/style, motion, and voice/music references respectively. Up to 9 images (JPEG, PNG, WEBP, HEIC, or HEIF; 30MB each; 256-5760px per side; width/height ratio 0.4-2.5), 3 videos (MP4 or MOV with H.264/H.265; 50MB each; 2-15s each and 15s combined; same pixel and ratio bounds; 23.976-60 fps), and 3 audio files (WAV or MP3; 15MB each; 2-15s each and 15s combined); audio never on its own. Optional: with the field absent, every uploaded input is a reference, in upload order."
				},
				"prompt": {
					"type": "string",
					"minLength": 3,
					"maxLength": 7000,
					"default": "a dancing cat under moonlight",
					"description": "Description of the desired video and audio."
				},
				"duration": {
					"type": "integer",
					"minimum": 4,
					"maximum": 15,
					"default": 6,
					"description": "Duration of the video in seconds. Rounded up to the model's frame grid (4 s returns 4.46 s, 15 s returns 15.08 s)."
				},
				"aspect_ratio": {
					"type": "string",
					"enum": [
						"adaptive",
						"16:9",
						"4:3",
						"1:1",
						"3:4",
						"9:16",
						"21:9"
					],
					"default": "adaptive",
					"description": "Controls the approximate aspect ratio of the resulting video. With \"adaptive\" (the default), the aspect ratio derives from the first visual reference (snapped to the nearest supported ratio); an explicit ratio overrides it."
				},
				"resolution": {
					"type": "string",
					"enum": [
						"768P"
					],
					"default": "768P",
					"description": "Only 768P is supported: the canvas targets a 768px short edge with a 768x1344 pixel area cap and 32px rounding; exact width and height follow the aspect ratio of the first visual reference."
				},
				"seed": {
					"type": "integer",
					"minimum": 0,
					"maximum": 2147483647,
					"description": "Seed for reproducible generation. Random when omitted."
				}
			}
		}
	}
}
```
