Skip to content

Datasets

A dataset is what gets sent to the provider. It has a type and comes from a source. The job form builds it once on save to report its size; drush ai-finetuner:dataset <id> lists the items.

Dataset types

Type Content Used for
image A list of image files LoRA and style training
text A list of records with id, title and text (plus entity_type and entity_id when they come from entities) Fine-tuning language models; plugins can export the records as JSONL

A plugin declares which types it accepts. The job form only offers those.

Sources

flowchart TD
    S{Source}
    S -->|upload| U[Image files on the job]
    S -->|entity_display| E[Entity query:<br>type, bundle, ids, limit]
    S -->|views| V[Execute the view display]
    U --> UI[image dataset]
    E -->|image| EI[Collect image, file and media<br>fields visible in the view mode]
    E -->|text| ET[Render each entity in the view mode,<br>convert HTML to text]
    V --> VS[Style plugin maps rows to<br>title, text, images]
    VS -->|image| VI[image dataset]
    VS -->|text| VT[text dataset]

Uploaded images

The simplest source: images attached to the job's Training images field. Every file becomes one dataset item. This source only produces image datasets.

Accepted extensions: png, jpg, jpeg, webp. Files are stored under ai-finetune/YYYY-MM.

Entities rendered through a view display

This source selects entities and lets an entity view mode decide which fields are training material.

Selection:

  • Entity type: any content entity type that can be rendered.
  • Bundle: a bundle machine name, or empty for all bundles of the type.
  • Entity IDs: an optional comma or newline separated list. When given, the bundle and limit are ignored and exactly these entities are loaded.
  • Maximum entities: the query limit, default 200. Entities are sorted by id descending, so the newest come first.

The bundle query runs with access checks for the current user, so entities they cannot see are left out. Under Drush that user is anonymous, which excludes unpublished content. An explicit Entity IDs list bypasses the query and loads exactly those entities.

Rendering:

  • View display (view mode): defaults to ai_finetuner, the AI fine-tuning dataset view mode the module ships for nodes and media. If the bundle has not customised that view mode, the default display is used instead. See Configuration for how to set it up.
  • For an image dataset the module walks the fields the display shows and collects images from every image, file and entity_reference field. File entities count when their MIME type starts with image/. Media entities contribute their source file, again only when it is an image. Other referenced entities are ignored.
  • For a text dataset each entity is rendered in the view mode and the HTML is converted to plain text. When the AI module's HTML to Markdown converter is available the output is Markdown; otherwise tags are stripped and block elements become line breaks. Scripts and styles are removed either way. The record title is the entity label.

Views

This source runs a View and lets its style plugin map the rows onto dataset records. Use it when you need filters, sorts, relationships or a hand picked field list.

Set up the view:

  1. Edit a view and add a display of type AI fine-tuning dataset. The display has no path, no pager and no exposed filters; it exists only to be picked by a job.
  2. Add the fields you need: something for the title, something for the text, and an image, file or media reference field for images. Add filters and sorts as usual.
  3. Under Format, the style is forced to AI fine-tuning dataset. Its settings map the fields:
    • Title field: used as the record title. Tags are stripped. Leave empty to use the row entity's label.
    • Text field: the training text. Leave empty to concatenate every non-excluded field of the row (except the image field), one per line.
    • Image field: an image, file or media reference field. Every image it references becomes a training image, using the same MIME type rules as the entity display source.
  4. The Views preview shows a table of the records (id, title, truncated text, image count), so you can check the mapping before saving.

Then select the display on the job (View display lists only displays of this type) or pass --view=VIEW_ID:DISPLAY_ID to Drush.

For an image dataset, the images of every row are flattened into one list. For a text dataset each row becomes one record with the mapped title and text, converted to plain text the same way as the entity display source.

Checks before training

When a training starts, the runner refuses to continue if:

  • the plugin's provider is not configured,
  • the source produces a dataset type the plugin does not accept,
  • the dataset has fewer items than the plugin's minimum,
  • the source is misconfigured (no entity type, a view without the right style, no matching entities).

Each of these is reported as an error message in the UI and as an exception in Drush.

Tips for image datasets

  • 5 to 30 consistent, high resolution images (1024px or larger) work better than a large noisy set.
  • Keep the subject or style consistent across the images; that is what the model learns.
  • Use the ai_finetuner view mode to expose only the image field of a bundle, so body images and thumbnails do not leak into the dataset.