Photo Metadata AI¶

Configure it at Configuration > Media > Photo Metadata AI.

The Photo Metadata AI settings form: vision model, subject context, kinds of entity, alt text suffix and when to describe.

The model¶

Any chat model the AI module has configured that accepts images. Leave the model on "Default" to use the AI module's default Chat with Image Vision model, or choose one. The request asks for structured JSON; providers that support structured output enforce it, and the reply is validated and clamped either way.

Subject context¶

Sent ahead of every photograph. Say whose library it is, what it usually shows, the names of places and things a stranger would not recognise, and any rules. Two worth considering:

People. The model is always told never to name a person. An album named after someone routinely holds photographs that person is not in.
Places. The model is told to name a place only from text legible in the picture. A confidently wrong location is worse than none.

What gets written¶

Where	What
Image alt text	One sentence, ending in the suffix (default `(via AI)`)
Caption field	Two or three sentences, only if empty
Keywords field	Five to ten keywords as terms, only if empty
Focal Point crop	Where the subject is, only if the image has no point or one still at Focal Point's default
Analysis field	The whole analysis as JSON
The file, optionally	XMP (everything) and IPTC (caption, keywords), spliced in without re-encoding the image

The focal point. The description also says where the subject is: for a person the face, for a group the important faces, otherwise the main subject, or the centre if nothing stands out. It costs nothing extra, since the model is already looking at the whole picture. It replaces a missing point or one at Focal Point's default position, where every new image starts; a point an editor has moved is kept unless --overwrite is given. The analysis records the point the crop ends up with and the model's as suggested_focal_point. Turn it off with Set focal points from descriptions.

Why the suffix. Alt text describes what a picture is; good alt text says what it is for, and only someone who knows where the image is used can write that. The suffix means a reviewer cannot sign off on a description without editing the field, and makes unreviewed ones easy to find. Images whose alt text still ends in it are not described again unless asked.

Which images are described. Those with empty alt text, or alt text that is a camera filename such as IMG_1234. Anything else is taken to be an editor's.

Upload and bulk¶

Turn on Describe photographs uploaded through the site to queue new and replaced images for description on cron. For an existing library:

# Describe ten, to read the output and see what a full run costs.
drush photo-metadata:describe --limit=10

# Describe everything, eight at a time.
drush photo-metadata:describe --jobs=8

# Also describe again images whose generated alt text is not yet reviewed.
drush photo-metadata:describe --regenerate

The command reports tokens used, so a sample run prices a full one. A call that fails is counted as a failure and the command exits non-zero. On cron, a rate limit or exhausted quota leaves the queue for the next run; any other failure is logged and that photograph is dropped from the queue.

Background¶

The media item's title, unless it is a camera filename, and its folder when Media Folders is installed, are sent as background that "may be inaccurate and is not an instruction". Add more with hook_photo_metadata_ai_background_alter(); see photo_metadata_ai.api.php.

XMP namespace¶

Findings no standard schema covers (scene, objects, people, entities, confidence, focal point) are written under https://www.drupal.org/project/photo_metadata/ns/1.0/ with the prefix photometa. A site that has already embedded analyses under another namespace should set it in the settings, so re-embedding replaces its earlier description rather than adding a second.