Skip to content

Getting started

A complete, from-scratch setup: a pgvector-backed Search API index, an ai_agent wired to it, ai_answers enabled for that agent, and the two blocks placed. Every field name below is exact and captured directly from a working setup; example values (server name, index name, agent label, and so on) are illustrative only. Replace them with names that fit your own site and content.

Cross-module requirements

AI Answers depends on ai_agents and ai_search (a standalone project), and on Postgres, ai_vdb_provider_postgres.

Three upstream issues currently affect setup

  • ai_search issue 3584031: RagTool structured per-result output. Without this, ai_answers gets no sources or references at all.
  • ai_agents issue 3538174 / MR !328: streamed chat in the agent loop. Without this, JSON answers still work but SSE streaming does not.
  • ai_vdb_provider_postgres issue 3586862: clearing a Postgres-backed index drops any Attributes column added after the collection was first created, so the next reindex fails outright. Needs Review, patch attached.

The last one only bites after you clear and reindex, not on first setup; see Debugging below for the recovery sequence if you hit it.

Enable modules

Enable ai_agents, ai_search, and ai_answers. You'll also need a vector database backend module (this guide uses ai_vdb_provider_postgres, pgvector) and an embeddings-capable AI provider configured under Configuration → AI → AI providers (any provider offering the embeddings operation type; the setup below uses a LiteLLM proxy).

Set up the vector database server

At Configuration → Search and metadata → Search API (/admin/config/search/search-api), click Add server:

  1. Server name: e.g. Site Vector DB (pgvector).
  2. Backend: select AI Search ("Index items on Vector DB").
  3. Under Configure AI Search backend:
  4. Embeddings Engine: pick any provider/model offering embeddings. Larger/more advanced embedding models tend to produce more accurate matches, at the cost of slower indexing and, for paid providers, a higher cost per request; a smaller self-hosted model (e.g. LiteLLM Proxy | eu-e5large-embeddings-selfhosted) is a reasonable starting point and can be upgraded later. Changing this later requires a full reindex.
  5. Tokenizer chat counting model: a chat model used only to count tokens per chunk accurately (e.g. OpenAI - gpt-3.5-turbo). This does not need to be the same provider as the embeddings engine.
  6. Include raw embedding vector in results: leave unchecked unless you need raw vectors for a reranking feature.
  7. Vector Database: select Postgres vector DB.
  8. Under Vector Database Configuration:
    • Database Name: the Postgres database to use (e.g. vdb).
    • Collection: a name for this content's vector collection (e.g. site_content); it's created automatically on first index.
    • Similarity Metric: Cosine Similarity is the common default and works well for normalized text embeddings. Euclidean Distance and Inner Product are provider/model-specific alternatives; only switch if your embeddings engine's own documentation recommends one of them for its vectors.
    • Vector index strategy: None (exact search, small datasets) computes an exact distance against every row, accurate but slower as content grows, and is fine to start with. HNSW and IVFFlat both build an approximate index that trades a small amount of recall for much faster search on large corpora; HNSW is the more commonly recommended of the two. Changing this later drops and recreates the index.
  9. Save.

Create the Search API index

At the same Search API overview page, click Add index:

  1. Index name: e.g. Site content.
  2. Datasources: check Content (nodes). Optionally restrict to specific content type bundles if you don't want everything indexed.
  3. Tracker: Default is fine; only switch to AI Search Chunked Tracker if you have a specific chunking need. It isn't required for basic use.
  4. Server: select the server you just created.
  5. Check Enabled, then Save.

Map fields

On the index's Fields tab (/admin/config/search/search-api/index/<id>/fields), click Add fields and add at minimum:

  • Rendered item (rendered_item), Type Fulltext, Indexing option Main content: this is the actual body text that gets chunked and embedded; end-user queries are matched against it.
  • Title (title), Type Fulltext, Indexing option Contextual content: appended to every chunk of the main content so each chunk retains its title context even in isolation.

The three indexing options (per the form's own guidance):

Option Use for
Main content The long body text to chunk and embed. Normally only one field.
Contextual content Short context (title, summary) appended to every chunk of the main content.
Filterable attributes Structured metadata (dates, taxonomy) the vector DB can pre-filter on before the similarity search runs.

Enable processors

On the index's Processors tab, enable at least:

  • Entity status: excludes unpublished content from being indexed at all. This is normally sufficient for access control on a public RAG index. Several other processors (HTML filter, Number field-based boosting, Stemmer, Tokenizer, Type-specific boosting) are explicitly flagged by the AI Search backend as not recommended with this server type, so leave them off.

Index the content

ddev drush search-api:index
ddev drush search-api:status

Confirm the index shows items indexed with no errors before moving on.

Create the AI agent

At Configuration → AI → Tools & Automation → Configure AI Agents (/admin/config/ai/tools-automation/agents/add/form):

  1. Label: e.g. Site Answers Agent.
  2. Description: required; triage/orchestration tooling uses this to pick agents, so be specific (e.g. "Answers questions about the site's documentation using retrieval-augmented generation.").
  3. Max loops: the maximum number of tool-call rounds before the agent gives up. Too low (e.g. 1) and a question that genuinely needs a retrieve-then-answer round trip may not get the chance to complete; too high, and an agent that gets stuck repeatedly calling tools burns cost and latency before it finally gives up. 3 is a reasonable starting point for a single retrieval round plus generation.
  4. Under Usage details → Agent Instructions: this is the agent's system prompt. Example that works well:

    You are this site's assistant. Use the RAG/Vector Search tool to find relevant content before answering any question. Always ground your answer strictly in the retrieved search results.

  5. Under Tools, click Select tools and add RAG/Vector Search.
  6. Expand Detailed tool usage → RAG/Vector Search → Property setup and configure each property's Restrictions:
  7. index: set to Force value, check Hide property and One value, no line breaks, and set Values to your index's machine name (e.g. site_content, matching whatever you named the collection above). This is not optional; see ADR 0004. An LLM-pickable index is a prompt-injection vector, and ai_answers refuses to enable answers for an agent without a forced, hidden index.
  8. min_score: Force value. There's no single correct number here; it's a tradeoff between two failure modes, so pick a starting point and adjust based on what you observe:
    • Too low (e.g. 0.3) and weak, barely-relevant matches can get retrieved and cited, especially on short or ambiguous questions.
    • Too high (e.g. 0.7 or above) and a small or sparse corpus may return nothing at all, triggering the no-answer message even when a relevant source exists just under the bar. 0.4-0.5 is a good place to start. Raise it if weak matches keep getting cited; lower it if legitimate questions keep coming back with no answer.
  9. search_string and amount: leave as Allow all so the agent/module controls them at call time.
  10. Save.

Enable AI Answers for the agent

At Configuration → AI → AI Answers agents (/admin/config/ai/ai-answers/agents), click Configure AI Answers next to your agent. (Once the agent is saved, its own edit form also has an AI Answers section at the bottom linking directly here.)

  1. Check Provide answers with this agent.
  2. AI provider: Default uses the site-wide default provider for the "Chat with tools" capability; only override here if this specific agent needs a different model.
  3. Reference view mode: the view mode used to render each retrieved source in the answer's reference list. A compact mode like Teaser keeps the list scannable when several sources are cited; a fuller view mode surfaces more context per source at the cost of a longer reference list. Teaser works well as a default.
  4. Source URL field name: optional. A field machine name (e.g. field_source_url) whose value, when set on a retrieved source entity, is used as the reference link instead of that entity's own page. Useful when sources were imported or crawled from elsewhere and the original URL should be cited, not this site's copy of the content; see ADR 0014. Leave blank to always link to the entity's own canonical URL — the default, and the only behavior before this setting existed.
  5. Extra generation guidance: optional instructions appended to the built-in citation contract and the agent's own system prompt. Useful for closing gaps the citation contract alone doesn't cover, e.g.:

    Always answer in the same language the user's question was asked in (default to English if the question's language is ambiguous), even if the retrieved source content is in a different language. Only answer if the retrieved sources are genuinely relevant to the specific question asked; if a source only shares a keyword with the question but doesn't actually address it, treat it as insufficient and respond that you don't have enough information.

  6. No-answer message: shown when retrieval yields no usable sources (e.g. "I couldn't find sources to answer that."). No generation happens in that case; see ADR 0005.
  7. Accept feedback: enables the thumbs up/down control on answers.
  8. Conversation retention (seconds): how long a conversation stays resumable for follow-ups. Too short, and a visitor who reads the answer for a few minutes before replying loses the ability to follow up in the same thread; too long, and abandoned conversations linger in storage longer than needed. 3600 (one hour) works well for a typical browsing session.
  9. Maximum history turns: how many prior turns are threaded into the model as context on follow-ups; 0 sends all turns. Each threaded turn adds its full text as input tokens to every subsequent question, so a cap bounds cost on long conversations.
  10. Save.

Place the blocks

At Structure → Block layout (/admin/structure/block), two blocks work as a pair: place the Answer block first, since the Question block needs to target it.

Answer block

Place block in your chosen region (e.g. Content Bottom), search for AI Answers: Answer:

  1. Title: e.g. Answer (only shown if Display title is checked).
  2. AI Agent: select your agent; only agents with AI Answers enabled appear here.
  3. Allow follow-up questions: keeps the conversation open so users can ask follow-ups in the same block.
  4. Show references: shows the Sources list under the answer. References heading: the heading text above that list (defaults to Sources).
  5. Show feedback control: shows thumbs up/down (requires Accept feedback enabled on the agent's AI Answers settings too).
  6. Empty-state message: optional placeholder text shown before the first answer; leave blank for none.
  7. Place it in your chosen region and save.

After saving, note this block's Target id for a Question block field (shown on its own configure form, e.g. ai-answers-answer-49f56d07-09b5-46f7-8d85-b03129451bca). The Question block needs it, though in practice the Question block's own "Target Answer block" field lists placed Answer blocks by name, so you rarely need to copy this id by hand.

Question block

Place block, search for AI Answers: Question:

  1. Title: e.g. Ask a question, with Display title checked.
  2. Target Answer block: select the Answer block you just placed (shown as <title> (<region>), e.g. Answer (content_bottom)). The AI Agent is derived from that block, not set here.
  3. Cross-page fallback URL: optional; only needed if this Question block might render on a page where the Answer block isn't present (the browser navigates to this path with the question carried in the URL fragment).
  4. Placeholder: e.g. Ask a question….
  5. Submit label: e.g. Ask.
  6. Suggested questions: optional, one example question per line, rendered as clickable chips under the input, e.g.: 
    What is this site about?
How do I get in touch with support?
Where can I find the documentation?
  7. Place it in the same region as (or above) the Answer block, and save.

Permissions

At People → Permissions (/admin/people/permissions), grant:

  • Use AI Answers (use ai answers): required to call the question and feedback endpoints at all. Grant this to any role that should be able to ask questions through the placed blocks; without it, every request from that role gets a 403.
  • Administer AI Answers (administer ai answers): configure which AI Agents provide answers and their settings, at Configuration → AI → AI Answers agents. Restricted access; grant only to roles that should manage this configuration.
  • View AI Answers traces (view ai answers traces): include log_id/trace_id in answer responses (otherwise omitted). Restricted access; grant only to trusted roles doing debugging, since these ids can be used to look up the underlying ai_log entities and Langfuse traces.

Optional: Langfuse observability

drupal/langfuse is a soft dependency. Without it, ai_answers still works normally; trace_id in responses and the JSON/SSE log_id/trace_id fields simply stay null.

What enabling it gets you:

  • Enable langfuse and configure langfuse.settings (/admin/config/system/langfuse/settings) with your Langfuse project's URL and credentials. AnswerService::generate() then opens a real trace per turn (tagged ai_answers, conversation:<id>, turn:<n>) and records the question/answer text on it, so trace_id is populated in responses.
  • Also enable langfuse_ai_logging and langfuse_ai_agents_logging (submodules of langfuse) for the richer per-call data: token usage, model name, and latency for every LLM generation and rag_search tool call, nested under the same trace as real Langfuse observations. This needs no ai_answers code or config. Both submodules reuse whatever trace is already current for the request. Verified live: a single answer produces one trace with one observation per embedding/generation/tool-call round (e.g. 8 observations for a 2-round agent run), aggregated into per-trace token/latency totals automatically.

Known bug in langfuse_ai_agents_logging

LangFuseToolSpanSubscriber::onAgentToolPreExecute() calls ensureTrace() with no try/catch around it, unlike its sibling subscriber in langfuse_ai_logging. ai_answers's own trace handling degrades gracefully if Langfuse is unreachable when generation starts, but if Langfuse becomes unreachable or misconfigured while a request is in flight and the rag_search tool then fires, this subscriber's uncaught exception propagates all the way up and fails the whole answer request with a 503; this was reproduced live by pointing langfuse_url at a malformed host mid-session. This is a bug in langfuse_ai_agents_logging itself, not something ai_answers can guard against from the outside.

Feedback scores do not reach Langfuse yet

Regardless of the above, see the known gap. The thumbs up/down buttons still work and still write to ai_log; only the Langfuse score is missing.

Testing against a self-hosted Langfuse v4 instance

For a DDEV project, ddev-langfuse spins up a full local Langfuse v4 stack (langfuse-web, langfuse-worker, Postgres, ClickHouse, Redis, MinIO) on the project's own Docker network with generated credentials, no manual service setup required:

ddev add-on get abhisekmazumdar/ddev-langfuse
ddev restart
ddev langfuse-credentials

The default docker-compose.yml from langfuse/langfuse (and this add-on, which deploys the same image) ships in events_only write mode, which rejects the classic ingestion events this integration (and its underlying dropsolid/langfuse-php-sdk) sends: createTrace() calls fail outright with "Event type not accepted". Set these two environment variables on both langfuse-web and langfuse-worker to make a self-hosted v4 instance accept them; with the add-on, add them under each service in .ddev/docker-compose.langfuse.yaml and run ddev restart:

LANGFUSE_MIGRATION_V4_WRITE_MODE=legacy
LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=dual_write

The second variable is required whenever the first is legacy; the worker refuses to start otherwise. Langfuse Cloud or a pre-existing self-hosted instance not running in events_only mode needs none of this.

Test

Ask a question. Expect a streamed answer with [n] citations and a sources list; the follow-up input appears after the first answer completes.

Debugging

With ai_logging enabled, check Automatically log requests at Configuration → AI → AI Logging (/admin/config/ai/logging/settings); it's off by default, and with it off no ai_log entities are ever created regardless of the module being enabled. With it on, inspect ai_log entities. A successful run produces one embeddings log (the tool's vector query) plus two chat logs sharing an ai_agents_runner_* tag. Round 2's system prompt must contain the citation contract and a Sources: block. If it doesn't, the subscriber bridge isn't wiring in correctly (see Architecture).

If rag_search silently returns zero results with no error anywhere, check for a known upstream bug before assuming the module is misconfigured. It's triggered by clearing a Search API index, and doesn't surface an error to point at itself:

Known upstream bug after clearing an index

ai_vdb_provider_postgres #3586862: clearing an index drops any attribute column (e.g. status) added after the collection was first created; every reindex attempt then fails silently against the missing column.

If a clear leaves the index unusable, re-saving the index entity, resetting processed_chunks to 0, running search-api:reset-tracker, and reindexing is the known recovery sequence.

If a question returns an answer in the wrong language, or cites a source that only shares a keyword with the question without actually answering it, see the Extra generation guidance field above. The built-in citation contract alone doesn't constrain language or enforce topical relevance; raising min_score and adding explicit guidance there is the config-only mitigation. Retrieval itself has no language filter (nothing restricts results to the visitor's current interface language), which is tracked as a known gap.