Skip to content

Installation

Requirements

  • Drupal 10.3 or later 10.x, or 11.2 or later. Drupal 11.0 and 11.1 lack the hook ordering attributes the module relies on.
  • PHP 8.1 or later.
  • Markdownify 1.2 or later.

Recommended:

  • Markdownify's Markdownify Path submodule, so .md works on path aliases (/recipes/oatmeal.md) and not only on system paths (/node/42.md).
  • Markdownify's Markdownify Metadata submodule, for richer front matter (author, dates, tags, images, Open Graph and Metatag values).
  • RobotsTxt, to write the directives to robots.txt. Without it, headers and front matter still carry them, and the status report says so.

Install

composer require drupal/agent_surface

drush en agent_surface agent_surface_fallback agent_surface_rsl markdownify_path markdownify_metadata robotstxt

agent_surface_fallback and agent_surface_rsl are optional; see Submodules.

robots.txt

With RobotsTxt enabled, delete the static robots.txt from the docroot, or the web server answers /robots.txt before Drupal does. Composer's scaffold restores the file on every composer install unless the project's composer.json excludes it:

"extra": {
    "drupal-scaffold": {
        "file-mapping": {
            "[web-root]/robots.txt": false
        }
    }
}

Review the defaults before going live

The shipped defaults declare the site's content available for AI training, AI inference and search, on every page and in robots.txt. Permissive declarations have legal weight in several jurisdictions. Enabling the module shows a warning saying so, and the status report keeps warning until an administrator saves the settings form at Configuration → Web services → Agent Surface (/admin/config/services/agent-surface). Saving the form is the record that the defaults were reviewed. See Configuration.

Uninstall

Uninstalling removes the module's configuration, the per-bundle overrides and the request log table. Uninstalling agent_surface_rsl removes the rsl vocabulary from the enabled list.

Hosting on Pantheon

  • Platform domains (*.pantheonsite.io) serve Pantheon's own robots.txt, which disallows crawlers. Under AIPREF, preferences apply only to crawlable paths, so the declarations mean nothing anywhere except Live on a custom domain. Test them there.
  • Core's cacheable branch, and therefore Cache-Control: public, max-age, applies only when Browser and proxy cache maximum age is above zero. The module sets its validators either way, so 304s work regardless.