Skip to content

GTFS 511

Provisions GTFS static and GTFS-RT feed entities from the 511 SF Bay Open Data API for agencies that are members of the Metropolitan Transportation Commission (MTC).

This module is a thin provisioning layer. All data storage and import logic is delegated to the GTFS and GTFS Realtime modules.


Requirements


Setup

1. Obtain a 511 API token

Request a token at https://511.org/open-data/token. The default rate limit is 60 requests per 3,600 seconds.

2. Store the token in a Key entity

  1. Navigate to Administration → Configuration → System → Keys.
  2. Add a new key of type Authentication → API Key.
  3. Enter your 511 token as the key value and save.

3. Configure the module

Navigate to Administration → GTFS → 511 Settings (/admin/gtfs/511/settings).

  1. Select the 511 API Key created in step 2.
  2. The form immediately queries https://api.511.org/transit/gtfsoperators and displays a checkbox list of all available MTC member operators.
  3. Check each operator whose feeds should be provisioned.
  4. Click Save and sync to persist the settings and immediately provision all feeds.

Use Save without syncing to update settings without triggering provisioning (e.g. when selecting operators you are not yet ready to import).


What the sync does

For each configured operator ID the module creates or updates:

Entity ID / Name Description
gtfs_feed {Agency Name} [511:{OPERATOR}] Static GTFS schedule zip download URL.
gtfs_rt_feed 511_{operator}_tripupdates GTFS-RT Trip Updates (protobuf).
gtfs_rt_feed 511_{operator}_vehiclepositions GTFS-RT Vehicle Positions (protobuf).
gtfs_rt_feed 511_{operator}_servicealerts GTFS-RT Service Alerts (protobuf).

All operations are idempotent. Running sync multiple times is safe.

511 API endpoints used

Feed URL pattern
Static GTFS https://api.511.org/transit/datafeeds?api_key={key}&operator_id={id}
Trip Updates https://api.511.org/transit/tripupdates?agency={id}
Vehicle Positions https://api.511.org/transit/vehiclepositions?agency={id}
Service Alerts https://api.511.org/transit/servicealerts?agency={id}

The realtime URLs carry no key. The module attaches it to each request through hook_gtfs_rt_feed_request_alter(), reading it from the configured Key entity at that moment, because a realtime feed is configuration and configuration is exported and committed. The static URL keeps its key: the GTFS module downloads it with no hook to add one, and a feed URL is content, not configuration. A post update takes the key out of existing realtime URLs once the Key entity can supply it.

Staying within the rate limit

511 allows each token 60 requests an hour unless it has raised that for you, and counts every feed the token fetches against it. GTFS Realtime fetches a feed whenever one is read after its refresh interval has passed, so on a busy site the intervals are what keep the token under the limit.

The module therefore sets them. Every enabled 511 realtime feed gets a share of Requests per hour on the settings form: trip updates and vehicle positions four parts each, service alerts one, because a prediction moves by the minute and an alert by the hour. One operator's trip updates and alerts at 60 an hour come to 75 and 300 seconds. Disable a feed you do not use (vehicle positions, often) and the others get its share.

The intervals are worked out again on every sync, on Save without syncing, and by the post update. Change a feed's status by hand or by configuration import and save the settings form afterwards, or export intervals you have worked out with the same weights.


Drush: provisioning without a browser

Everything above is also available from the command line, so a 511 region can be provisioned by a script rather than by a person clicking through a form.

Command Purpose
drush gtfs:511:set-key <ENV_VAR> Point this site at an API key held in an environment variable.
drush gtfs:511:operators List the operators the configured key can see.
drush gtfs:511:sync Provision feeds for the configured operators.
drush gtfs:511:set-key GTFS_511_API_KEY
drush gtfs:511:operators
drush gtfs:511:sync --operators=AC,SF --import

gtfs:511:sync options:

  • --operators=AC,SF — operator IDs to provision. Defaults to whatever the settings form saved. The IDs are checked against the live operator list, so a typo is refused rather than silently creating a feed that 404s.
  • --import — also download and import each operator's static feed. Without it the feed entities exist but hold no data, exactly as after a form sync.
  • --dry-run — report what would change and change nothing.

Keeping the token out of configuration

gtfs:511:set-key creates a Key entity that uses the env provider, so the entity records only the name of an environment variable. The token itself lives in the environment of the web and cron processes and never reaches configuration, the database, or an exported YAML file — which matters because the 511 static feed URL embeds the key as a query parameter, and that URL is stored on the feed entity.

For the same reason --import runs the GTFS module's import chain in process rather than shelling out to drush gtfs:feed:refresh <alias> <url>: passing the URL on a command line would put a live credential in the process list of every machine that runs it.

The variable must be set for the processes that actually fetch feeds, not only for the shell running drush. Under Docker that means the app container's environment; under a traditional host, whatever php-fpm and cron inherit.


Importing static GTFS data

After syncing, the settings page displays a Provisioned Feeds table with an import link for each operator's static feed. Click the link to open the GTFS import form for that feed.

Alternatively, navigate to Administration → GTFS → Feeds, find the feed named {Agency Name} [511:{OPERATOR}], and use its Import tab.


Removing an operator

Uncheck the operator in the Operators list and click Save and sync. The module will:

  1. Delete the three gtfs_rt_feed config entities for that operator.
  2. Queue a batch deletion of the gtfs_feed entity and all associated GTFSRow entities via the GTFS module's batch-delete mechanism.

Note: Batch deletion runs after the form submission redirect. Large feeds may take several page loads to complete.

Maintainers

  • Tyler Brown Cifu Shuster - tbcs
  • Jason P. Salter - jpsalter

This project has been sponsored by: