PHPUnit¶

The phpunit jobs run the PHPUnit tests defined by your module, including Functional, FunctionalJavascript, Kernel and Unit tests. You have a choice of two ways that the tests can be executed:

Using the phpunit binary directly and running the tests in sequential mode. You can specify it using:
```
variables:
  _PHPUNIT_CONCURRENT: 0
```
Using the Drupal core script run-tests.sh to execute the tests in concurrent mode. This is the default, and it would be the same as:
```
variables:
  _PHPUNIT_CONCURRENT: 1
```

The PHPUnit 'variants' do not all have to be run the same way. For example, you could leave the default as _PHPUNIT_CONCURRENT: 1 but run the 'next minor' job using the phpunit binary:

phpunit (next minor):
  variables:
    _PHPUNIT_CONCURRENT: 0

Using `phpunit` binary directly (non-concurrent mode)¶

The tests will be run calling the phpunit binary directly and in a sequential order. As there is no concurrency, the tests are likely to take longer to run.

You can use the _PHPUNIT_EXTRA variable to add extra options to the phpunit binary. For example:

variables:
  _PHPUNIT_EXTRA: "--debug"

TestDox¶

An example of one of the phpunit-only options is TestDox. If you want nicer CI terminal output you can set or add the option --testdox to the variable _PHPUNIT_EXTRA. The output will change from the typical

.............. (14/14)

to a more detailed output showing each test name and class, for example:

Admin Pages (Drupal\Tests\api\Functional\AdminPages)
 ✔ Admin pages
Api Access (Drupal\Tests\api\Functional\ApiAccess)
 ✔ Access
 ✔ Branch objects
 ...

Note that this option conflicts with the Drupal HtmlOutputPrinter printer and, if used, no .html artifacts will be generated after the run.

Running a subset of the tests¶

There may be times when you temporarily only need to run one or two tests, instead of the whole test suite. This can be achieved using:

variables:
  _PHPUNIT_EXTRA: "--filter='/thisTest|thatTest/' "

The --filter parameter is only valid when running the phpunit binary directly.

Using `run-tests.sh` core script (concurrent mode)¶

This script is included in Drupal core and leverages concurrency and also runs the tests in the same way that Drupal core tests are run. As there is concurrency, the job should run quicker than with the sequential option. You can set the _CONCURRENCY_THREADS variable to tweak the concurrency level used by run-tests.sh. The default value is 32.

To add options compatible with the run-tests.sh script use the _RUNTESTS_EXTRA variable. For example, to run a subset of the tests:

variables:
  _RUNTESTS_EXTRA: "--class 'Drupal\Tests\{module}\Functional\ModuleSomeParticularTest'"

You can add parallel or parallel:matrix options to the job definition, to split out the tests into multiple jobs running in parallel. If you want to split evenly the tests that are run in each parallel job, you can do it like this:

variables:
  _RUNTESTS_EXTRA: "--ci-parallel-node-index $CI_NODE_INDEX --ci-parallel-node-total $CI_NODE_TOTAL"

phpunit:
  parallel: 3

Variants¶

The phpunit job has multiple variants to check previous, current and future versions of Drupal. See the OPT_IN variables for more details.

If you want to tweak the value of the options per variant, you can do something like this:

variables:
  _PHPUNIT_EXTRA: "value for all variants unless overridden..."

phpunit (next major):
  variables:
    _PHPUNIT_EXTRA: "value for next major variant only"

Lenient core compatibility for dependent projects in 'Next Major' variant¶

If your project has opted in to test against the next major core version, via OPT_IN_TEST_NEXT_MAJOR: 1 and you also are also using the Composer Lenient Support functionality, then the dependencies defined in _LENIENT_ALLOW_LIST are adjusted to be compatible with Next Major so that your PHPUnit tests can be run. To disable this feature set _PHPUNIT_LENIENT_COMPATIBILITY: 0

PHPUnit versions¶

Drupal 10 uses PHPUnit 9, whereas Drupal 11 uses PHPUnit 10. Because of this, not all phpunit options are valid in both versions. If you use _PHPUNIT_CONCURRENT: 1, then run-tests.sh takes care of these cases for you. However, if you use _PHPUNIT_CONCURRENT: 0, you might need to adapt the options per variant as shown in the previous section.

If you do not provide a PHPUnit configuration file in your module, the default file from core/phpunit.xml will be used.

If you do provide a PHPUnit configuration file in your module, you might need to provide a different one depending on the PHPUnit version. You can achieve it like this (where phpunit9.xml and phpunit10.xml are files in the root of your repository):

variables:
  _PHPUNIT_EXTRA: '-c $CI_PROJECT_DIR/phpunit9.xml'

phpunit (next major):
  variables:
    _PHPUNIT_EXTRA: '-c $CI_PROJECT_DIR/phpunit10.xml'

The variants inherit customizations made in the phpunit job. If you don't want these to propagate to the variants, you could extend the variants from the .phpunit-base job, like this:

phpunit (next major):
  extends: !reference [.phpunit-base]

Customizing the Exit Code processing¶

If your project needs to manipulate the EXIT_CODE value that is returned, or react to certain exit code values, it can sometimes be too late to do this in the after_script section, and it has to be done in the script section instead. To do this, you can reference the phpunit-base-script, then add your customizations, like this:

phpunit:
  script:
    - !reference [ .phpunit-base-script ]
    # Do your customized processing here. No 'exit' will be executed in the above
    # .phpunit-base-script so you should probably end this with 'exit $EXIT_CODE'
    - exit $EXIT_CODE

Web driver versions¶

Tests on Drupal 11 and above will use Selenium and the latest Chrome W3C-compliant browser. Tests on Drupal 10 and below will continue to use the legacy Chrome driver which is not W3C-compliant. If your javascript tests at Drupal 11 still need to use the legacy driver, add the following lines to a PHPUnit job to override the default behavior:

  # Set the services and web driver variables to the legacy versions.
  services:
    - !reference [ .with-database ]
    - !reference [ .with-chrome-legacy ]
  variables:
    MINK_DRIVER_ARGS_WEBDRIVER: $MINK_DRIVER_ARGS_WEBDRIVER_LEGACY

Deprecations¶

From Drupal Core 11.5 and above, deprecation handling uses the variable DRUPAL_DEPRECATION_FILTER_CONFIG and this is empty by default. See the Change Record for more details.

Core uses a .deprecation-ignore.txt file to specify some deprecation messages that have to be ignored. If your project needs to ignore additional deprecation messages, for example those created by 3^rd-party modules that are used in tests, you can create a similar file stored in the root of your project, and set DRUPAL_DEPRECATION_FILTER_CONFIG to use this file:

phpunit (next minor):
  variables:
    DRUPAL_DEPRECATION_FILTER_CONFIG: '{"projectIgnoreFile":"$CI_PROJECT_DIR/.deprecation-ignore.txt"}'

If you name the file .deprecation-ignore.txt then the core file will be automatically appended to it, so that all the core deprecations will be ignored in addition to the ones specified in the project's file.

If you do not want any deprecations to be reported you can use the run-tests.sh option --suppress-deprecations by setting:

phpunit (next minor):
  variables:
    _RUNTESTS_EXTRA: '--suppress-deprecations'

This is only available in Core 11.5+ and when using _PHPUNIT_CONCURRENT:1.

The legacy variable SYMFONY_DEPRECATIONS_HELPER is used in core versions up to and including 11.4. This is set to disabled by default so that warnings and deprecations are not reported. The same process as above applies if you need to ignore additional deprecations. The syntax is slightly different:

phpunit (next minor):
  variables:
    SYMFONY_DEPRECATIONS_HELPER: "ignoreFile=$CI_PROJECT_DIR/.deprecation-ignore.txt"

Deprecation warnings are not shown by default when using the phpunit binary directly (_PHPUNIT_CONCURRENT:0) and when using the default phpunit.xml config file provided by Core. If your project has a custom configuration then the deprecation handling might be different.

If you are using a Drupal core version that comes with PHPUnit 10 or above (this would be Drupal 11 or above), you can set the flag --display-deprecations in the _PHPUNIT_EXTRA variable to show details of the deprecations. You can set --fail-on-deprecation to make the job end with a failure exit code.

variables:
  _PHPUNIT_EXTRA: '--display-deprecations --fail-on-deprecation'

If your project has its own phpunit.xml(.dist) file and you are testing with Drupal 10 (which uses PHPUnit 9), your file will need to contain the <listener class="\Drupal\Tests\Listeners\DrupalListener"> class in order to get deprecation warnings.

Skipped Tests¶

If a test is skipped (for example a Javascript test cannot run because the webdriver is not available) then this will be indicated in the log, but the job will not fail. To change this default behavior, you can add --fail-on-skipped to the _PHPUNIT_EXTRA variable or set this option in your projects own phpunit.xml configuration file. It will only have the required effect when using _PHPUNIT_CONCURRENT=0 and running the phpunit binary directly, as this option is invalid when using run-tests.sh