GitLab tests in the Continuous Integration (CI) context

Test suite parallelization on the CI

Our current CI parallelization setup is as follows:

The retrieve-tests-metadata job in the prepare stage ensures we have a knapsack/report-master.json file:
- The knapsack/report-master.json file is fetched from the latest main pipeline which runs update-tests-metadata (for now it’s the 2-hourly scheduled master pipeline), if it’s not here we initialize the file with {}.
Each [rspec|rspec-ee] [unit|integration|system|geo] n m job are run with knapsack rspec and should have an evenly distributed share of tests:
- It works because the jobs have access to the knapsack/report-master.json since the “artifacts from all previous stages are passed by default”.
- the jobs set their own report path to "knapsack/${TEST_TOOL}_${TEST_LEVEL}_${DATABASE}_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json".
- if knapsack is doing its job, test files that are run should be listed under Report specs, not under Leftover specs.
The update-tests-metadata job (which only runs on scheduled pipelines for the canonical project takes all the knapsack/rspec*_pg_*.json files and merge them all together into a single knapsack/report-master.json file that is saved as artifact.

After that, the next pipeline uses the up-to-date knapsack/report-master.json file.

The GitLab test suite is monitored for the main branch, and any branch that includes rspec-profile in their name.

Rails logging to log/test.log is disabled by default in CI for performance reasons. To override this setting, provide the RAILS_ENABLE_TEST_LOG environment variable.