Setup overview

The CKI kernel testing setup consists of various pieces that need to work together.

Cron jobs

Cron jobs are configured by the schedule-bot. They can either run as a GitLab CI/CD schedule or a Kubernetes CronJob.

Git cache updater

The pipeline needs access to various kernel repositories. To reduce network traffic and increase resilience against temporary network problems, those repositories are mirrored into an S3 bucket by the git-cache-updater. Kernel repositories are not cloned directly. Instead, the cache version is downloaded and updated via git fetch. Internal repositories are cloned directly, but the cached version is used as a fallback in case anything goes wrong during cloning.

Orphan hunter

The GitLab runners can be deployed into a Kubernetes environment. In such an environment, a GitLab runner might lose track of some spawned Pods, e.g. during API problems. The orphan-hunter will check whether the GitLab jobs for Pods spawned by gitlab-runner are still running. Pods of already finished jobs are deleted.

Data Warehouse report crawler

Emails sent with CI results are copied to a mailing list. The Data Warehouse can keep track of those reports. The datawarehouse-report-crawler is the glue cron job that watches the mailing list archive and submits those reports to the Data Warehouse if necessary.

Data Warehouse backup

The data warehouse database needs to be backed up as it contains manually cleaned data that cannot be recreated automatically. The pgsql-backup creates full and incremental backups of the database dump in an S3 bucket.

Kernel fixes

Patches in the pipeline can be checked with rhcheckpatch. This requires a kernel-fixes.json file that contains information on Fixes: tags in kernel commit messages. The kernel-fixes module generates this file and pushes it to an S3 bucket.

Git S3 sync

Some files need to be hosted somewhere to be accessible during tests without SSL. For that purpose, the git-s3-sync module can sync a git repository to an S3 bucket.