ci: modernise CI pipeline with improved tooling and workflow structure
Some checks failed
CI Checks / Prek & Format (push) Successful in 50s
CI Checks / Tests (push) Has been cancelled
CI Checks / Clippy (push) Has been cancelled
Release Builds / Prepare Build Matrix (push) Successful in 4s
Release Builds / Build Images & Binaries (push) Failing after 1m57s
Release Builds / Publish Multi-platform Manifest (push) Has been skipped

Pipeline Modernisation:
- Updated to prek (prefligit renamed to avoid typosquatting)
- Adopted uvx for tool invocation (prek, rustup) without install scripts
- Consolidated workflow steps to run in parallel where possible
- Removed redundant custom actions in favour of direct uvx commands
- Made sccache optional (based on secrets availability) for fork compatibility

Workflow Restructuring:
- Renamed workflows for clarity and consistency:
  * rust-checks.yml → ci-checks.yml (now includes prek linting)
  * clients-element.yml → deploy-element.yml
  * documentation.yml → deploy-docs.yml
  * release-image.yml → release-builds.yml (builds images AND binaries)
  * mirror-images.yml → docker-mirror.yml
- Simplified job and step naming throughout for better readability

Performance Improvements:
- Parallel execution of independent steps (formatting and prek checks)
- Optimised caching strategy for uv and Rust dependencies
- Reduced CI execution time through better parallelisation

Dependency Management:
- Added Renovate configuration for automated dependency updates
- Configured to monitor both .forgejo/ and .github/ workflows
- Set up automatic PR creation for non-major GitHub Actions updates
This commit is contained in:
Tom Foster 2025-08-12 12:11:33 +01:00
parent 583cb924f1
commit 62795067c9
10 changed files with 319 additions and 213 deletions

View file

@ -0,0 +1,175 @@
name: CI Checks
on:
push:
pull_request:
# Cancel in-progress runs when a new push is made to the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
fast-checks:
name: Prek & Format
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install uv
uses: https://github.com/astral-sh/setup-uv@v6
with:
enable-cache: true
ignore-nothing-to-cache: true
cache-dependency-glob: ''
- name: Run prek (formerly prefligit)
run: uvx prek run --show-diff-on-failure --color=always -v --all-files --hook-stage manual
- name: Install rust nightly with rustfmt
run: |
uvx rustup override set nightly
uvx rustup component add rustfmt
- name: Check formatting
run: |
cargo +nightly fmt --all -- --check
clippy:
name: Clippy
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install uv
uses: https://github.com/astral-sh/setup-uv@v6
with:
enable-cache: true
ignore-nothing-to-cache: true
cache-dependency-glob: '' # Disable Python dependency tracking for Rust project
- name: Install Rust toolchain
run: |
# Install toolchain from rust-toolchain.toml
uvx rustup show # This will auto-install from rust-toolchain.toml
# cache-apt-pkgs-action requires apt lists to be initialised first
- name: Update APT package lists
run: sudo apt-get update
- name: Cache system packages
uses: https://github.com/awalsh128/cache-apt-pkgs-action@latest
with:
packages: clang liburing-dev
version: 1.0
- name: Cache Rust registry
uses: actions/cache@v4
with:
path: |
~/.cargo/git
!~/.cargo/git/checkouts
~/.cargo/registry
!~/.cargo/registry/src
key: rust-registry-${{hashFiles('**/Cargo.lock') }}
- name: Run Clippy lints
run: |
cargo clippy \
--workspace \
--features full \
--locked \
--no-deps \
--profile test \
-- \
-D warnings
tests:
name: Tests
runs-on: ubuntu-latest
env:
SCCACHE_ENABLED: ${{ vars.GH_APP_ID != '' && secrets.GH_APP_PRIVATE_KEY != '' }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install uv
uses: https://github.com/astral-sh/setup-uv@v6
with:
enable-cache: true
ignore-nothing-to-cache: true
cache-dependency-glob: '' # Disable Python dependency tracking for Rust project
- name: Install Rust toolchain
run: |
# Install toolchain from rust-toolchain.toml
uvx rustup show # This will auto-install from rust-toolchain.toml
# cache-apt-pkgs-action requires apt lists to be initialised first
- name: Update APT package lists
run: sudo apt-get update
- name: Cache system packages
uses: https://github.com/awalsh128/cache-apt-pkgs-action@latest
with:
packages: clang liburing-dev
version: 1.0
- name: Cache Rust registry
uses: actions/cache@v4
with:
path: |
~/.cargo/git
!~/.cargo/git/checkouts
~/.cargo/registry
!~/.cargo/registry/src
key: rust-registry-${{hashFiles('**/Cargo.lock') }}
- name: Create GitHub App token for sccache
if: env.SCCACHE_ENABLED == 'true'
uses: https://github.com/actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ vars.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
github-api-url: https://api.github.com
owner: ${{ vars.GH_APP_OWNER }}
repositories: ""
- name: Setup sccache
if: env.SCCACHE_ENABLED == 'true'
uses: ./.forgejo/actions/sccache
with:
token: ${{ steps.app-token.outputs.token }}
- name: Setup Timelord
if: env.SCCACHE_ENABLED == 'true'
uses: ./.forgejo/actions/timelord
with:
key: sccache-v0
path: .
- name: Run Cargo tests
run: |
cargo test \
--workspace \
--features full \
--locked \
--profile test \
--all-targets \
--no-fail-fast
- name: Display sccache statistics
if: always() && env.SCCACHE_ENABLED == 'true'
run: sccache --show-stats

View file

@ -1,22 +0,0 @@
name: Checks / Prefligit
on:
push:
pull_request:
permissions:
contents: read
jobs:
prefligit:
runs-on: ubuntu-latest
env:
FROM_REF: ${{ github.event.pull_request.base.sha || (!github.event.forced && ( github.event.before != '0000000000000000000000000000000000000000' && github.event.before || github.sha )) || format('{0}~', github.sha) }}
TO_REF: ${{ github.sha }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- uses: ./.forgejo/actions/prefligit
with:
extra_args: --all-files --hook-stage manual

View file

@ -1,6 +1,8 @@
name: Release Docker Image
name: Release Builds
# Cancel in-progress runs when a new push is made to the same branch
concurrency:
group: "release-image-${{ github.ref }}"
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false # Don't cancel release builds
on:
push:
@ -17,11 +19,11 @@ on:
workflow_dispatch:
env:
BUILTIN_REGISTRY: forgejo.ellis.link
BUILTIN_REGISTRY_ENABLED: "${{ ((vars.BUILTIN_REGISTRY_USER && secrets.BUILTIN_REGISTRY_PASSWORD) || (github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false)) && 'true' || 'false' }}"
BUILTIN_REGISTRY_ENABLED: "${{ env.BUILTIN_REGISTRY != '' && ((vars.BUILTIN_REGISTRY_USER && secrets.BUILTIN_REGISTRY_PASSWORD) || (github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false)) && 'true' || 'false' }}"
jobs:
define-variables:
prepare:
name: Prepare Build Matrix
runs-on: ubuntu-latest
outputs:
@ -30,7 +32,7 @@ jobs:
build_matrix: ${{ steps.var.outputs.build_matrix }}
steps:
- name: Setting variables
- name: Define build matrix and registries
uses: https://github.com/actions/github-script@v7
id: var
with:
@ -39,26 +41,39 @@ jobs:
const repoId = githubRepo.split('/')[1]
core.setOutput('github_repository', githubRepo)
const builtinImage = '${{ env.BUILTIN_REGISTRY }}/' + githubRepo
console.log('GitHub repository:', githubRepo)
const registry = '${{ env.BUILTIN_REGISTRY }}'
console.log('Registry:', registry || '(not set)')
const builtinImage = registry ? `${registry}/${githubRepo}` : ''
console.log('Built-in image:', builtinImage || '(registry not configured)')
let images = []
if (process.env.BUILTIN_REGISTRY_ENABLED === "true") {
images.push(builtinImage)
}
console.log('Registry enabled:', process.env.BUILTIN_REGISTRY_ENABLED)
console.log('Images:', images.length > 0 ? images : '(none)')
core.setOutput('images', images.join("\n"))
core.setOutput('images_list', images.join(","))
const platforms = ['linux/amd64', 'linux/arm64']
core.setOutput('build_matrix', JSON.stringify({
const buildMatrix = {
platform: platforms,
target_cpu: ['base'],
include: platforms.map(platform => { return {
platform,
slug: platform.replace('/', '-')
}})
}))
}
console.log('Build matrix:', JSON.stringify(buildMatrix, null, 2))
core.setOutput('build_matrix', JSON.stringify(buildMatrix))
build-image:
build:
name: Build Images & Binaries
runs-on: dind
needs: define-variables
needs: prepare
permissions:
contents: read
packages: write
@ -78,16 +93,16 @@ jobs:
}
steps:
- name: Echo strategy
run: echo '${{ toJSON(fromJSON(needs.define-variables.outputs.build_matrix)) }}'
- name: Echo matrix
run: echo '${{ toJSON(matrix) }}'
- name: Display build matrix
run: |
echo "Strategy: ${{ toJSON(fromJSON(needs.prepare.outputs.build_matrix)) }}"
echo "Matrix: ${{ toJSON(matrix) }}"
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install rust
- name: Install Rust toolchain
id: rust-toolchain
uses: ./.forgejo/actions/rust-toolchain
@ -95,35 +110,34 @@ jobs:
uses: docker/setup-buildx-action@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Login to builtin registry
- name: Login to container registry
if: env.BUILTIN_REGISTRY != ''
uses: docker/login-action@v3
with:
registry: ${{ env.BUILTIN_REGISTRY }}
username: ${{ vars.BUILTIN_REGISTRY_USER || github.actor }}
password: ${{ secrets.BUILTIN_REGISTRY_PASSWORD || secrets.GITHUB_TOKEN }}
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
- name: Extract metadata (labels, annotations) for Docker
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{needs.define-variables.outputs.images}}
images: ${{needs.prepare.outputs.images}}
# default labels & annotations: https://github.com/docker/metadata-action/blob/master/src/meta.ts#L509
env:
DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,index
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see "[Usage](https://github.com/docker/build-push-action#usage)" in the README of the `docker/build-push-action` repository.
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
# It will not push images generated from a pull request
- name: Get short git commit SHA
id: sha
run: |
calculatedSha=$(git rev-parse --short ${{ github.sha }})
echo "Short SHA: $calculatedSha (from full SHA: ${{ github.sha }})"
echo "COMMIT_SHORT_SHA=$calculatedSha" >> $GITHUB_ENV
- name: Get Git commit timestamps
run: echo "TIMESTAMP=$(git log -1 --pretty=%ct)" >> $GITHUB_ENV
- name: Get commit timestamp
run: |
timestamp=$(git log -1 --pretty=%ct)
echo "Commit timestamp: $timestamp ($(date -d @$timestamp))"
echo "TIMESTAMP=$timestamp" >> $GITHUB_ENV
- uses: ./.forgejo/actions/timelord
with:
@ -160,7 +174,7 @@ jobs:
path: |
var-lib-apt-${{ matrix.slug }}
key: var-lib-apt-${{ matrix.slug }}
- name: inject cache into docker
- name: Inject build cache
uses: https://github.com/reproducible-containers/buildkit-cache-dance@v3.1.0
with:
cache-map: |
@ -176,7 +190,7 @@ jobs:
}
skip-extraction: ${{ steps.cache.outputs.cache-hit }}
- name: Build and push Docker image by digest
- name: Build Docker image
id: build
uses: docker/build-push-action@v6
with:
@ -193,26 +207,25 @@ jobs:
cache-from: type=gha
# cache-to: type=gha,mode=max
sbom: true
outputs: type=image,"name=${{ needs.define-variables.outputs.images_list }}",push-by-digest=true,name-canonical=true,push=true
outputs: type=image,"name=${{ needs.prepare.outputs.images_list }}",push-by-digest=true,name-canonical=true,push=true
env:
SOURCE_DATE_EPOCH: ${{ env.TIMESTAMP }}
# For publishing multi-platform manifests
- name: Export digest
- name: Export image digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Extract binary from container (image)
- name: Create container from image
id: extract-binary-image
run: |
mkdir -p /tmp/binaries
digest="${{ steps.build.outputs.digest }}"
echo "container_id=$(docker create --platform ${{ matrix.platform }} ${{ needs.define-variables.outputs.images_list }}@$digest)" >> $GITHUB_OUTPUT
- name: Extract binary from container (copy)
echo "container_id=$(docker create --platform ${{ matrix.platform }} ${{ needs.prepare.outputs.images_list }}@$digest)" >> $GITHUB_OUTPUT
- name: Extract binary from container
run: docker cp ${{ steps.extract-binary-image.outputs.container_id }}:/sbin/conduwuit /tmp/binaries/conduwuit-${{ matrix.target_cpu }}-${{ matrix.slug }}-${{ matrix.profile }}
- name: Extract binary from container (cleanup)
- name: Clean up container
run: docker rm ${{ steps.extract-binary-image.outputs.container_id }}
- name: Upload binary artifact
@ -230,9 +243,10 @@ jobs:
if-no-files-found: error
retention-days: 5
merge:
publish:
name: Publish Multi-platform Manifest
runs-on: dind
needs: [define-variables, build-image]
needs: [prepare, build]
steps:
- name: Download digests
uses: forgejo/download-artifact@v4
@ -240,8 +254,8 @@ jobs:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Login to builtin registry
- name: Login to container registry
if: env.BUILTIN_REGISTRY != ''
uses: docker/login-action@v3
with:
registry: ${{ env.BUILTIN_REGISTRY }}
@ -251,7 +265,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Extract metadata (tags) for Docker
- name: Extract Docker tags
id: meta
uses: docker/metadata-action@v5
with:
@ -263,15 +277,15 @@ jobs:
type=ref,event=pr
type=sha,format=long
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
images: ${{needs.define-variables.outputs.images}}
images: ${{needs.prepare.outputs.images}}
# default labels & annotations: https://github.com/docker/metadata-action/blob/master/src/meta.ts#L509
env:
DOCKER_METADATA_ANNOTATIONS_LEVELS: index
- name: Create manifest list and push
- name: Create and push manifest
working-directory: /tmp/digests
env:
IMAGES: ${{needs.define-variables.outputs.images}}
IMAGES: ${{needs.prepare.outputs.images}}
shell: bash
run: |
IFS=$'\n'
@ -287,7 +301,7 @@ jobs:
- name: Inspect image
env:
IMAGES: ${{needs.define-variables.outputs.images}}
IMAGES: ${{needs.prepare.outputs.images}}
shell: bash
run: |
IMAGES_LIST=($IMAGES)

View file

@ -0,0 +1,64 @@
name: Renovate
on:
schedule:
# Run at 2am UTC daily
- cron: '0 2 * * *'
workflow_dispatch:
inputs:
dryRun:
description: 'Dry run mode'
required: false
default: 'false'
type: choice
options:
- 'true'
- 'false'
logLevel:
description: 'Log level'
required: false
default: 'info'
type: choice
options:
- 'debug'
- 'info'
- 'warn'
- 'error'
push:
branches:
- main
paths:
- '.forgejo/workflows/renovate.yml'
- 'renovate.json'
jobs:
renovate:
name: Renovate
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Run Renovate
uses: renovatebot/github-action@v40.1.0
with:
token: ${{ secrets.RENOVATE_TOKEN }}
configurationFile: renovate.json
env:
# Platform settings
RENOVATE_PLATFORM: gitea
RENOVATE_ENDPOINT: ${{ github.server_url }}/api/v1
RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }}
# Repository settings
RENOVATE_REPOSITORIES: '["${{ github.repository }}"]'
# Behaviour settings
RENOVATE_DRY_RUN: ${{ inputs.dryRun || 'false' }}
LOG_LEVEL: ${{ inputs.logLevel || 'info' }}
# Forgejo/Gitea specific
RENOVATE_GIT_AUTHOR: 'Renovate Bot <renovate@noreply.${{ github.server_url }}>'
# PR settings
RENOVATE_PR_HOURLY_LIMIT: '2'
RENOVATE_PR_CONCURRENT_LIMIT: '3'

View file

@ -1,144 +0,0 @@
name: Checks / Rust
on:
push:
jobs:
format:
name: Format
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install rust
uses: ./.forgejo/actions/rust-toolchain
with:
toolchain: "nightly"
components: "rustfmt"
- name: Check formatting
run: |
cargo +nightly fmt --all -- --check
clippy:
name: Clippy
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install rust
uses: ./.forgejo/actions/rust-toolchain
- uses: https://github.com/actions/create-github-app-token@v2
id: app-token
with:
app-id: ${{ vars.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
github-api-url: https://api.github.com
owner: ${{ vars.GH_APP_OWNER }}
repositories: ""
- name: Install sccache
uses: ./.forgejo/actions/sccache
with:
token: ${{ steps.app-token.outputs.token }}
- run: sudo apt-get update
- name: Install system dependencies
uses: https://github.com/awalsh128/cache-apt-pkgs-action@v1
with:
packages: clang liburing-dev
version: 1
- name: Cache Rust registry
uses: actions/cache@v3
with:
path: |
~/.cargo/git
!~/.cargo/git/checkouts
~/.cargo/registry
!~/.cargo/registry/src
key: rust-registry-${{hashFiles('**/Cargo.lock') }}
- name: Timelord
uses: ./.forgejo/actions/timelord
with:
key: sccache-v0
path: .
- name: Clippy
run: |
cargo clippy \
--workspace \
--features full \
--locked \
--no-deps \
--profile test \
-- \
-D warnings
- name: Show sccache stats
if: always()
run: sccache --show-stats
cargo-test:
name: Cargo Test
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install rust
uses: ./.forgejo/actions/rust-toolchain
- uses: https://github.com/actions/create-github-app-token@v2
id: app-token
with:
app-id: ${{ vars.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
github-api-url: https://api.github.com
owner: ${{ vars.GH_APP_OWNER }}
repositories: ""
- name: Install sccache
uses: ./.forgejo/actions/sccache
with:
token: ${{ steps.app-token.outputs.token }}
- run: sudo apt-get update
- name: Install system dependencies
uses: https://github.com/awalsh128/cache-apt-pkgs-action@v1
with:
packages: clang liburing-dev
version: 1
- name: Cache Rust registry
uses: actions/cache@v3
with:
path: |
~/.cargo/git
!~/.cargo/git/checkouts
~/.cargo/registry
!~/.cargo/registry/src
key: rust-registry-${{hashFiles('**/Cargo.lock') }}
- name: Timelord
uses: ./.forgejo/actions/timelord
with:
key: sccache-v0
path: .
- name: Cargo Test
run: |
cargo test \
--workspace \
--features full \
--locked \
--profile test \
--all-targets \
--no-fail-fast
- name: Show sccache stats
if: always()
run: sccache --show-stats

View file

@ -9,7 +9,7 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-byte-order-marker
- id: fix-byte-order-marker
- id: check-case-conflict
- id: check-symlinks
- id: destroyed-symlinks

View file

@ -22,5 +22,24 @@
"tikv-jemalloc-ctl",
"opentelemetry-rust",
"tracing-opentelemetry"
]
],
"github-actions": {
"enabled": true,
"fileMatch": [
"(^|/)\\.forgejo/workflows/[^/]+\\.ya?ml$",
"(^|/)\\.forgejo/actions/[^/]+/action\\.ya?ml$",
"(^|/)\\.github/workflows/[^/]+\\.ya?ml$",
"(^|/)\\.github/actions/[^/]+/action\\.ya?ml$"
]
},
"packageRules": [
{
"description": "Group all non-major GitHub Actions updates",
"matchManagers": ["github-actions"],
"matchUpdateTypes": ["minor", "patch"],
"groupName": "github-actions-non-major"
}
],
"prConcurrentLimit": 3,
"prHourlyLimit": 2
}