It's unclear what the version is supposed to be on this runner, but
work around any ambiguity by installing CMake explicitly. Previously,
the version was less than 3.30, as a CMake preset update requiring at
least 3.30 caused the workflow to fail.
Starting with CMake 3.30 and CMakePresets version 9, the include field
supports interesting macro expansions. We can now define platform
specific presets in separate files and include them in the top-level
CMakePresets.json, while keeping the same preset names across all
platforms. This avoids some preset explosion at the cost of some mostly
empty json files for each unix platform.
The CMake minimum required in the top-level and Lagom CMakeLists.txt
have not been adjusted in this patch, as that would have the effect of
changing the default policy versions and is a bit out of scope.
ENABLE_WINDOWS_CI and the *_CI presets were initially added back when
the AK library and all the AK Test* executables were the only targets
that supported building and running in CI. Since then, almost all the
targets in the codebase are built on Windows besides the following:
- LibLine
- test-262-runner
Since these targets above are not required to actually run or test the
browser on Windows in its current experimental state, fully disabling
them should be fine for now.
ENABLE_WINDOWS_CI was also used to exclude test-web from ctest. This
can be fully disabled on Windows for now until proper runtime support
is added.
The remaining locations were all using ENABLE_WINDOWS_CI as a proxy for
ENABLE_ADDRESS_SANITIZER, so we can just be explicit instead.
The new presets map much more directly to the unix Release, Debug, and
Sanitizer presets which should make setting up ladybird on Windows less
confusing.
We also make the new Windows_Experimental_Release preset the default in
ladybird.py to match Unix.
We discovered the issue with failed deployments was due to an update to
the checkout action that did not interact well with the deploy action.
There is a fix in the deploy action already. See:
1d137fdcc7
However, a tagged release for the deploy action is not yet available,
and pulling the action at a specific commit hash does not work.
For now, let's pin the checkout action at a known-good version. We can
update both actions when the deploy action has a tagged release. In the
meantime, this just removes the token with write-permissions from the
Ladybird checkout.
Our CI commit linter now evaluates commit messages line by line, similar
to `Meta/lint-commit.sh`.
The URL rule was updated to allow any freestanding URL, optionally
prefixed by whitespace.
We already check if CR is present in the first rule, and although it
would be nice to see all errors at once, we expect people to run
`Meta/lint-commit.sh` before pushing as well. This simplifies our
patterns a bit.
If we run js-benchmarks against older builds that did not yet include
the wasm repl, make sure we skip extracting the file and passing the
`--wasm-executable` argument.
Older commits also have older setup code, such as requiring older XCode
versions - which fails on newer macOS systems. Let's always use the
latest custom actions so we can retroactively support these older
builds.
This allows us to run js-benchmarks against older commits and have the
workflow correctly identify the commit used to build the binaries. In
order to actually build commits where the wasm binary was not yet built,
we have to account for missing archives as well.
The upstream workflow `head_sha` contains the wrong hash in case the
workflow was run manually against a provided commit hash. We use the
COMMIT file that cpack adds to the archive to determine the actual
commit hash that was used to build the binaries.
Note that this will only work for builds since the introduction of that
file, so we cannot process benchmarks for older commits, unfortunately.
Currently we default to GITHUB_REF for the checkout of js-benchmarks,
and if we've built up a large backlog of benchmark jobs to run, this can
mean we're running against a severely outdated version of js-benchmarks.
This new behavior allows us to see a failed js-benchmarks job, fix the
bug, and restart the failed job. This is important since we're testing a
specific commit and build from the upstream JS/WASM workflow.
If we ever want to rerun benchmarks for an older commit, e.g. if we
added new benchmarks that we would want historical performance data on,
we need to consider that some benchmarks might fail because of a bug or
missing feature. For manual runs of the upstream build, pass
`--continue-on-failure` so we simply skip these failing benchmarks.
This allows us to manually kick off jobs for the JS/WASM artifacts
build with an explicit reference to build, which is useful if we want to
generate benchmark results for older commits.
Note that this workflow does not actually produce benchmark results -
that's the job of another workflow. The webhook storing the benchmark
results only appends new benchmarks; existing ones will be kept.
Also note that the GitHub checkout action defaults to the default
reference if `inputs.reference_to_build` is not provided, such as in the
case of a push to the master branch.
This reverts commit 8d46a11748.
The updated webhook supports the newer output format of our
js-benchmarks tool, allowing us to store multiple result types per
benchmark and subtest combination.
We currently have the issue that our macOS builds are being hosted by
both Apple M1 and M4 runners. By default, these target `-mcpu=apple-m1`
and `-mcpu=apple-m3` respectively, causing their ccache artifacts to be
incompatible and effectively thrashing each other's ccache cache on each
run.
Add a new CMake option `ENABLE_CI_BASELINE_CPU` that toggles a baseline
`-mcpu/-march` compile option for each of the supported build platforms.
When disabled (the default), we now default to `-march=native` which on
Linux could theoretically lead to better performance.
The new CPU baseline is used by the `lagom-template.yml` and
`js-and-wasm-artifacts.yml` workflows, since they both produce artifacts
or caches that might be picked up by other runners. We also enable it
for Flatpak builds, so they target a fixed architecture instead of
whatever architecture the action runner that picked up the job has.
The output format for js-benchmarks is going to change, and while the
webhook parsing the results has not yet been updated pin the
js-benchmark checkout to a specific commit.
This will build both aarch64 and x86_64 flatpaks nightly. The test jobs
I did with the default GitHub runners took ~1h30m for aarch64 and
~2h30m for x86_64, but this will hopefully improve over time as the
cache is built up. And be better time-wise on the blacksmith runners.
Disk space is a large concern with these flatpak builds. A naive script
on my local machine showed that we needed a max of 10.3 GiB of free
space for the x86_64 build, with clang as the compiler. For some reason,
building with default gcc 14 uses more disk space than that, bumping
up against the default 14 GiB free space limit guaranteed by GitHub for
their default runners.
This was a cache race condition between the fuzzers and sanitizer
builds, where the vcpkg binary cache could have been updated before the
sanitizer builds started doing their vcpkg install, causing the source
assets to never be updated at all.
These existed because of their `vcpkg_ci` base preset, whose goal was to
enable vcpkg-supported caching on GitHub. We now handle vcpkg caching in
the CI steps, removing the necessity for all these *_CI preset variants.
This allows us to reuse `inputs.build_preset` in the test step, and we
can do away with the Windows-specific test step since it is now almost
identical to the Linux/macOS test step.
I've left in the Windows_CI presets, because that one actually results
in a different set of compiled files compared to the
Windows_Experimental presets.
I've set up a shared vcpkg source assets cache using Azurite. By
default, it runs in read-only mode allowing anyone who builds Ladybird
to use the cache to speed up downloads of source tarballs. This should
alleviate some of the more slower vcpkgs downloads I've been seeing
lately.
CI runs on master put vcpkg in readwrite mode, updating the cache for
everyone.
For macOS, we now default to ENABLE_QT=OFF and use the same build step
as for the other platforms. We keep a single separate macOS + Qt build
step to prevent bitrotting that part of the code.
The Windows step ran in a different directory and skipped the install
step; we can just use the Linux behavior here.
The CI runs are supposed to upload test dumps for failed LibWeb tests as
artifacts. However, the path for the artifact upload was wrong, it
likely regressed in 9a5b31ccd1.
Having the pull_request 'labeled' event trigger the main CI is much too
impactful, since it cancels running jobs and does not start anything if
the label is anything else than 'windows'. Let's go with a different
approach and put the Windows CI job in a separate workflow.
How this works:
* Jobs are only run if the 'windows' label is present on the PR.
* If a PR is opened or updated, existing jobs are canceled.
* If a PR is (un)labeled, existing jobs are only canceled if the label
was 'windows'. Other labels cause the job to be rescheduled.
As far as I can see, there's no way to prevent the job from being
rescheduled when labels other than 'windows' are being added or removed.
However, by not canceling the existing Windows job, we can try to create
a cache so the next run will be much quicker.
Each matrix item now defines whether it is enabled, and if not, the job
conditionally disables itself. Apparantly we cannot use expressions in
`matrix.exclude`, but it should work in the items' definitions.
This has the added benefit that this makes the job show up as "skipped",
which could serve as a hint that it could be run on a PR.
Our CI matrix now includes Windows if the 'windows' label is added to
the PR. The job still takes too long to include it for every PR, but it
is certainly useful to have it run on Windows-specific PRs and PRs that
might break Windows builds.
We were seeing lower ccache hitrates using Blacksmith's cache, and after
some investigation it turned out that actions running on our `master`
branch were pulling the caches from PRs. Blacksmith is looking into
this - for now revert to GitHub's cache until the issue is resolved.
Github Actions just updated windows-2025 to LLVM 20, which is the
minimum version required for us to build and run tests with sanitizers
Now that we've added support, enable the Sanitizer build in CI.
This makes it easier to ensure everyone is using the same version of
Swift when building the project, especially for CI environments.
`swiftly install` will automatically read this file and install the
specified version if it is not already installed. It also tells swiftly
what version to use for the project, independent of the global default
version.
As we now are only officially supporting the Ninja generator for
Windows, the old preset structure can now be simplified. Also, we
now follow the naming conventions of the non-hidden presets.
We could use flake8 for linting, but ruff is compatible with black
formatting out-of-the-box. It also seems to catch more than flake8,
such as unnecessary f-strings.
By default, matrix jobs generate a name for themselves by concatenating
their job name and all matrix variables into a big string. Changing the
runner labels causes the job name to change, which means we need to go
into GitHub and change the required checks since those are name-based.
Give all matrix workflows an explicit, more stable name.
The GitHub-provided runner frequently times out and is plain slow most
of the time. And since Blacksmith does not yet offer macOS runners,
let's use our self-hosted runner that is currently idle most of the time
(when it's not running JS benchmarks).
This is now always a JSON array of runner labels, inside a string. We
need it to be a string because `workflow_call` works with inputs that
do not natively support an array type.
No functional changes.
We haven't built a universal binary in over 11 months. The name is
confusing, and actually breaks esvu on macOS. The fact that nobody
has complained suggests that this is not a common use case..
The way we echo the cache keys for output variables strips these quotes.
So when we save the caches at the end of CI, the keys are sans quotes.
This patch adds an extra cache restore key without quotes to allow jobs
to fetch their own caches again. This will become moot once all runners
are able to use the Blacksmith cache action.
Swift jobs were failing a test due to the llvm-symbolizer not being
available in the default location next to the clang binary. swift.org
toolchains don't ship this tool, so LSAN suppressions were not being
applied, failing TestWOFF2.
This was hard to reproduce locally, because I have always had a set of
alternatives set up for the full suite of LLVM tools on my machine.
Their cache action only works on their runners. For jobs that run on
other runners, we have use the default cache action. At least until they
update their cache product to work or fallback on other runners.
Everything that's not self-hosted or macOS is now pointing to
Blacksmith.sh. Nightly jobs and JS artifact builds use 8VCPU machines,
while regular integration builds & tests use 16VCPU machines.
The people over at Blacksmith.sh have generously offered usage of their
runners for our organization, so let's try to switch over some simple
workflows. The runners should be drop-in replacements.
When we try to retrieve benchmark results in the webhook call, we cannot
use the `head_sha` parameter since the workflow run might have a
different `head_sha` associated with it than the upstream workflow run.
This can happen when the JS repl binary workflow runs, a new commit is
pushed to master, followed by a JS benchmarks workflow run causing this
latter run to be associated with a different commit ID.
This extends the webhook payload to include the current run ID, which
can eventually be used by the webhook script to specifically download
the benchmark results associated with the current run.
Additionally, this changes the JS artifact download to use the upstream
run ID which seems nicer to do anyway.
This job uses the windows_ci_ninja preset to build just the
components and unit tests that are known to work with ClangCL on the
amd64-pc-windows-msvc target triple.
As a nightly job, its failures are non-blocking for any PRs, though
they should be fixed eventually or the job will get turned off by
email-annoyed maintainers.
For our js-benchmarks and libjs-test262 workflow runs, we already know
that they're provisioned with these repositories and can skip adding the
key and repo altogether.
We were using both wget and curl arbitrarily; use curl exclusively since
that is installed by default on our machines and containers. Fixes the
js-benchmarks workflow.
This is a workaround for the deprecation of the cache v1 REST API that
was replaced with a new protobuf RPC based API this month. vcpkg was
using the private cache backend API without the knowledge of the GitHub
actions team, and was thus broken by the deprecation.
While we wait for Microsoft to talk to Microsoft to get this feature
restored, we can use the raw actions/cache step to get almost the same
cache behavior. The only difference is that the cache will be less fine
grained than the per-package cache that VCPKG_BINARY_SOURCES of x-gha
was giving us before.
Chaining workflows does not cause the subsequently spawned workflow runs
to use the same event, but rather it uses the latest head SHA based on
the branch it runs on. This would cause the JS benchmarks jobs to not be
able to find artifacts (if a new JS repl workflow was started before the
previous one could finish) and/or assign the wrong commit SHA to the
benchmark results.
Since `github.event` contains information about the original workflow
run that spawned the JS benchmarks jobs, we can take the commit SHA from
there and use it to download the correct artifact.
We had concurrency set on the JS artifacts and JS benchmarks workflows
causing them to not run in parallel for the same combination of
(workflow, OS name). You'd expect that this causes a FIFO queue to exist
of the jobs to run sequentially, but in reality GitHub maintains a
single job to prioritize and cancels all others. We don't want that for
our artifacts and benchmarks: we want them to run on each push.
For example, a new push could have workflows getting cancelled because
someone restarted a previously failed workflow, resulting in the
following message:
"Canceling since a higher priority waiting request for [..] exists"
By removing the concurrency setting from these workflows, we make use of
all available runners to execute the jobs and potentially run some of
them in parallel. For the benchmarks however, we currently only have one
matching self-hosted runner per job, and as such they are still not run
in parallel.
This introduces a matrix for the js-benchmarks workflow and runs both
the Linux x86_64 and macOS arm64 JS repl builds against our benchmarks
repository.
The workflow-webhook action that was being used didn't work on macOS or
machines without Docker, so let's create the payload ourselves, sign it
and send it over using plain old `curl`.
It might be useful to have these artifacts, even for older commits. As
an added bonus, this causes the JS benchmarks to run as well giving us
more datapoints.
In practice this does not make a big difference, but technically it
could happen that a second JS Repl artifact was built before the first
JS Benchmarks job is executed. So make sure to filter on commit ID.
This workflow starts after a successful js-artifacts workflow, picks up
the JS repl binary and runs our js-benchmarks tool. It does not yet
publish or otherwise store the benchmark results, but it's a start!
This reverts commits bf333eaea2 and
6f69a445bd.
The commit linter needs to run on event `pull_request_target` to have
access to its secret token, which means we cannot have a dependency on
that job from another workflow that is run as a result of the
`pull_request` event.
Additionally, the linters were no longer run for first-time
contributors. This isn't a huge problem but it was nice that a
preliminary check took place before running the full CI on their PRs.
This requires us to always run the CI job and check the individual jobs'
results, since only having `needs:` will not work when `lint_commits` is
potentially skipped.
We can prevent running builds unnecessarily by requiring the linters to
succeed first. If either the code or commit linter fails, it means the
author of the PR needs to rework their branch and after pushing their
changes, we need to do a full new CI run anyway.
