Skip to content

CI: Better way of limiting CPU usage #10716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 9, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 25 additions & 10 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,26 @@ on:
env:
DOTTY_CI_RUN: true

# In this file, we set `--cpu-shares 4096` on every job. This might seem useless
# since it means that every container has the same weight which should be
# equivalent to doing nothing, but it turns out that OpenJDK computes
# `Runtime.getRuntime.availableProcessors` by dividing the cpu-shares value if
# it exists by 1024 (cf
# http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-January/036087.html),
# so this means that we effectively run every job with 4 cores. This is much
# nicer than setting `--cpus 4` because the latter enforces CPU quotas and ends
# up slowing our jobs more than needed. It's equivalent to running the JVM with
# `-XX:ActiveProcessorCount=4`, but since our tests can spawn new JVM in many
# places, it would be very hard to ensure that this option is always passed to
# `java` (we could use the `_JAVA_OPTIONS` environment variable, but this prints
# text on stderr and so can break tests which check the output of a program).

jobs:
test_non_bootstrapped:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -54,7 +68,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -139,7 +153,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -175,7 +189,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -211,7 +225,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -247,7 +261,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -290,7 +304,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -325,7 +339,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -367,7 +381,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -450,7 +464,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpus=4
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down Expand Up @@ -495,6 +509,7 @@ jobs:
runs-on: [self-hosted, Linux]
container:
image: lampepfl/dotty:2020-11-19
options: --cpu-shares 4096
volumes:
- ${{ github.workspace }}/../../cache/sbt:/root/.sbt
- ${{ github.workspace }}/../../cache/ivy:/root/.ivy2/cache
Expand Down