Skip to content

Optimize CLI image building #666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 91 additions & 6 deletions .github/workflows/build-and-run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ on:
branches: [main]
pull_request:
branches: [main]

env:
REGISTRY: ghcr.io
IMAGE_NAME: utbot_java_cli
DOCKERFILE_PATH: docker/Dockerfile_java_cli

jobs:
build_and_run_tests:
Expand All @@ -16,14 +21,30 @@ jobs:
java-version: '8'
distribution: 'zulu'
java-package: jdk+fx

- uses: gradle/gradle-build-action@v2
with:
gradle-version: 6.8

- name: Set project version
run: echo "VERSION="$(date +%Y).$(date +%-m)"" >> $GITHUB_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many double quotes look ambiguous


- name: Build and run tests in UTBot Java
run: |
export KOTLIN_HOME="/usr"
gradle clean build --no-daemon
gradle clean build --no-daemon -PsemVer=${{ env.VERSION }}

- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1
if: always()
with:
files: "**/build/test-results/**/*.xml"

- name: Cache utbot-cli-${{ env.VERSION }}.jar
uses: actions/cache@v3
with:
path: utbot-cli/build/libs/utbot-cli-${{ env.VERSION }}.jar
key: utbot-cli

- name: Upload utbot-framework logs
if: ${{ always() }}
Expand All @@ -39,9 +60,73 @@ jobs:
name: utbot_framework_tests_report
path: utbot-framework/build/reports/tests/test/*

- name: Upload utbot-intellij tests report artifacts if tests have failed
if: ${{ failure() }}
uses: actions/upload-artifact@v2

publish_cli_image:
needs: build_and_run_tests
if: ${{ github.event_name != 'pull_request' }}
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2

- name: Set timezone
uses: szenius/set-timezone@v1.0
with:
timezoneLinux: "Europe/Moscow"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need time zone here? And why is it set to Moscow?


- name: Set environment variables
run: |
echo "COMMIT_SHORT_SHA="$(git rev-parse --short HEAD)"" >> $GITHUB_ENV
echo "VERSION="$(date +%Y).$(date +%-m)"" >> $GITHUB_ENV

- uses: actions/cache@v3
with:
name: utbot_intellij_tests_report
path: utbot-intellij/build/reports/tests/test/*
path: utbot-cli/build/libs/utbot-cli-${{ env.VERSION }}.jar
key: utbot-cli

- name: Set docker tag
run:
echo "DOCKER_TAG="$(date +%Y).$(date +%-m).$(date +%-d)-${{ env.COMMIT_SHORT_SHA }}"" >> $GITHUB_ENV

- name: Log in to the Container registry
uses: docker/login-action@v1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-

- name: Docker meta
id: meta
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ github.repository }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=${{ env.DOCKER_TAG }}

- name: Docker Buildx (build and push)
run: |
docker buildx build \
-f ${{ env.DOCKERFILE_PATH }} \
--cache-from "type=local,src=/tmp/.buildx-cache" \
--cache-to "type=local,dest=/tmp/.buildx-cache-new" \
--tag ${{ steps.meta.outputs.tags }} \
--build-arg UTBOT_JAVA_CLI=utbot-cli/build/libs/utbot-cli-${{ env.VERSION }}.jar \
--push .
# Temp fix
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896

- name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the cache matter when we run on clean runner each time?

73 changes: 0 additions & 73 deletions .github/workflows/publish-cli-image.yml

This file was deleted.

21 changes: 4 additions & 17 deletions docker/Dockerfile_java_cli
Original file line number Diff line number Diff line change
@@ -1,25 +1,12 @@
FROM openjdk:8

ARG ACCESS_TOKEN
ARG UTBOT_JAVA_CLI

WORKDIR /usr/src/

RUN apt-get update \
&& apt-get install -y curl \
unzip \
python3 \
python3-requests \
&& apt-get clean

# Install UTBot Java CLI
COPY docker/get_java_cli_download_url.py .

ENV JAVA_CLI_ZIP_NAME "utbot_java_cli.zip"
COPY ${UTBOT_JAVA_CLI} .

RUN curl -H "Authorization: Bearer ${ACCESS_TOKEN}" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dockerfile looks much better

-L "$(python3 get_java_cli_download_url.py)" \
-o "${JAVA_CLI_ZIP_NAME}" \
&& unzip "${JAVA_CLI_ZIP_NAME}" \
&& rm "${JAVA_CLI_ZIP_NAME}" \
&& JAVA_CLI_PATH="$(find /usr/src -type f -name 'utbot-cli*')" \
&& ln -s "${JAVA_CLI_PATH}" /usr/src/utbot-cli.jar
RUN UTBOT_JAVA_CLI_PATH="$(find /usr/src -type f -name 'utbot-cli*')" \
&& ln -s "${UTBOT_JAVA_CLI_PATH}" /usr/src/utbot-cli.jar
14 changes: 0 additions & 14 deletions docker/get_java_cli_download_url.py

This file was deleted.