Skip to content

Commit 61d65a9

Browse files
z103cbkevin85421asm582Maxusmustianishasthana
authored
Upgrade the golang version for the quota-management branch. (#313)
* update (#298) * Upgraded the golang version Updated travis config for the new version Updated build.sh scripts for the new version * Updates to go.mod Updates to travis.yml, to set dist Added the generated.deepcopy files * Removed unused package. * e2e test script updates * Queue.go updates Other fixes to the makefile and e2e test * Updates to the e2e script Updates to util.go to allow for different images to be used. * Updates to to local host setup * Changes to the run-e2e-kind.sh locally (#290) * Changes to the run-e2e-kind.sh to * Changed out echo server image, small edits * Backed out change in the clean up code. * Updated the makefile to build the executable * Address code review comments. * * Small changes to the e2e script * Build documentation updates. * Documentation updates per code review Fixed errors in the e2e script to load ubuntu:latest image correctly. * Cleaned up false errors Rectified use of undefined variable. * Downgraded go to version 1.18 to allow for use of ubi images. * Downgraded travis ubuntu distro * Added use of docker builder using ubi8 images Makefile updates * Cleaned up the obsolete build scripts Updated documentation Minor changes to makefile * avoid preempt of failed co-schd pods (#317) * Update CONTROLLER_VERSION * Updates to travis and devcontainer * Small updates to the run-e2e test * Updated go-restful dependency * More go.mod updates debug messages for the e2e script. * Added back in the removal of taints from head node Cleanup of junk files. * Remove whitesapce * Small corrections. * Refactoring of env-setup * moe arch issues with the kuttl binnary * Updated the go version in go.mod (#316) * Updated the go version in go.mod Ran `go mod tidy` Updated travis.yml Deleted staled code, disabled the quota manager in the controller. * Added travis changes. * Removed linter install Ported changes to .travis.yml from quota-management-branch * Removed linter install Ported changes to .travis.yml from quota-management-branch * Downgraded go to version 1.18 to allow for use of ubi images. * Added use of docker builder using ubi8 images Makefile updates * Cleaned up the obsolete build scripts Updated documentation Minor changes to makefile * Small fix to the docker file. * More updates to go.mod to fix vulnerabilities * Removed text files. * Yet another text file removal * Small fix to address PR review comments. * Create 1.29.57 release Signed-off-by: Anish Asthana <anishasthana1@gmail.com> * Adding Performance test script (#289) * Adding Performance test script Signed-off-by: James Busche <jbusche@us.ibm.com> * red hat image and more checking Signed-off-by: James Busche <jbusche@us.ibm.com> * Check for mcad install Signed-off-by: James Busche <jbusche@us.ibm.com> * remove oc and formatting Signed-off-by: James Busche <jbusche@us.ibm.com> * fix to run from any namespace Signed-off-by: James Busche <jbusche@us.ibm.com> --------- Signed-off-by: James Busche <jbusche@us.ibm.com> * fix typo (#326) * Document updates. * remove invalid characters from branch name an replace with - (#328) Signed-off-by: Kevin <kpostlet@redhat.com> * Bump golang.org/x/text from 0.3.4 to 0.3.8 (#321) Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.4 to 0.3.8. - [Release notes](https://github.com/golang/text/releases) - [Commits](golang/text@v0.3.4...v0.3.8) --- updated-dependencies: - dependency-name: golang.org/x/text dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump golang.org/x/net from 0.0.0-20210405180319-a5a99cb37ef4 to 0.7.0 (#324) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20210405180319-a5a99cb37ef4 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/commits/v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/gogo/protobuf from 1.3.1 to 1.3.2 (#323) Bumps [github.com/gogo/protobuf](https://github.com/gogo/protobuf) from 1.3.1 to 1.3.2. - [Release notes](https://github.com/gogo/protobuf/releases) - [Commits](gogo/protobuf@v1.3.1...v1.3.2) --- updated-dependencies: - dependency-name: github.com/gogo/protobuf dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added .devcontainer to gitignore Removed directory from commit. * Codereview comments. * Bump golang.org/x/crypto from 0.0.0-20201002170205-7f63de1d35b0 to 0.1.0 (#322) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20201002170205-7f63de1d35b0 to 0.1.0. - [Release notes](https://github.com/golang/crypto/releases) - [Commits](https://github.com/golang/crypto/commits/v0.1.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fixes to the setup part of the script. * change schedSpec to schedulingSpec and note M1 image for echo server in tutorial (#333) Signed-off-by: Sunyanan Choochotkaew <sunyanan.choochotkaew1@ibm.com> * Changes user to 1000 (#329) * non-root 1000 user Signed-off-by: James Busche <jbusche@us.ibm.com> * cleanup dockerfile Signed-off-by: James Busche <jbusche@us.ibm.com> * moving curl into builder Signed-off-by: James Busche <jbusche@us.ibm.com> --------- Signed-off-by: James Busche <jbusche@us.ibm.com> * Address race condition in XController variable (#330) * Update Dockerfile: * To not to use root user to build * To allow for passing of golang build args Updated Makefile: * To selectively run the generate code * To allow for passing of golang build args * Updates to ensure that docker builds flags are used Update to the e2e script to allow for not caching test runs and cluster setup * Updates to e2e script and travis for log level Fixed 1 race condition issue. * Reverted needless changes. * Documentation updates. Minor fixes to make file. * Fixed bug in docker file Reduced noise in e2e test. * Fixed potential regression. * Added back Dockerfile * Fix bad merge. --------- Signed-off-by: Anish Asthana <anishasthana1@gmail.com> Signed-off-by: James Busche <jbusche@us.ibm.com> Signed-off-by: Kevin <kpostlet@redhat.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Sunyanan Choochotkaew <sunyanan.choochotkaew1@ibm.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Abhishek Malvankar <asm582@users.noreply.github.com> Co-authored-by: Mustafa Eyceoz <maxusmusti@gmail.com> Co-authored-by: Anish Asthana <anishasthana1@gmail.com> Co-authored-by: James Busche <101845348+jbusche@users.noreply.github.com> Co-authored-by: Lionel Villard <villard@us.ibm.com> Co-authored-by: Kevin Postlethwait <k.postlethwait24@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sunyanan Choochotkaew <sunya-ch@ist.osaka-u.ac.jp>
1 parent 0601164 commit 61d65a9

38 files changed

+1068
-1630
lines changed

‎.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,3 +125,9 @@ kubernetes.tar.gz
125125
/bazel-*
126126
*.pyc
127127

128+
# .devcontainer files
129+
.devcontainer
130+
131+
# .devcontainer files
132+
.devcontainer
133+

‎.travis.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,15 @@ services:
99
- docker
1010

1111
go:
12-
- "1.16.3"
12+
- "1.18.10"
1313

1414
go_import_path: github.com/project-codeflare/multi-cluster-app-dispatcher
1515

16-
install:
17-
- go get -u golang.org/x/lint/golint
16+
install: []
17+
1818

1919
before_script:
20-
- export TEST_LOG_LEVEL=4
20+
- export TEST_LOG_LEVEL=2
2121

2222
script:
2323
- BLUE='\033[34m'

‎CONTROLLER_VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.29.55
1+
1.29.58

‎Dockerfile

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
FROM registry.access.redhat.com/ubi8/go-toolset:1.18.10-1 AS BUILDER
2+
ARG GO_BUILD_ARGS
3+
WORKDIR /workdir
4+
USER root
5+
6+
COPY Makefile Makefile
7+
COPY go.mod go.mod
8+
COPY go.sum go.sum
9+
COPY cmd cmd
10+
COPY pkg pkg
11+
COPY hack hack
12+
COPY CONTROLLER_VERSION CONTROLLER_VERSION
13+
14+
RUN cd /workdir && curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/$(go env GOARCH)/kubectl && chmod +x kubectl
15+
ENV GO_BUILD_ARGS=$GO_BUILD_ARGS
16+
RUN echo "Go build args: $GO_BUILD_ARGS" && \
17+
make mcad-controller
18+
19+
FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
20+
21+
COPY --from=BUILDER /workdir/_output/bin/mcad-controller /usr/local/bin
22+
COPY --from=BUILDER /workdir/kubectl /usr/local/bin
23+
24+
RUN true \
25+
&& microdnf update \
26+
&& microdnf clean all \
27+
&& true
28+
29+
WORKDIR /usr/local/bin
30+
31+
RUN chown -R 1000:1000 /usr/local/bin
32+
33+
USER 1000

‎Makefile

Lines changed: 35 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@ VERSION_FILE=./CONTROLLER_VERSION
44
RELEASE_VER=v$(shell $(CAT_CMD) $(VERSION_FILE))
55
CURRENT_DIR=$(shell pwd)
66
GIT_BRANCH:=$(shell git symbolic-ref --short HEAD 2>&1 | grep -v fatal)
7+
#define the GO_BUILD_ARGS if you need to pass additional arguments to the go build
8+
GO_BUILD_ARGS?=
9+
710
# Reset branch name if this a Travis CI environment
811
ifneq ($(strip $(TRAVIS_BRANCH)),)
912
GIT_BRANCH:=${TRAVIS_BRANCH}
@@ -18,14 +21,23 @@ endif
1821
# Check for current branch name
1922
ifneq ($(strip $(GIT_BRANCH)),)
2023
TAG:=${TAG}${GIT_BRANCH}-
24+
25+
# replace invalid characters that might exist in the branch name
26+
TAG:=$(shell echo ${TAG} | sed 's/[^a-zA-Z0-9]/-/g')
2127
endif
2228
TAG:=${TAG}${RELEASE_VER}
2329

2430
.PHONY: print-global-variables
2531

32+
# Build the controler executable for use in docker image build
2633
mcad-controller: init generate-code
34+
ifeq ($(strip $(GO_BUILD_ARGS)),)
2735
$(info Compiling controller)
28-
CGO_ENABLED=0 GOARCH=amd64 go build -o ${BIN_DIR}/mcad-controller ./cmd/kar-controllers/
36+
CGO_ENABLED=0 go build -o ${BIN_DIR}/mcad-controller ./cmd/kar-controllers/
37+
else
38+
$(info Compiling controller with build arguments: '${GO_BUILD_ARGS}')
39+
go build $(GO_BUILD_ARGS) -o ${BIN_DIR}/mcad-controller ./cmd/kar-controllers/
40+
endif
2941

3042
print-global-variables:
3143
$(info "---")
@@ -34,6 +46,7 @@ print-global-variables:
3446
$(info " "GIT_BRANCH="$(GIT_BRANCH)")
3547
$(info " "RELEASE_VER="$(RELEASE_VER)")
3648
$(info " "TAG="$(TAG)")
49+
$(info " "GO_BUILD_ARGS="$(GO_BUILD_ARGS)")
3750
$(info "---")
3851

3952
verify: generate-code
@@ -48,32 +61,42 @@ verify-tag-name: print-global-variables
4861
# Check for invalid tag name
4962
t=${TAG} && [ $${#t} -le 128 ] || { echo "Target name $$t has 128 or more chars"; false; }
5063

51-
generate-code:
52-
$(info Compiling deepcopy-gen...)
53-
go build -o ${BIN_DIR}/deepcopy-gen ./cmd/deepcopy-gen/
64+
generate-code: pkg/apis/controller/v1beta1/zz_generated.deepcopy.go
65+
66+
pkg/apis/controller/v1beta1/zz_generated.deepcopy.go: ${BIN_DIR}/deepcopy-gen
5467
$(info Generating deepcopy...)
5568
${BIN_DIR}/deepcopy-gen -i ./pkg/apis/controller/v1beta1/ -O zz_generated.deepcopy
5669
${BIN_DIR}/deepcopy-gen -i ./pkg/apis/quotaplugins/quotasubtree/v1 -O zz_generated.deepcopy
5770

58-
images: verify-tag-name
71+
${BIN_DIR}/deepcopy-gen:
72+
$(info Compiling deepcopy-gen...)
73+
go build -o ${BIN_DIR}/deepcopy-gen ./cmd/deepcopy-gen/
74+
75+
images: verify-tag-name generate-code
5976
$(info List executable directory)
6077
$(info repo id: ${git_repository_id})
6178
$(info branch: ${GIT_BRANCH})
62-
ls -l ${CURRENT_DIR}/_output/bin
6379
$(info Build the docker image)
64-
docker build --quiet --no-cache --tag mcad-controller:${TAG} -f ${CURRENT_DIR}/deployment/Dockerfile.both ${CURRENT_DIR}/_output/bin
80+
ifeq ($(strip $(GO_BUILD_ARGS)),)
81+
docker build --quiet --no-cache --tag mcad-controller:${TAG} -f ${CURRENT_DIR}/Dockerfile ${CURRENT_DIR}
82+
else
83+
docker build --no-cache --tag mcad-controller:${TAG} --build-arg GO_BUILD_ARGS=$(GO_BUILD_ARGS) -f ${CURRENT_DIR}/Dockerfile ${CURRENT_DIR}
84+
endif
6585

66-
images-podman: verify-tag-name
86+
images-podman: verify-tag-name generate-code
6787
$(info List executable directory)
6888
$(info repo id: ${git_repository_id})
6989
$(info branch: ${GIT_BRANCH})
70-
ls -l ${CURRENT_DIR}/_output/bin
7190
$(info Build the docker image)
72-
podman build --quiet --no-cache --tag mcad-controller:${TAG} -f ${CURRENT_DIR}/deployment/Dockerfile.both ${CURRENT_DIR}/_output/bin
91+
ifeq ($(strip $(GO_BUILD_ARGS)),)
92+
podman build --quiet --no-cache --tag mcad-controller:${TAG} -f ${CURRENT_DIR}/Dockerfile ${CURRENT_DIR}
93+
else
94+
podman build --no-cache --tag mcad-controller:${TAG} --build-arg GO_BUILD_ARGS=$(GO_BUILD_ARGS) -f ${CURRENT_DIR}/Dockerfile ${CURRENT_DIR}
95+
endif
7396

7497
push-images: verify-tag-name
7598
ifeq ($(strip $(quay_repository)),)
76-
$(info No registry information provide. To push images to a docker registry please set)
99+
$(info No registry information provided. To push images to a docker registry please set)
77100
$(info environment variables: quay_repository, quay_token, and quay_id. Environment)
78101
$(info variables do not need to be set for github Travis CICD.)
79102
else
@@ -94,7 +117,7 @@ run-test:
94117
$(info Running unit tests...)
95118
hack/make-rules/test.sh $(WHAT) $(TESTS)
96119

97-
run-e2e: mcad-controller verify-tag-name
120+
run-e2e: verify-tag-name
98121
ifeq ($(strip $(quay_repository)),)
99122
echo "Running e2e with MCAD local image: mcad-controller ${TAG} IfNotPresent."
100123
hack/run-e2e-kind.sh mcad-controller ${TAG} IfNotPresent
@@ -103,13 +126,8 @@ else
103126
hack/run-e2e-kind.sh ${quay_repository}/mcad-controller ${TAG}
104127
endif
105128

106-
mcad-controller-private: init generate-code
107-
$(info Compiling controller)
108-
CGO_ENABLED=0 GOARCH=amd64 GOPRIVATE=github.ibm.com/* go build -tags private -modfile ./private.mod -o ${BIN_DIR}/mcad-controller ./cmd/kar-controllers/
109-
110129
coverage:
111130
# KUBE_COVER=y hack/make-rules/test.sh $(WHAT) $(TESTS)
112131

113132
clean:
114133
rm -rf _output/
115-
rm -f mcad-controllers

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Multi-Cluster App Dispatcher
22
==
33

4-
The `multi-cluster-app-dispatcher` is a Kubernetes controller providing mechanisms for applications to manage batch jobs in a single or mult-cluster environment.
4+
The `multi-cluster-app-dispatcher` is a Kubernetes controller providing mechanisms for applications to manage batch jobs in a single or multi-cluster environment.
55

66
The `multi-cluster-app-dispatcher` (`MCAD`) controller is capable of (i) providing an abstraction for wrapping all resources of the job/application and treating them holistically, (ii) queuing job/application creation requests and applying different queuing policies, e.g., First In First Out, Priority, (iii) dispatching the job to one of multiple clusters, where a `MCAD` queuing agent runs, using configurable dispatch policies, and (iv) auto-scaling pod sets, balancing job demands and cluster availability (future work).
77

‎deployment/Dockerfile.both

Lines changed: 0 additions & 21 deletions
This file was deleted.

‎deployment/build-inside-container-private.sh

Lines changed: 0 additions & 31 deletions
This file was deleted.

‎deployment/build-inside-container.sh

Lines changed: 0 additions & 15 deletions
This file was deleted.

‎deployment/build-podman.sh

Lines changed: 0 additions & 9 deletions
This file was deleted.

‎deployment/build-private.sh

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎deployment/build.sh

Lines changed: 0 additions & 9 deletions
This file was deleted.

‎deployment/image-podman.sh

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎deployment/image.sh

Lines changed: 0 additions & 8 deletions
This file was deleted.

0 commit comments

Comments
 (0)