Skip to content

Copying MCAD CRDs in Kustomize config instead of referencing them #357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 20, 2023
Merged

Copying MCAD CRDs in Kustomize config instead of referencing them #357

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
fd00167
copying mcad crds in kustomize config instead of referencing them
Fiona-Waters Oct 19, 2023
07f6abf
addressing feedback
Fiona-Waters Oct 19, 2023
81400ae
build: split crds into their own files
dimakis Oct 20, 2023
2dffc9e
refactor: isntall yq if not exists
dimakis Oct 20, 2023
b2966ee
updating kustomization.yaml and verify_generated_files workflow
Fiona-Waters Oct 20, 2023
80835d3
remove restore
Fiona-Waters Oct 20, 2023
72524ca
adding crds
Fiona-Waters Oct 20, 2023
e1f3759
update check_yq target
Fiona-Waters Oct 20, 2023
79159cd
Update Makefile
dimakis Oct 20, 2023
29d544e
Update Makefile
dimakis Oct 20, 2023
f6fdbc3
Update Makefile
dimakis Oct 20, 2023
97efe89
commenting out lines
Fiona-Waters Oct 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/verify_generated_files.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,15 @@ on:
- '**.go'
- '**go.mod'
- '**go.sum'
- 'config/**'
tags-ignore:
- 'v*'
pull_request:
paths:
- '**.go'
- '**go.mod'
- '**go.sum'
- 'config/**'
jobs:
verify-imports:
runs-on: ubuntu-latest
Expand Down
39 changes: 34 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,9 +146,13 @@ defaults:

gofmt -w $(DEFAULTS_TEST_FILE)

# this encounters sed issues on MacOS, quick fix is to use gsed or to escape the parentheses i.e. \( \)
.PHONY: manifests
manifests: controller-gen ## Generate RBAC objects.
manifests: controller-gen kustomize ## Generate RBAC objects.
$(CONTROLLER_GEN) rbac:roleName=manager-role webhook paths="./..."
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

$(KUSTOMIZE) build config/crd/mcad > config/crd/mcad.yaml && make split_yaml FILE=config/crd/mcad.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont call this

git restore config/*

.PHONY: fmt
fmt: ## Run go fmt against code.
Expand Down Expand Up @@ -192,19 +196,16 @@ endif

.PHONY: install
install: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config.
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
$(KUSTOMIZE) build config/crd | kubectl apply -f -
git restore config/*

.PHONY: uninstall
uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
$(KUSTOMIZE) build config/crd | kubectl delete --ignore-not-found=$(ignore-not-found) -f -
git restore config/*

.PHONY: deploy
deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG}
$(KUSTOMIZE) build config/${ENV} | kubectl apply -f -
git restore config/*
Expand Down Expand Up @@ -284,7 +285,6 @@ validate-bundle: install-operator-sdk
.PHONY: bundle
bundle: defaults manifests kustomize install-operator-sdk ## Generate bundle manifests and metadata, then validate generated files.
$(OPERATOR_SDK) generate kustomize manifests -q
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
cd config/manifests && $(KUSTOMIZE) edit add patch --patch '[{"op":"add", "path":"/metadata/annotations/containerImage", "value": "$(IMG)" }]' --kind ClusterServiceVersion
cd config/manifests && $(KUSTOMIZE) edit add patch --patch '[{"op":"add", "path":"/spec/replaces", "value": "codeflare-operator.$(PREVIOUS_VERSION)" }]' --kind ClusterServiceVersion
Expand Down Expand Up @@ -388,3 +388,32 @@ verify-imports: openshift-goimports ## Run import verifications.
.PHONY: scorecard-bundle
scorecard-bundle: install-operator-sdk ## Run scorecard tests on bundle image.
$(OPERATOR_SDK) scorecard bundle


FILE ?= input.yaml # Default value, it isn't a file, but the make cmds fill hang for longer without it
temp_dir := temp_split
output_dir := 'config/crd/'

.PHONY: check_yq
check_yq:
@command -v yq >/dev/null 2>&1 || (echo "Installing wget..."; yum install -y wget)
Copy link
Contributor

@dimakis dimakis Oct 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be
@command -v wget

@command -v yq >/dev/null 2>&1 || (echo "Installing yq..."; wget https://github.com/mikefarah/yq/releases/download/v4.2.0/yq_linux_amd64.tar.gz -O - |\
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be YQ

tar xz && mv yq_linux_amd64 /usr/bin/yq)

# this works on a MacOS by replacing awk with gawk
.PHONY: split_yaml
split_yaml:
@$(MAKE) check_yq
@mkdir -p $(temp_dir)
@awk '/apiVersion: /{if (x>0) close("$(temp_dir)/section_" x ".yaml"); x++}{print > "$(temp_dir)/section_"x".yaml"}' $(FILE)
@$(MAKE) process_sections

.PHONY: process_sections
process_sections:
@mkdir -p $(output_dir)
@for section_file in $(temp_dir)/section_*; do \
metadata_name=$$(yq e '.metadata.name' $$section_file); \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you're calling YQ

file_name=$$(echo $$metadata_name | awk -F'.' '{print $$2"."$$3"_"$$1".yaml"}'); \
mv $$section_file $(output_dir)/$$file_name; \
done
@rm -r $(temp_dir)
5 changes: 4 additions & 1 deletion config/crd/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,8 @@
# since it depends on service name and namespace that are out of this kustomize package.
# It should be run by config/default
resources:
- mcad
- quota.codeflare_quotasubtrees.yaml
- workload.codeflare_appwrappers.yaml
- workload.codeflare_schedulingspecs.yaml

#+kubebuilder:scaffold:crdkustomizeresource
Loading