This is work in progress
Introduction
The helm manager is a service for managing helm charts. The service provides a REST API for onboarding of charts as well as installation and uninstallation of applications based on these charts.
In addition to the helm manager, a chart repository is used to store the available. charts for onboarding and installation.
The Helm manager (and chart repo) can be executed in one of the following deployments:
- As docker container on a local machine with kubernbetes
- As kubernetes services/pods on a local machine with kubernetes
- As kubernetes services/pods on a kubernetes cluster
Preparation
Clone the nonrtric repo from gerrit and change dir. Make sure to use the correct branch when cloning.
All needed scripts for start and run the helm manager, in docker or kubernetes are in this directory.
If the Helm Manager shall be installed in a kubernetes cluster the actions below shall be made on a node in cluster.
$ git clone "https://gerrit.o-ran-sc.org/r/nonrtric" -b <branch> $ cd helm-manager
Build image
There is no need to build the image for the helm manager unless changes are made to the Dockerfile in the current dir.
If no changes are needed, skip this section.
Instruction to build the image:
$ mvn clean package
The built image should be named: o-ran-sc/nonrtric-helm-manager:1.0.0-SNAPSHOT
$ docker images | grep helm o-ran-sc/nonrtric-helm-manager 1.0.0-SNAPSHOT 56e50ade8c37 3 minutes ago 498MB
Note: Note, replace the image name in the docker-hm.sh and kube-hm.sh if this image shall be used.
Note: Locally built images are not available to a kubernetes cluster unless the image is made available in an image repo accessible from within the cluster.
Create helm chart for test
Create a helm for this test. Package the chart into an archive
$ helm create simple-app Successfully packaged chart and saved it to: /nonrtric/helm-manager/simple-app-0.1.0.tgz $ helm package simple-app
Run in docker with local kubernetes
The helm-manger is possible to run as a docker container. However, a local kubernetes must be running where the application can be installed.
Create a private docker network, unless it already exists, for the containers to run in.
$ docker network create nonrtric-docker-net
Start the chartmuseum container in a separate window. This will be used as a chart repository. Make note of the port, 8222, which will be available on local host. Charts uploaded to the chartmuseum container will be availed in the mounted dir 'charts' on your host.
$ ./docker-cm.sh
Add the chart, created in the section 'Create helm for test', to the repo
$ curl --data-binary "@simple-app-0.1.0.tgz" -X POST http://localhost:8222/api/charts {"saved":true}
Start the helm manager in a separate window. Make note of the port, 8112, which will be available on local host. This is the port to the Helm manager REST API.
$ ./docker-cm.sh
The chartmusem repo need to added to helm. This operation must be called with a url accessible from the helm manager container.
Go into the helm manager container and add the repo.
$ docker exec -it helmmanagerservice sh # helm repo add cm http://chartmuseum:8080 "cm" has been added to your repositories $ exit
The helm manager is now running and configured with a chart repo.
Run the script test.sh to execute the sequence for installing the application 'simpleapp' namespace 'ckhm':
- Namespace 'ckhm' is created in kubernetes if not existing
- Onboard chart
- Install chart
- Uninstall chart
- Remove (the onboarded) chart
All operations should report "OK".
$ ./test.sh docker
Run in kubernetes
This instruction is valid for running both in a local kubernetes and in a kubernetes ccluster
If running in a local kubernetes, set the env KUBE_HOST to localhost
$ KUBE_HOST="localhost"
If running in a kubernetes cluster, set the env KUBE_HOST to the ip of the cluster.
$ kubectl cluster-info Kubernetes master is running at https://10.2.0.103:6443 $ KUBE_HOST=10.2.0.103
Check if the nonrtric names space exists. If not, create the namespace
$ kubectl get ns nonrtric $ kubectl create ns nonrtric
Start the chartmuseum service and pod
$ kubectl apply -f kube-cm.yaml
Add the chart, created in the section 'Create helm for test', to the chartmuseum repo. The node port of the chartmuseum service is obtained and env var CM_PORT is assigned that port number.
$ CM_PORT=$(kubectl get svc chartrepo -n nonrtric -o jsonpath='{...ports[?(@.name=="'http'")].nodePort}') $ curl --data-binary @simple-app-0.1.0.tgz -X POST http://$KUBE_HOST:$CM_PORT/api/charts {"saved":true}
Create a service account for the helm manager. This example service account bind to the "cluster-admin" role which normally has full permissions to the add/change/read/delete any kubernetes object. It is advisable to bind the service account to a ClusterRole with less permissions if desired.
$ kubectl apply -f helm-manager-sa.yaml serviceaccount/helm-manager-sa created clusterrolebinding.rbac.authorization.k8s.io/helm-manager-sa-clusterrolebinding created
Start the helm manager. Four objects will be created.
$ kubectl apply -f helm-manager.yaml service/helmmanagerservice created pod/helmmanagerservice created persistentvolume/helm-manager-service-pv created persistentvolumeclaim/helm-manager-service-pvc created
The chartmuseum repo need to added to helm. This operation must be called with a url accessible from the helm manager pod.
Go into the helm manager container and add the repo.
$ kubectl exec -it helmmanagerservice -n nonrtric -- sh # helm repo add cm http://chartrepo.nonrtric:8080 "cm" has been added to your repositories $ exit
The helm manager is now running and configured with a chart repo.
Run the script test.sh to execute the sequence for installing the application 'simpleapp' namespace 'ckhm':
- Namespace 'ckhm' is created in kubernetes if not existing
- Onboard chart
- Install chart
- Uninstall chart
- Remove (the onboarded) chart
All operations should report "OK".
If running in a local kubernetes
$ ./test.sh kube-local
if running in a kubernetes cluster. Use the previously set env var KUBE_HOST as second arg to the script
$ ./test.sh kube-cluster $KUBE_HOST
Example output of the script
Start test ================ Get apps - empty ================ curl -sw %{http_code} http://localhost:32743/helm/charts Curl OK Response: 200 Body: {"charts":[]} ============ Onboard app =========== curl -sw %{http_code} http://localhost:32743/helm/charts -X POST -F chart=@simple-app-0.1.0.tgz -F values=@simple-app-values.yaml -F info=<simple-app.json Curl OK Response: 200 Body: ===================== Get apps - simple-app ===================== curl -sw %{http_code} http://localhost:32743/helm/charts Curl OK Response: 200 Body: {"charts":[{"releaseName":"simpleapp","chartName":"simple-app","version":"0.1.0","namespace":"ckhm","repository":"cm"}]} =========== Install app =========== curl -sw %{http_code} http://localhost:32743/helm/install -X POST -H Content-Type:application/json -d @simple-app-installation.json Curl OK Response: 201 Body: ===================== Get apps - simple-app ===================== curl -sw %{http_code} http://localhost:32743/helm/charts Curl OK Response: 200 Body: {"charts":[{"releaseName":"simpleapp","chartName":"simple-app","version":"0.1.0","namespace":"ckhm","repository":"cm"}]} ============================= helm ls to list installed app ============================= NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION hm-app hm-app-ns 1 2021-05-19 17:15:26.068017436 +0000 UTC deployed hm-app-0.1.0 1.16.0 kall1 default 2 2021-05-16 21:32:46.710349 +0200 CEST deployed app1-0.1.0 1.16.0 kall11 default 1 2021-05-16 21:40:57.073819 +0200 CEST deployed app1-0.1.0 1.16.0 kalle3 default 1 2021-05-17 01:17:03.671637 +0200 CEST deployed app3-0.1.0 1.16.0 kalle3 app3-ns 1 2021-05-17 01:22:29.514917 +0200 CEST deployed app3-0.1.0 1.16.0 simpleapp ckhm 1 2021-06-01 22:24:12.797624073 +0000 UTC deployed simple-app-0.1.0 1.16.0 ========================================== sleep 30 - give the app some time to start ========================================== ============================ List svc and pod of the app ============================ NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE simpleapp-simple-app ClusterIP 10.105.1.129 <none> 80/TCP 30s NAME READY STATUS RESTARTS AGE simpleapp-simple-app-858c798f97-k6vsl 1/1 Running 0 30s ======================== Uninstall app simple-app ======================== curl -sw %{http_code} http://localhost:32743/helm/uninstall/simple-app/0.1.0 -X DELETE Curl OK Response: 204 Body: sleep 10 - give the app some time to remove ============================================= List svc and pod of the app - should be gone ============================================= No resources found in ckhm namespace. NAME READY STATUS RESTARTS AGE simpleapp-simple-app-858c798f97-k6vsl 0/1 Terminating 0 41s ===================== Get apps - simple-app ===================== curl -sw %{http_code} http://localhost:32743/helm/charts Curl OK Response: 200 Body: {"charts":[{"releaseName":"simpleapp","chartName":"simple-app","version":"0.1.0","namespace":"ckhm","repository":"cm"}]} ============ Delete chart =========== curl -sw %{http_code} http://localhost:32743/helm/charts/simple-app/0.1.0 -X DELETE Curl OK Response: 204 Body: ================ Get apps - empty ================ curl -sw %{http_code} http://localhost:32743/helm/charts Curl OK Response: 200 Body: {"charts":[]} Test result All tests ok End of test
Cleanup of all created kubernetes object
$ kubectl delete -f kube-hm.yaml service "helmmanagerservice" deleted pod "helmmanagerservice" deleted persistentvolume "helm-manager-service-pv" deleted persistentvolumeclaim "helm-manager-service-pvc" deleted $ kubectl delete -f kube-cm.yaml service "chartrepo" deleted pod "chartrepo" deleted persistentvolume "chartrepo-pv" deleted persistentvolumeclaim "chartrepo-pvc" deleted $ kubectl delete -f helm-manager-sa.yaml serviceaccount "helm-manager-sa" deleted clusterrolebinding.rbac.authorization.k8s.io "helm-manager-sa-clusterrolebinding" deleted