
This document describes how to deploy an openstack-helm environment based on the INF with Ceph.

1. Preparation

Setup INF O-Cloud. Install INF(StarlingX R8.0) AIO-SX by following the procedure. Enable the Ceph as persistent storage.
- https://docs.starlingx.io/r/stx.8.0/deploy_install_guides/release/virtual/aio_simplex.html
- https://docs.starlingx.io/r/stx.8.0/deploy_install_guides/release/virtual/aio_simplex_install_kubernetes.html#optionally-finish-configuration-of-ceph-based-persistent-storage-backend
Use the following command to make sure you have the ceph as backend on the INF

sysadmin@controller-1:~$ source /etc/platform/openrc
[sysadmin@controller-1 ~(keystone_admin)]$ system storage-backend-list
+--------------------------------------+------------+---------+------------+-------------------+----------+---------------+
| uuid                                 | name       | backend | state      | task              | services | capabilities  |
+--------------------------------------+------------+---------+------------+-------------------+----------+---------------+
| da70e0b7-34c8-488a-9e1f-08e057d6a4be | ceph-store | ceph    | configured | provision-storage | None     | replication:  |
|                                      |            |         |            |                   |          | 2 min_replica |
|                                      |            |         |            |                   |          | tion: 1       |
|                                      |            |         |            |                   |          |               |
+--------------------------------------+------------+---------+------------+-------------------+----------+---------------+


[sysadmin@controller-1 ~(keystone_admin)]$ ceph -s
  cluster:
    id:     c5663990-249a-4b71-988f-19b402784429
    health: HEALTH_OK

  services:
    mon: 1 daemons, quorum controller (age 8h)
    mgr: controller-1(active, since 8h), standbys: controller-0
    mds: kube-cephfs:1 {0=controller-1=up:active} 1 up:standby
    osd: 2 osds: 2 up (since 8h), 2 in (since 8h)

  data:
    pools:   3 pools, 192 pgs
    objects: 181 objects, 258 MiB
    usage:   2.4 GiB used, 269 GiB / 271 GiB avail
    pgs:     192 active+clean

Prepare a Linux environment in which the network is reachable to the INF AIO-SX with Bash CLI. And, install the required packages.

$ sudo apt-get install git make patch jq
# Make sure your CLI has the 'kubectl', 'openstack' and 'helm' installed

Clone openstack-helm and openstack-helm-infra code on your Linux.

# The commit id is 82a6aa8ce96b1669af0b9e8da85b537d02fc5fd3 that used in this demo.
$ git clone --depth 1 --branch master https://opendev.org/openstack/openstack-helm.git
# The commit id is 07c735f632147378c4af8e7b4ce6f390d38e3d69 that used in this demo.
$ git clone --depth 1 --branch master https://opendev.org/openstack/openstack-helm-infra.git

Copy the "/etc/kubernetes/admin.conf" from the INF controller node to your local Linux.

$ scp <INF-controller-0>:/etc/kubernetes/admin.conf ~/.kube/config
# Change the IP address in the ~/.kube/config
# server: https://<INF-OAM-IP>:6443
# You can get the OAM IP through this command on controller node
# system addrpool-show `system addrpool-list | grep oam | awk '{print $2}'` | grep floating

Add labels to controller-0 node.

$ kubectl label node controller-0 openstack-control-plane=enabled

Create namespaces.

$ kubectl create namespace openstack

2. Deploy Ingress Controller

Original procedure: https://docs.openstack.org/openstack-helm/latest/install/developer/kubernetes-and-common-setup.html#deploy-the-ingress-controller

Modify openstack-helm/tools/deployment/component/common/ingress.sh file as follows:

diff --git a/tools/deployment/component/common/ingress.sh b/tools/deployment/component/common/ingress.sh
index 9ae0371..3229dcb 100755
--- a/tools/deployment/component/common/ingress.sh
+++ b/tools/deployment/component/common/ingress.sh
@@ -29,6 +29,23 @@ deployment:
   type: DaemonSet
 network:
   host_namespace: true
+endpoints:
+  ingress:
+    port:
+      http:
+        default: 10080
+      https:
+        default: 10443
+      healthz:
+        default: 11254
+      status:
+        default: 11246
+      stream:
+        default: 11247
+      profiler:
+        default: 11245
+      server:
+        default: 18181
 EOF

 touch /tmp/ingress-component.yaml
@@ -48,21 +65,21 @@ pod:
 EOF
 fi

-helm upgrade --install ingress-kube-system ${HELM_CHART_ROOT_PATH}/ingress \
-  --namespace=kube-system \
-  --values=/tmp/ingress-kube-system.yaml \
-  ${OSH_EXTRA_HELM_ARGS} \
-  ${OSH_EXTRA_HELM_ARGS_INGRESS} \
-  ${OSH_EXTRA_HELM_ARGS_INGRESS_KUBE_SYSTEM}
-
-#NOTE: Wait for deploy
-./tools/deployment/common/wait-for-pods.sh kube-system
+#helm upgrade --install ingress-kube-system ${HELM_CHART_ROOT_PATH}/ingress \
+#  --namespace=kube-system \
+#  --values=/tmp/ingress-kube-system.yaml \
+#  ${OSH_EXTRA_HELM_ARGS} \
+#  ${OSH_EXTRA_HELM_ARGS_INGRESS} \
+#  ${OSH_EXTRA_HELM_ARGS_INGRESS_KUBE_SYSTEM}
+#
+##NOTE: Wait for deploy
+#./tools/deployment/common/wait-for-pods.sh kube-system

 #NOTE: Deploy namespace ingress
 helm upgrade --install ingress-openstack ${HELM_CHART_ROOT_PATH}/ingress \
   --namespace=openstack \
   --values=/tmp/ingress-component.yaml \
-  --set deployment.cluster.class=nginx \
+  --set deployment.cluster.class=nginx-openstack \
   ${OSH_EXTRA_HELM_ARGS} \
   ${OSH_EXTRA_HELM_ARGS_INGRESS} \
   ${OSH_EXTRA_HELM_ARGS_INGRESS_OPENSTACK}
@@ -70,13 +87,13 @@ helm upgrade --install ingress-openstack ${HELM_CHART_ROOT_PATH}/ingress \
 #NOTE: Wait for deploy
 ./tools/deployment/common/wait-for-pods.sh openstack

-helm upgrade --install ingress-ceph ${HELM_CHART_ROOT_PATH}/ingress \
-  --namespace=ceph \
-  --values=/tmp/ingress-component.yaml \
-  --set deployment.cluster.class=nginx-ceph \
-  ${OSH_EXTRA_HELM_ARGS} \
-  ${OSH_EXTRA_HELM_ARGS_INGRESS} \
-  ${OSH_EXTRA_HELM_ARGS_INGRESS_CEPH}
-
-#NOTE: Wait for deploy
-./tools/deployment/common/wait-for-pods.sh ceph
+#helm upgrade --install ingress-ceph ${HELM_CHART_ROOT_PATH}/ingress \
+#  --namespace=ceph \
+#  --values=/tmp/ingress-component.yaml \
+#  --set deployment.cluster.class=nginx-ceph \
+#  ${OSH_EXTRA_HELM_ARGS} \
+#  ${OSH_EXTRA_HELM_ARGS_INGRESS} \
+#  ${OSH_EXTRA_HELM_ARGS_INGRESS_CEPH}
+#
+##NOTE: Wait for deploy
+#./tools/deployment/common/wait-for-pods.sh ceph

Execute ingress.sh.

$ cd $HOME/openstack-helm/
$ ./tools/deployment/component/common/ingress.sh

3. Deploy Other Component

Original Procedure: https://docs.openstack.org/openstack-helm/latest/install/developer/deploy-with-ceph.html#deploy-mariadb

Install the following components to use Tacker:

* MariaDB

* RabbitMQ

* Memcached

* Keystone

* Glance

Modify openstack-helm/tools/deployment/component/glance/glance.sh file as follows:

diff --git a/tools/deployment/component/glance/glance.sh b/tools/deployment/component/glance/glance.sh
index b388ec04..4d50c2c5 100755
--- a/tools/deployment/component/glance/glance.sh
+++ b/tools/deployment/component/glance/glance.sh
@@ -27,7 +27,7 @@ make glance
 tee /tmp/glance.yaml <<EOF
 storage: ${GLANCE_BACKEND}
 volume:
-  class_name: standard
+  class_name: general
 bootstrap:
   structured:
     images:

Execute script files.

$ ./tools/deployment/developer/ceph/050-mariadb.sh
$ ./tools/deployment/developer/ceph/060-rabbitmq.sh
$ ./tools/deployment/developer/ceph/070-memcached.sh
$ ./tools/deployment/developer/ceph/080-keystone.sh
$ ./tools/deployment/component/glance/glance.sh

4. Deploy Barbican and Tacker

Modify openstack-helm/tacker/templates/pvc.yaml file as follows:

diff --git a/tacker/templates/pvc.yaml b/tacker/templates/pvc.yaml
index 8b1678b3..c0599b45 100644
--- a/tacker/templates/pvc.yaml
+++ b/tacker/templates/pvc.yaml
@@ -23,7 +23,7 @@ metadata:
   name: {{ $name }}
 spec:
   accessModes:
-    - "ReadWriteMany"
+    - "ReadWriteOnce"
   resources:
     requests:
       storage: {{ $size }}

Modify openstack-helm/tacker/values.yaml file as follows:

diff --git a/tacker/values.yaml b/tacker/values.yaml
index 90702f95..3d2f2621 100644
--- a/tacker/values.yaml
+++ b/tacker/values.yaml
@@ -105,12 +105,12 @@ pod:
   security_context:
     server:
       pod:
-        runAsUser: 42424
-        runAsNonRoot: true
+        runAsUser: 0
+        runAsNonRoot: false
     conductor:
       pod:
-        runAsUser: 42424
-        runAsNonRoot: true
+        runAsUser: 0
+        runAsNonRoot: false
   lifecycle:
     termination_grace_period:
       server:

Execute script files.

$ ./tools/deployment/developer/common/085-barbican.sh
$ ./tools/deployment/component/tacker/tacker.sh

5. Verify successful deployment

The helm releases are deployed as follows:

sysadmin@controller-0:~$ helm list -n openstack
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
barbican                openstack       1               2023-10-04 14:11:54.122228604 +0000 UTC deployed        barbican-0.3.5                  v1.0.0     
glance                  openstack       1               2023-10-05 01:14:44.18606719 +0000 UTC  deployed        glance-0.4.13                   v1.0.0     
ingress-openstack       openstack       1               2023-10-02 07:59:30.823441021 +0000 UTC deployed        ingress-0.2.17                  v1.5.1     
keystone                openstack       2               2023-10-04 13:58:36.81624535 +0000 UTC  deployed        keystone-0.3.4                  v1.0.0     
mariadb                 openstack       1               2023-10-04 13:36:33.178219784 +0000 UTC deployed        mariadb-0.2.33                  v10.6.7    
memcached               openstack       1               2023-10-04 13:44:40.7788406 +0000 UTC   deployed        memcached-0.1.13                v1.5.5     
rabbitmq                openstack       1               2023-10-04 13:39:44.683045128 +0000 UTC deployed        rabbitmq-0.1.29                 v3.9.0     
tacker                  openstack       1               2023-10-05 10:03:19.033603307 +0000 UTC deployed        tacker-0.1.1                    v1.0.0

The pods are read as follows (Check all pods are "Completed" or "Running" status):

sysadmin@controller-0:~/openstack-helm$ kubectl get pod -n openstack
NAME                                                        READY   STATUS      RESTARTS      AGE
barbican-api-75fd4d79d7-ncz2c                               1/1     Running     0             46h
barbican-db-init-mvhs4                                      0/1     Completed   0             46h
barbican-db-sync-2hn96                                      0/1     Completed   0             46h
barbican-ks-endpoints-57rm2                                 0/3     Completed   0             46h
barbican-ks-service-x2jqn                                   0/1     Completed   0             46h
barbican-ks-user-ds9h6                                      0/1     Completed   0             46h
barbican-rabbit-init-gz647                                  0/1     Completed   0             46h
barbican-test                                               0/1     Completed   0             46h
glance-api-97df56ddb-pr598                                  1/1     Running     0             35h
glance-bootstrap-fbmpq                                      0/1     Completed   0             35h
glance-db-init-gtmdc                                        0/1     Completed   0             35h
glance-db-sync-9jkb8                                        0/1     Completed   0             35h
glance-ks-endpoints-dkb6m                                   0/3     Completed   0             35h
glance-ks-service-xdhfk                                     0/1     Completed   0             35h
glance-ks-user-9xhvf                                        0/1     Completed   0             35h
glance-metadefs-load-rw2kc                                  0/1     Completed   0             35h
glance-rabbit-init-c4wvr                                    0/1     Completed   0             35h
glance-storage-init-lzn72                                   0/1     Completed   0             35h
ingress-5448bbd7d-7rz99                                     1/1     Running     1 (47h ago)   4d4h
ingress-error-pages-54c8fdfb4d-wgktt                        1/1     Running     1 (47h ago)   4d4h
keystone-api-6cb7d765ff-srpwg                               1/1     Running     0             46h
keystone-bootstrap-f9s5n                                    0/1     Completed   0             46h
keystone-credential-setup-27qkx                             0/1     Completed   0             46h
keystone-db-init-sr9dj                                      0/1     Completed   0             46h
keystone-db-sync-7hnj8                                      0/1     Completed   0             46h
keystone-domain-manage-2n6sf                                0/1     Completed   0             46h
keystone-fernet-rotate-28275120-djbg7                       0/1     Completed   0             24h
keystone-fernet-rotate-28275840-z2wnq                       0/1     Completed   0             12h
keystone-fernet-rotate-28276560-z6rmr                       0/1     Completed   0             30m
keystone-fernet-setup-x8px7                                 0/1     Completed   0             46h
keystone-rabbit-init-w5h9q                                  0/1     Completed   0             46h
mariadb-ingress-7f9bcfd79b-6flfw                            1/1     Running     0             46h
mariadb-ingress-7f9bcfd79b-tlwkc                            1/1     Running     0             46h
mariadb-ingress-error-pages-557b55c45f-tw8sw                1/1     Running     0             46h
mariadb-server-0                                            1/1     Running     0             46h
memcached-memcached-785bbdd4d8-zxh76                        1/1     Running     0             46h
rabbitmq-cluster-wait-49khp                                 0/1     Completed   0             46h
rabbitmq-rabbitmq-0                                         1/1     Running     0             46h
rabbitmq-rabbitmq-1                                         1/1     Running     0             46h
tacker-conductor-9f977f5b4-tx58c                            1/1     Running     0             26h
tacker-db-init-4d7xz                                        0/1     Completed   0             26h
tacker-db-sync-vwzg2                                        0/1     Completed   0             26h
tacker-ks-endpoints-426wd                                   0/3     Completed   0             26h
tacker-ks-service-lltsv                                     0/1     Completed   0             26h
tacker-ks-user-5vpws                                        0/1     Completed   0             26h
tacker-rabbit-init-2jkgb                                    0/1     Completed   0             26h
tacker-server-76d9bbf6c8-skk8h                              1/1     Running     0             26h

Test if Tacker is working properly

$ TACKER_SERVER_POD=tacker-server-76d9bbf6c8-skk8h
$ TACKER_ENDPOINT=tacker-api.openstack.svc.cluster.local

# Issue token from keystone
$ kubectl exec -n openstack -it $TACKER_SERVER_POD \
  -- curl -i -X POST -H "Content-Type: application/json" \
  -d '{"auth":{"identity":{"methods":["password"],"password":{"user":{"domain":{"name":"default"},"name":"admin","password":"password"}}},"scope":{"project":{"domain":{"name":"default"},"name":"admin"}}}}' \
  http://keystone-api.openstack.svc.cluster.local:5000/v3/auth/tokens

HTTP/1.1 201 CREATED
Date: Fri, 06 Oct 2023 12:46:40 GMT
Content-Type: application/json
Content-Length: 3175
Connection: keep-alive
X-Subject-Token: gAAAAABlIAGv1RqxqMJ7rt_VyAtPTxF0XjMG19zp-0zaZmHdFkKmEjLfUus09GkPUdcbCeVuR8ZfmMjqg9C2kRCWWX4Llfdwld1lKM-beqQ7s127kjhpilf28e1oXh351CmBFy97PaZ9D5WBoe3fRrDkhhB_cEsB76Pyj6P2KQuNeMIhGmb1fKA
Vary: X-Auth-Token
x-openstack-request-id: req-408ef1f6-2b61-4a8d-89b0-0d987878cbbb

# Set `X-Subject-Token` retrieved as TOKEN 
$ TOKEN=gAAAAABlIAGv1RqxqMJ7rt_VyAtPTxF0XjMG19zp-0zaZmHdFkKmEjLfUus09GkPUdcbCeVuR8ZfmMjqg9C2kRCWWX4Llfdwld1lKM-beqQ7s127kjhpilf28e1oXh351CmBFy97PaZ9D5WBoe3fRrDkhhB_cEsB76Pyj6P2KQuNeMIhGmb1fKA

$ kubectl exec -n openstack -it $TACKER_SERVER_POD \
  -- curl -X GET http://${TACKER_ENDPOINT}:9890/vnflcm/v2/vnf_instances \
  -H "X-Auth-Token:$TOKEN" -H "Version: 2.0.0"
[]    *** Success if you can get an empty list ***

Show all results

Space shortcuts

Page tree

1. Preparation

2. Deploy Ingress Controller

3. Deploy Other Component

4. Deploy Barbican and Tacker

5. Verify successful deployment

Space shortcuts

Page tree

ETSI-DMS on INF O-Cloud, Tacker installation guide

1. Preparation

2. Deploy Ingress Controller

3. Deploy Other Component

4. Deploy Barbican and Tacker

5. Verify successful deployment