Release G: JWT Sidecar Injection

Introduction

This repository represents rapp service exposure prototyping in O-RAN.

It can be downloaded using the following command: git clone "https://gerrit.o-ran-sc.org/r/nonrtric"

The relevant code is in the service-exposure folder.

How it works

The helm charts include an "rapp" section that is read by the installer and used during setup.

  securityEnabled: true
  type: provider
  realm: demo
  client: demoprovider-cli
  authenticator: client-jwt
  roles:
  - role : provider-viewer
    grants:
      - GET
  - role : provider-admin
    grants:
      - GET
      - POST
      - PUT
      - DELETE

The rapp type "provider" tells the installer to setup a realm named demo and a client named demoprovider-cli and to use the client-jwt method to authenticate the token request in Keycloak.

The invoker will wrap the client certificate in a JWT token and send it to Keycloak in order to obtain an access token which will authorize it to access the provider.

The appropriate Istio template files are also applied to secure the service.

The rapp type "invoker" only needs to setup one istio resource - the envoy filter

The following code snippet shows how the script is called with the appropriate parameters:

-- Make an HTTP call to an upstream host with the following headers, body, and timeout.
local headers, body = request_handle:httpCall(
 "jwt_cluster",
 {
  [":method"] = "GET",
  [":path"] = "/token",
  [":authority"] = "jwt-proxy",
  ["realm"] = "{{.Realm}}",
  ["client"] = "{{.Client}}",
  ["authenticator"] = "{{.Authenticator}}",
  ["ns"] = "{{.Namespace}}"
 },
"jwt call",
5000)

The following diagram illustrates the flow:

The flow only applies when the side car has been injected.

This is configured in the MutatingWebhookConfiguration.yaml file.

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: jwt-proxy-webhook
  namespace: default
webhooks:
  - name: rapps-webhook.default.svc.cluster.local
    admissionReviewVersions:
      - "v1beta1"
    sideEffects: "None"
    timeoutSeconds: 30
    objectSelector:
      matchLabels:
        app.kubernetes.io/name: rapp-helloworld-invoker1

The objectSelector field determine where to apply the webhook.

Prerequisites

Istio

Istio should be installed on your cluster ith the demo profile and istioctl should be added to your $PATH variable.

istioctl install --set profile=demo

Please refer to the Istio documentation for more information: Installation Guides.

Create the istio-nonrtric namespace and enable it for istio injection

kubectl create ns istio-nonrtric

kubectl label namespace istio-nonrtric istio-injection=enabled

Note: The deployments have been implemented and tested using minikube.
If you are not using minikube, references to "minikube ip" should be changed to the appropiate value for you host.

File paths

To replicate these tests you will need to setup the various host path referenced in the yaml files on your own machine.

or change them to match your own setup.

Certificates

You will need cfssl installed on your system: sudo apt install golang-cfssl
Please refer to the K8s documentation: Manage TLS Certificates in a Cluster

The certs directory contains 3 shell scripts for creating the server, client and webhook certs: server_certs.sh, client_certs.sh and webhook_certs.sh
Certs generated by the server_certs.sh script: rootCA.crt, tls.crt and tls.key go in the "/var/keycloak/certs" directory
Certs generated by the client_certs.sh script: client.crt, client.key and rootCA.crt go in the "/var/rapps/certs" directory

Run ./server_certs.sh

Run ./client_certs.sh

Copy generated certs to the appropriate directory

The webhook_certs.sh script generates certs for use in the MutatingWebhookConfiguration.yaml and the rapps-webhook.yaml files.
To configure MutatingWebhookConfiguration.yaml run the following commands:
1. ca_pem_b64="$(openssl base64 -A <"./certs/ca.pem")"
2. sed -i 's/${CA_PEM_B64}/'"$ca_pem_b64"'/g' MutatingWebhookConfiguration.yaml

To configure rapps-webhook.yaml append the rapps-webhook-tls.yaml file to the end of it
1. cat rapps-webhook.yaml ./certs/rapps-webhook-tls.yaml >> rapps-webhook.yaml.tmp
2. mv rapps-webhook.yaml.tmp rapps-webhook.yaml

Go & Docker

All go programs need to be built prior to running the Dockerfiles

   go build rapps-helm-installer.go
   go build rapps-keycloak-mgr.go
   go build rapps-istio-mgr.go
   go build rapps-webhook.go
   go build rapps-jwt.go
   go build rapps-rapp-helloworld-provider.go
   go build rapps-rapp-helloworld-invoker1.go
   go build rapps-rapp-helloworld-invoker2.go

Once the go programs have been compile you then need to build a docker image for each of them.

   docker build -f Dockerfile_rim . -t <tag prefix>/rapps-istio-mgr
   docker build -f Dockerfile_rkm . -t <tag prefix>/rapps-keycloak-mgr
   docker build -f Dockerfile_rhi . -t <tag prefix>/rapps-helm-installer
   docker build -f Dockerfile_wh . -t <tag prefix>/rapps-webhook
   docker build -f Dockerfile_jwt . -t <tag prefix>/rapps-jwt
   docker build -f Dockerfile_rhwp  . -t <tag prefix>/rapps-rapp-helloworld-provider
   docker build -f Dockerfile_rhwi1  . -t <tag prefix>/rapps-rapp-helloworld-invoker1
   docker build -f Dockerfile_rhwi2  . -t <tag prefix>/rapps-rapp-helloworld-invoker2

These images can then be pushed to your own repository.

Image references in the yaml files/helm charts should be changed to match your own tagged images.

Chartmuseum

You will need to package your rapp charts and copy them to the /var/chartmuseum/charts directory before starting.

   cd charts/
   helm package rapp-helloworld-provider
   scp -i $(minikube ssh-key) rapp-helloworld-provider-0.1.0.tgz docker@$(minikube ip):/var/chartmuseum/charts

   helm package rapp-helloworld-invoker1
   scp -i $(minikube ssh-key) rapp-helloworld-invoker1-0.1.0.tgz docker@$(minikube ip):/var/chartmuseum/charts

   helm package rapp-helloworld-invoker2
   scp -i $(minikube ssh-key) rapp-helloworld-invoker2-0.1.0.tgz docker@$(minikube ip):/var/chartmuseum/charts

Running the demo

Note: All management pods run in he default namespace.

Start keycloak and postgres in the default namespace with istio injection run:

  ./keycloak.sh deploy 

To start the management pods run:

 ./start_pods.sh

Once all pods have been started a list of running pods is displayed at the end of the script.

To deploy the provider rapp run:

./deploy_rapp.sh rapp-helloworld-provider

To deploy the first invoker rapp run:

./deploy_rapp.sh rapp-helloworld-invoker1

Sidecar injection has been enabled in rapp-helloworld-invoker1 so it is authorized to communicate with the provider.

To deploy the second invoker rapp run:

./deploy_rapp.sh rapp-helloworld-invoker2

Sidecar injection has not been enabled in rapp-helloworld-invoker2 so it is not authorized to communicate with the provider.

Stopping the demo

Run ./stop_pods.sh

To undeploy keycloak and postgres run:

./keycloak.sh undeploy