Table of Contents |
---|
JIRA Ticket
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
ISTIO
Istio is a service mesh which provides a dedicated infrastructure layer that you can add to your applications. It adds capabilities like observability, traffic management, and security, without adding them to your own code.
...
Code Block | ||||
---|---|---|---|---|
| ||||
apiVersion: "security.istio.io/v1beta1" kind: "PeerAuthentication" metadata: name: "default" namespace: "istio-system" spec: mtls: mode: STRICT |
JWT
JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object.
...
Code Block | ||||
---|---|---|---|---|
| ||||
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: httpbin namespace: foo spec: selector: matchLabels: app: httpbin rules: - from: - source: requestPrincipals: ["*"] - to: - operation: paths: ["/healthz"] |
Use Case
KUBERNETES
K8S RBAC (Role Based Access Control) - supported by default in kubernetes.
K8S Service Account
- Kube use service account (sa) to validate api access
- SAs can be lined to a role via a binding
- A default sa, with no permissions (no bindings), is created in each namespace – pods uses this namespace unless otherwise specified
- Pods can be specified to run with a specific sa
- The helm manger needs a SA with cluster wide permission (to be able to list installed charts etc)
However, during installation the pod running the helm install/upgrade/delete should run with a sa with only namespace permission to ensure not other modification is made to kube objects outside the namespace
K8S RBAC - controlling access to kubernetes resources
- Role and and rolebinding objects defines who (sa, user or group) is allowed to do what in the kubernetes api
- Role object
- Sets permissions on resources in a specific namespace
- ClusterRole object
- Sets permission on non-namespaced resources or across namespaces
- RoleBinding object
- Binds one or more Role or a ClusterRole object(s) to a user, group or service account
- ClusterRoleBinding object
- Binds one or more ClusterRole object(s) to a user, group or service account
- No installation requried – RBAC enabled by default
Network Policies
- K8S supports Network Policy objects but a provider need to be install for the polcies to take effect, e.g. Calico
- Network policies can control ingress and/or egress traffic by selecting applicable pods - bacially controlling traffic between pods and/or network endpoints
- Several providers: Calico, Cillium etc
...
- Allow traffic from ns: kong (the gateway), nonrtric and onap
- Deny traffic from any other ns
KONG
Kong is Orchestration Microservice API Gateway. Kong provides a flexible abstraction layer that securely manages communication between clients and microservices via API. Also known as an API Gateway, API middleware or in some cases Service Mesh.
...
With client-side discovery, the client or API gateway making the request is responsible for identifying the location of the service instance and routing the request to it. The client begins by querying the service registry to identify the location of the available instances of the service and then determines which instance to use. See https://konghq.com/learning-center/microservices/service-discovery-in-a-microservices-architecture/
Kong datastore
Kong uses an external datastore to store its configuration such as registered APIs, Consumers and Plugins. Plugins themselves can store every bit of information they need to be persisted, for example rate-limiting data or Consumer credentials. See https://konghq.com/faqs/#:~:text=PostgreSQL%20is%20an%20established%20SQL%20database%20for%20use,Cassandra%20or%20PostgreSQL%2C%20Kong%20maintains%20its%20own%20cache.
Kong Demo
Demo of Kong gateway access control already available.
...
Code Block | ||||
---|---|---|---|---|
| ||||
apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: app-jwt-kp namespace: nonrtric plugin: jwt --- apiVersion: v1 kind: Secret metadata: name: pms-jwt-sec namespace: nonrtric type: Opaque stringData: kongCredType: jwt key: pms-issuer algorithm: RS256 rsa_public_key: | -----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwetu4+suoz6c7e1kQz7I Jmujci8zHpp4qh3nsmEL8e3QOKzMVsLuQPcF8lO1bBoChSA+KMNJ5rEixGWSxClp 9XroBSgrvjDsKtpPIlBQMnyOUYRSXWnIodmN+7wA72pTxo7JtAypPzRscSgi0OZt 9dtmv50RLr9Wph5cI+IE9OtgW58OKtdFRGigGHfdUEwrT/MPw2rOU85YRFaEgT/i wcuQCe+Zmf2S2gVgK62u51ZFFn2VycJT1LcOt9cdqrSXYZAPfVKnQ/EgYvDdzFL1 x73JkrrSEP3pfrN4bXOnc7cS/S9Y2qk/I+QCR6a6XKmqk5SnWJSyXvKdYQJrgxJp lQIDAQAB -----END PUBLIC KEY----- --- apiVersion: v1 kind: Secret metadata: name: ecs-jwt-sec namespace: nonrtric type: Opaque stringData: kongCredType: jwt key: ecs-issuer algorithm: RS256 rsa_public_key: | -----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtminzTtNs5oqPCbg4uC1 L7MfR3B+uyYvkSKr3NFieRCxp6VhrgodJJXYc3SqXbaTVBkTwU24wG4UvJCnoRQd 0VhSawtLkN8XNAdCiD831dKUYMJPs43ZY/gO5CHVqUMdSHlp8dn7jNren59dvRRS 3xC1D3etXuEU01XGuLi/5qJLAKqDbYs3bH1vslTjndg1WTsrkU8GEIT1NphSYg25 s6rSLTIBfk8FjKquYHw3wYVSQK9rg2mqddJpRWkfZnazMHTmSNjOJpiNb77VLGSx 9qDbbLjurCl2mAG5Z+w76uKfKGgOo68SU0TL1sPybsKhAoZZg1gF06mvMln5eq5C RQIDAQAB -----END PUBLIC KEY----- --- apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: pms-group-acl-kp namespace: nonrtric plugin: acl config: whitelist: ['pms-group'] --- apiVersion: v1 kind: Secret metadata: name: pms-group-acl-sec namespace: nonrtric type: Opaque stringData: kongCredType: acl group: pms-group --- apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: ecs-group-acl-kp namespace: nonrtric plugin: acl config: whitelist: ['ecs-group'] --- apiVersion: v1 kind: Secret metadata: name: ecs-group-acl-sec namespace: nonrtric type: Opaque stringData: kongCredType: acl group: ecs-group --- apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: all-group-acl-kp namespace: nonrtric plugin: acl config: whitelist: ['ecs-group', 'pms-group'] --- apiVersion: configuration.konghq.com/v1 kind: KongConsumer metadata: name: pms-user-kc namespace: nonrtric annotations: kubernetes.io/ingress.class: kong username: pms-user credentials: - pms-jwt-sec - pms-group-acl-sec --- apiVersion: configuration.konghq.com/v1 kind: KongConsumer metadata: name: ecs-user-kc namespace: nonrtric annotations: kubernetes.io/ingress.class: kong username: ecs-user credentials: - ecs-jwt-sec - ecs-group-acl-sec --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: r1-pms-ing namespace: nonrtric annotations: konghq.com/plugins: app-jwt-kp,pms-group-acl-kp konghq.com/strip-path: "false" spec: ingressClassName: kong rules: - http: paths: - path: /a1-policy pathType: ImplementationSpecific backend: service: name: policymanagementservice port: number: 8081 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: r1-ecs-ing namespace: nonrtric annotations: konghq.com/plugins: app-jwt-kp,ecs-group-acl-kp konghq.com/strip-path: "false" spec: ingressClassName: kong rules: - http: paths: - path: /data-consumer pathType: ImplementationSpecific backend: service: name: enrichmentservice port: number: 8083 - path: /data-producer pathType: ImplementationSpecific backend: service: name: enrichmentservice port: number: 8083 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: r1-echo-ing namespace: nonrtric annotations: konghq.com/plugins: app-jwt-kp,all-group-acl-kp konghq.com/strip-path: "true" spec: ingressClassName: kong rules: - http: paths: - path: /echo pathType: ImplementationSpecific backend: service: name: httpecho port: number: 80 |
ISTIO Demo
- Install ISTIO on minikube using instruction here: Istio Installation - Simplified Learning (waytoeasylearn.com)
cd to the istio directory and install the demo application
kubectl create ns foo
kubectl apply -f <(istioctl kube-inject -f samples/httpbin/httpbin.yaml) -n foo
Create a python script to generate a JWT token using the code from here: https://medium.com/intelligentmachines/istio-jwt-step-by-step-guide-for-micro-services-authentication-690b170348fc . Install python_jwt using pip if it's not already installed.
- Create jwt-example.yaml using the public key generated by the python script:
kubectl create -f jwt-example.yaml
Code Block language yml title jwt-example.yaml apiVersion: "security.istio.io/v1beta1" kind: "RequestAuthentication" metadata: name: "jwt-example" namespace: istio-system spec: selector: matchLabels: istio: ingressgateway jwtRules: - issuer: "ISSUER" jwks: | { "keys":[{"e":"AQAB","kty":"RSA","n":"x_yYl4uW5c6NHOA-bDDh0MThFggBWl-vYJr77b9F1LmAtTlJVM0rL5klTfv2DmlAmD9eZPrWeUOoOGhSpe58XiSAvxyeaOrZhtyUjT3aglrSys0YBsB19ItNGMuoIuzPpWOrdtKwHa9rPbrdc6q7vb93qu2UVaIz-3FJmGFtSA5t8FK_5bZKF-oOzRLwqeVQ3n0Bu_dFDuGeZjQWMZF32QupyA-GF-tDGGriPLy9sutlB1NQyZ4qiSZx5UMxcfLwsWfQxHemdwLeZXWKWNBov8RmbZy2Jz-dwg6XjHBWAjTnCGG9p-bp63nUlnELI3LcEGhGOugZBqcpNT5dEAQ0fQ"}]}
Export the JWT token generated by the python script as an environment variable:
export TOKEN="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJBVURJRU5DRSIsImV4cCI6MTYzNzI1NDkxNSwiaWF0IjoxNjM3MjUxOTE1LCJpc3MiOiJJU1NVRVIiLCJqdGkiOiJCcmhDdEstcC00ZTF0RlBrZmpuSmhRIiwibmJmIjoxNjM3MjUxOTE1LCJwZXJtaXNzaW9uIjoicmVhZCIsInJvbGUiOiJ1c2VyIiwic3ViIjoiU1VCSkVDVCJ9.HrQCLPZXf0VkFe7JUVGXq-sHJQhVibqhToG4r63py-iwHWlUL02_WfoWRoxapgqGwImDdSlt1uG8RR-6VMqzWwGlcqBIRhFTG0nmzmtQjnOUs6QAKSUpA3PyWBIYHV0BwZbpo8Zq1Bo-sELy400fU-MCQ_054fSsG7JMBMmrnj8NyJmD2lNN0VSFGO53SPl2tQSVlc9OwAr8Uu0jfLPfUmh6yq43qFuxnVRfBGLLPNOt29aOfAetKLc72qlphtnbDx2a9teP5AIbkIWyIlhTytEnQRCwU4x8gDrEdkrHui4qCtzpl_uoITSwPe3AFsi7gQHB6rJoDj-j2zPc4rUTAA"export INGRESS_HOST=$(minikube ip)
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
Test the service:
curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST:$INGRESS_PORT/headers -s -o /dev/null -w "%{http_code}\n"You should get a response code of 200
Update the token to something invalid
The response will be 401
Istio Service JWT Test
istio-test.yaml (uses the default namespace)
...
See the latest version here: istio-test-latest.yaml
Istio with Keycloak
If you are using minikube on Ubuntu WSL you need to run "minikube service keycloak" to see keycloak ui.
...
Retrieve public key using : http(s)://<hostname>/auth/realms/<realm name>
Anchor keycloak keycloak
Enable keycloak with Istio
Setup a new realm, user and client as shown here : https://www.keycloak.org/getting-started/getting-started-kube
...
Note: The iss of the token will differ depending on how you retrieve it. If it's retrieved from within the cluster for URL will start with http://keycloak.default:8080/ otherwise it will be something like : http://192.168.49.2:31560/ (http://(minikube ip): (keycloak service nodePort))
Keycloak database
Keycloak uses the H2 database by default.
...
Code Block | ||||
---|---|---|---|---|
| ||||
spec: initContainers: - name: init-keycloak image: busybox command: ['sh', '-c', 'until nc -vz keycloak.default 8080; do echo waiting for keycloak; sleep 2; done;'] containers: - name: a1-policy image: hashicorp/http-echo ports: - containerPort: 5678 args: - -text - "Hello a1-policy" |
Istio mTLS
Test: Istio / Mutual TLS Migration
...
Change namespace to istio-system to apply mTLS for the entire cluster.
Istio cert manager
https://istio.io/latest/docs/ops/integrations/certmanager/
Go Http Request Handler for Testing
Anchor | ||||
---|---|---|---|---|
|
nonrtric-server-go
Code Block | ||||
---|---|---|---|---|
| ||||
package main import ( "fmt" "log" "github.com/gorilla/mux" "net/http" "encoding/json" "io/ioutil" "strings" ) func requestHandler(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") params := mux.Vars(r) var id = params["id"] var data = params["data"] var prefix = strings.Split(r.URL.Path, "/")[1] switch r.Method { case "GET": if id == "" { fmt.Println( "Received get request for "+ prefix +", params: nil\n") fmt.Fprintf(w, "Response to get request for "+ prefix +", params: nil\n") }else { fmt.Println("Received get request for "+ prefix +", params: id=" + id + "\n") fmt.Fprintf(w, "Response to get request for "+ prefix +", params: id=" + id + "\n") } case "POST": body, err := ioutil.ReadAll(r.Body) if err != nil { panic(err.Error()) } keyVal := make(map[string]string) json.Unmarshal(body, &keyVal) id := keyVal["id"] data := keyVal["data"] fmt.Println("Received post request for "+ prefix +", params: id=" + id +", data=" + data + "\n") fmt.Fprintf(w, "Response to post request for "+ prefix +", params: id=" + id +", data=" + data + "\n") case "PUT": fmt.Println("Received put request for "+ prefix +", params: id=" + id +", data=" + data + "\n") fmt.Fprintf(w, "Response to put request for "+ prefix +", params: id=" + id +", data=" + data + "\n") case "DELETE": fmt.Println("Received delete request for "+ prefix +", params: id=" + id + "\n") fmt.Fprintf(w, "Response to delete request for "+ prefix +", params: id=" + id + "\n") default: fmt.Println("Received request for unsupported method, only GET, POST, PUT and DELETE methods are supported.") fmt.Fprintf(w, "Error, only GET, POST, PUT and DELETE methods are supported.") } } func main() { router := mux.NewRouter() var prefixArray [3]string = [3]string{"/a1-policy", "/data-consumer", "/data-producer"} for _, prefix := range prefixArray { router.HandleFunc(prefix, requestHandler) router.HandleFunc(prefix+"/{id}", requestHandler) router.HandleFunc(prefix+"/{id}/{data}", requestHandler) } log.Fatal(http.ListenAndServe(":8080", router)) } |
...
Code Block | ||||
---|---|---|---|---|
| ||||
FROM golang:1.15.2-alpine3.12 as build RUN apk add git RUN mkdir /build ADD . /build WORKDIR /build RUN go get github.com/gorilla/mux RUN go build -o nonrtric-server-go . FROM alpine:latest RUN mkdir /app WORKDIR /app/ # Copy the Pre-built binary file from the previous stage COPY --from=build /build . # Expose port 8080 EXPOSE 8080 # Run Executable CMD ["/app/nonrtric-server-go"] |
Testing
Update AuthorizationPolicy to only allow certain operations:
...
-----------------------------------------------------------------------
Number of Tests: 10, Tests Passed: 10, Tests Failed: 0
Date: 2021-12-06-14:48:33
-----------------------------------------------------------------------
Go Http Client for running inside cluster
Anchor | ||||
---|---|---|---|---|
|
nonrtric-client-go
Code Block | ||||
---|---|---|---|---|
| ||||
package main import ( "fmt" "net/http" "net/url" "encoding/json" "time" "io/ioutil" "math/rand" "strings" "bytes" "strconv" "flag" ) type Jwttoken struct { Access_token string Expires_in int Refresh_expires_in int Refresh_token string Token_type string Not_before_policy int Session_state string Scope string } var gatewayHost string var gatewayPort string var keycloakHost string var keycloakPort string var useGateway string var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ") func randSeq(n int) string { b := make([]rune, n) for i := range b { b[i] = letters[rand.Intn(len(letters))] } return string(b) } func getToken(user string, password string, clientId string, realmName string) string { keycloakUrl := "http://"+keycloakHost+":"+keycloakPort+"/auth/realms/"+realmName+"/protocol/openid-connect/token" resp, err := http.PostForm(keycloakUrl, url.Values{"username": {user}, "password": {password}, "grant_type": {"password"}, "client_id": {clientId}}) if err != nil { fmt.Println(err) panic("Something wrong with the credentials or url ") } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) var jwt Jwttoken json.Unmarshal([]byte(body), &jwt) return jwt.Access_token; } func MakeRequest(client *http.Client, prefix string, method string, ch chan<-string) { var id = rand.Intn(1000) var data = randSeq(10) var service = strings.Split(prefix, "/")[1] var gatewayUrl = "http://"+gatewayHost+":"+gatewayPort var token = "" var jsonValue []byte = []byte{} var restUrl string = "" if strings.ToUpper(useGateway) != "Y" { gatewayUrl = "http://"+service+".istio-nonrtric:80" //fmt.Println(gatewayUrl) } if service == "a1-policy" { token = getToken("pmsuser", "secret","pmsclient", "pmsrealm") }else{ token = getToken("icsuser", "secret","icsclient", "icsrealm") } if method == "POST" { values := map[string]string{"id": strconv.Itoa(id), "data": data} jsonValue, _ = json.Marshal(values) restUrl = gatewayUrl+prefix } else if method == "PUT" { restUrl = gatewayUrl+prefix+"/"+strconv.Itoa(id)+"/"+data } else { restUrl = gatewayUrl+prefix+"/"+strconv.Itoa(id) } req, err := http.NewRequest(method, restUrl, bytes.NewBuffer(jsonValue)) if err != nil { fmt.Printf("Got error %s", err.Error()) } req.Header.Set("Content-type", "application/json") req.Header.Set("Authorization", "Bearer "+token) resp, err := client.Do(req) if err != nil { fmt.Printf("Got error %s", err.Error()) } defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) respString := string(body[:]) if respString == "RBAC: access denied"{ respString += " for "+service+" "+strings.ToLower(method)+" request\n" } ch <- fmt.Sprintf("%s", respString) } func main() { flag.StringVar(&gatewayHost, "gatewayHost", "192.168.49.2", "Gateway Host") flag.StringVar(&gatewayPort, "gatewayPort" , "32162", "Gateway Port") flag.StringVar(&keycloakHost, "keycloakHost", "192.168.49.2", "Keycloak Host") flag.StringVar(&keycloakPort, "keycloakPort" , "31560", "Keycloak Port") flag.StringVar(&useGateway, "useGateway" , "Y", "Connect to services hrough API gateway") flag.Parse() client := &http.Client{ Timeout: time.Second * 10, } ch := make(chan string) var prefixArray [3]string = [3]string{"/a1-policy", "/data-consumer", "/data-producer"} var methodArray [4]string = [4]string{"GET", "POST", "PUT", "DELETE"} for true { for _,prefix := range prefixArray{ for _,method := range methodArray{ go MakeRequest(client, prefix, method, ch) } } for i := 0; i < len(prefixArray); i++ { for j := 0; j < len(methodArray); j++ { fmt.Println(<-ch) } } time.Sleep(30 * time.Second) } } |
...
Code Block | ||||
---|---|---|---|---|
| ||||
apiVersion: "security.istio.io/v1beta1" kind: "AuthorizationPolicy" metadata: name: "pms-policy" namespace: istio-nonrtric spec: selector: matchLabels: apptype: nonrtric-pms action: ALLOW rules: - from: - source: requestPrincipals: ["http://192.168.49.2:31560/auth/realms/pmsrealm/fab53fd0-3315-4e2f-bd17-6984fb7745f2"] - source: requestPrincipals: ["http://keycloak.default:8080/auth/realms/pmsrealm/fab53fd0-3315-4e2f-bd17-6984fb7745f2"] to: - operation: methods: ["GET", "POST", "PUT", "DELETE"] paths: ["/a1-policy*"] when: - key: request.auth.claims[role] values: ["pms_admin"] - from: - source: requestPrincipals: ["http://192.168.49.2:31560/auth/realms/pmsrealm/f96255ec-d553-4c2e-b106-0ed586ccab70"] - source: requestPrincipals: ["http://keycloak.default:8080/auth/realms/pmsrealm/f96255ec-d553-4c2e-b106-0ed586ccab70"] to: - operation: methods: ["GET"] paths: ["/a1-policy*"] when: - key: request.auth.claims[role] values: ["pms_viewer"] |
pms_admin role:
Test 1: Testing GET /a1-policy
Received get request for a1-policy, params: nil
...
Test 5: Testing POST /a1-policy
Received post request for a1-policy, params: id=1003, data=abc
pms_viewer role:
Test 1: Testing GET /a1-policy
Received get request for a1-policy, params: nil
...
Anchor grafana grafana
Grafana
Istio also comes with grafana, to start it run : istioctl dashboard grafana
...
This does not really work for a single shard instance like the one we are using.
Prometheus
Start the prometheus dashboard by running: istioctl dashboard prometheus
...
You can then create your own dashboard in grafana using these metrics: rapps-requests.json
OAuth2 Proxy
Welcome to OAuth2 Proxy | OAuth2 Proxy (oauth2-proxy.github.io)
Calico network policy
https://docs.projectcalico.org/security/calico-network-policy
...
Following the example in the link above I installed the test application in a separate namespace (calico-test). Using curl I was able to access the database prior to applying the GlobalNetworkPolicy. After applying the policy the request timed out rather than return a 403 forbidden message.
Logging
Anchor | ||||
---|---|---|---|---|
|
...
Code Block | ||||
---|---|---|---|---|
| ||||
PUT /_ilm/policy/cleanup_policy { "policy": { "phases": { "hot": { "actions": {} }, "delete": { "min_age": "1d", "actions": { "delete": {} } } } } } PUT /logstash-*/_settings { "lifecycle.name": "cleanup_policy" } PUT /_template/logging_policy_template { "index_patterns": ["logstash-*"], "settings": { "index.lifecycle.name": "cleanup_policy" } } |
Quick Installation Guide
- Download and install istio: istioctl install --set profile=demo
- cd to the samples/addons/ directory and install the dashboards e.g. kubectl create -f kiali.yaml
- Install postgres: istioctl kube-inject -f postgres.yaml | kubectl apply -f - (change the hostPath path value to a path on your host)
- Install keycloak: istioctl kube-inject -f keycloak.yaml | kubectl apply -f -
- Open the keycloak admin console and setup the required realms, users and clients
- Setup the "pms_admin" and "pms_viewer" roles for pmsuser and pmsuser2 respectively.
- Install Release E: Coordinated Service Exposure: nonrtric-server-go: docker build -t nonrtric-server-go:latest .
- Create the istio-nonrtric namespace: kubectl create namespace istio-nonrtric
- Enable istio for the istio-nonrtric namespace: kubectl label namespace istio-nonrtric istio-injection=enabled
- Edit the istio-test.yaml so the host ip specified matches yours.
- Also change the userid in the requestPrincipals field to match yours
- Install istio-test.yaml : kubectl create -f istio-test.yaml
- Install Release E: Coordinated Service Exposure nonrtric-client-go: docker build -t nonrtric-client-go:latest .
- Install the test client: istioctl kube-inject -f client.yaml | kubectl apply -f -
- Open the kiali dashboard to check your services are up and running
- Open the grafana to view the istio dashboard
- Optionally install Release E: Coordinated Service Exposure elasticsearch
ONAP
ONAP Next Generation Security & Logging Architecture
GO Client
Create kubernetes jobs in golang
...
There is also a helm sdk you can use:
HELM SDK
Code Block | ||||
---|---|---|---|---|
| ||||
package main import ( "fmt" "os" "log" "k8s.io/cli-runtime/pkg/genericclioptions" "k8s.io/client-go/rest" "helm.sh/helm/v3/pkg/action" "helm.sh/helm/v3/pkg/chart/loader" ) func main() { chartPath := "/tmp/wordpress-12.3.3.tgz" chart, err := loader.Load(chartPath) releaseName := "wordpress" releaseNamespace := "default" actionConfig, err := getActionConfig(releaseNamespace) if err != nil { panic(err) } listAction := action.NewList(actionConfig) releases, err := listAction.Run() if err != nil { log.Println(err) } for _, release := range releases { log.Println("Release: " + release.Name + " Status: " + release.Info.Status.String()) } iCli := action.NewInstall(actionConfig) iCli.Namespace = releaseNamespace iCli.ReleaseName = releaseName rel, err := iCli.Run(chart, nil) if err != nil { fmt.Println(err) } fmt.Println("Successfully installed release: ", rel.Name) } func getActionConfig(namespace string) (*action.Configuration, error) { actionConfig := new(action.Configuration) // Create the rest config instance with ServiceAccount values loaded in them config, err := rest.InClusterConfig() if err != nil { // fallback to kubeconfig home, exists := os.LookupEnv("HOME") if !exists { home = "/root" } kubeconfigPath := filepath.Join(home, ".kube", "config") if envvar := os.Getenv("KUBECONFIG"); len(envvar) >0 { kubeconfigPath = envvar } if err := actionConfig.Init(kube.GetConfig(kubeconfigPath, "", namespace), namespace, os.Getenv("HELM_DRIVER"), func(format string, v ...interface{}) { fmt.Sprintf(format, v) }); err != nil { panic(err) } } else { // Create the ConfigFlags struct instance with initialized values from ServiceAccount var kubeConfig *genericclioptions.ConfigFlags kubeConfig = genericclioptions.NewConfigFlags(false) kubeConfig.APIServer = &config.Host kubeConfig.BearerToken = &config.BearerToken kubeConfig.CAFile = &config.CAFile kubeConfig.Namespace = &namespace if err := actionConfig.Init(kubeConfig, namespace, os.Getenv("HELM_DRIVER"), func(format string, v ...interface{}) { fmt.Sprintf(format, v) }); err != nil { panic(err) } } return actionConfig, nil } |
...
The method getActionConfig works for both in-cluster deployments and from the localhost. It determines which one to use by calling the rest.InClusterConfig() function.
GO CLIENT SDK
Here are another couple of programs that demonstrate how to use the go client:
...
There is also support for istio in client go Istio client-go
ISTIO SDK
Code Block | ||||
---|---|---|---|---|
| ||||
package main import ( "context" "bytes" "fmt" "os" "log" "path/filepath" k8Yaml "k8s.io/apimachinery/pkg/util/yaml" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" clientcmd "k8s.io/client-go/tools/clientcmd" versioned "istio.io/client-go/pkg/clientset/versioned" v1beta1 "istio.io/client-go/pkg/apis/security/v1beta1" securityv1beta1 "istio.io/api/security/v1beta1" typev1beta1 "istio.io/api/type/v1beta1" ) const ( NAMESPACE = "default" ) const authorizationPolicyManifest = ` apiVersion: "security.istio.io/v1beta1" kind: "AuthorizationPolicy" metadata: name: "pms-policy" namespace: default spec: selector: matchLabels: apptype: nonrtric-pms action: ALLOW rules: - from: - source: principals: ["cluster.local/ns/default/sa/goclient"] to: - operation: methods: ["GET", "POST", "PUT", "DELETE"] paths: ["/a1-policy*"] hosts: ["a1-policy*"] ports: ["8080"] when: - key: request.auth.claims[role] values: ["pms_admin"] ` func connectToK8s() *versioned.Clientset { home, exists := os.LookupEnv("HOME") if !exists { home = "/root" } configPath := filepath.Join(home, ".kube", "config") config, err := clientcmd.BuildConfigFromFlags("", configPath) if err != nil { log.Fatalln("failed to create K8s config") } ic, err := versioned.NewForConfig(config) if err != nil { log.Fatalf("Failed to create istio client: %s", err) return ic } func createAuthorizationPolicy(clientset *versioned.Clientset) { authClient := clientset.SecurityV1beta1().AuthorizationPolicies(NAMESPACE) auth := &v1beta1.AuthorizationPolicy{} dec := k8Yaml.NewYAMLOrJSONDecoder(bytes.NewReader([]byte(authorizationPolicyManifest)), 1000) if err := dec.Decode(&auth); err != nil { fmt.Println(err) } result, err := authClient.Create(context.TODO(), auth, metav1.CreateOptions{}) if err!=nil { panic(err.Error()) } fmt.Printf("Create Authorization Policy %s \n", result.GetName()) } func createAuthorizationPolicy2(clientset *versioned.Clientset) { authClient := clientset.SecurityV1beta1().AuthorizationPolicies(NAMESPACE) auth := &v1beta1.AuthorizationPolicy{ ObjectMeta: metav1.ObjectMeta{ Name: "ics-policy", }, Spec: securityv1beta1.AuthorizationPolicy { Selector: &typev1beta1.WorkloadSelector{ MatchLabels: map[string]string{ "apptype" : "nonrtric-ics", }, }, Action: securityv1beta1.AuthorizationPolicy_ALLOW, Rules: []*securityv1beta1.Rule{{ From: []*securityv1beta1.Rule_From{{ Source: &securityv1beta1.Source{ Namespaces : []string{ "default", }, }, },}, To: []*securityv1beta1.Rule_To{{ Operation: &securityv1beta1.Operation{ Methods : []string{ "GET", "POST", "PUT", "DELETE", }, Paths : []string{ "/data-*", }, Hosts : []string{ "data-consumer*", "data-producer*", }, Ports : []string{ "8080", }, }, },}, },}, }, } result, err := authClient.Create(context.TODO(), auth, metav1.CreateOptions{}) if err!=nil { panic(err.Error()) } fmt.Printf("Create Authorization Policy %s \n", result.GetName()) } func main() { clientset := connectToK8s() createAuthorizationPolicy(clientset) createAuthorizationPolicy2(clientset) } |
...
keycloak aslo has a client called gocloak
GOCLOAK SDK
Code Block | ||||
---|---|---|---|---|
| ||||
package main import ( "github.com/Nerzal/gocloak/v10" "context" "fmt" ) func main(){ client := gocloak.NewClient("http://192.168.49.2:31560") ctx := context.Background() token, err := client.LoginAdmin(ctx, "admin", "admin", "master") if err != nil { fmt.Println(err) panic("Something wrong with the credentials or url") } realmRepresentation := gocloak.RealmRepresentation{ ID: gocloak.StringP("testRealm"), Realm: gocloak.StringP("testRealm"), DisplayName: gocloak.StringP("testRealm"), Enabled: gocloak.BoolP(true), } realm, err := client.CreateRealm(ctx, token.AccessToken, realmRepresentation) if err != nil { fmt.Println(err) panic("Oh no!, failed to create realm :(") } else { fmt.Println("Created new realm", realm) } newClient := gocloak.Client{ ClientID: gocloak.StringP("testClient"), Enabled: gocloak.BoolP(true), DirectAccessGrantsEnabled: gocloak.BoolP(true), BearerOnly: gocloak.BoolP(false), PublicClient: gocloak.BoolP(true), } clientId, err := client.CreateClient(ctx, token.AccessToken, realm, newClient) if err != nil { fmt.Println(err) panic("Oh no!, failed to create client :(") } else { fmt.Println("Created new client", clientId) } newUser := gocloak.User{ FirstName: gocloak.StringP("Bob"), LastName: gocloak.StringP("Uncle"), Email: gocloak.StringP("something@really.wrong"), Enabled: gocloak.BoolP(true), Username: gocloak.StringP("testUser"), } userId, err := client.CreateUser(ctx, token.AccessToken, realm, newUser) if err != nil { fmt.Println(err) panic("Oh no!, failed to create user :(") } else { fmt.Println("Created new user", userId) } err = client.SetPassword(ctx, token.AccessToken, userId, realm, "secret", false) if err != nil { fmt.Println(err) panic("Oh no!, failed to set password :(") } else { fmt.Println("Set password for user") } removeRoles := []gocloak.Role{} origRoles, err := client.GetRealmRoles(ctx, token.AccessToken, realm, gocloak.GetRoleParams{}) if err != nil { fmt.Println(err) panic("Oh no!, failed to retreive roles :(") } else { fmt.Println("Retrieved roles") } for _, r := range origRoles { removeRoles = append(removeRoles, *r) } newRole := gocloak.Role{ Name: gocloak.StringP("testRole"), } roleName, err := client.CreateRealmRole(ctx, token.AccessToken, realm, newRole) if err != nil { fmt.Println(err) panic("Oh no!, failed to create role :(") } else { fmt.Println("Created new role", roleName) } role, err := client.GetRealmRole(ctx, token.AccessToken, realm, roleName) if err != nil { fmt.Println(err) panic("Oh no!, failed to retrieve role :(") } else { fmt.Println("Retrieved role") } roles := []gocloak.Role{} roles = append(roles, *role) err = client.AddRealmRoleToUser(ctx, token.AccessToken, realm, userId, roles) if err != nil { fmt.Println(err) panic("Oh no!, failed to add role to user :(") } else { fmt.Println("Role added to user") } err = client.DeleteRealmRoleFromUser(ctx, token.AccessToken, realm, userId, removeRoles) if err != nil { fmt.Println(err) panic("Oh no!, failed to remove roles from user :(") } else { fmt.Println("Roles removed from user") } newMapper := gocloak.ProtocolMapperRepresentation{ ID: gocloak.StringP("testMapper"), Name: gocloak.StringP("testMapper"), Protocol: gocloak.StringP("openid-connect"), ProtocolMapper: gocloak.StringP("oidc-usermodel-realm-role-mapper"), Config: &map[string]string{ "access.token.claim": "true", "aggregate.attrs": "", "claim.name": "role", "id.token.claim": "true", "jsonType.label": "String", "multivalued": "", "userinfo.token.claim": "true", }, } _, err = client.CreateClientProtocolMapper(ctx, token.AccessToken, realm, clientId, newMapper) if err != nil { fmt.Println(err) panic("Oh no!, failed to add roleampper to client :(") } else { fmt.Println("Rolemapper added to client") } } |
...
Code Block | ||||
---|---|---|---|---|
| ||||
when: - key: request.auth.claims[clientRole] values: ["hwclientrole"] |
Using X509 certificates
Create the server side certificates using the following script
...
Java example available here: X.509 Authentication in Spring Security
Istio CA Certs
To allow istio to work with keycloak you must add your certificate to the istio certs when you're installing.
...
Further instruction are available here: Custom CA Integration using Kubernetes CSR
Using istio-gateway to obtain JWT tokens.
You may want to avoid connecting directly to the keycloak server for security reasons.
...
Note: The file above also allows for http calls to keycloak through the gateway, the ISS in this case is: "http://istio-ingressgateway.istio-system:80/auth/realms/<realm>". In this case the jwksUri ahould be set to the default URI for in-cluster keycloak calls i.e. "http://keycloak.default:8080/auth/realms/<realm>/protocol/openid-connect/certs"
Client authentication with signed JWT
Another option for retrieving JWT tokens for confidentail clients is using client authentication with signed JWT.
...
Note: we can also call this using https with some small modifications.
Client authentication with signed JWT with client secret
This is similar to the option above except we sign with the client secret instead of the private key.
...
Code Block | ||||
---|---|---|---|---|
| ||||
secret := "NKTh1bfR9HNwllMhdWrDMKhVJHTvwreC" token, err := jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString([]byte(secret)) if err != nil { return "", fmt.Errorf("create: sign token: %w", err) } |
Client keys tab
The client offer many different ways of configuring the authenticatio keys.
...
Code Block | ||||
---|---|---|---|---|
| ||||
claims["iss"] = "jwtclient3" claims["aud"] = "https://192.168.49.2:31561/auth/realms/x509" token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims) token.Header["kid"] = "AKAwbsKtqu9OmIwIsPOUf5zTJkIC73hzY9Myv4srjTs" tokenString, err := token.SignedString(key) if err != nil { return "", fmt.Errorf("create: sign token: %w", err) } return tokenString, nil } |
Authorization code grant
The OAuth Authorization Code Grant flow is recommended if your application support redirects.
e.g. your application is a Web application or a mobile application.
...
Login will trigger a call to the /callback endpoint whcih in turn gets the JWT token and returns it to the user.
Keycloak Rest API
Documentation for the keycloak Rest API is available here: Keycloak Admin REST API
...