...

Code Block

language	py
title	Minio Python

#!/usr/bin/python3
from keycloak import KeycloakOpenID
import requests
import xmltodict
from minio import Minio
from minio.error import (ResponseError, BucketAlreadyOwnedByYou, BucketAlreadyExists)
import uuid
import io

def get_token():
   # Configure client
   keycloak_openid = KeycloakOpenID(server_url="http://keycloak:8080",
                                    client_id="minio-cli",
                                    realm_name="minio",
                                    client_secret_key="YVZGoUdprHmqexlX30stzGWwZtT2SNll")
   return keycloak_openid.token(grant_type='client_credentials')

def get_credentials():
   token = get_token()
   r = requests.post("http://minio:9000",
                     data={
                         'Action': "AssumeRoleWithWebIdentity",
                         'Version': "2011-06-15",
                         'WebIdentityToken': token['id_token'],
                         'DurationSeconds': token['expires_in']
                     })
   tree = xmltodict.parse(r.content)
   return dict(tree['AssumeRoleWithWebIdentityResponse']['AssumeRoleWithWebIdentityResult']['Credentials'])

# Retrieve credenstials
credenstials = get_credentials()

# Initialize Minio client
client = Minio(
        "minio:9000",
        access_key=credenstials['AccessKeyId'],
        secret_key=credenstials['SecretAccessKey'],
        session_token=credenstials['SessionToken'],
        secure=False,
        region=None,
        http_client=None,
        credentials=None
)

# Create bucket if it doesn't already exist
print("Create Bucket")
print("-------------")
bucket_name = "py-bucket"
found = client.bucket_exists(bucket_name)
if not found:
    client.make_bucket(bucket_name)
    print(f"Bucket '{bucket_name}' created")
else:
    print(f"Bucket '{bucket_name}' already exists")

buckets = client.list_buckets()

# Retrieve list of buckets
print("\nBucket List")
print("-----------")
for bucket in buckets:
    print(f"{bucket.name} created on {bucket.creation_date}")

# Upload an object to a bucket
data = "I want to stream some data to minio"
object_bytes = data.encode('utf-8')
object_stream = io.BytesIO(object_bytes)
object_name =  f"{uuid.uuid1()}.txt"

print("\nUpload To Bucket")
print("----------------")
try:
    client.put_object(bucket_name, object_name, object_stream , len(object_bytes))
    print(f"{object_name} uploaded to {bucket_name}")
except Exception as ex:
    raise ex

...

A similar program written in go is available here: Minio Go Client

Links

Configure MinIO for Authentication using OpenID

AssumeRoleWithWebIdentity

AssumeRoleWithClientGrants

Keycloak over SSL

If you are using keycloak over SSL you'll need to copy the CA certifcate for keycloak in to the /root/.minio/certs/CAs directory.

This is required so Minio can read the keycloak endpoints over https.

You can create a secret to store and CA and then mount it in the /root/.minio/certs/CAs directory

Code Block

language	text
title	CA secret

kubectl create secret generic keycloak-ca-secret --from-file=keycloak-ca.crt=rootCA.crt

Minio SSE (Server Side Encryption)

MinIO SSE uses Key Encryption Service (KES) to secure objects at the storage layer.

We can test this using the KMS server at play.min.io

Download the root.key and root.cert fom the play.min.io server:

Code Block

language	text
title	curl

curl -sSL --tlsv1.2  -O 'https://raw.githubusercontent.com/minio/kes/master/root.key'  -O 'https://raw.githubusercontent.com/minio/kes/master/root.cert'

Use these to create a secret, mount the secret in the pod, then setup some environment variables to enable SSE.

Code Block

language	yml
title	Minio SSE

apiVersion: v1
kind: Secret
metadata:
  name: kms-ssl
  namespace: default
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJLRENCMjZBREFnRUNBaEI2dmViR01VZktubUJLeXFvQXBSU09NQVVHQXl0bGNEQWJNUmt3RndZRFZRUUQKREJCeWIyOTBRSEJzWVhrdWJXbHVMbWx2TUI0WERUSXdNRFF6TURFMU1qSXlOVm9YRFRJMU1EUXlPVEUxTWpJeQpOVm93R3pFWk1CY0dBMVVFQXd3UWNtOXZkRUJ3YkdGNUxtMXBiaTVwYnpBcU1BVUdBeXRsY0FNaEFMem43MzVXCmZtU0gvZ2hLcys0aVBXemlaTW1XZGlXci9zcXZxZVcrV3dTeG96VXdNekFPQmdOVkhROEJBZjhFQkFNQ0I0QXcKRXdZRFZSMGxCQXd3Q2dZSUt3WUJCUVVIQXdJd0RBWURWUjBUQVFIL0JBSXdBREFGQmdNclpYQURRUURaT3JHSwpiMkFUa0RsdTJwVGNQM0x5aFNCRHBZaDdWNFR2alJrQlRSZ2prYWNDendGTG0rbWgrN1VTOFY0ZEJwSURzSjR1CnVXb0YweTZ2YkxWR0lsa0cKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1DNENBUUF3QlFZREsyVndCQ0lFSUQ5RTdGU1lXck1EK1ZqaEk2cTU0NWNZVDlZT3lGeFpiN1VuakVlcFlEUmMKLS0tLS1FTkQgUFJJVkFURSBLRVktLS0tLQo=
---
        env:
        - name: MINIO_KMS_KES_ENDPOINT
          value: https://play.min.io:7373
        - name: MINIO_KMS_KES_CERT_FILE
          value: /etc/kms/ssl/tls.crt
        - name: MINIO_KMS_KES_KEY_FILE
          value: /etc/kms/ssl/tls.key
        - name: MINIO_KMS_KES_KEY_NAME
          value: my-first-key
        volumeMounts:
        - mountPath: "/etc/kms/ssl"
          name: kms-ssl
          readOnly: true
      volumes:      
      - name: kms-ssl
        secret:
          secretName: kms-ssl

Check the encryption has been enabled by running the following command:

Code Block

language	text
title	Key Status

mc admin kms key status <minio alias>
Key: my-first-key
   - Encryption ✔
   - Decryption ✔

Create a bucket on your minio server and upload a file to it: mc cp test.txt myminio/encrypt

Run the following command to check the file has been encrypted:

Code Block

language	text
title	mc statt

mc stat myminio/encrypt/test.txt
Name      : test.txt
Date      : 2023-04-11 15:24:48 IST
Size      : 13 B
ETag      : 8c3be95b9d3517d5b9e5d699f2692437
Type      : file
Metadata  :
  Content-Type: text/plain
Encrypted :
  X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id: arn:aws:kms:my-first-key
  X-Amz-Server-Side-Encryption               : aws:kms

If you try to view the file on the hard drive you will see the following:

Code Block

language	text
title	test.txt

cat test.txt/xl.meta
XL2 ��$���T�l[��8����!�䃤Type�V2Obj��ID��DDir���|C�\DBO8�EcAlgo�EcM�EcN�EcBSize��EcIndex�EcDist��CSumAlgo�Pa�Size-�MTime�T�l[��8�MetaSys��x-minio-internal-inline-data�true�9X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-KeyĴlbFBRVMyNTYtR0NNX1NIQTI1NtkgNmRlYWM3N2QxYTM4OWViMGUyOGQ3OWFmMTg1NTRkYTPEEPO6T631t4/N7XQ6Fe/F06zEDBPSiv8jcucioK+AuMQwiSKMoCs6ccJ3OAFoeHJ+5N1gWwaBaMaL18JNXXfQVaHb8ed9Gj6W566lXcDMpWTd�6X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm�DAREv2-HMAC-SHA256�*X-Minio-Internal-Server-Side-Encryption-Iv�,7TMvCi8frGGEx7qFSmJyL/xSoPJsom2E3eN8AaVwcD0=�6X-Minio-Internal-Server-Side-Encryption-Kms-Sealed-Key�XIAAfALq9UzWoSFkk3jZ3aCUso5gjvB0NymrfanM91hRx09V1vaJW4BsO82NXMH+RLm7ChfM1ErG6Ya6ABIb3EQ==�5X-Minio-Internal-Server-Side-Encryption-S3-Kms-Key-Id�
        my-first-key�MetaUsr��content-type�text/plain�etag�`20000f00f4a8e3628e7f7f3823c26a0b9d4fb2526422ea3dbdedbc64a63d607664f3e7c390bf8fe75eadded80909e086�v�c����&@�.��null�M�2;Z        �����ݘ�
                                                               ��� ��x8��)���
                                                                                �5�c:L������s~�#C�r*i�UR�@bPdocker@minikube:/