Recording: link to zoom recording and then two small demos we had later: demo on OID support in RIC, demo on hello-world python xapp
Schedule, see calendar for Dec-21.
Zoom = Zoom 1 bridge: https://zoom.us/j/9644759813
Near-RT RIC platform (we might change the order of demos in the beginning of the meeting)
- (10min) demo on deployment of optional platform components (Deepanshu)
- (10min) deploying the RIC, deploying an xApp and connecting an E2 simulator (Abdulwahid and Naidu (Appalanaidu Chandaka))
- (10min) demo the subscription duplication scenario using nanobot automation (Sangeetha)
- Demo to be handled in Jan-18 project meeting (10min) E2APv1.1 and support for OID in function definitions RIC-640 (Subhash+Naman)
(10min) Initial a1 mediator implementation in go (Subhash+Naman)
(10min) DBaaS/SDL: findkeys/getallkeys with glob-style patterns, multinamespace API, SDLCLI (Timo, Petri)
- (10min) E2 restart in submgr (Anssi)
Near-RT RIC xApp project
- (10min) RC demo (Dummy xApp to RC to RAN SIM) (Nitin & Team)
- (10min) HW GO xApp (Subhash)
- (10min) Bouncer xApp Subscription Delete (Amit)
Please upload slides as attachment to this demo page.
19 Comments
Mohammad Sadegh Ghasrizadeh
Hello, I just tried the demo on hello-world python xapp and followed the commands.
I cloned this repository:
for hw-python xapp.
But I get this error in the hw-xapp logs:
This error occurs when the xapp tries to get the gNB list.
Do you have any clue to fix this error?
Thanks
Mohammad Sadegh Ghasrizadeh
Update: after running docker image of hw-python, I get this error:
"ricsdl.exceptions.NotConnected: SDL not connected to backend: Error -2 connecting to service-ricplt-dbaas-tcp.ricplt.svc.cluster.local:6379. Name does not resolve."
the full log of the raised error is attached.
NearRT RIC version: E release
OS: Ubuntu 20.04 LTS
Python version: 3.8.10
Thoralf Czichy
there's an environment variable DBAAS_SERVICE_HOST whcih you might set to teh IP address of the DBAAS service (headless service. Use the IP of the pod as per "kubectl get pods -A -o wide" (see column "IP")
Mohammad Sadegh Ghasrizadeh
Thank you for your reply.
I followed the steps below:
1) copy the IP address returned by "kubectl get pods -A -o wide | grep statefulset-ricplt-dbaas-server-0" (which returned 10.244.0.43)
2) paste the above IP address in "Dockerfile" as the value of "ENV DBAAS_SERVICE_HOST". ( i.e. ENV DBAAS_SERVICE_HOST = 10.244.0.43)
3) build the Dockerfile
4) run the created docker image.
It seems the previous error (i.e. Name does not resolve) is solved but again I have this error: "msgpack.exceptions.ExtraData: unpack(b) received extra data."
Thoralf Czichy
could you check which Redis server version you have?
kubectl exec -ti statefulset-ricplt-dbaas-server-0 -n ricplt -- /bin/sh
redis-cli
INFO
we changed this in F (to Redis 6 from Redis 5).
Python client libraries need to be aligned. I understand this was changed on Jan-21-2022. Worth checking first that there's no issue because of hw-py using old python redis libs. Though, (without checking) newer redis libs should still work with old Redis servers.
https://gerrit.o-ran-sc.org/r/q/project:ric-plt%252Fsdlpy
Thoralf Czichy
Also unless you really need to use the E release (because of E2APv1.1), I suggest to use teh F release (E2APv2.0).
Mohammad Sadegh Ghasrizadeh
After switching to the F release, it is the output of the commands you mentioned in the previous comment:
----------------
/data # redis-cli
127.0.0.1:6379> INFO
# Server
redis_version:6.2.6
----------------
in the hw-python container :
The error does not appear when no gNB is connected but right after connecting gNB (by running e2sim) the error reraised:
Thoralf Czichy
It seems that the GNB data in database is not MessagePack data. when trying to msgpack.unpackb the GNB data we get the similar error. No idea what has been changed. Worth trying if the data can be unpacked using e.g. golang. This is to make sure whether it's a client decoding problem or one where teh data isn't correctly encoded.
msgpack.unpackb(b'\n\x12gnb_244_007_00332b(\x022"\n\x0642F470\x12\x180000000000110011001010118\x02h\x01J\xb0\'\x12\xdb\x03\x08\x04\x12\xd4\x0368304F52414E2D4532534D2D.......04C4D4E204964656E746974791088C300805344\x18\x01\x18\x01', raw=False)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "msgpack/_unpacker.pyx", line 201, in msgpack._cmsgpack.unpackb
msgpack.exceptions.ExtraData: unpack(b) received extra data.
Thoralf Czichy
Not sure how this could ever have worked. I understand the values in SDL are stored in proto format. How could decoding it using msgpack format have ever worked?
I checked from rNibReader.go in nodeb-rnib repo and it uses getByKeyAndUnmarshal() which is protobuf decoding the data. Something similar probably needs to be done in python.
The weird code is in the sample hello-world python xapp: ric-app/hw-python
hw-python/src/manager/SdlManager.py: self._rmr_xapp.sdl_find_and_get(self.__namespace, "GNB") which invokes this function with usemsgpack=True (you could try setting it to false, but better way -> see below):
def sdl_find_and_get(self, namespace, prefix, usemsgpack=True):
You could try change the HW xapp to use functions like get_list_gnb_ids(). I presume this is the correct function to use. These functions all use usemsgpack=false and then decode the protobuf data using the proto decoder.
Mohammad Sadegh Ghasrizadeh
Thank you Thoralf Czichy,
Both methods worked for me and now the error is solved but the xApp cannot send subscription request; i.e. nothing is changed in the e2sim logs. I'm trying to understand the code and appreciate any help.
steve ratti
Hi i am looking for E release ricplt and its docker images . please provide me the link of E release
Abdul Wahid W
I think this is the one you are looking for - Near-RT RIC (E release)
steve ratti
thanks for your response , yet i was looking for this , because there are command list for F and Other releases
steve ratti
steve ratti
hi everyoine my A1 mediator is deployment-ricplt-a1mediator-69cfc57b87-slmg4 0/1 CrashLoopBackOff , giveimg me this error can anybody help me what to do
Abdul Wahid W
The k8s healthcheck is failing for A1. Could you provide following from A1 mediator pod ?
kubectl logs -p <a1mediatorpod> -n ricplt
kubectl describe pods <a1mediatorpod> -n ricplt
steve ratti
here are describe pods
root@k2:~# kubectl describe pods deployment-ricplt-a1mediator-69cfc57b87-slmg4 -n ricplt
Name: deployment-ricplt-a1mediator-69cfc57b87-slmg4
Namespace: ricplt
Priority: 0
Node: k2/10.0.2.15
Start Time: Mon, 15 Apr 2024 13:19:40 +0200
Labels: app=ricplt-a1mediator
pod-template-hash=69cfc57b87
release=r4-a1mediator
Annotations: <none>
Status: Running
IP: 10.244.0.45
IPs:
IP: 10.244.0.45
Controlled By: ReplicaSet/deployment-ricplt-a1mediator-69cfc57b87
Containers:
container-ricplt-a1mediator:
Container ID: docker://5e2e4a4607ea3533f9c1096fc3f064bb1147661ce873c02e251b0f42be94a85a
Image: nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-a1:2.5.1
Image ID: docker-pullable://nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-a1@sha256:9f3830b25766602849cda7f545a836320f26ca135c79bae95832c75c3db1513b
Ports: 10000/TCP, 4561/TCP, 4562/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Mon, 15 Apr 2024 14:31:09 +0200
Finished: Mon, 15 Apr 2024 14:32:09 +0200
Ready: False
Restart Count: 17
Liveness: http-get http://:http/A1-P/v2/healthcheck delay=0s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:http/A1-P/v2/healthcheck delay=0s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
configmap-ricplt-a1mediator-env ConfigMap Optional: false
configmap-ricplt-dbaas-appconfig ConfigMap Optional: false
Environment: <none>
Mounts:
/opt/route from a1conf (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-8j482 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
a1conf:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: configmap-ricplt-a1mediator-a1conf
Optional: false
default-token-8j482:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-8j482
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning BackOff 37m (x90 over 61m) kubelet, k2 Back-off restarting failed container
Normal Pulled 5m50s (x14 over 66m) kubelet, k2 Container image "nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-a1:2.5.1" already present on machine
Normal Created 5m49s (x15 over 67m) kubelet, k2 Created container container-ricplt-a1mediator
Normal Started 5m48s (x15 over 67m) kubelet, k2 Started container container-ricplt-a1mediator
Warning Unhealthy 5m39s (x76 over 67m) kubelet, k2 Readiness probe failed: HTTP probe failed with statuscode: 404
Warning Unhealthy 2m40s (x51 over 67m) kubelet, k2 Liveness probe failed: HTTP probe failed with statuscode: 404
steve ratti
hi here are the logs
root@k2:~# kubectl logs -p deployment-ricplt-a1mediator-69cfc57b87-slmg4 -n ricplt
Invalid Log Level defined in ConfigMap
Invalid Log Level defined in ConfigMap
1713184270099 1/RMR [INFO] ric message routing library on SI95 p=4562 mv=3 flg=02 id=a (d5197f5 4.8.0 built: Dec 2 2021)
1713184271402 1/RMR [INFO] sends: ts=1713184271 src=service-ricplt-a1mediator-rmr.ricplt:4562 target=service-ricxapp-admctrl-rmr.ricxapp:4563 open=0 succ=0 fail=0 (hard=0 soft=0)
::ffff:10.244.0.1 - - [2024-04-15 12:31:19] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.001288
::ffff:10.244.0.1 - - [2024-04-15 12:31:20] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.001077
::ffff:10.244.0.1 - - [2024-04-15 12:31:29] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.002875
::ffff:10.244.0.1 - - [2024-04-15 12:31:30] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.000902
::ffff:10.244.0.1 - - [2024-04-15 12:31:39] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.001696
::ffff:10.244.0.1 - - [2024-04-15 12:31:40] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.000616
1713184302436 1/RMR [INFO] sends: ts=1713184302 src=service-ricplt-a1mediator-rmr.ricplt:4562 target=service-ricxapp-admctrl-rmr.ricxapp:4563 open=0 succ=0 fail=0 (hard=0 soft=0)
::ffff:10.244.0.1 - - [2024-04-15 12:31:50] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.000793
::ffff:10.244.0.1 - - [2024-04-15 12:32:00] "GET /A1-P/v2/healthcheck HTTP/1.1" 404 329 0.000663
root@k2:~#
Abdul Wahid W
You can check this issue - RIC-1045 - Getting issue details... STATUS . It seems the URL for liveness check is incorrect there. It should be "/a1-/healthcheck" instead of "/A1-P/v2/healthcheck"
[root@master bin]# curl -i http://172.16.1.34:10000/A1-P/v2/healthcheck
HTTP/1.1 404 NOT FOUND
Content-Type: application/problem+json
Content-Length: 205
Date: Mon, 15 Apr 2024 19:35:19 GMT
{
"detail": "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.",
"status": 404,
"title": "Not Found",
"type": "about:blank"
}
[root@master bin]# curl -i http://172.16.1.34:10000/a1-p/healthcheck
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 3
Date: Mon, 15 Apr 2024 19:35:25 GMT
""