Skip to end of metadata
Go to start of metadata


Overview

 

Pre requisite
  • Near Realtime RIC installation completed
  • Near Realtime RIC up and running

   # kubectl get pods -n ricplt   # There should be ~16 pods running in the ricplt namespace.  

Step 1: Prep Tasks (Preparing for xApp on-boarding)

Each xApp is required to provide a "descriptor".   The descriptor is used by the Near RT RIC xApp On-boarder to generate a Helm chart and other deployment related artifacts for the xapp.  At this step we are preparing for API calls into the xApp On-Boarder by providing thee locations of the xApp descriptors.

Run ...

$ sudo -i
# Working directory will be dep where near realtime ric is deployed

$ cd dep
$ echo '{"config-file.json_url": "https://gerrit.o-ran-sc.org/r/gitweb?p=ric-app/qp.git;a=blob_plain;f=xapp-descriptor/config.json;hb=HEAD" }' > onboard.qp.url
$ echo '{"config-file.json_url": "https://gerrit.o-ran-sc.org/r/gitweb?p=ric-app/qp-driver.git;a=blob_plain;f=xapp-descriptor/config.json;hb=HEAD" }' > onboard.qpd.url
$ echo '{"config-file.json_url": "https://gerrit.o-ran-sc.org/r/gitweb?p=ric-app/ts.git;a=blob_plain;f=xapp-descriptor/config.json;hb=HEAD" }' > onboard.ts.url
Step 2: Onboard xApps

Now we invoke API calls into the xApp On-boarder, providing it the locations of the xApp descripters.

Run:

$ curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download"      --header 'Content-Type: application/json' --data-binary "@./onboard.ts.url
$ curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download"      --header 'Content-Type: application/json' --data-binary "@./onboard.qp.url"
$ curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download"      --header 'Content-Type: application/json' --data-binary "@./onboard.qpd.url"
# checking what have been on-boarded
$ curl --location --request GET "http://$(hostname):32080/onboard/api/v1/charts"
Step 3:  Deploy xApp

At this step, we deploy the xApps by invoking the xApp Manager's API.  Note that the names of the xApp to be deployed must match with what the on-boarder has.  Once receiving the deploy API call, the xApp Manager will make API call into Helm/Kubernetes to deploy the xApp's Helm chart.  The Routing Manager is also involved if the xApp is to process RMR messages, – it will compote the routes and send out route updates.

Run ...

$ curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps"      --header 'Content-Type: application/json' --data-raw '{"xappName": "qp"}'
$ curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps"      --header 'Content-Type: application/json' --data-raw '{"xappName": "qpdriver"}'
$ curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps"      --header 'Content-Type: application/json' --data-raw '{"xappName": "trafficxapp"}'
# now check the running state of these xApps
$ kubectl get pods -n ricxapp
Step 4: Populate the Database with test RAN data

This step is needed for the phase one of the Traffic Steering use case.  In phase one, the RAN data is artificial and the data points need to be injected into RNIB database.  After these artificial data points are injected, we should see the Traffic Steering xApp starting to process these data, using the default threshold of 0.

Run ...

$ git clone http://gerrit.o-ran-sc.org/r/ric-app/ts -b bronze
$ cd ts/test/populatedb
$  ./populate_db.sh
Step 5:  Create a Policy Type

Here we define a new policy type and call the A1 mediator to create such a policy type.

Run ...

$ cd ~/dep/
$ POLICY_TYPE_ID="20008"
$ echo '{ "name": "tsapolicy", "description": "tsa parameters", "policy_type_id": 20008, "create_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "title": "TS Policy", "description": "TS policy type", "type": "object", "properties": { "threshold": { "type": "integer", "default": 0 } }, "additionalProperties": false } } ' > ts-policy-type-20008.json
$ curl -v -X PUT "http://$(hostname):32080/a1mediator/a1-p/policytypes/${POLICY_TYPE_ID}"   -H "accept: application/json" -H "Content-Type: application/json"   -d @./ts-policy-type-${POLICY_TYPE_ID}.json
Step 6:  List Policy Types

We now should see the newly created policy type when querying the A1 mediator.

Run ...

$ curl -X GET --header "Content-Type: application/json" --header "accept: application/json"   http://$(hostname):32080/a1mediator/a1-p/policytypes
Step 7:  Create a Policy Instance

Policy type is merely a declaration.  Now we create a policy instance, filling the properties with actual values.  This is also carried out by the A1 mediator. 

A1 mediator will distribute the new policy instance to the Traffic Steering xApp. 

And then we should see that the TS xApp receives this policy instancee and its behavior of "filtering RAN data by threshold" changing to using the new threshold value.

Run...

$ POLICY_ID="tsapolicy145"
$ curl -X PUT --header "Content-Type: application/json"   --data "{\"threshold\" : 5}"   http://$(hostname):32080/a1mediator/a1-p/policytypes/${POLICY_TYPE_ID}/policies/${POLICY_ID}
Tutorial Videos

Hello World xApp xapp-hw.mp4


Helpful Hints

Kubectl commads:

kubectl get pods -n nampespace - gets a list of Pods running

kubectl get logs -n namespace name_of_running_pod






Complete these tasks to get started


Recent space activity

Space contributors

{"mode":"list","scope":"descendants","limit":"5","showLastTime":"true","order":"update","contextEntityId":20875211}

  • No labels

35 Comments

  1. Hi Lusheng,

    Thanks for the RIC XAPP deployment guide.

    I was able to deploy all the xAPPs except QP driver.

    I have the following error when I try to onboard using  curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps"      --header 'Content-Type: application/json' --data-raw '{"xappName": "qpdriver"}'

    {
    "error_source": "config-file.json",
    "error_message": "'__empty_control_section__' is a required property",
    "status": "Input payload validation failed"
    }

    I think the error is from the descriptor of the version 1.0.9 located at onboard.qpd.url

    But when I changed the qpdriver version to 1.0.0 it works  

    https://gerrit.o-ran-sc.org/r/gitweb?p=ric-app/qp-driver.git;a=blob_plain;f=xapp-descriptor/config.json;h=94582086d02df6299dd902556aa2e2111ed94268;hb=4074973dbbe90e11643c229fff39bb8a6f662dc8



    Thanks

    1. Anonymous

      Hello!

      This can be resolved by deleting lines 38-41.

      "controls": {
      "example_int": 10000,
      "example_str": "value"
      }


      --Cornellius Dagmang

  2. Hi Luseng,

    I have created multiple policies while testing & want to delete those policies from Nonrt dashboard GUI. I have used below command to delete

    curl -X DELETE --header "Content-Type: application/json" --header "accept: application/json"   http://192.168.122.31:32080/a1mediator/a1-p/policytypes/20020

    By using above command it is getting deleted from VM but not Nonrt dashboard GUI. Please guide me the command to delete policies from Nonrt dashboard GUI.


    Thanks ,

    Gautam


  3. After onboarding all the xApps, there is a error sending message from ts-xapp

    In get_sdl_ue_data()
    message body {"UEPredictionSet": ["12345"]}
    payload length 30
    <SNDR> send failed: 2

    Any known fixes/workarounds?

      1. Here is the normal output:

        In get_sdl_ue_data()
        message body {"UEPredictionSet": ["12345"]}
        payload length 30
        Prediction Callback got a message, type=30002 , length=182
        payload is { "12345" : { "310-680-200-555001" : [ 2000000 , 1200000 ], "310-680-200-555002" : [ 800000 , 400000 ], "310-680-200-555003" : [ 800000 , 400000 ] } }
        Prediction for 12345


        from the source code:

        void send_prediction_request(vector<string> ues_to_predict) {

        int mtype = 30000;

        // payload updated in place, nothing to copy from, so payload parm is nil
        if ( ! msg->Send_msg( mtype, Message::NO_SUBID, strlen( (char *) send_payload.get() ), NULL )) {
            fprintf( stderr, "<SNDR> send failed: %d\n", msg->Get_state() );
        }

        }

        void prediction_callback( Message& mbuf, int mtype, int subid, int len, Msg_component payload, void* data ) {

        cout << "Prediction Callback got a message, type=" << mtype << " , length=" << len << "\n";
        cout << "payload is " << payload.get() << "\n";

        }

        which means that prediction_callback is not correctly called.


        in tsxapp.main, prediction_callback is binded to MSG_ID=30002.

        extern int main( int argc, char** argv ) {
            xfw->Add_msg_cb( 20010, policy_callback, NULL );
            xfw->Add_msg_cb( 30002, prediction_callback, NULL );
        }


        RMR will route msg with MSG_ID=30000 to qpdriver xapp. qpdriver will process it , and generate another msg with MSG_ID=30001

        rmr_xapp.register_callback(steering_req_handler, 30000)

        success = self.rmr_send(payload, 30001)


        RMR will further route msg with MSG_ID=30001 to qp xapp. qp will process it, and generate another msg with MSG_ID=30002, which will be routed back to tsxapp by RMR.

        qp_xapp.register_callback(qp_predict_handler, 30001)

        success = self.rmr_send(mock_msg.encode(), 30002)


        Based on above analysis, my best guess is that, qp-driver and qp xapp has not been deployed successfully.

  4. Hi Experts,

    I got RMR_ERR_NOENDPT after creating policy instance. Can you pls help? thanks

    RMR_ERR_NOENDPTsend/call could not find an endpoint based on msg type


    here is part of 'kubectl logs -f deployment-ricplt-a1mediator-66fcf76c66-8dxcj -n ricplt':

    ::ffff:10.244.0.53 - - [2020-08-07 07:41:28] "PUT /a1-p/policytypes/20008/policies/tsapolicy145 HTTP/1.1" 202 116 0.007207

    {"ts": 1596786088608, "crit": "DEBUG", "id": "a1.a1rmr", "mdc": {}, "msg": "_send_msg: sending: {'payload': b'{\"operation\": \"CREATE\", \"policy_type_id\": 20008, \"policy_instance_id\": \"tsapolicy145\", \"payload\": {\"threshold\": 5}}', 'payload length': 115, 'message type': 20010, 'subscription id': 20008, 'transaction id': b'67712138d88111eab3bcd2cb2aea4bae', 'message state': 0, 'message status': 'RMR_OK', 'payload max size': 4096, 'meid': b'', 'message source': 'service-ricplt-a1mediator-rmr.ricplt:4562', 'errno': 0}"}

    {"ts": 1596786088608, "crit": "WARNING", "id": "a1.a1rmr", "mdc": {}, "msg": "RMR send failed; pre-send summary: {'payload': b'{\"operation\": \"CREATE\", \"policy_type_id\": 20008, \"policy_instance_id\": \"tsapolicy145\", \"payload\": {\"threshold\": 5}}', 'payload length': 115, 'message type': 20010, 'subscription id': 20008, 'transaction id': b'67712138d88111eab3bcd2cb2aea4bae', 'message state': 0, 'message status': 'RMR_OK', 'payload max size': 4096, 'meid': b'', 'message source': 'service-ricplt-a1mediator-rmr.ricplt:4562', 'errno': 0}, post-send summary: {'payload': None, 'payload length': 115, 'message type': 20010, 'subscription id': 20008, 'transaction id': b'67712138d88111eab3bcd2cb2aea4bae', 'message state': 2, 'message status': 'RMR_ERR_NOENDPT', 'payload max size': 4096, 'meid': b'', 'message source': 'service-ricplt-a1mediator-rmr.ricplt:4562', 'errno': 6}"}

    {"ts": 1596786088608, "crit": "DEBUG", "id": "a1.a1rmr", "mdc": {}, "msg": "_send_msg: result message state: 2"}

    {"ts": 1596786088608, "crit": "WARNING", "id": "a1.a1rmr", "mdc": {}, "msg": "_send_msg: failed after 20 retries"}

    1. Anonymous

      hi all,

      i rebuild the docker image from "ric-app-ts 1.0.11" of Gerrit, and re-deploy trafficxapp, there is no RMR_ERR_NOENDPT error now.

      ::ffff:10.244.0.61 - - [2020-08-07 14:12:02] "PUT /a1-p/policytypes/20008/policies/tsapolicy145 HTTP/1.1" 202 116 0.008321
      {"ts": 1596809522972, "crit": "DEBUG", "id": "a1.a1rmr", "mdc": {}, "msg": "_send_msg: sending: {'payload': b'{\"operation\": \"CREATE\", \"policy_type_id\": 20008, \"policy_instance_id\": \"tsapolicy145\", \"payload\": {\"threshold\": 5}}', 'payload length': 115, 'message type': 20010, 'subscription id': 20008, 'transaction id': b'f769383ad8b711eaacd4cac483485e64', 'message state': 0, 'message status': 'RMR_OK', 'payload max size': 4096, 'meid': b'', 'message source': 'service-ricplt-a1mediator-rmr.ricplt:4562', 'errno': 0}"}
      {"ts": 1596809522977, "crit": "DEBUG", "id": "a1.a1rmr", "mdc": {}, "msg": "_send_msg: result message state: 0"}

  5. Anonymous

    Would someone provide prerequisite and instructions to create the xApp??

    1. Please post after you log in to Confluence so people know who you are.

  6. Anonymous

    Can anyone let me know the same steps for Admission Control xAPP

  7. Was the image(nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:8-u18.04) used ts/test/Dockerfile removed in nexus?

    I can't find it or its other version for ubuntu18 in nexus


    1. pls refer to ORAN Base Docker Images for CI Builds#Imagebldr-ubuntu18-c-go

      you need: 

      $docker pull nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:9-u18.04


      1. I had tried it before and I could find  bldr-ubuntu18-c-go:9-u18.04 in nexus at that time,

        but there is not bldr-ubuntu18-c-go in nexus now.


        1. Yeah, there is no bldr-ubuntu18-c-go image any more in the repository.

          I propose two solutions:

          (1) you can build the docker image manually and retag it:

          https://gerrit.o-ran-sc.org/r/gitweb?p=it/dev.git;a=blob;f=bldr-imgs/bldr-ubuntu18-c-go/Dockerfile;h=c7be3e4a4456a74ddfa5e4e222b77d9fbdc03fec;hb=HEAD

          $git clone https://gerrit.o-ran-sc.org/r/it/dev.git

          $cd bldr-imgs/bldr-ubuntu18-c-go

          $sudo docker build -t nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:9-u18.04 .

          (2) you may pull the image from my docker-hub and retag it:

          $docker pull zhenggao2/bldr-ubuntu18-c-go:9-u18.04

          $docker tag zhenggao2/bldr-ubuntu18-c-go:9-u18.04 nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:9-u18.04

          $docker rmi zhenggao2/bldr-ubuntu18-c-go:9-u18.04

          1. It works, thanks!


            If someone meet issue when running populate_db.sh

            You can try the patch

            diff --git a/test/populatedb/Dockerfile b/test/populatedb/Dockerfile

            index 479fcd5..bb44b5a 100644

            --- a/test/populatedb/Dockerfile

            +++ b/test/populatedb/Dockerfile

            @@ -13,7 +13,7 @@

             #   See the License for the specific language governing permissions and

             #   limitations under the License.

             # ==================================================================================

            -FROM nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:8-u18.04 as buildenv

            +FROM zhenggao2/bldr-ubuntu18-c-go:9-u18.04 as buildenv


             RUN mkdir /playpen

             RUN mkdir /playpen/src

            @@ -22,7 +22,8 @@ ENV LD_LIBRARY_PATH=/usr/local/lib


             RUN apt-get install -y cpputest

             RUN apt-get remove -y libboost-all-dev

            -RUN apt-get install -y  libboost-all-dev

            +RUN apt-get update

            +RUN apt-get install -y libboost-all-dev

             RUN apt-get install -y libhiredis-dev

             RUN apt-get install -y valgrind


        2. Late response but the image is tagged as 1.9.0 rather than 9-u18.04

  8. Anonymous

    Can anyone help me why my pod is not running after running the script i.e populate_db.sh.

    kubectl get pods -n ricplt
    NAME READY STATUS RESTARTS AGE
    dbprepopjob-ws99s 0/1 ImagePullBackOff 0 83s


    1. Anonymous

      Dear all:

      I have a same problem, anyone help?

  9. Anonymous

    Hi everyone,

    Thanks for the RIC XAPP deployment guide.

    I was able to deploy all the xAPPs except QP driver.

    I have the following error when I try to onboard using  

    curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download" --header 'Content-Type: application/json' --data-binary "@./onboard.qpd.url"


    error message:

    {
    "error_source": "config-file.json",
    "error_message": "'__empty_control_section__' is a required property",
    "status": "Input payload validation failed"
    }

  10. Anonymous

    Hi everyone,

    Thanks for the RIC XAPP deployment guide.

    I was able to deploy all the xAPPs except QP driver.

    I have the following error in this step: 


    root@near-VirtualBox:~/dep/ts/test/populatedb# ./populate_db.sh
    release "dbprepop" deleted
    Sending build context to Docker daemon 19.97kB
    Step 1/17 : FROM nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:8-u18.04 as buildenv
    manifest for nexus3.o-ran-sc.org:10004/o-ran-sc/bldr-ubuntu18-c-go:8-u18.04 not found: manifest unknown: manifest unknown
    NAME: dbprepop
    LAST DEPLOYED: Tue Feb 23 16:15:45 2021
    NAMESPACE: ricplt
    STATUS: DEPLOYED

    RESOURCES:
    ==> v1/Job
    NAME COMPLETIONS DURATION AGE
    dbprepopjob 0/1 0s 0s

    ==> v1/Pod(related)
    NAME READY STATUS RESTARTS AGE
    dbprepopjob-667t5 0/1 ContainerCreating 0 0s

  11. Hi,


    I'm working with Bronze release.

    I have a question regarding the use of private docker registries for xApp image hosting.

    As far as I understand, in the hello world xApp scenario, the image is pulled from a public repository : nexus3.o-ran-sc.org:10002, which is specified in the config file.

    I tried to use a private repository instead, so I replaced the "registry" name in the config file by my own registry : 10.0.0.1:5000 , which hosts my app : testApp

    When I try to pull my image using :

            docker pull 10.0.0.1:5000/testApp

    the image is successfully pulled.

    However, when I try to load the configuration using :

            curl --location --request POST "http://$KONG_IP:32080/onboard/api/v1/onboard/download" --header 'Content-Type: application/json' --data-binary "@./onboard.hw.url"

    I receive an error indicating :

            10.0.0.1:5000 does not match ^[A-Za-z0-9\\\\.-]{1,}\\\\.[A-Za-z]{1,}(?:\\\\:\\\\d+)?$

    I agree that my input does not match the required format, but I do not understand why this format is required. I cannot put at least one alphabetical letter between the IP address and the port, as required (at least I didn't find a way to do so, but I'm not a docker expert).

    Do you know the reason for this format check ? Is there something that I am missing, or do I misuse the registry field ?


    Thank you in advance,

    Best regards,

    Cedric


    EDIT : I found a workaround : I edited the /etc/host file of the machine hosting the near RT RIC, and added an entry with a compliant name for the repo : 10.0.0.1 docker-repository.local

    However this solution requires to have admin access to the near RT RIC machine, I think that the validity check of the config file should accept IP addresses, unless there is a reason not to do so.

    1. Modify the regex to accept any location ie: .* and then this issue goes away. you will need to rebuild the corresponding image and replace it in the running kubernetes cluster.


  12. Anonymous

    HI every , I have one question about Policy Type.

    if it's only one policy type for only one xapp?  


  13. I have successfully ported the RIC from x86 to ARM64 but having issues debugging xapp. Modified the regex to accept my local registry. Running python http server to serve the file to for xapp.

    NAMESPACE     NAME                                                         READY   STATUS      RESTARTS   AGE
    kube-system   coredns-5644d7b6d9-s9sqw                                     1/1     Running     3          7d1h
    kube-system   coredns-5644d7b6d9-z5g9p                                     1/1     Running     3          7d1h
    kube-system   etcd-kubernetes-master                                       1/1     Running     3          7d1h
    kube-system   kube-apiserver-kubernetes-master                             1/1     Running     3          7d1h
    kube-system   kube-controller-manager-kubernetes-master                    1/1     Running     3          7d1h
    kube-system   kube-flannel-ds-ztz5s                                        1/1     Running     3          7d1h
    kube-system   kube-proxy-sr75b                                             1/1     Running     3          7d1h
    kube-system   kube-scheduler-kubernetes-master                             1/1     Running     3          7d1h
    kube-system   tiller-deploy-68bf6dff8f-hlg22                               1/1     Running     8          7d1h
    ricinfra      deployment-tiller-ricxapp-6b6b4c787-89w5q                    1/1     Running     3          7d1h
    ricinfra      tiller-secret-generator-5fzwj                                0/1     Completed   0          7d1h
    ricplt        deployment-ricplt-a1mediator-cb47dc85d-b9nb4                 1/1     Running     135        7d1h
    ricplt        deployment-ricplt-appmgr-5fbcf5c7f7-tmqrj                    1/1     Running     3          7d1h
    ricplt        deployment-ricplt-e2mgr-7dbfbbb796-mxjl4                     1/1     Running     1          5d20h
    ricplt        deployment-ricplt-e2term-alpha-8665fc56f6-mkqqr              1/1     Running     5          7d1h
    ricplt        deployment-ricplt-jaegeradapter-85cbdfbfbc-xgthz             1/1     Running     4          7d1h
    ricplt        deployment-ricplt-o1mediator-86587dd94f-9gp7c                1/1     Running     3          7d1h
    ricplt        deployment-ricplt-rtmgr-67c9bdccf6-769rh                     1/1     Running     5          7d1h
    ricplt        deployment-ricplt-submgr-6ffd499fd5-zfhm2                    1/1     Running     3          7d1h
    ricplt        deployment-ricplt-vespamgr-68b68b78db-5lkvf                  1/1     Running     3          7d1h
    ricplt        deployment-ricplt-xapp-onboarder-57f78cfdf-769ls             2/2     Running     6          7d1h
    ricplt        r4-infrastructure-kong-84cd44455-pjxmm                       2/2     Running     14         7d1h
    ricplt        r4-infrastructure-prometheus-alertmanager-75dff54776-bjwsk   2/2     Running     6          7d1h
    ricplt        r4-infrastructure-prometheus-server-5fd7695-pr2pw            1/1     Running     3          7d1h
    ricplt        statefulset-ricplt-dbaas-server-0                            1/1     Running     1          5d20h


    I ported the ric-app-hw:1.0.6 app over to arm64 and ran the following command:

    curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download" --header 'Content-Type: application/json' --data-binary "@./onboard.hw.url"

    response:

    {

    "status": "Created"

    }

    command:

    curl --location --request GET "http://$(hostname):32080/onboard/api/v1/charts"

    response:
    {
    "hwxapp": [
    {
    "name": "hwxapp",
    "version": "1.0.0",
    "description": "Standard xApp Helm Chart",
    "apiVersion": "v1",
    "appVersion": "1.0",
    "urls": [
    "charts/hwxapp-1.0.0.tgz"
    ],
    "created": "2021-03-30T19:38:11.209682186Z",
    "digest": "aa48978de777bb2c04cd843778696a4b116fe3c134aafbae9d674f97b78a23df"
    }
    ]
    }

    But I never see the xapp start on kubernetes. 

    command:

    kubectl get pods -n ricxapp

    response:

    No resources found in ricxapp namespace.


    EDIT: Sorry forgot the most important part.

    curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps" --header 'Content-Type: application/json' --data-raw '{"xappName": "hwxapp"}'

    "operation XappDeployXapp has not yet been implemented"


    UPDATE:

    Looks like appmgr under the master branch is not working correctly. Ive gone back to cherry and now im having cert issues:


    {"ts":1617300249780,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T18:04:09Z","xm":"0.4.3"},"msg":"Running command: [/bin/sh -c helm list --all --deployed --output yaml --namespace=ricxapp] "}
    {"ts":1617300249842,"crit":"ERROR","id":"appmgr","mdc":{"time":"2021-04-01T18:04:09Z","xm":"0.4.3"},"msg":"Command failed: exit status 2 - could not read x509 key pair (cert: \"/opt/ric/secret/helm-client.cert\", key: \"/opt/ric/secret/helm-client.key\"): can't load key pair from cert /opt/ric/secret/helm-client.cert and key /opt/ric/secret/helm-client.key: tls: failed to find any PEM data in certificate input\n, retrying"}
    {"ts":1617300251842,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T18:04:11Z","xm":"0.4.3"},"msg":"Listing deployed xapps failed: could not read x509 key pair (cert: \"/opt/ric/secret/helm-client.cert\", key: \"/opt/ric/secret/helm-client.key\"): can't load key pair from cert /opt/ric/secret/helm-client.cert and key /opt/ric/secret/helm-client.key: tls: failed to find any PEM data in certificate input\n"}
    {"ts":1617300251842,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T18:04:11Z","xm":"0.4.3"},"msg":"Helm list failed: could not read x509 key pair (cert: \"/opt/ric/secret/helm-client.cert\", key: \"/opt/ric/secret/helm-client.key\"): can't load key pair from cert /opt/ric/secret/helm-client.cert and key /opt/ric/secret/helm-client.key: tls: failed to find any PEM data in certificate input\n"}

    I do not see this issue with master, I redeployed the entire ric once more and still see the issue.


    Any ideas on how to debug this?

    1. Anonymous

      You're missing step 3.

      curl --location --request POST "http://$(hostname):32080/appmgr/ric/v1/xapps"      --header 'Content-Type: application/json' --data-raw '{"xappName": "hwxapp"}'
    2. Anonymous

      Is tiller added to k8s and ready before deploy? I had the same issue and adding a --wait to helm init (with tiller installed) worked

  14. The ric-plt-appmgr was working no issues but I kept getting “operation XappDeployXapp has not yet been implemented” or some variant when deploying an xapp. I pulled changes and rebuilt but when I redeploy the secrets are mising. The certs and keys were empty:

    {"ts":1617306072432,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:41:12Z","xm":"0.4.3"},"msg":"helm init failed, retyring ..."}
    {"ts":1617306082432,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:41:22Z","xm":"0.4.3"},"msg":"Init for Version 2"}
    {"ts":1617306082432,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:41:22Z","xm":"0.4.3"},"msg":"Running command: [/bin/sh -c helm init -c --skip-refresh] "}
    {"ts":1617306082534,"crit":"ERROR","id":"appmgr","mdc":{"time":"2021-04-01T19:41:22Z","xm":"0.4.3"},"msg":"Command failed: exit status 2 - could not read x509 key pair (cert: \"/opt/ric/secret/helm-client.cert\", key: \"/opt/ric/secret/helm-client.key\"): can't load key pair from cert /opt/ric/secret/helm-client.cert and key /opt/ric/secret/helm-client.key: asn1: structure error: tags don't match (16 vs {class:0 tag:13 length:45 isCompound:true}) {optional:false explicit:false application:false private:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false} certificate @2\n, retrying"}


    This was corrected by running the

    /dep/ric-dep/helm/appmgr/templates/bin/_appmgr-tiller-secret-copier.sh.tpl


    This created three items:

    • helm-client.cert
    • helm-client.key
    • tiller-ca.cert


    Now I have issues with the certs.

    {"ts":1617306548369,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:49:08Z","xm":"0.4.3"},"msg":"Running command: [/bin/sh -c helm list --all --deployed --output yaml --namespace=ricxapp] "}
    {"ts":1617306548485,"crit":"ERROR","id":"appmgr","mdc":{"time":"2021-04-01T19:49:08Z","xm":"0.4.3"},"msg":"Command failed: exit status 1 - Error: x509: certificate signed by unknown authority\n, retrying"}
    {"ts":1617306550485,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:49:10Z","xm":"0.4.3"},"msg":"Listing deployed xapps failed: Error: x509: certificate signed by unknown authority\n"}
    {"ts":1617306550485,"crit":"INFO","id":"appmgr","mdc":{"time":"2021-04-01T19:49:10Z","xm":"0.4.3"},"msg":"Helm list failed: Error: x509: certificate signed by unknown authority\n"}


    How should this process be completed?

  15. Anonymous

    Hi Everyone,

        I am trying to bring up the XAPP on bronze release, and all the pods are up and running.

    But when I try to onboard Xapp
    curl --location --request POST "http://$(hostname):32080/onboard/api/v1/onboard/download" --header 'Content-Type: application/json' --data-binary "@./onboard.ts.url"

    The curl command just waits

    When I try to read the /var/log/syslog

    I see these logs


    Apr 7 08:21:41 bronze kubelet[699]: W0407 08:21:41.202464 699 kubelet_pods.go:849] Unable to retrieve pull secret ricplt/secret-nexus3-o-ran-sc-org-10002-o-ran-sc for ricplt/deployment-ricplt-vespamgr-7458d9b5d-kpv87 due to secret "secret-nexus3-o-ran-sc-org-10002-o-ran-sc" not found. The image pull may not succeed.
    Apr 7 08:21:44 bronze kubelet[699]: W0407 08:21:44.208763 699 kubelet_pods.go:849] Unable to retrieve pull secret ricplt/secret-nexus3-o-ran-sc-org-10002-o-ran-sc for ricplt/deployment-ricplt-rtmgr-9d4847788-t9ht4 due to secret "secret-nexus3-o-ran-sc-org-10002-o-ran-sc" not found. The image pull may not succeed.
    Apr 7 08:21:44 bronze dhclient[14518]: DHCPDISCOVER on enp0s3 to 255.255.255.255 port 67 interval 12 (xid=0x19f17239)
    Apr 7 08:21:45 bronze kubelet[699]: E0407 08:21:45.015589 699 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
    Apr 7 08:21:47 bronze kubelet[699]: W0407 08:21:47.203279 699 kubelet_pods.go:849] Unable to retrieve pull secret ricplt/secret-nexus3-o-ran-sc-org-10002-o-ran-sc for ricplt/deployment-ricplt-alarmadapter-64d559f769-4tzjx due to secret "secret-nexus3-o-ran-sc-org-10002-o-ran-sc" not found. The image pull may not succeed.
    Apr 7 08:21:55 bronze kubelet[699]: E0407 08:21:55.055905 699 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"


    Can anyone let me know, why the onboarding of xapp is not responding pls.







    1. Anonymous

      Thank you very much Moogan Wilson, It worked for me

Write a comment…