Move products from nonrtric repo to separate repos

To make maintenance easier a decision has been made to move all products in the nonrtric repo to separate product repos. This page describes the steps needed to perform such a move for a product

Set up ci-management

Do the following steps to configure the ci-management for the new repo:

• In the "ci-management" repo, go to the folder under the "jjb" folder with the name the same as the new product.
• In the new folder, edit the file named "<product name>.yaml".
• From the file "jjb/nonrtr/nonrtric.yaml", copy the things listed below and replace "<full product name>" with the product's full name, e.g. "nonrtric-plt-rappcatalogue". Replace "<product name>" with the short product name, e.g. "rappcatalogue".
• For Go products, remove "jdk_" from the "jdk_common" tag and also remove the "mvn-version" and "java-version" tags.

- <product name>_jdk_common: &<product name>_jdk_common
    name: <product name>-jdk-common
    # git repo
    project: <full product name>
    # jenkins job name prefix
    project-name: <product name>
    # maven settings file has docker credentials
    mvn-settings: <full product name>-settings
    mvn-version: mvn36
    java-version: openjdk11

- project:
    <<: *<product name>_jdk_common
    mvn-params: -Dbuild.number=${{BUILD_NUMBER}}
    jobs:
      - gerrit-maven-docker-verify
      - gerrit-maven-docker-merge
      - gerrit-maven-docker-stage
    stream:
      - master:
          branch: master
 
- project:
    <<: *<product name>_jdk_common
    name: <full product name>-release
    # maven release requires sigul which requires centos
    # container release requires docker
    build-node: centos7-docker-2c-8g
    jobs:
      - '{project-name}-gerrit-release-jobs'

- project:
    <<: *<product name>_jdk_common
    name: <full product name>-sonar
    project-name: '{name}'
    # template goal install builds docker image unnecessarily
    mvn-params: '-Ddocker.skip=true'
    sonarcloud: true
    sonarcloud-api-token: '{sonarcloud_api_token}'
    sonarcloud-project-key: '{sonarcloud_project_organization}_nonrtric'
    sonarcloud-project-organization: '{sonarcloud_project_organization}'
    jobs:
      - gerrit-maven-sonar

• For products where we release jar files the following part also needs to be copied and adapted:

# nonrtric:<product name>
# Maven artifacts for nonrtric-<product name>
- project:
    <<: *<product name>_jdk_common
    name: nonrtric-<full product name>
    project-name: '{name}'
    mvn-opts: '-Ddocker.skip=true'
    build-node: ubuntu1804-builder-4c-4g
    jobs:
      - gerrit-maven-verify
      - gerrit-maven-merge
      - gerrit-maven-stage
      - gerrit-maven-clm
    stream:
      - master:
          branch: master

• For Go based products, instead of using "<product name>_jdk_common" use "<product name>_common" only. Replace the "-project" part for the "docker" and "sonar" parts described above with the following parts:

- project:
    <<: *<product name>_common
    name: <full product name>
    project-name: '{name}'
    # image name
    docker-name: 'o-ran-sc/<full product name>'
    # get tag from YAML
    container-tag-method: yaml-file
    # use host network to clone from our gerrit
    docker-build-args: '--network=host'
    # Directory with Dockerfile
    docker-root: '<path_to_Dockerfile>'
    # Directory where go test should be run
    go-root: '<path_to_go_code>'
    build-node: ubuntu1804-docker-4c-4g
    # Includes verify and merge jobs
    jobs:
      - "{project-name}-gerrit-docker-jobs"
      - gerrit-go-verify
    stream:
      - master:
          branch: master

- project:
    <<: *<product name>_common
    name: <full product name>-sonar
    project-name: nonrtric-<full product name>
    # prescan script requires ubuntu
    # golang is only on docker-enabled nodes
    build-node: ubuntu1804-docker-4c-4g
    sonar-prescan-script: !include-raw-escape: prescan-<product name>-ubuntu.sh
    sonar-project-file: ""
    java-version: openjdk11
    # use sonarcloud values from defaults.yaml
    sonar-properties: !include-raw: sonar-project.properties
    jobs:
      - gerrit-sonar-prescan-script

• For Go based projects, copy the "prescan-<product name>-ubuntu.sh" and "sonar-project.properties" files, used for Sonar, from the "nonrtric" jjb folder to the new product's jjb folder. Adapt the paths in the files to fit the new project structure.
• Commit and push the new jjb file and push it upstream from Nordix.

Copy resources from old repo to new repo

1. Copy folders and files from product's folder in the nonrtric repo. Things to copy, if they exist, e.g.:
   1. "src" folder
   2. "api" folder
   3. "config" folder
   4. Dockerfile
   5. eclipse-formatter.xml
   6. pom.xml
   7. README.md
   8. tox.ini
   9. .readthedocs.yaml
2. For Java projects, remove ".nonrtric" from the "groupId" tag in the pom file.
3. For Java projects, change the name of the product and change the image name in the "docker-maven-plugin" configuration in the pom file.
4. For Go projects, copy the "build-<product name>-ubuntu.sh", "container-tag.yaml", and "generate_swagger_docs.sh" (if it exists) files. Adapt the paths in the files to fit the new project structure.
5. Copy the top level ".gitignore" file from the "nonrtric" repo and add the content of the ".gitignore" file in the original product's folder.
6. Create a folder named "docs" in the new repo.
7. Copy the "docs/images" folder to the docs folder of the new repo, and remove images that are not used in the product's documentation.
8. Copy the "docs/_static" folder to the docs folder of the new repo.
9. Copy the "docs/conf.py" file to the docs folder of the new repo. Edit it, and from the "linkcheck_ignore" remove the paths not relevant for the product. Also remove unused things from the "redoc" part and any unused "intersphinx_mapping".
10. Copy the "docs/conf.yaml" file to the docs folder of the new repo. Edit it and change the "project" to fit the name of the product.
11. Copy the "docs/favicon.ico" to the docs folder of the new repo.
12. Copy the "requirements-docs.txt" to the docs folder of the new repo.
13. Copy all the "rst" files, except for "use-cases.rst" and "requirements.rst" to the docs folder of the new repo.
14. Remove "./use-cases.rst" and "./requirements.rst" from the "index.rst" file.
15. Adapt the paths to the Swagger generated api files in "conf.py" and "api-docs.rst". Example: "../r-app-catalogue/api/rac-api.json" should be "../api/rac-api.json".
16. Go through and adapt or remove all the other "rst" files to fit the product. Remember to update the "index.rst" file for all removed files. Run a search for the product's name in all files to see if anything more needs fixing, "grep -rnw '.' -e 'r-app-catalogue'".
17. Adapt paths in Docker files if needed.
18. Build the documentation from the top level with the command "tox". Fix any problems that might arise.
19. Build the project and fix any problems that might arise:
    For Java projects: Run "mvn install" from the top level.
    For Go projects: Run "go build", "go test ./...", and "docker build".
20. Stage everything, commit and push to Nordix and upstream.
21. Make sure the " rtdv3-global-verify-master" job that verifies the readthedocs is successful in Gerrit in o-ran-sc.

Verify ci-management setup

When the commit with the seed code for the new repo has been upstreamed, do the following to verify that the ci-management setup is correct:

• On the change in ci-management in Gerrit, answer " jjb-deploy nonrtric-<product name>*".
• Note! You cannot login to the sandbox with your Linux Foundation ID and get the correct permissions, you must request that specifically. If you haven't requested permissions in the sandbox, follow instructions on this page: https://confluence.linuxfoundation.org/pages/viewpage.action?pageId=28477040.
• Go to https://jenkins.o-ran-sc.org/sandbox/ and log in.
• When the new jobs for the product show up, click on the "verify-master" job and in the left menu on the job's page, select the "Build with parameters" option.
• From the change with the seed code for the repo, press ""Download" and click copy on the "Checkout" option.
• From the copied URL, select the "refs/changes/61/9861/2" part and copy it.
• Paste the copied value into the " GERRIT_BRANCH" and " GERRIT_REFSPEC" fields in the job configuration page.
• Start the job by pressing the "Build" button.
• Hopefully the result will be ok. Otherwise you need to troubleshoot the jjb file. Look at the console output of the job in the sandbox for clues on what went wrong.
• When the job finishes with success, paste the URL to the job in the comments of the jjb file change in Gerrit and add Jessica Wagantall as reviewer.
• When the jjb file has been merged, make sure the seed code commit has been verified.
• When the seed code commit has been committed, make sure the merge, stage and documentation jobs are successful.
• Check the generated documentation at "https://docs.o-ran-sc.org/projects/<full product name>/en/latest/index.html", e.g. "https://docs.o-ran-sc.org/projects/o-ran-sc-nonrtric-plt-rappcatalogue/en/latest/index.html".

Cleanup

When the new repo is working properly, remove the product from the "nonrtric" repo. The following steps gives an idea about how to do this:

1. Edit the "docs/api-docs.rst" file to remove any existance of the product.
2. Edit the "docs/conf.py" file to remove any existance of the product.
3. Edit the "docs/developer-guide.rst" file to remove any existance of the product.
4. Edit the "docs/overview.rst" file and make the info about the product look like the info for the "rApp Catalogue".
5. Edit the "docs/release-notes.rst" file to remove any existance of the product.
6. Build the documentation from the top level with the command "tox". Fix any problems that might arise.
7. If it is a Java product, remove it from the top "pom.xml" file.
8. Remove the subfolder containing the product.
9. Stage everything, commit and push to Nordix and upstream.

Also, remove any product named configurations from the jjb file for the "nonrtric" repo in the "ci-management" repo. Stage the file, commit and push to Nordix and upstream. Add Jessica Wagantall as reviewer. Note! If the product has a released Maven artifact, the verification job for the removal in the "nonrtric" repo will fail until this change has been merged.