...
This document provides an overview of how to write JJB definition files into the ci-management repo. The repo contains a global-job submodule that has a large body of existing templates and scripts that O-RAN SC jobs can reference. Documentation from Linux Foundation for global-job can be found at: https://docs.releng.linuxfoundation.org/projects/global-jjb/en/latest/index.html.
Table of Contents |
---|
Directory Structure
Below is the directory structure of the ci-management repo, where the most interesting potion is the jjb directory where the per-repo job definitions reside. This is where we want to make changes for defining new or modifying existing Jenkins jobs. The global-jjb directory is initially empty when the repo is cloned. It is a submodule that can be populated by running: "git submodule update --init". Once initialized, this directory contains the global-jjb templates managed by the Linux Foundation Release Engineering team.
Code Block | ||
---|---|---|
| ||
. |-- README.md |-- docker | |-- README.md | |-- bldr-alpine3 | |-- bldr-debian-python | |-- bldr-ubuntu16-c-go | `-- bldr-ubuntu18-c-go |-- global-jjb |-- jenkins-config | |-- clouds | |-- global-vars-production.sh | `-- global-vars-sandbox.sh |-- jjb | |-- ci-management | |-- com-asn1 | |-- com-golog | |-- com-log | |-- com-pylog | |-- common-views.yaml | |-- defaults.yaml | |-- doc | |-- global-jjb | |-- it-dep | |-- it-otf | |-- it-test | |-- oam | |-- oran-jjb | |-- portal-ric-dashboard | |-- pti-rtp | |-- read-the-docs | |-- ric-app-admin | |-- ric-app-mc | |-- ric-plt-a1 | |-- ric-plt-appmgr | |-- ric-plt-dbaas | |-- ric-plt-e2 | |-- ric-plt-e2mgr | |-- ric-plt-lib-rmr | |-- ric-plt-resource-status-manager | |-- ric-plt-rtmgr | |-- ric-plt-sdl | |-- ric-plt-sdlgo | |-- ric-plt-submgr | |-- ric-plt-tracelibcpp | |-- ric-plt-tracelibgo | |-- ric-plt-vespamgr | `-- shell |-- packer | |-- common-packer | |-- provision | `-- templates `-- tox.ini |
Docker based Building
Many O-RAN SC repos have adopted the docker based building process. In this process, repo dev team provides a Dockerfile that completes all the steps of installing building artifacts, including installing additional tools, configuring the build, executing the building, and generating the result artifacts. The JJB build job will simply invoke docker build and if applicable publishes result artifact to relevant repository. This is the process we focus here in this document. For more "conventional" approach of defining JJB jobs, there are other documents such as this page for the ONAP project: https://wiki.onap.org/display/DW/Using+Standard+Jenkins+Job+%28JJB%29+Templates.
...
The following subsections explain what needs to be done at each location using existing O-RAN-SC repos as examples.
Dockerfile
This is where the actual building happens. Depends on the type of artifact to be built, docker container image or Linux software package (rpm/deb), the process differs.
Building for Docker Container Image
Here we use the ric-plt/e2mgr repo as example because it builds into a docker container image for deployment.
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
FROM nexus3.o-ran-sc.org:10004/bldr-ubuntu16-c-go:2-u16.04-nng as ubuntu WORKDIR /opt/E2Manager COPY . . ENV PATH=$PATH:/usr/local/go/bin:/usr/lib/go-1.12/bin # Install RMr library and dev files RUN wget --content-disposition https://packagecloud.io/o-ran-sc/staging/packages/debian/stretch/rmr_1.10.0_amd64.deb/download.deb RUN dpkg -i rmr_1.10.0_amd64.deb RUN wget --content-disposition https://packagecloud.io/o-ran-sc/staging/packages/debian/stretch/rmr-dev_1.10.0_amd64.deb/download.deb RUN dpkg -i rmr-dev_1.10.0_amd64.deb RUN cd asn1codec && make RUN go build app/main.go # Execute UT ENV LD_LIBRARY_PATH=/usr/local/lib ENV GODEBUG=cgocheck=2,clobberfree=1,gcstoptheworld=2,allocfreetrace=0 ENV RIC_ID="bbbccc-abcd0e/20" RUN go test ./... FROM ubuntu:16.04 RUN apt-get update && apt-get install -y \ net-tools \ iputils-ping \ curl \ tcpdump COPY --from=ubuntu /opt/E2Manager/router.txt /opt/E2Manager/router.txt COPY --from=ubuntu /opt/E2Manager/main /opt/E2Manager/main COPY --from=ubuntu /opt/E2Manager/resources /opt/E2Manager/resources COPY --from=ubuntu /usr/local/lib/librmr_nng.so.1 /usr/local/lib/librmr_nng.so.1 COPY --from=ubuntu /usr/local/lib/libnng.so.1 /usr/local/lib/libnng.so.1 WORKDIR /opt/E2Manager ENV LD_LIBRARY_PATH=/usr/local/lib \ port=3800 ENV RMR_SEED_RT=router.txt EXPOSE 3800 CMD ["sh", "-c", "./main -port=$port"] |
Building for Linux Software Package
Below is the Dockerfile for the ric-plt/lib/rmr repo, which builds into a deb and a rpm package, as an example for how to build Linux Software package in the docker based building process.
...
Code Block | ||||
---|---|---|---|---|
| ||||
FROM buildpack-deps:stretch RUN apt-get update && apt-get -q -y install cmake ksh alien ADD . /tmp WORKDIR /tmp # build RMr, run unit tests, and generate packages and package lists RUN ksh ci/ci_build.ksh # Executing the container "as a binary" will cause the CI publish # script to execute. This will take the simple package list generated # by the ci_build script and copy the list of packages to the target # directory. The target directory is /export by default, but can be # overridden from the docker run command line. In either case, the # assumption is that the target directory is mounted as a volume. # ENTRYPOINT [ "ci/publish.sh" ] |
JJB Definition
Building for Docker Container Image
Again, the JJB definition file for the E2 manager is used as example for illustrating how to define docker image building jobs. The JJB definition for the E2 Manager can be found at the jjb/ric-plt-e2mgr directory of the ci-management repo.
...
Obviously, when copying such a block into a JJB definition file for a new repo, all the references to e2mgr need to be modified. In addition, line 19 is another important line. It specifies where the Dockerfile and container-tag.yaml files are provided. In this example, they are under the E2Manager subdirectory from the repo root. It must be customized to point where in the new repo the Dockerfile and container-tag.yaml files are stored.
Building for Linux Software Package
Again, the the JJB definition file for RMR library is used as example for illustrating how to define Linux software package building jobs. The JJB definition for the RMR library can be found under jjb/ric-plt-lib-rmr of the ci-management repo.
...
line 25-27: two different job templates are referenced here. In particular, the oran-gerrit-docker-ci-pc-merge job template has the steps for executing the docker build, and publishing result files placed in "/export" directory of the docker container by the building process to PackageCloud.
How to Add CI for My Repo
With an understanding of how CI is defined, new Jenkins jobs can be added by following what the examples above are doing:
...