diff options
Diffstat (limited to 'docs/userguides/developer-guide')
14 files changed, 463 insertions, 0 deletions
diff --git a/docs/userguides/developer-guide/index.rst b/docs/userguides/developer-guide/index.rst new file mode 100644 index 000000000..129ff7cd6 --- /dev/null +++ b/docs/userguides/developer-guide/index.rst @@ -0,0 +1,35 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 +.. International License. http://creativecommons.org/licenses/by/4.0 +.. Copyright (C) 2020 Deutsche Telekom AG. + +Developer Guide +================= + +Blueprints Processor Microservice +---------------------------------- + +Micro service to Manage Controller Blueprint Models, such as Resource Dictionary, Service Models, Velocity Templates etc, which will serve service for Controller Design Studio and Controller runtimes. + +This microservice is used to deploy Controller Blueprint Archive file in Run time database. This also helps to test the Valid CBA. + +.. toctree:: + :caption: Guide how to run Blueprint Processor in an IDE: + :maxdepth: 1 + + running-bp-processor-in-ide + +Architecture +~~~~~~~~~~~~~ + +|image0| + +.. |image0| image:: media/blueprintprocessor.jpg + :width: 400pt + + +Testing in local environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Point your browser to http://localhost:8000/api/v1/execution-service/ping (please note that the port is 8000, not 8080) + +To authenticate, use ccsdkapps/ccsdkapps login user id and password.
\ No newline at end of file diff --git a/docs/userguides/developer-guide/media/blueprintprocessor.jpg b/docs/userguides/developer-guide/media/blueprintprocessor.jpg Binary files differnew file mode 100644 index 000000000..429876a13 --- /dev/null +++ b/docs/userguides/developer-guide/media/blueprintprocessor.jpg diff --git a/docs/userguides/developer-guide/media/build_logs.png b/docs/userguides/developer-guide/media/build_logs.png Binary files differnew file mode 100644 index 000000000..558fd60a8 --- /dev/null +++ b/docs/userguides/developer-guide/media/build_logs.png diff --git a/docs/userguides/developer-guide/media/create_run_config_java.png b/docs/userguides/developer-guide/media/create_run_config_java.png Binary files differnew file mode 100644 index 000000000..5d006e2ac --- /dev/null +++ b/docs/userguides/developer-guide/media/create_run_config_java.png diff --git a/docs/userguides/developer-guide/media/create_run_config_kt.png b/docs/userguides/developer-guide/media/create_run_config_kt.png Binary files differnew file mode 100644 index 000000000..566ff609c --- /dev/null +++ b/docs/userguides/developer-guide/media/create_run_config_kt.png diff --git a/docs/userguides/developer-guide/media/expand_vm_options.PNG b/docs/userguides/developer-guide/media/expand_vm_options.PNG Binary files differnew file mode 100644 index 000000000..9cb98d3f9 --- /dev/null +++ b/docs/userguides/developer-guide/media/expand_vm_options.PNG diff --git a/docs/userguides/developer-guide/media/import_project.png b/docs/userguides/developer-guide/media/import_project.png Binary files differnew file mode 100644 index 000000000..ce7eb3ac5 --- /dev/null +++ b/docs/userguides/developer-guide/media/import_project.png diff --git a/docs/userguides/developer-guide/media/reimport_maven.png b/docs/userguides/developer-guide/media/reimport_maven.png Binary files differnew file mode 100644 index 000000000..bc49b3dc8 --- /dev/null +++ b/docs/userguides/developer-guide/media/reimport_maven.png diff --git a/docs/userguides/developer-guide/media/run_config_java.png b/docs/userguides/developer-guide/media/run_config_java.png Binary files differnew file mode 100644 index 000000000..4da5d7f34 --- /dev/null +++ b/docs/userguides/developer-guide/media/run_config_java.png diff --git a/docs/userguides/developer-guide/media/run_config_kt.png b/docs/userguides/developer-guide/media/run_config_kt.png Binary files differnew file mode 100644 index 000000000..4e88d2d0a --- /dev/null +++ b/docs/userguides/developer-guide/media/run_config_kt.png diff --git a/docs/userguides/developer-guide/media/run_config_set_up.png b/docs/userguides/developer-guide/media/run_config_set_up.png Binary files differnew file mode 100644 index 000000000..3c2e5fb4f --- /dev/null +++ b/docs/userguides/developer-guide/media/run_config_set_up.png diff --git a/docs/userguides/developer-guide/media/run_debug.png b/docs/userguides/developer-guide/media/run_debug.png Binary files differnew file mode 100644 index 000000000..3aa90577f --- /dev/null +++ b/docs/userguides/developer-guide/media/run_debug.png diff --git a/docs/userguides/developer-guide/media/vsc_logs.png b/docs/userguides/developer-guide/media/vsc_logs.png Binary files differnew file mode 100644 index 000000000..886d1b3c5 --- /dev/null +++ b/docs/userguides/developer-guide/media/vsc_logs.png diff --git a/docs/userguides/developer-guide/running-bp-processor-in-ide.rst b/docs/userguides/developer-guide/running-bp-processor-in-ide.rst new file mode 100644 index 000000000..ab6ae2314 --- /dev/null +++ b/docs/userguides/developer-guide/running-bp-processor-in-ide.rst @@ -0,0 +1,428 @@ +.. This work is a derivative of https://wiki.onap.org/display/DW/Running+Blueprints+Processor+Microservice+in+an+IDE +.. This work is licensed under a Creative Commons Attribution 4.0 +.. International License. http://creativecommons.org/licenses/by/4.0 +.. Copyright (C) 2020 Deutsche Telekom AG. + +Running Blueprints Processor Microservice in an IDE +==================================================== + +Objective +~~~~~~~~~~~~ + +Run the blueprint processor locally in an IDE, while having the database running in a container. +This way, code changes can be conveniently tested and debugged. + +Check out the code +~~~~~~~~~~~~~~~~~~~ + +Check out the code from Gerrit: https://gerrit.onap.org/r/#/admin/projects/ccsdk/cds + +Build it locally +~~~~~~~~~~~~~~~~~~ + +In the checked out directory, type + +.. code-block:: bash + + mvn clean install -Pq -Dadditionalparam=-Xdoclint:none + +.. note:: + If an error ``invalid flag: --release`` appears when executing the maven install command, you need to upgrade Java version of your local + Maven installation. Use something like ``export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64``. + +Wait for the maven install command to finish until you go further. + +Spin up a Docker container with the database +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Blueprints Processor project uses a database to store information about the blueprints +and therefore it needs to be online before attempting to run it. + +One way to create the database is by using the :file:`docker-compose.yaml` file. +This database will require a local directory to mount a volume, therefore before running docker-compose create following directory: + +.. code-block:: bash + + mkdir -p -m 755 /opt/app/cds/mysql/data + +Navigate to the docker-compose file in the distribution module: + +.. tabs:: + + .. group-tab:: Frankfurt - Latest + + .. code-block:: bash + + cd ms/blueprintsprocessor/application/src/main/dc + + .. group-tab:: El Alto - Dublin + + .. code-block:: bash + + ms/blueprintsprocessor/distribution/src/main/dc + +And run docker-composer: + +.. code-block:: bash + + docker-compose up -d db + +This should spin up a container of the MariaDB image in the background. +To check if it has worked, this command can be used: + +.. code-block:: bash + + docker-compose logs -f + +The phrase ``mysqld: ready for connections`` indicates that the database was started correctly. + +From now on, the Docker container will be available on the computer; if it ever gets stopped, +it can be started again by the command: + +.. code-block:: bash + + docker start <id of mariadb container> + +Set permissions on the local file system +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Blueprints processor uses the local file system for some operations and, therefore, +need some existing and accessible paths to run properly. + +Execute the following commands to create the needed directories, and grant access to the current user to modify them: + +.. code-block:: bash + + mkdir -p -m 755 /opt/app/onap/blueprints/archive + mkdir -p -m 755 /opt/app/onap/blueprints/deploy + mkdir -p -m 755 /opt/app/onap/scripts + sudo chown -R $(id -u):$(id -g) /opt/app/onap/ + +Import the project into the IDE +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. tabs:: + + .. tab:: IntelliJ IDEA + + .. note:: + This is the recommended IDE for running CDS blueprint processor. + + Go to *File | Open* and choose the :file:`pom.xml` file of the cds/ms/blueprintprocessor directory: + + |imageImportProject| + + Import as a project. Sometimes it may be necessary to reimport Maven project, e.g. if some dependencies can't be found: + + |imageReimportMaven| + + **Override some application properties:** + + Next steps will create a run configuration profile overriding some application properties with custom values, + to reflect the local environment characteristics. + + .. tabs:: + + .. group-tab:: Frankfurt - Latest + + Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class: + + ``ms/blueprintsprocessor/application/src/main/kotlin/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.kt``. + + After dependencies are imported and indexes are set up you will see a green arrow + next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be + created. Right-click inside the class at any point to load the context menu and select create + a run configuration from context: + + |imageCreateRunConfigKt| + + **The following window will open:** + + |imageRunConfigKt| + + **Add the following in the field `VM Options`:** + + .. code-block:: bash + :caption: **Custom values for properties** + + -Dspring.profiles.active=dev + + Optional: You can override any value from **application-dev.properties** file here. In this case use the following pattern: + + .. code-block:: java + + -D<application-dev.properties key>=<application-dev.properties value> + + .. group-tab:: El Alto + + Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class: + + ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java.`` + + After dependencies are imported and indexes are set up you will see a green arrow + next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be + created. Right-click inside the class at any point to load the context menu and select create + a run configuration from context: + + |imageCreateRunConfigJava| + + **The following window will open:** + + |imageRunConfigJava| + + **Add the following in the field `VM Options`:** + + .. code-block:: bash + :caption: **Custom values for properties** + + -Dspring.profiles.active=dev + + Optional: You can override any value from **application-dev.properties** file here. In this case use the following pattern: + + .. code-block:: java + + -D<application-dev.properties key>=<application-dev.properties value> + + .. group-tab:: Dublin + + Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class: + + ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java``. + + After dependencies are imported and indexes are set up you will see a green arrow + next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be + created. Right-click inside the class at any point to load the context menu and select create + a run configuration from context: + + |imageCreateRunConfigJava| + + **The following window will open:** + + |imageRunConfigJava| + + **Add the following in the field `VM Options`** + + .. code-block:: java + :caption: **Custom values for properties** + + -DappName=ControllerBluePrints + -Dms_name=org.onap.ccsdk.apps.controllerblueprints + -DappVersion=1.0.0 + -Dspring.config.location=opt/app/onap/config/ + -Dspring.datasource.url=jdbc:mysql://127.0.0.1:3306/sdnctl + -Dspring.datasource.username=sdnctl + -Dspring.datasource.password=sdnctl + -Dcontrollerblueprints.loadInitialData=true + -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/ + -Dblueprintsprocessor.db.primary.url=jdbc:mysql://localhost:3306/sdnctl + -Dblueprintsprocessor.db.primary.username=sdnctl + -Dblueprintsprocessor.db.primary.password=sdnctl + -Dblueprintsprocessor.db.primary.driverClassName=org.mariadb.jdbc.Driver + -Dblueprintsprocessor.db.primary.hibernateHbm2ddlAuto=update + -Dblueprintsprocessor.db.primary.hibernateDDLAuto=none + -Dblueprintsprocessor.db.primary.hibernateNamingStrategy=org.hibernate.cfg.ImprovedNamingStrategy + -Dblueprintsprocessor.db.primary.hibernateDialect=org.hibernate.dialect.MySQL5InnoDBDialect + -Dblueprints.processor.functions.python.executor.executionPath=./components/scripts/python/ccsdk_blueprints + -Dblueprints.processor.functions.python.executor.modulePaths=./components/scripts/python/ccsdk_blueprints,./components/scripts/python/ccsdk_netconf,./components/scripts/python/ccsdk_restconf + -Dblueprintsprocessor.restconfEnabled=true + -Dblueprintsprocessor.restclient.sdncodl.type=basic-auth + -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/ + -Dblueprintsprocessor.restclient.sdncodl.username=admin + -Dblueprintsprocessor.restclient.sdncodl.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U + -Dblueprintsprocessor.grpcEnable=false + -Dblueprintsprocessor.grpcPort=9111 + -Dblueprintsprocessor.blueprintDeployPath=/opt/app/onap/blueprints/deploy + -Dblueprintsprocessor.blueprintArchivePath=/opt/app/onap/blueprints/archive + -Dblueprintsprocessor.blueprintWorkingPath=/opt/app/onap/blueprints/work + -Dsecurity.user.password={bcrypt}$2a$10$duaUzVUVW0YPQCSIbGEkQOXwafZGwQ/b32/Ys4R1iwSSawFgz7QNu + -Dsecurity.user.name=ccsdkapps + -Dblueprintsprocessor.messageclient.self-service-api.kafkaEnable=false + -Dblueprintsprocessor.messageclient.self-service-api.topic=producer.t + -Dblueprintsprocessor.messageclient.self-service-api.type=kafka-basic-auth + -Dblueprintsprocessor.messageclient.self-service-api.bootstrapServers=127.0.0.1:9092 + -Dblueprintsprocessor.messageclient.self-service-api.consumerTopic=receiver.t + -Dblueprintsprocessor.messageclient.self-service-api.groupId=receiver-id + -Dblueprintsprocessor.messageclient.self-service-api.clientId=default-client-id + -Dspring.profiles.active=dev + -Dblueprintsprocessor.httpPort=8080 + -Dserver.port=55555 + + + **In the field 'Working Directory' browse to your application path** ``.../cds/ms/blueprintsprocessor/application`` + **if path is not already specified correctly.** + + Run configuration should now look something like this: + + |imageRunConfigSetUp| + + **Add/replace the following in Blueprint's application-dev.properties file.** + + .. code-block:: java + + blueprintsprocessor.grpcclient.remote-python.type=token-auth + blueprintsprocessor.grpcclient.remote-python.host=localhost + blueprintsprocessor.grpcclient.remote-python.port=50051 + blueprintsprocessor.grpcclient.remote-python.token=Basic Y2NzZGthcHBzOmNjc2RrYXBwcw== + + blueprintprocessor.remoteScriptCommand.enabled=true + + Take care that if a parameter already exist you need to change the value of the existing parameter to avoid duplicates. + + + **Run the application:** + + Before running Blueprint Processor check that you use the correct Java version in IntelliJ. + Select either run or debug for the created Run Configuration to start the Blueprints Processor: + + |imageRunDebug| + + |imageBuildLogs| + + .. tab:: Visual Studio Code + + .. tabs:: + + .. group-tab:: Frankfurt - Latest + + * **Step #1** - Make sure your installation of Visual Studio Code is up to date. This guide was writen using version 1.48 + * **Step #2** - Install `Kotlin extension from the Visual Studio Code Marketplace <https://marketplace.visualstudio.com/items?itemName=fwcd.kotlin>`_ + * **Step #3** - On the top menu click *Run | Open Configurations* + + .. warning:: This should open the file called `launch.json` but in some cases you'll need to wait for the Kotlin Language Server to be installed before you can do anything. + Please watch the bottom bar in Visual Studio Code for messages about things getting installed. + + * **Step #4** - add configuration shown below to your configurations list. + + .. code-block:: json + + { + "type": "kotlin", + "request": "launch", + "name": "Blueprint Processor", + "projectRoot": "${workspaceFolder}/ms/blueprintsprocessor/application", + "mainClass": "-Dspring.profiles.active=dev org.onap.ccsdk.cds.blueprintsprocessor.BlueprintProcessorApplicationKt" + } + + .. warning:: The `projectRoot` path assumes that you created your Workspace in the main CDS repository folder. If not - please change the path accordingly + + .. note:: The `mainClass` contains a spring profile param before the full class name. This is done because `args` is not supported by Kotlin launch.json configuration. + If you have a cleaner idea how to solve this - please let us know. + + **Add/replace the following in Blueprint's application-dev.properties file:** + + .. code-block:: java + + blueprintsprocessor.grpcclient.remote-python.type=token-auth + blueprintsprocessor.grpcclient.remote-python.host=localhost + blueprintsprocessor.grpcclient.remote-python.port=50051 + blueprintsprocessor.grpcclient.remote-python.token=Basic Y2NzZGthcHBzOmNjc2RrYXBwcw== + + blueprintprocessor.remoteScriptCommand.enabled=true + + **Currently the following entries need to be added in VSC too:** + + .. code-block:: java + + logging.level.web=DEBUG + logging.level.org.springframework.web: DEBUG + + #Encrypted username and password for health check service + endpoints.user.name=eHbVUbJAj4AG2522cSbrOQ== + endpoints.user.password=eHbVUbJAj4AG2522cSbrOQ== + + #BaseUrls for health check blueprint processor services + blueprintprocessor.healthcheck.baseUrl=http://localhost:8080/ + blueprintprocessor.healthcheck.mapping-service-name-with-service-link=[Execution service,/api/v1/execution-service/health-check],[Resources service,/api/v1/resources/health-check],[Template service,/api/v1/template/health-check] + + #BaseUrls for health check Cds Listener services + cdslistener.healthcheck.baseUrl=http://cds-sdc-listener:8080/ + cdslistener.healthcheck.mapping-service-name-with-service-link=[SDC Listener service,/api/v1/sdclistener/healthcheck] + + #Actuator properties + management.endpoints.web.exposure.include=* + management.endpoint.health.show-details=always + management.info.git.mode=full + + In VSC the properties are read from target folder, thats why the following maven command needs to be rerun: + + .. code-block:: bash + + mvn clean install -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none + + Click Run in Menu bar. + + |imageLogsVSC| + + +Testing the application +~~~~~~~~~~~~~~~~~~~~~~~ + +There are two main features of the Blueprints Processor that can be of interest of a developer: +blueprint publish and blueprint process. + +To upload custom blueprints, the endpoint ``api/v1/execution-service/publish`` is used. + +To process, the endpoint is ``api/v1/execution-service/process``. + +Postman is a software that can be used to send these request, and an example of +them is present on https://www.getpostman.com/collections/b99863b0cde7565a32fc. + +A detailed description of the usage of different APIs of CDS will follow. + + +Possible Fixes +~~~~~~~~~~~~~~ + +Imported packages or annotiations are not found, Run Config not available? +***************************************************************************** + +1. Rebuild with ``maven install ...`` (see above) +2. Potentially change Maven home directory in Settings +3. Maven reimport in IDE + +Compilation error? +******************* +* Change Java Version to 11 + + +.. image alignment inside tabs doesn't work + +.. |imageRunConfigJava| image:: media/run_config_java.png + :width: 500pt + :align: middle + +.. |imageRunConfigKt| image:: media/run_config_kt.png + :width: 500pt + :align: middle + +.. |imageCreateRunConfigJava| image:: media/create_run_config_java.png + :width: 500pt + :align: middle + +.. |imageCreateRunConfigKt| image:: media/create_run_config_kt.png + :width: 500pt + :align: middle + +.. |imageImportProject| image:: media/import_project.png + :width: 300pt + :align: middle + +.. |imageReimportMaven| image:: media/reimport_maven.png + :width: 400pt + :align: middle + +.. |imageRunDebug| image:: media/run_debug.png + :width: 500pt + :align: middle + +.. |imageBuildLogs| image:: media/build_logs.png + :width: 500pt + :align: middle + +.. |imageLogsVSC| image:: media/vsc_logs.png + :width: 500pt + :align: middle + +.. |imageRunConfigSetUp| image:: media/run_config_set_up.png + :width: 500pt + :align: middle |
