summaryrefslogtreecommitdiffstats
path: root/docs/development/devtools/devtools.rst
blob: c626966f1452597d9ae63fbdc5b9967a3baad9f0 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
.. This work is licensed under a
.. Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0

.. _policy-development-tools-label:

Policy Platform Development Tools
#################################

.. contents::
    :depth: 3


This article explains how to build the ONAP Policy Framework for development purposes and how to run stability/
performance tests for a variety of components. To start, the developer should consult the latest ONAP Wiki to
familiarize themselves with developer best practices and how-tos to setup their environment,
see `https://wiki.onap.org/display/DW/Developer+Best+Practices`.

This article assumes that:

* You are using a *\*nix* operating system such as linux or macOS.
* You are using a directory called *git* off your home directory *(~/git)* for your git repositories
* Your local maven repository is in the location *~/.m2/repository*
* You have copied the settings.xml from oparent to *~/.m2/* directory
* You have added settings to access the ONAP Nexus to your M2 configuration,
  see `Maven Settings Example <https://wiki.onap.org/display/DW/Setting+Up+Your+Development+Environment>`_
  (bottom of the linked page)

The procedure documented in this article has been verified to work on a MacBook laptop running macOS Mojave Version
10.14.6 and an Ubuntu 18.06 VM.

Cloning All The Policy Repositories
***********************************

Run a script such as the script below to clone the required modules from the
`ONAP git repository <https://gerrit.onap.org/r/admin/repos/q/filter:policy>`_.
This script clones all the ONAP Policy Framework repositories.

ONAP Policy Framework has dependencies to the ONAP Parent *oparent* module, the ONAP ECOMP SDK *ecompsdkos* module,
and the A&AI Schema module.


.. code-block:: bash
   :caption: Typical ONAP Policy Framework Clone Script
   :linenos:

    #!/usr/bin/env bash

    ## script name for output
    MOD_SCRIPT_NAME=`basename $0`

    ## the ONAP clone directory, defaults to "onap"
    clone_dir="onap"

    ## the ONAP repos to clone
    onap_repos="\
    policy/parent \
    policy/common \
    policy/models \
    policy/docker \
    policy/api \
    policy/pap \
    policy/apex-pdp \
    policy/drools-pdp \
    policy/drools-applications \
    policy/xacml-pdp \
    policy/distribution \
    policy/gui \
    policy/clamp "

    ##
    ## Help screen and exit condition (i.e. too few arguments)
    ##
    Help()
    {
        echo ""
        echo "$MOD_SCRIPT_NAME - clones all required ONAP git repositories"
        echo ""
        echo "       Usage:  $MOD_SCRIPT_NAME [-options]"
        echo ""
        echo "       Options"
        echo "         -d          - the ONAP clone directory, defaults to '.'"
        echo "         -h          - this help screen"
        echo ""
        exit 255;
    }

    ##
    ## read command line
    ##
    while [ $# -gt 0 ]
    do
        case $1 in
            #-d ONAP clone directory
            -d)
                shift
                if [ -z "$1" ]; then
                    echo "$MOD_SCRIPT_NAME: no clone directory"
                    exit 1
                fi
                clone_dir=$1
                shift
            ;;

            #-h prints help and exists
            -h)
                Help;exit 0;;

            *)    echo "$MOD_SCRIPT_NAME: undefined CLI option - $1"; exit 255;;
        esac
    done

    if [ -f "$clone_dir" ]; then
        echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as file"
        exit 2
    fi
    if [ -d "$clone_dir" ]; then
        echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as directory"
        exit 2
    fi

    mkdir $clone_dir
    if [ $? != 0 ]
    then
        echo cannot clone ONAP repositories, could not create directory '"'$clone_dir'"'
        exit 3
    fi

    for repo in $onap_repos
    do
        repoDir=`dirname "$repo"`
        repoName=`basename "$repo"`

        if [ ! -z $dirName ]
        then
            mkdir "$clone_dir/$repoDir"
            if [ $? != 0 ]
            then
                echo cannot clone ONAP repositories, could not create directory '"'$clone_dir/repoDir'"'
                exit 4
            fi
        fi

        git clone https://gerrit.onap.org/r/${repo} $clone_dir/$repo
    done

    echo ONAP has been cloned into '"'$clone_dir'"'


Execution of the script above results in the following directory hierarchy in your *~/git* directory:

    *  ~/git/onap
    *  ~/git/onap/policy
    *  ~/git/onap/policy/parent
    *  ~/git/onap/policy/common
    *  ~/git/onap/policy/models
    *  ~/git/onap/policy/api
    *  ~/git/onap/policy/pap
    *  ~/git/onap/policy/gui
    *  ~/git/onap/policy/docker
    *  ~/git/onap/policy/drools-applications
    *  ~/git/onap/policy/drools-pdp
    *  ~/git/onap/policy/clamp
    *  ~/git/onap/policy/apex-pdp
    *  ~/git/onap/policy/xacml-pdp
    *  ~/git/onap/policy/distribution


Building ONAP Policy Framework Components
*****************************************

**Step 1:** Optionally, for a completely clean build, remove the ONAP built modules from your local repository.

    .. code-block:: bash

        rm -fr ~/.m2/repository/org/onap


**Step 2:**  A pom such as the one below can be used to build the ONAP Policy Framework modules. Create the *pom.xml*
file in the directory *~/git/onap/policy*.

.. code-block:: xml
   :caption: Typical pom.xml to build the ONAP Policy Framework
   :linenos:

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>org.onap</groupId>
        <artifactId>onap-policy</artifactId>
        <version>1.0.0-SNAPSHOT</version>
        <packaging>pom</packaging>
        <name>${project.artifactId}</name>
        <inceptionYear>2017</inceptionYear>
        <organization>
            <name>ONAP</name>
        </organization>

        <modules>
            <module>parent</module>
            <module>common</module>
            <module>models</module>
            <module>api</module>
            <module>pap</module>
            <module>apex-pdp</module>
            <module>xacml-pdp</module>
            <module>drools-pdp</module>
            <module>drools-applications</module>
            <module>distribution</module>
            <module>gui</module>
            <module>clamp</module>
        </modules>
    </project>

**Policy Architecture/API Transition**

In Dublin, a new Policy Architecture was introduced. The legacy architecture runs in parallel with the new
architecture. It will be deprecated after Frankfurt release. If the developer is only interested in working with the
new architecture components, the engine sub-module can be ommitted.


**Step 3:** You can now build the Policy framework.

Java artifacts only:

    .. code-block:: bash

       cd ~/git/onap
       mvn clean install

With docker images:

    .. code-block:: bash

       cd ~/git/onap
       mvn clean install -P docker

Developing and Debugging each Policy Component
**********************************************

Running a MariaDb Instance
++++++++++++++++++++++++++

The Policy Framework requires a MariaDb instance running. The easiest way to do this is to run a docker image locally.

One example on how to do this is to use the scripts used by the policy/api S3P tests.

`Simulator Setup Script Example <https://gerrit.onap.org/r/gitweb?p=policy/api.git;a=tree;f=testsuites/stability/src/main/resources/simulatorsetup;h=9038413f67cff2e2a79d6345f198f96ee0c57de1;hb=refs/heads/master>`_

    .. code-block:: bash

       cd ~/git/onap/api/testsuites/stability/src/main/resources/simulatorsetup
       ./setup_components.sh

Another example on how to run the MariaDb is using the docker compose file used by the Policy API CSITs:

`Example Compose Script to run MariaDB <https://gerrit.onap.org/r/gitweb?p=integration/csit.git;a=blob;f=scripts/policy/docker-compose-api.yml;h=e32190f1e6cb6d9b64ddf53a2db2c746723a0c6a;hb=refs/heads/master>`_

Running the API component standalone
++++++++++++++++++++++++++++++++++++

Assuming you have successfully built the codebase using the instructions above. The only requirement for the API
component to run is a running MariaDb database instance. The easiest way to do this is to run the docker image, please
see the mariadb documentation for the latest information on doing so. Once the mariadb is up and running, a
configuration file must be provided to the api in order for it to know how to connect to the mariadb. You can locate
the default configuration file in the packaging of the api component:

`Default Policy API Configuration <https://gerrit.onap.org/r/gitweb?p=policy/api.git;a=blob;f=packages/policy-api-tarball/src/main/resources/etc/apiParameters.yaml;h=2c19199a8a889cb0ab203334182662fe15e1635e;hb=refs/heads/master>`_

You will want to change the fields pertaining to "host", "port" and "databaseUrl" to your local environment settings
and start the policy-api springboot application either using your IDE of choice or using the run goal from Spring Boot
Maven plugin: *mvn spring-boot:run*.

Running the API component using Docker Compose
++++++++++++++++++++++++++++++++++++++++++++++

An example of running the api using a docker compose script is located in the Policy Integration CSIT test repository.

`Policy CSIT API Docker Compose <https://gerrit.onap.org/r/gitweb?p=integration/csit.git;a=blob;f=scripts/policy/docker-compose-api.yml;h=e32190f1e6cb6d9b64ddf53a2db2c746723a0c6a;hb=refs/heads/master>`_

Running the PAP component standalone
++++++++++++++++++++++++++++++++++++

Once you have successfully built the PAP codebase, a running MariaDb database and DMaaP instance will also be required
to start up the application. For MariaDb instance, the easiest way is to run the docker image, please see the mariadb
documentation for the latest information on doing so. For DMaaP, the easiest way during development is to run the DMaaP
simulator which is explained in the below sections. Once the mariadb and DMaaP are running, a configuration file must
be provided to the PAP component in order for it to know how to connect to the mariadb and DMaaP along with other
relevant configuration details. You can locate the default configuration file in the packaging of the PAP component:

`Default PAP Configuration <https://gerrit.onap.org/r/gitweb?p=policy/pap.git;a=blob;f=packages/policy-pap-tarball/src/main/resources/etc/papParameters.yaml;h=06dd45f4946fd0a11ed8ef859f8fc5bcf409a3f0;hb=HEAD>`_

Update the fields related to MariaDB, DMaaP and the RestServer for the application as per your local environment settings.
Then to start the application, just run the Spring Boot application using IDE or command line.


Running the Smoke Tests
***********************

The following links contain instructions on how to run the smoke tests. These may be helpful to developers to become
familiar with the Policy Framework components and test any local changes.

.. toctree::
   :maxdepth: 1

   policy-gui-acm-smoke.rst
   db-migrator-smoke.rst
   acm-participants-smoke.rst
   clamp-smoke.rst
   clamp-cl-participant-protocol-smoke.rst
   policy-participant-smoke.rst
   api-smoke.rst
   pap-smoke.rst
   apex-smoke.rst
   drools-smoke.rst
   xacml-smoke.rst
   distribution-smoke.rst


Running the Stability/Performance Tests
***************************************

The following links contain instructions on how to run the S3P Stability and Performance tests. These may be helpful to developers to become
familiar with the Policy Framework components and test any local changes.

.. toctree::
   :maxdepth: 2

   testing/s3p/run-s3p.rst
   testing/s3p/api-s3p.rst
   testing/s3p/pap-s3p.rst
   testing/s3p/apex-s3p.rst
   testing/s3p/drools-s3p.rst
   testing/s3p/xacml-s3p.rst
   testing/s3p/distribution-s3p.rst
   testing/s3p/clamp-s3p.rst


Running the Pairwise Tests
**************************

The following links contain instructions on how to run the pairwise tests. These may be helpful to developers check that
the Policy Framework works in a full ONAP deployment.

.. toctree::
   :maxdepth: 1

   clamp-policy.rst

   clamp-dcae.rst

   policy-cds.rst

   clamp-sdc.rst

..
   api-pairwise.rst

..
   pap-pairwise.rst

..
   apex-pairwise.rst

..
   drools-pairwise.rst

..
   xacml-pairwise.rst

..
   distribution-pairwise.rst


Testing OpenSuse docker images
******************************

Policy Framework offers docker images in two flavors: Alpine and OpenSuse.
Alpine images are used in OOM for ONAP deployments.
The OpenSuse images are built manually if needed, by running Maven with the -Pdockersuse profile.
To test these images, CSITs will be run.

1. Build the OpenSuse image you want by running Maven with -Pdockersuse:

    .. code-block:: bash

        cd policy/apex-pdp
        mvn clean install -Pdockersuse

    The image onap/policy-apex-pdp:latest will be produced.

2. To avoid ambiguity, tag the image as opensuse:

    .. code-block:: bash

        docker tag onap/policy-apex-pdp:latest onap/policy-apex-pdp:opensuse

3. Clone policy/docker repo.

4. Modify docker/csit/docker-compose.yml to use the tagged OpenSuse image.

    Replace:

    .. code-block:: yaml

        apex-pdp:
          image: nexus3.onap.org:10001/onap/policy-apex-pdp:${POLICY_APEX_PDP_VERSION}

    with:

    .. code-block:: yaml

        apex-pdp:
          image: onap/policy-apex-pdp:opensuse

5. Run the project CSIT. For apex-pdp:

    .. code-block:: bash

        cd docker/csit
        ./run-project-csit.sh apex-pdp

    Automated tests will be run, and log files displayed.


Generating Swagger Documentation
********************************

1. Accessing Swagger documentation for springboot based policy applications
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Springfox Swagger2 maven dependency aids with auto-generation of Swagger documentation.

Using the Swagger-UI maven dependency Swagger HTML documentation can be accessed at the root url.

- The generated swagger.json can be accessed at: *https://service_IP:service_port/v2/api-docs*
- Swagger UI can be accessed at: *https://service_IP:service_port/swagger-ui/index.html*

Running the DMaaP Simulator during Development
**********************************************
It is sometimes convenient to run the DMaaP simulator during development. You can run it from the command line using Maven or from within your IDE.

Running on the Command Line
+++++++++++++++++++++++++++
1. Check out the policy models repository
2. Go to the *models-sim/policy-models-simulators* subdirectory in the policy-models repo
3. Run the following Maven command:

   .. code-block:: bash

      mvn exec:java  -Dexec.mainClass=org.onap.policy.models.simulators.Main -Dexec.args="src/test/resources/simParameters.json"

Running in Eclipse
++++++++++++++++++
1. Check out the policy models repository
2. Go to the *models-sim/policy-models-simulators* module in the policy-models repo
3. Specify a run configuration using the class *org.onap.policy.models.simulators.Main* as the main class
4. Specify an argument of *src/test/resources/simParameters.json* to the run configuration
5. Run the configuration

Specifying a local configuration file
+++++++++++++++++++++++++++++++++++++

You may specify a local configuration file instead of *src/test/resources/simParameters.json* on the command line or as an argument in the run configuration in eclipse:

.. code-block:: json

   {
     "dmaapProvider": {
       "name": "DMaaP simulator",
       "topicSweepSec": 900
     },
     "restServers": [
       {
         "name": "DMaaP simulator",
         "providerClass": "org.onap.policy.models.sim.dmaap.rest.DmaapSimRestControllerV1",
         "host": "localhost",
         "port": 3904,
         "https": false
       }
     ]
   }

Bringing up Strimzi-Kafka Deploment with Policy Framework
*********************************************************

This page will explain how to setup a local Kubernetes cluster and minimal helm setup to run and deploy Policy Framework on a single host.

This is meant for a development purpose only as we are going to use microk8s in this page

.. toctree::
   :maxdepth: 1

   strimzi-policy.rst