Added JMeter scripts for perf test

Change-Id: I4286c6defdef331ee5c40399d17381d3c8b8a544
Issue-ID: POLICY-836
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>

4 files changed, 530 insertions, 0 deletions

diff --git a/s3p/README.perf.md b/s3p/README.perf.md
new file mode 100644
index 00000000..aaa9d47d
--- /dev/null
+++ b/s3p/README.perf.md
@@ -0,0 +1,55 @@
+# Performance Test for Policy Distribution
+## Steps to Run performance test
+1. Download Apache JMeter
+Download JMeter 5.0 from https://jmeter.apache.org/download_jmeter.cgi, and extracted it locally.
+ 
+2. Launch the policy distribution service
+Launch the policy distribution service with the configuration from s3pConfig.json, you need to make sure when the service is being launched, the directory configured in s3pConfig.json by receptionHandlerConfigurationParameters.fileConfiguration.parameters.watchPath is a valid local directory(Default directory is /tmp/policy_distribution/csar/).
+The policy distribution service will use the FileSystemReceptionHandler plugin to monitor the local directory specified by the 'watchPath' parameter for newly added csar files, parse them to generate policies and forward it to PDP.
+```
+java -cp "<comma separated directories containing jar files>" org.onap.policy.distribution.main.startstop.Main -c s3pConfig.json
+```
+  Or if you want to launch it from docker, please do the followings:
+```
+mkdir -p /tmp/policy_distribution/csar/
+docker run -d -e "CONFIG_FILE=/opt/app/policy/distribution/etc/s3pConfig.json" \
+           -v /tmp/policy_distribution/csar/:/tmp/policy_distribution/csar/ \
+           -p 6969:6969 \
+           --name policy-distribution policy-distribution
+```
+Here we use -e option to "docker run" to pass the config file which the policy distribution service will be launched upon and use -v option to map the local host /tmp/policy_distribution/csar/ directory as the directory of /tmp/policy_distribution/csar/ within the policy-distribution docker.
+
+**NOTED:**
+Please make sure when you launch the policy distribution service, the following requirements are met:
+a. The policy PDP service is active, and can be reached using the parameters configured by policyForwarderConfigurationParameters.xacmlPdpConfiguration.parameters in the s3pConfig.json file.
+
+b. Make sure you have installed the AAF root CA either in local host or in the docker image(running as root):
+```
+curl -s https://git.onap.org/dmaap/buscontroller/plain/misc/cert-client-init.sh | bash --
+```
+
+3. Get the Jmeter configuration ans scripts
+Git clone the policy distribution code, and goto the s3p directory:
+```
+git clone https://git.onap.org/policy/distribution policy-distribution
+cd policy-distribution/s3p
+```
+
+4. Run the JMeter stability test
+```
+rm -f perf.log; <jmeter_dir>/bin/jmeter.sh -t perf.jmx -n -Jhost=<pdp service hostname> -Jtotal=5 -l perf.log
+```
+In the meantime, you can run various system tools i.e. top, atop, etc. to monitor the cpu/memory usage of the policy-distribution service.
+
+After jmeter finished, search for 'Fail at idx' in the perf.log file to see if there is any errors found during the stability test.
+
+## JMeter properties
+We can configure the following properties when running the JMeter stability test for policy distribution 
+| Property Name | Default Value | Description |
+|---------------|-------------|---------------|
+| host | pdp | PDP service host name or ip  |
+| csardir | /tmp/policy_distribution/csar | Directory where to store the generated csar files |
+| total | 5 | Total number of csar files to be generated in bulk to test performance
+| retry | 100 | Number of retry to retrieve the policy of each csar
+
+
diff --git a/s3p/generate_perf.py b/s3p/generate_perf.py
new file mode 100644
index 00000000..dfd62fc8
--- /dev/null
+++ b/s3p/generate_perf.py
@@ -0,0 +1,40 @@
+import argparse
+import csv
+import os
+import shutil
+import sys
+import subprocess
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Prepare CSAR for policy distrition performance test')
+    parser.add_argument('--total', required=True, type=int, help='total number of CSAR to be generated')
+    parser.add_argument('--dest', required=True, help='dest directory where the CSAR files will be stored')
+    parser.add_argument('--out',  default='perf_data.csv', help='list of generated CSAR identifier')
+    return parser.parse_args(sys.argv[1:])
+
+
+def main():
+    args = parse_args()
+    
+    # create dest dir
+    shutil.rmtree(args.dest, ignore_errors=True)
+    os.makedirs(args.dest)
+
+    # prepartion
+    count = 0
+    maxwidth = len(str(args.total))
+    scripts = os.path.dirname(os.path.abspath(__file__))
+    scripts = os.path.join(scripts, 'generate.sh')
+    
+    with open(args.out, 'w') as out_file:
+        out_writer = csv.writer(out_file, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
+        while (count < args.total):
+            seed = str(count).zfill(maxwidth)
+            subprocess.check_call([scripts, args.dest, seed])
+            out_writer.writerow(["s3p_" + seed])
+            count += 1
+    return 0
+
+if __name__ == '__main__':
+    main()
diff --git a/s3p/generate_perf.sh b/s3p/generate_perf.sh
new file mode 100755
index 00000000..beabb783
--- /dev/null
+++ b/s3p/generate_perf.sh
@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+
+function print_usage_and_exit {
+    [ -z "$1" ] || echo "Error: $1"
+    echo "Usage: $0 <output_dir> <total>"
+    echo "    - <output_dir>: directory where the generated csar file will be put into"
+    echo "    - <total>: total number of csar files to be generated"
+    exit 1
+}
+
+[ "$#" -ne 2 ] && print_usage_and_exit
+OUTPUT=$1
+TOTAL=$2
+
+[ -d $OUTPUT ] || mkdir -p $OUTPUT
+[ -d $OUTPUT ] || print_usage_and_exit "$OUTPUT is not a valid directory"
+[[ $TOTAL =~ ^[0-9]+$ ]] || print_usage_and_exit "$SEED is not a integer"
+
+
+ROOT_DIR=`dirname $(readlink -f $0)`
+TMP_DIR=$ROOT_DIR/perf_tmp
+
+python $ROOT_DIR/generate_perf.py --dest $TMP_DIR --total $TOTAL --out $ROOT_DIR/perf_data.csv
+
+rm -f $OUTPUT/*.csar
+cp -fr $TMP_DIR/*.csar $OUTPUT/
+
+rm -rf $TMP_DIR
diff --git a/s3p/perf.jmx b/s3p/perf.jmx
new file mode 100644
index 00000000..ac934776
--- /dev/null
+++ b/s3p/perf.jmx
@@ -0,0 +1,407 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.0 r1840935">
+  <hashTree>
+    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="perf test plan" enabled="true">
+      <stringProp name="TestPlan.comments"></stringProp>
+      <boolProp name="TestPlan.functional_mode">false</boolProp>
+      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
+      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
+      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
+        <collectionProp name="Arguments.arguments">
+          <elementProp name="PDP_HOST" elementType="Argument">
+            <stringProp name="Argument.name">PDP_HOST</stringProp>
+            <stringProp name="Argument.value">${__P(host,pdp)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+          <elementProp name="RETRY" elementType="Argument">
+            <stringProp name="Argument.name">RETRY</stringProp>
+            <stringProp name="Argument.value">${__P(retry, 100)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+          <elementProp name="PERF_DATA" elementType="Argument">
+            <stringProp name="Argument.name">PERF_DATA</stringProp>
+            <stringProp name="Argument.value">${__P(perf_data, perf_data.csv)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+          <elementProp name="BASE_DIR" elementType="Argument">
+            <stringProp name="Argument.name">BASE_DIR</stringProp>
+            <stringProp name="Argument.value">${__BeanShell(import org.apache.jmeter.services.FileServer; FileServer.getFileServer().getBaseDir();)}${__BeanShell(File.separator,)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+          <elementProp name="CSAR_DIR" elementType="Argument">
+            <stringProp name="Argument.name">CSAR_DIR</stringProp>
+            <stringProp name="Argument.value">${__P(csardir, /tmp/policy_distribution/csar)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+          <elementProp name="TOTAL" elementType="Argument">
+            <stringProp name="Argument.name">TOTAL</stringProp>
+            <stringProp name="Argument.value">${__P(total, 5)}</stringProp>
+            <stringProp name="Argument.metadata">=</stringProp>
+          </elementProp>
+        </collectionProp>
+      </elementProp>
+      <stringProp name="TestPlan.user_define_classpath"></stringProp>
+    </TestPlan>
+    <hashTree>
+      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
+        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
+        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
+          <boolProp name="LoopController.continue_forever">false</boolProp>
+          <stringProp name="LoopController.loops">1</stringProp>
+        </elementProp>
+        <stringProp name="ThreadGroup.num_threads">1</stringProp>
+        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
+        <boolProp name="ThreadGroup.scheduler">false</boolProp>
+        <stringProp name="ThreadGroup.duration">${DURATION}</stringProp>
+        <stringProp name="ThreadGroup.delay">1</stringProp>
+      </ThreadGroup>
+      <hashTree>
+        <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="GET_POLICY_HTTP_HEADER" enabled="true">
+          <collectionProp name="HeaderManager.headers">
+            <elementProp name="" elementType="Header">
+              <stringProp name="Header.name">Content-Type</stringProp>
+              <stringProp name="Header.value">application/json</stringProp>
+            </elementProp>
+            <elementProp name="" elementType="Header">
+              <stringProp name="Header.name">Accept</stringProp>
+              <stringProp name="Header.value">application/json</stringProp>
+            </elementProp>
+            <elementProp name="" elementType="Header">
+              <stringProp name="Header.name">ClientAuth</stringProp>
+              <stringProp name="Header.value">cHl0aG9uOnRlc3Q=</stringProp>
+            </elementProp>
+            <elementProp name="" elementType="Header">
+              <stringProp name="Header.name">Authorization</stringProp>
+              <stringProp name="Header.value">Basic dGVzdHBkcDphbHBoYTEyMw==</stringProp>
+            </elementProp>
+            <elementProp name="" elementType="Header">
+              <stringProp name="Header.name">Environment</stringProp>
+              <stringProp name="Header.value">TEST</stringProp>
+            </elementProp>
+          </collectionProp>
+        </HeaderManager>
+        <hashTree/>
+        <GenericController guiclass="LogicControllerGui" testclass="GenericController" testname="check if existing policy" enabled="true"/>
+        <hashTree>
+          <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="list_policy" enabled="true">
+            <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
+            <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
+              <collectionProp name="Arguments.arguments">
+                <elementProp name="" elementType="HTTPArgument">
+                  <boolProp name="HTTPArgument.always_encode">false</boolProp>
+                  <stringProp name="Argument.value">{&quot;policyName&quot;: &quot;oofCasablanca.*s3p.*&quot;}</stringProp>
+                  <stringProp name="Argument.metadata">=</stringProp>
+                </elementProp>
+              </collectionProp>
+            </elementProp>
+            <stringProp name="HTTPSampler.domain">${PDP_HOST}</stringProp>
+            <stringProp name="HTTPSampler.port">8081</stringProp>
+            <stringProp name="HTTPSampler.protocol">https</stringProp>
+            <stringProp name="HTTPSampler.contentEncoding"></stringProp>
+            <stringProp name="HTTPSampler.path">/pdp/api/listConfig</stringProp>
+            <stringProp name="HTTPSampler.method">POST</stringProp>
+            <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
+            <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
+            <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
+            <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
+            <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
+            <stringProp name="HTTPSampler.connect_timeout"></stringProp>
+            <stringProp name="HTTPSampler.response_timeout"></stringProp>
+          </HTTPSamplerProxy>
+          <hashTree>
+            <JSR223PostProcessor guiclass="TestBeanGUI" testclass="JSR223PostProcessor" testname="JSR223 PostProcessor" enabled="true">
+              <stringProp name="scriptLanguage">groovy</stringProp>
+              <stringProp name="parameters"></stringProp>
+              <stringProp name="filename"></stringProp>
+              <stringProp name="cacheKey">true</stringProp>
+              <stringProp name="script">import groovy.json.JsonSlurper;
+
+def to_delete = [];
+if (prev.getResponseCode() == &apos;200&apos;) {
+	def jsonSlurper = new JsonSlurper();
+	to_delete = jsonSlurper.parseText(prev.getResponseDataAsString());
+}
+vars.putObject(&apos;to_delete&apos;, to_delete);
+</stringProp>
+            </JSR223PostProcessor>
+            <hashTree/>
+          </hashTree>
+          <WhileController guiclass="WhileControllerGui" testclass="WhileController" testname="While Controller" enabled="true">
+            <stringProp name="WhileController.condition">${__groovy(vars.getObject(&apos;to_delete&apos;).size() &gt; 0)}</stringProp>
+          </WhileController>
+          <hashTree>
+            <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="delete_policy_pdp" enabled="true">
+              <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
+              <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
+                <collectionProp name="Arguments.arguments">
+                  <elementProp name="" elementType="HTTPArgument">
+                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
+                    <stringProp name="Argument.value">{&#xd;
+	&quot;pdpGroup&quot;: &quot;default&quot;,&#xd;
+	&quot;policyComponent&quot;: &quot;PDP&quot;,&#xd;
+	&quot;policyType&quot;: &quot;Optimization&quot;,&#xd;
+	&quot;policyName&quot;: &quot;${policyname}&quot;&#xd;
+}</stringProp>
+                    <stringProp name="Argument.metadata">=</stringProp>
+                  </elementProp>
+                </collectionProp>
+              </elementProp>
+              <stringProp name="HTTPSampler.domain">${PDP_HOST}</stringProp>
+              <stringProp name="HTTPSampler.port">8081</stringProp>
+              <stringProp name="HTTPSampler.protocol">https</stringProp>
+              <stringProp name="HTTPSampler.contentEncoding"></stringProp>
+              <stringProp name="HTTPSampler.path">/pdp/api/deletePolicy</stringProp>
+              <stringProp name="HTTPSampler.method">DELETE</stringProp>
+              <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
+              <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
+              <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
+              <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
+              <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
+              <stringProp name="HTTPSampler.connect_timeout"></stringProp>
+              <stringProp name="HTTPSampler.response_timeout"></stringProp>
+            </HTTPSamplerProxy>
+            <hashTree>
+              <JSR223PreProcessor guiclass="TestBeanGUI" testclass="JSR223PreProcessor" testname="JSR223 PreProcessor" enabled="true">
+                <stringProp name="scriptLanguage">groovy</stringProp>
+                <stringProp name="parameters"></stringProp>
+                <stringProp name="filename"></stringProp>
+                <stringProp name="cacheKey">true</stringProp>
+                <stringProp name="script">def to_delete = vars.getObject(&apos;to_delete&apos;);
+def policyname = to_delete.pop()
+vars.put(&apos;policyname&apos;, policyname.toString());
+vars.putObject(&apos;to_delete&apos;, to_delete);
+</stringProp>
+              </JSR223PreProcessor>
+              <hashTree/>
+            </hashTree>
+            <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="delete_policy_pap" enabled="true">
+              <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
+              <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
+                <collectionProp name="Arguments.arguments">
+                  <elementProp name="" elementType="HTTPArgument">
+                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
+                    <stringProp name="Argument.value">{&#xd;
+	&quot;deleteCondition&quot;:&quot;ALL&quot;,&#xd;
+	&quot;policyComponent&quot;: &quot;PAP&quot;,&#xd;
+	&quot;policyType&quot;: &quot;Optimization&quot;,&#xd;
+	&quot;policyName&quot;: &quot;${policyname}&quot;&#xd;
+}</stringProp>
+                    <stringProp name="Argument.metadata">=</stringProp>
+                  </elementProp>
+                </collectionProp>
+              </elementProp>
+              <stringProp name="HTTPSampler.domain">${PDP_HOST}</stringProp>
+              <stringProp name="HTTPSampler.port">8081</stringProp>
+              <stringProp name="HTTPSampler.protocol">https</stringProp>
+              <stringProp name="HTTPSampler.contentEncoding"></stringProp>
+              <stringProp name="HTTPSampler.path">/pdp/api/deletePolicy</stringProp>
+              <stringProp name="HTTPSampler.method">DELETE</stringProp>
+              <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
+              <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
+              <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
+              <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
+              <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
+              <stringProp name="HTTPSampler.connect_timeout"></stringProp>
+              <stringProp name="HTTPSampler.response_timeout"></stringProp>
+            </HTTPSamplerProxy>
+            <hashTree/>
+          </hashTree>
+        </hashTree>
+        <GenericController guiclass="LogicControllerGui" testclass="GenericController" testname="prepare_perf_data" enabled="true"/>
+        <hashTree>
+          <SystemSampler guiclass="SystemSamplerGui" testclass="SystemSampler" testname="OS Process Sampler" enabled="true">
+            <boolProp name="SystemSampler.checkReturnCode">false</boolProp>
+            <stringProp name="SystemSampler.expectedReturnCode">0</stringProp>
+            <stringProp name="SystemSampler.command">${BASE_DIR}generate_perf.sh</stringProp>
+            <elementProp name="SystemSampler.arguments" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
+              <collectionProp name="Arguments.arguments">
+                <elementProp name="" elementType="Argument">
+                  <stringProp name="Argument.name"></stringProp>
+                  <stringProp name="Argument.value">${CSAR_DIR}</stringProp>
+                  <stringProp name="Argument.metadata">=</stringProp>
+                </elementProp>
+                <elementProp name="" elementType="Argument">
+                  <stringProp name="Argument.name"></stringProp>
+                  <stringProp name="Argument.value">${TOTAL}</stringProp>
+                  <stringProp name="Argument.metadata">=</stringProp>
+                </elementProp>
+              </collectionProp>
+            </elementProp>
+            <elementProp name="SystemSampler.environment" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
+              <collectionProp name="Arguments.arguments"/>
+            </elementProp>
+            <stringProp name="SystemSampler.directory"></stringProp>
+          </SystemSampler>
+          <hashTree/>
+          <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="Load_all_data" enabled="true">
+            <stringProp name="cacheKey">true</stringProp>
+            <stringProp name="filename"></stringProp>
+            <stringProp name="parameters"></stringProp>
+            <stringProp name="script">def data_file=new File(vars.get(&apos;BASE_DIR&apos;) + vars.get(&apos;PERF_DATA&apos;));
+def data=data_file.readLines();
+def total=data.size();
+
+vars.putObject(&apos;data&apos;, data);
+vars.putObject(&apos;total&apos;, total);
+vars.putObject(&apos;idx&apos;, 0);
+vars.putObject(&apos;total_counter&apos;, vars.get(&apos;RETRY&apos;).toInteger() * total);
+
+SampleResult.setIgnore();</stringProp>
+            <stringProp name="scriptLanguage">groovy</stringProp>
+          </JSR223Sampler>
+          <hashTree/>
+        </hashTree>
+        <TransactionController guiclass="TransactionControllerGui" testclass="TransactionController" testname="verify policy created" enabled="true">
+          <boolProp name="TransactionController.includeTimers">false</boolProp>
+          <boolProp name="TransactionController.parent">true</boolProp>
+        </TransactionController>
+        <hashTree>
+          <Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
+            <collectionProp name="Arguments.arguments">
+              <elementProp name="innter_counter" elementType="Argument">
+                <stringProp name="Argument.name">innter_counter</stringProp>
+                <stringProp name="Argument.value">1</stringProp>
+                <stringProp name="Argument.metadata">=</stringProp>
+              </elementProp>
+              <elementProp name="rescode" elementType="Argument">
+                <stringProp name="Argument.name">rescode</stringProp>
+                <stringProp name="Argument.value"></stringProp>
+                <stringProp name="Argument.metadata">=</stringProp>
+              </elementProp>
+              <elementProp name="counter" elementType="Argument">
+                <stringProp name="Argument.name">counter</stringProp>
+                <stringProp name="Argument.value">1</stringProp>
+                <stringProp name="Argument.metadata">=</stringProp>
+              </elementProp>
+            </collectionProp>
+          </Arguments>
+          <hashTree/>
+          <WhileController guiclass="WhileControllerGui" testclass="WhileController" testname="OUTER" enabled="true">
+            <stringProp name="WhileController.condition">${__javaScript(${counter} &lt; ${total_counter} &amp;&amp; ${idx} &lt; ${total})}</stringProp>
+          </WhileController>
+          <hashTree>
+            <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="Reset_Inner_Loop" enabled="true">
+              <stringProp name="scriptLanguage">groovy</stringProp>
+              <stringProp name="parameters"></stringProp>
+              <stringProp name="filename"></stringProp>
+              <stringProp name="cacheKey">true</stringProp>
+              <stringProp name="script">//log.info(&quot;idx: &quot; + vars.getObject(&apos;idx&apos;));
+//log.info(&quot;counter: &quot; + vars.get(&apos;counter&apos;));
+//log.info(&quot;total_counter: &quot; + vars.get(&apos;total_counter&apos;));
+
+vars.putObject(&apos;inner_counter&apos;, 0);
+vars.put(&apos;rescode&apos;, &quot;&quot;);
+
+SampleResult.setIgnore();</stringProp>
+            </JSR223Sampler>
+            <hashTree/>
+            <WhileController guiclass="WhileControllerGui" testclass="WhileController" testname="INNER" enabled="true">
+              <stringProp name="WhileController.condition">${__javaScript(${inner_counter} &lt; ${RETRY} &amp;&amp; &quot;${rescode}&quot; != &apos;200&apos;)}</stringProp>
+            </WhileController>
+            <hashTree>
+              <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Get_Policy" enabled="true">
+                <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
+                <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
+                  <collectionProp name="Arguments.arguments">
+                    <elementProp name="" elementType="HTTPArgument">
+                      <boolProp name="HTTPArgument.always_encode">false</boolProp>
+                      <stringProp name="Argument.value">{&quot;policyName&quot;: &quot;oofCasablanca.*${resid}.*&quot;}</stringProp>
+                      <stringProp name="Argument.metadata">=</stringProp>
+                    </elementProp>
+                  </collectionProp>
+                </elementProp>
+                <stringProp name="HTTPSampler.domain">${PDP_HOST}</stringProp>
+                <stringProp name="HTTPSampler.port">8081</stringProp>
+                <stringProp name="HTTPSampler.protocol">https</stringProp>
+                <stringProp name="HTTPSampler.contentEncoding"></stringProp>
+                <stringProp name="HTTPSampler.path">/pdp/api/getConfig</stringProp>
+                <stringProp name="HTTPSampler.method">POST</stringProp>
+                <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
+                <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
+                <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
+                <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
+                <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
+                <stringProp name="HTTPSampler.connect_timeout"></stringProp>
+                <stringProp name="HTTPSampler.response_timeout"></stringProp>
+              </HTTPSamplerProxy>
+              <hashTree>
+                <JSR223PreProcessor guiclass="TestBeanGUI" testclass="JSR223PreProcessor" testname="JSR223 PreProcessor" enabled="true">
+                  <stringProp name="scriptLanguage">groovy</stringProp>
+                  <stringProp name="parameters"></stringProp>
+                  <stringProp name="filename"></stringProp>
+                  <stringProp name="cacheKey">true</stringProp>
+                  <stringProp name="script">def idx=vars.getObject(&apos;idx&apos;);
+def data=vars.getObject(&apos;data&apos;);
+
+vars.put(&apos;resid&apos;, data[idx]);
+//log.error(data[idx]);
+</stringProp>
+                </JSR223PreProcessor>
+                <hashTree/>
+                <JSR223PostProcessor guiclass="TestBeanGUI" testclass="JSR223PostProcessor" testname="JSR223 PostProcessor" enabled="true">
+                  <stringProp name="cacheKey">true</stringProp>
+                  <stringProp name="filename"></stringProp>
+                  <stringProp name="parameters"></stringProp>
+                  <stringProp name="script">def rescode = prev.getResponseCode();
+def idx=vars.getObject(&apos;idx&apos;);
+def total=vars.getObject(&apos;total&apos;);
+def inner_counter=vars.getObject(&apos;inner_counter&apos;);
+
+vars.put(&apos;rescode&apos;, rescode);
+//log.info(&apos;inner_counter: &apos; + inner_counter);
+//log.info(&apos;idx: &apos;+ idx);
+
+inner_counter += 1;
+vars.putObject(&apos;inner_counter&apos;, inner_counter);
+
+if (rescode == &apos;200&apos;){
+    idx += 1;
+    vars.putObject(&apos;idx&apos;, idx);
+}else {
+    
+}
+</stringProp>
+                  <stringProp name="scriptLanguage">groovy</stringProp>
+                </JSR223PostProcessor>
+                <hashTree/>
+              </hashTree>
+              <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter" enabled="true">
+                <stringProp name="CounterConfig.start">1</stringProp>
+                <stringProp name="CounterConfig.end"></stringProp>
+                <stringProp name="CounterConfig.incr">1</stringProp>
+                <stringProp name="CounterConfig.name">counter</stringProp>
+                <stringProp name="CounterConfig.format"></stringProp>
+                <boolProp name="CounterConfig.per_user">true</boolProp>
+                <boolProp name="CounterConfig.reset_on_tg_iteration">true</boolProp>
+              </CounterConfig>
+              <hashTree/>
+            </hashTree>
+          </hashTree>
+        </hashTree>
+        <GenericController guiclass="LogicControllerGui" testclass="GenericController" testname="final_check" enabled="true"/>
+        <hashTree>
+          <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="final_check" enabled="true">
+            <stringProp name="scriptLanguage">groovy</stringProp>
+            <stringProp name="parameters"></stringProp>
+            <stringProp name="filename"></stringProp>
+            <stringProp name="cacheKey">true</stringProp>
+            <stringProp name="script">def idx=vars.getObject(&apos;idx&apos;);
+def total=vars.getObject(&apos;total&apos;);
+
+if (idx&lt;total) {
+	SampleResult.setSuccessful(false);
+	SampleResult.setResponseCode(&quot;410&quot;);
+	SampleResult.setResponseMessage(&quot;Fail at idx &quot; + idx);
+}else {
+	SampleResult.setSuccessful(true);
+	SampleResult.setResponseCode(&quot;200&quot;);
+	SampleResult.setResponseMessage(&quot;Success&quot;);
+}
+</stringProp>
+          </JSR223Sampler>
+          <hashTree/>
+        </hashTree>
+      </hashTree>
+    </hashTree>
+  </hashTree>
+</jmeterTestPlan>