summaryrefslogtreecommitdiffstats
path: root/plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka
diff options
context:
space:
mode:
authorliamfallon <liam.fallon@est.tech>2020-10-16 13:09:11 +0100
committerliamfallon <liam.fallon@est.tech>2020-10-16 13:09:16 +0100
commit0cf967c0239a8ab9c8b8831b700b72d9a08f7b03 (patch)
treea4fbcd97008769d55ac443bc22abf517308bf6a7 /plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka
parent9833876720ff14517ee78bda557e6021df723800 (diff)
Remove apex asciidoc documents
Apex documentation has now all been ported to use the ONAP recommended rst format. This review removes the old asciidoc documents. Issue-ID: POLICY-2824 Change-Id: I562bd344cb7d6ff36e7d54bdb8f95e3b656468f8 Signed-off-by: liamfallon <liam.fallon@est.tech>
Diffstat (limited to 'plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka')
-rw-r--r--plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka/src/site-docs/adoc/fragments/ct-kafka-io.adoc114
1 files changed, 0 insertions, 114 deletions
diff --git a/plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka/src/site-docs/adoc/fragments/ct-kafka-io.adoc b/plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka/src/site-docs/adoc/fragments/ct-kafka-io.adoc
deleted file mode 100644
index 94586e9bd..000000000
--- a/plugins/plugins-event/plugins-event-carrier/plugins-event-carrier-kafka/src/site-docs/adoc/fragments/ct-kafka-io.adoc
+++ /dev/null
@@ -1,114 +0,0 @@
-//
-// ============LICENSE_START=======================================================
-// Copyright (C) 2016-2018 Ericsson. All rights reserved.
-// Modifications Copyright (C) 2019 Nordix Foundation.
-// ================================================================================
-// This file is licensed under the CREATIVE COMMONS ATTRIBUTION 4.0 INTERNATIONAL LICENSE
-// Full license text at https://creativecommons.org/licenses/by/4.0/legalcode
-//
-// SPDX-License-Identifier: CC-BY-4.0
-// ============LICENSE_END=========================================================
-//
-// @author Sven van der Meer (sven.van.der.meer@ericsson.com)
-//
-
-== Kafka IO
-
-Kafka IO is supported by the APEX Kafka plugin.
-The configurations below are examples.
-APEX will take any configuration inside the parameter object and forward it to Kafka.
-More information on Kafka specific configuration parameters can be found in the Kafka documentation:
-
-* link:https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html[Kafka Consumer Class]
-* link:https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html[Kafka Producer Class]
-
-
-
-=== Kafka Input
-
-APEX will receive events from the Apache Kafka messaging system.
-The input is uni-directional, an engine will only receive events from the input but not send any event to the input.
-
-[source%nowrap,json]
-----
-"carrierTechnologyParameters" : {
- "carrierTechnology" : "KAFKA", <1>
- "parameterClassName" :
- "org.onap.policy.apex.plugins.event.carrier.kafka.KAFKACarrierTechnologyParameters",
- "parameters" : {
- "bootstrapServers" : "localhost:49092", <2>
- "groupId" : "apex-group-id", <3>
- "enableAutoCommit" : true, <4>
- "autoCommitTime" : 1000, <5>
- "sessionTimeout" : 30000, <6>
- "consumerPollTime" : 100, <7>
- "consumerTopicList" : ["apex-in-0", "apex-in-1"], <8>
- "keyDeserializer" :
- "org.apache.kafka.common.serialization.StringDeserializer", <9>
- "valueDeserializer" :
- "org.apache.kafka.common.serialization.StringDeserializer", <10>
- "kafkaProperties" : [] <11>
- }
-}
-----
-
-<1> set Kafka as carrier technology
-<2> bootstrap server and port
-<3> a group identifier
-<4> flag for auto-commit
-<5> auto-commit timeout in milliseconds
-<6> session timeout in milliseconds
-<7> consumer poll time in milliseconds
-<8> consumer topic list
-<9> key for the Kafka de-serializer
-<10> value for the Kafka de-serializer
-<11> an optional list of name value pairs of properties to be passed transparently to Kafka.
-This field need not be specified, can be set to null, or to an empty list as here.
-
-
-=== Kafka Output
-
-APEX will send events to the Apache Kafka messaging system.
-The output is uni-directional, an engine will send events to the output but not receive any event from the output.
-
-
-[source%nowrap,json]
-----
-"carrierTechnologyParameters" : {
- "carrierTechnology" : "KAFKA", <1>
- "parameterClassName" :
- "org.onap.policy.apex.plugins.event.carrier.kafka.KAFKACarrierTechnologyParameters",
- "parameters" : {
- "bootstrapServers" : "localhost:49092", <2>
- "acks" : "all", <3>
- "retries" : 0, <4>
- "batchSize" : 16384, <5>
- "lingerTime" : 1, <6>
- "bufferMemory" : 33554432, <7>
- "producerTopic" : "apex-out", <8>
- "keySerializer" :
- "org.apache.kafka.common.serialization.StringSerializer", <9>
- "valueSerializer" :
- "org.apache.kafka.common.serialization.StringSerializer", <10>
- "kafkaProperties": [ <11>
- ["message.max.bytes", 1000000],
- ["compression.codec", "none"]
- ]
- }
-}
-----
-
-<1> set Kafka as carrier technology
-<2> bootstrap server and port
-<3> acknowledgement strategy
-<4> number of retries
-<5> batch size
-<6> time to linger in milliseconds
-<7> buffer memory in byte
-<8> producer topic
-<9> key for the Kafka serializer
-<10> value for the Kafka serializer
-<11> an optional list of name value pairs of properties to be passed transparently to Kafka. If a property appears in
-the _kafkaProperties_ field and is also explicitly specified to a non-default value (such as _lingerTime_
-and _linger.ms_) the explictly specified value of the property is used rather than the value specified in the
-_kafkaProperties_ list.
or: #336699; font-style: italic } /* Name.Label */ .highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .highlight .py { color: #336699; font-weight: bold } /* Name.Property */ .highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #336699 } /* Name.Variable */ .highlight .ow { color: #008800 } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */ .highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */ .highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */ .highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */ .highlight .vc { color: #336699 } /* Name.Variable.Class */ .highlight .vg { color: #dd7700 } /* Name.Variable.Global */ .highlight .vi { color: #3333bb } /* Name.Variable.Instance */ .highlight .vm { color: #336699 } /* Name.Variable.Magic */ .highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ }
#!/bin/bash

REPO_URL_BLOB=$(cat /opt/config/repo_url_blob.txt)
REPO_URL_ARTIFACTS=$(cat /opt/config/repo_url_artifacts.txt)
DEMO_ARTIFACTS_VERSION=$(cat /opt/config/demo_artifacts_version.txt)
INSTALL_SCRIPT_VERSION=$(cat /opt/config/install_script_version.txt)
CLOUD_ENV=$(cat /opt/config/cloud_env.txt)

# Convert Network CIDR to Netmask
cdr2mask () {
	# Number of args to shift, 255..255, first non-255 byte, zeroes
	set -- $(( 5 - ($1 / 8) )) 255 255 255 255 $(( (255 << (8 - ($1 % 8))) & 255 )) 0 0 0
	[ $1 -gt 1 ] && shift $1 || shift
	echo ${1-0}.${2-0}.${3-0}.${4-0}
}

# OpenStack network configuration
if [[ $CLOUD_ENV == "openstack" ]]
then
	echo 127.0.0.1 $(hostname) >> /etc/hosts

	# Allow remote login as root
	mv /root/.ssh/authorized_keys /root/.ssh/authorized_keys.bk
	cp /home/ubuntu/.ssh/authorized_keys /root/.ssh

	MTU=$(/sbin/ifconfig | grep MTU | sed 's/.*MTU://' | sed 's/ .*//' | sort -n | head -1)

	IP=$(cat /opt/config/local_private_ipaddr.txt)
	BITS=$(cat /opt/config/pktgen_private_net_cidr.txt | cut -d"/" -f2)
	NETMASK=$(cdr2mask $BITS)
	echo "auto eth1" >> /etc/network/interfaces
	echo "iface eth1 inet static" >> /etc/network/interfaces
	echo "    address $IP" >> /etc/network/interfaces
	echo "    netmask $NETMASK" >> /etc/network/interfaces
	echo "    mtu $MTU" >> /etc/network/interfaces

	IP=$(cat /opt/config/oam_private_ipaddr.txt)
	BITS=$(cat /opt/config/onap_private_net_cidr.txt | cut -d"/" -f2)
	NETMASK=$(cdr2mask $BITS)
	echo "auto eth2" >> /etc/network/interfaces
	echo "iface eth2 inet static" >> /etc/network/interfaces
	echo "    address $IP" >> /etc/network/interfaces
	echo "    netmask $NETMASK" >> /etc/network/interfaces
	echo "    mtu $MTU" >> /etc/network/interfaces
fi

# Download required dependencies
echo "deb http://ppa.launchpad.net/openjdk-r/ppa/ubuntu $(lsb_release -c -s) main" >>  /etc/apt/sources.list.d/java.list
echo "deb-src http://ppa.launchpad.net/openjdk-r/ppa/ubuntu $(lsb_release -c -s) main" >>  /etc/apt/sources.list.d/java.list
apt-get update
apt-get install --allow-unauthenticated -y make wget openjdk-8-jdk gcc libcurl4-openssl-dev python-pip bridge-utils apt-transport-https ca-certificates git maven
pip install jsonschema

# Download vFirewall demo code for packet generator
cd /opt
wget $REPO_URL_BLOB/org.onap.demo/vnfs/vlbms/$INSTALL_SCRIPT_VERSION/v_packetgen_init.sh
wget $REPO_URL_BLOB/org.onap.demo/vnfs/vlbms/$INSTALL_SCRIPT_VERSION/vpacketgen.sh
wget $REPO_URL_BLOB/org.onap.demo/vnfs/vlbms/$INSTALL_SCRIPT_VERSION/run_streams_dns.sh
wget $REPO_URL_BLOB/org.onap.demo/vnfs/vlbms/$INSTALL_SCRIPT_VERSION/properties.conf -O /opt/config/properties.conf
wget $REPO_URL_BLOB/org.onap.demo/vnfs/vlbms/$INSTALL_SCRIPT_VERSION/run_health.sh
wget $REPO_URL_ARTIFACTS/org/onap/demo/vnf/vlb/vlb_dns_streams/$DEMO_ARTIFACTS_VERSION/vlb_dns_streams-$DEMO_ARTIFACTS_VERSION-demo.tar.gz 

sed -i 's/primary=.*/primary=false/g' /opt/config/properties.conf
sed -i 's/vnfc=.*/vnfc=vPacketGen/g' /opt/config/properties.conf

tar -zmxvf vlb_dns_streams-$DEMO_ARTIFACTS_VERSION-demo.tar.gz
mv vlb_dns_streams-$DEMO_ARTIFACTS_VERSION dns_streams

# Clone Honeycomb interface for the VNF component
mkdir honeycomb-api
git init honeycomb-api
cd honeycomb-api
git remote add origin https://gerrit.onap.org/r/p/demo.git
git config core.sparsecheckout true
echo "vnfs/vLBMS/apis" >> .git/info/sparse-checkout
git pull --depth=1 origin master

cat > ~/.m2/settings.xml << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!-- vi: set et smarttab sw=2 tabstop=2: -->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
 
  <profiles>
    <profile>
      <id>fd.io-release</id>
      <repositories>
        <repository>
          <id>fd.io-mirror</id>
          <name>fd.io-mirror</name>
          <url>https://nexus.fd.io/content/groups/public/</url>
          <releases>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>fd.io-mirror</id>
          <name>fd.io-mirror</name>
          <url>https://nexus.fd.io/content/repositories/public/</url>
          <releases>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
        </pluginRepository>
      </pluginRepositories>
    </profile>
 
    <profile>
      <id>fd.io-snapshots</id>
      <repositories>
        <repository>
          <id>fd.io-snapshot</id>
          <name>fd.io-snapshot</name>
          <url>https://nexus.fd.io/content/repositories/fd.io.snapshot/</url>
          <releases>
            <enabled>false</enabled>
          </releases>
          <snapshots>
            <enabled>true</enabled>
          </snapshots>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>fd.io-snapshot</id>
          <name>fd.io-snapshot</name>
          <url>https://nexus.fd.io/content/repositories/fd.io.snapshot/</url>
          <releases>
            <enabled>false</enabled>
          </releases>
          <snapshots>
            <enabled>true</enabled>
          </snapshots>
        </pluginRepository>
      </pluginRepositories>
    </profile>
    <profile>
      <id>opendaylight-snapshots</id>
      <repositories>
        <repository>
          <id>opendaylight-snapshot</id>
          <name>opendaylight-snapshot</name>
          <url>https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/</url>
          <releases>
            <enabled>false</enabled>
          </releases>
          <snapshots>
            <enabled>true</enabled>
          </snapshots>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>opendaylight-shapshot</id>
          <name>opendaylight-snapshot</name>
          <url>https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/</url>
          <releases>
            <enabled>false</enabled>
          </releases>
          <snapshots>
            <enabled>true</enabled>
          </snapshots>
        </pluginRepository>
      </pluginRepositories>
    </profile>
  </profiles>
 
  <activeProfiles>
    <activeProfile>fd.io-release</activeProfile>
    <activeProfile>fd.io-snapshots</activeProfile>
    <activeProfile>opendaylight-snapshots</activeProfile>
  </activeProfiles>
</settings>
EOF

cd /opt/honeycomb-api/vnfs/vLBMS/apis
mvn clean install

#wget $REPO_URL_ARTIFACTS/org/onap/demo/vnf/vlb/vlb-vnf-onap-distribution/$DEMO_ARTIFACTS_VERSION/vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION-hc.tar.gz
#tar -zmxvf vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION-hc.tar.gz
sed -i 's/"restconf-binding-address": "127.0.0.1",/"restconf-binding-address": "0.0.0.0",/g' /opt/honeycomb-api/vnfs/vLBMS/apis/vlb-vnf-onap-distribution/target/vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION-hc/vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION/config/honeycomb.json
sed -i 's/"netconf-tcp-binding-address": "127.0.0.1",/"netconf-tcp-binding-address": "0.0.0.0",/g' /opt/honeycomb-api/vnfs/vLBMS/apis/vlb-vnf-onap-distribution/target/vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION-hc/vlb-vnf-onap-distribution-$DEMO_ARTIFACTS_VERSION/config/honeycomb.json

cd /opt
rm *.tar.gz
chmod +x v_packetgen_init.sh
chmod +x vpacketgen.sh
chmod +x run_streams_dns.sh
chmod +x run_health.sh

echo "vpp" > config/service.txt

# Install VPP
export UBUNTU="xenial"
export RELEASE=".stable.1707"
rm /etc/apt/sources.list.d/99fd.io.list
echo "deb [trusted=yes] https://nexus.fd.io/content/repositories/fd.io$RELEASE.ubuntu.$UBUNTU.main/ ./" | tee -a /etc/apt/sources.list.d/99fd.io.list
apt-get update
apt-get install -y vpp vpp-dpdk-dkms vpp-lib vpp-dbg vpp-plugins vpp-dev
sleep 1

# Run instantiation script
cd /opt
mv vpacketgen.sh /etc/init.d
update-rc.d vpacketgen.sh defaults

# Rename network interface in openstack Ubuntu 16.04 images. Then, reboot the VM to pick up changes
if [[ $CLOUD_ENV != "rackspace" ]]
then
	sed -i "s/GRUB_CMDLINE_LINUX=.*/GRUB_CMDLINE_LINUX=\"net.ifnames=0 biosdevname=0\"/g" /etc/default/grub
	grub-mkconfig -o /boot/grub/grub.cfg
	sed -i "s/ens[0-9]*/eth0/g" /etc/network/interfaces.d/*.cfg
	sed -i "s/ens[0-9]*/eth0/g" /etc/udev/rules.d/70-persistent-net.rules
	echo 'network: {config: disabled}' >> /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
	echo "APT::Periodic::Unattended-Upgrade \"0\";" >> /etc/apt/apt.conf.d/10periodic
	reboot
fi

./v_packetgen_init.sh