Clean up .sh scripts

Removed .sh scripts that are not needed.
Fill readme with a description of some of the things
that might be not visible at first glance
Refactor .sh scripts, remove not used or not needed parts
Fix a bug where pidof <package> does not work.
Change logging of the scripts and they way that app is run so
that the logs are in one place -> collector.logs
Fixed a bug where tomcat startup problems are not logged
Added log rotation with logrotate tool

Change-Id: I156328ab23d18a20072177ae8c5420772c3e2c7c
Signed-off-by: PawelSzalapski <pawel.szalapski@nokia.com>
Issue-ID: DCAEGEN2-610

11 files changed, 253 insertions, 336 deletions

diff --git a/README.md b/README.md
index 6dc31611..b7bbdb83 100644
--- a/README.md
+++ b/README.md
@@ -17,13 +17,57 @@ git clone ssh://vv770d@gerrit.onap.org:29418/dcaegen2/collectors/ves
 mvn clean install
 ```
 
-### Docker Image
+### Running Locally
 
+Build the image (it will go into your local docker repository)
 ```
-git clone ssh://vv770d@gerrit.onap.org:29418/dcaegen2/collectors/ves
-mvn clean deploy
+mvn clean package
+```
+
+Run the image using docker-compose.yml
+```
+docker-compose up
 ```
 
+### Environment variables in Docker Container
+Most of the configuration of how VESCollector should be started and managed is done through environment variables.
+Some of them are set during the image build process and some of them are defined manually or by
+a particular deployment system.
+
+Variables set manually / coming from deployment system:
+- COLLECTOR_IP
+- DMAAPHOST - should contain an address to DMaaP, so that event publishing can work
+- CBSPOLLTIMER - it should be put in here if we want to automatically fetch configuration from CBS.
+- CONSUL_HOST - used with conjunction with CBSPOLLTIMER, should be a host address (without port! e.g http://my-ip-or-host) where Consul service lies
+- CONFIG_BINDING_SERVICE - used with conjunction with CBSPOLLTIMER, should be a name of CBS as it is registered in Consul
+- HOSTNAME - used with conjunction with CBSPOLLTIMER, should be a name of VESCollector application as it is registered in CBS catalog
+
+### Docker file system layout
+The main directory where all code resides in docker container
+looks like this and is located in /opt/app/VESCollector
+```
+<host>:/opt/app/VESCollector# ls
+Dockerfile  bin  etc  lib  logs  specs	tomcat.8080
+```
+- bin contains sh scripts (path here is denoted by env var $SCRIPTS_PATH)
+- etc contains various application configuration, most notably it reflects 'etc' directory from repository
+- lib contains all libraries that are pulled into the app during maven build
+- logs contains all application logs, especially collector.log file which is a main log file denoted by $MAIN_LOG_FILE variable
+- specs contains json schemas specs for ves-collector
+
+## Managing application in Docker container
+Scripts directory contain .sh scripts that are used to start & stop & configure the VESCollector application
+inside the docker image.
+These scripts are packaged inside the docker image by a mvn assembly & docker plugins.
+
+## How the application starts inside container
+General flow goes like this
+- Docker image is build, and it points docker-entry.sh as the entrypoint.
+- Docker-entry point, depending on the deployment type,
+configures a bunch of things and starts the application in a separate process
+and loops indefinitely to hold the docker container process.
+
+### Release images
 For R1 - image/version  pushed to nexus3 
 ```
 nexus3.onap.org:10003/snapshots/onap/org.onap.dcaegen2.collectors.ves.vescollector   1.1 
diff --git a/etc/log4j.xml b/etc/log4j.xml
index baf03559..460004c6 100644
--- a/etc/log4j.xml
+++ b/etc/log4j.xml
@@ -27,18 +27,7 @@ ECOMP is a trademark and service mark of AT&T Intellectual Property.
 	<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
 		<param name="threshold" value="INFO" />
 		<layout class="org.apache.log4j.PatternLayout">
-			<param name="ConversionPattern" value="[%d{ABSOLUTE}][%-5p][%-10t]%m%n" />
-		</layout>
-	</appender>
-
-	<appender name="FILE" class="org.apache.log4j.RollingFileAppender">
-		<param name="threshold" value="INFO" />
-		<param name="File" value="logs/collector.log" />
-		<param name="MaxFileSize" value="32MB"/>
-		<param name="MaxBackupIndex" value="20"/>
-		<layout class="org.apache.log4j.PatternLayout">
-			<!-- param name="ConversionPattern" value="[%d{ABSOLUTE}][%-5p][%-10t][%-5c][%4L]%m%n" / -->
-			<param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-10t][%-5c]%m%n" />
+			<param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-10t][%c] - %m%n"/>
 		</layout>
 	</appender>
 
@@ -48,7 +37,6 @@ ECOMP is a trademark and service mark of AT&T Intellectual Property.
 		<param name="MaxFileSize" value="32MB"/>
 		<param name="MaxBackupIndex" value="10"/>
 		<layout class="org.apache.log4j.PatternLayout">
-			<!-- param name="ConversionPattern" value="[%d{ABSOLUTE}][%-5p][%-10t][%-5c][%4L]%m%n" / -->
 			<param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-10t][%-5c]%m%n" />
 		</layout>
 	</appender>
@@ -59,33 +47,30 @@ ECOMP is a trademark and service mark of AT&T Intellectual Property.
 		<param name="MaxFileSize" value="32MB"/>
 		<param name="MaxBackupIndex" value="10"/>
 		<layout class="org.apache.log4j.PatternLayout">
-			<!-- param name="ConversionPattern" value="[%d{ABSOLUTE}][%-5p][%-10t][%-5c][%4L]%m%n" / -->
 			<param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-10t][%-5c]%m%n" />
 		</layout>
 	</appender>
-	
+
 	<appender name="EFILE" class="org.apache.log4j.RollingFileAppender">
 		<param name="threshold" value="INFO" />
 		<param name="File" value="logs/error.log" />
 		<param name="MaxFileSize" value="32MB"/>
 		<param name="MaxBackupIndex" value="5"/>
 		<layout class="org.apache.log4j.PatternLayout">
-			<!-- param name="ConversionPattern" value="[%d{ABSOLUTE}][%-5p][%-10t][%-5c][%4L]%m%n" / -->
 			<param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-10t][%-5c]%m%n" />
 		</layout>
 	</appender>
-	
+
 	<!--
 		ECOMP logging setup
-	
+
 		NOTES:
-		
+
 		1. files are written to "./logs/<filename>". You must setup the environment
 			so that ./logs is a symlink to the correct location according to the ECOMP
 			log standard. For example, "/opt/logs/DCAE/highlandParkVcScope". If that's
 			not possible, change the File setting in each appender appropriately.
 	  -->
-
 	<appender name="ECOMP_AUDIT" class="org.apache.log4j.RollingFileAppender">
 		<param name="threshold" value="DEBUG" />
 		<param name="File" value="./logs/ecomp/audit.log" />
@@ -165,25 +150,20 @@ ECOMP is a trademark and service mark of AT&T Intellectual Property.
     <logger name="org.onap.dcae.commonFunction.EventPublisher" additivity="false">
 		<level value="debug"/>
 		<appender-ref ref="CONSOLE" />
-		<appender-ref ref="FILE" />
 	</logger>
     
-    
 	<logger name="com.att.nsa.apiClient.http.HttpClient" additivity="false">
 		<level value="info"/>
-		<appender-ref ref="FILE" />
 		<appender-ref ref="CONSOLE" />
 	</logger>
     
     <logger name="com.att.nsa.cambria.client.impl.CambriaSimplerBatchPublisher" additivity="false">
 		<level value="info"/>
-		<appender-ref ref="FILE" />
 		<appender-ref ref="CONSOLE" />
 	</logger>
 	
 	<root>
 		<level value="DEBUG" />
-		<appender-ref ref="FILE" />
 		<appender-ref ref="CONSOLE" />
 		<appender-ref ref="ECOMP_AUDIT" />
 		<appender-ref ref="ECOMP_DEBUG" />
diff --git a/etc/logrotate.conf b/etc/logrotate.conf
new file mode 100644
index 00000000..41bf4edc
--- /dev/null
+++ b/etc/logrotate.conf
@@ -0,0 +1,7 @@
+/opt/app/VESCollector/logs/collector.log {
+  size 64M
+  rotate 10
+  copytruncate
+  dateext
+  dateformat .%Y-%m-%dT%H:%M:%S
+}
\ No newline at end of file
diff --git a/pom.xml b/pom.xml
index b534379f..c87a9d24 100644
--- a/pom.xml
+++ b/pom.xml
@@ -18,8 +18,8 @@ limitations under the License.
 -->
 
 <project xmlns="http://maven.apache.org/POM/4.0.0"
-         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>
 
     <parent>
@@ -191,7 +191,6 @@ limitations under the License.
                     <imageTags>
                         <tag>latest</tag>
                     </imageTags>
-
                     <baseImage>openjdk:8-jre-slim</baseImage>
                     <env>
                         <HOSTALIASES>/etc/host.aliases</HOSTALIASES>
@@ -205,6 +204,10 @@ limitations under the License.
                     </resources>
                     <runs>
                         <run>chmod +x bin/*.sh</run> <!-- Maven is loosing file permissions during artifacts copy -->
+                        <run>
+                            <![CDATA[apt-get update && apt-get install -y --no-install-recommends logrotate && apt-get clean all]]>
+                        </run>
+                        <run>chmod 0644 etc/logrotate.conf</run>
                     </runs>
                     <exposes>
                         <expose>8080</expose>
@@ -241,7 +244,8 @@ limitations under the License.
                         </goals>
                         <configuration>
                             <image>${docker.image.name}:latest</image>
-                            <newName>${onap.nexus.dockerregistry.daily}/${docker.image.name}:${project.version}</newName>
+                            <newName>${onap.nexus.dockerregistry.daily}/${docker.image.name}:${project.version}
+                            </newName>
                             <pushImage>true</pushImage>
                         </configuration>
                     </execution>
@@ -253,7 +257,9 @@ limitations under the License.
                         </goals>
                         <configuration>
                             <image>${docker.image.name}:latest</image>
-                            <newName>${onap.nexus.dockerregistry.daily}/${docker.image.name}:${project.version}-${maven.build.timestamp}Z</newName>
+                            <newName>
+                                ${onap.nexus.dockerregistry.daily}/${docker.image.name}:${project.version}-${maven.build.timestamp}Z
+                            </newName>
                             <pushImage>true</pushImage>
                         </configuration>
                     </execution>
@@ -262,6 +268,25 @@ limitations under the License.
         </plugins>
     </build>
 
+    <profiles>
+        <profile>
+            <id>with-system-proxy</id>
+            <build>
+                <plugins>
+                    <plugin>
+                        <groupId>com.spotify</groupId>
+                        <artifactId>docker-maven-plugin</artifactId>
+                        <configuration>
+                            <buildArgs>
+                                <http_proxy>${env.http_proxy}</http_proxy>
+                            </buildArgs>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+    </profiles>
+
     <reporting>
         <plugins>
             <plugin>
diff --git a/src/main/scripts/VESrestfulCollector.sh b/src/main/scripts/VESrestfulCollector.sh
index c883f6de..8462f4e2 100644
--- a/src/main/scripts/VESrestfulCollector.sh
+++ b/src/main/scripts/VESrestfulCollector.sh
@@ -1,10 +1,11 @@
-#!/bin/sh

+#!/bin/bash

 

 ###

 # ============LICENSE_START=======================================================

 # PROJECT

 # ================================================================================

 # Copyright (C) 2017-2018 AT&T Intellectual Property. All rights reserved.

+# Copyright (C) 2018 Nokia Networks Intellectual Property. All rights reserved.

 # ================================================================================

 # Licensed under the Apache License, Version 2.0 (the "License");

 # you may not use this file except in compliance with the License.

@@ -19,175 +20,83 @@
 # limitations under the License.

 # ============LICENSE_END=========================================================

 ###

+source bin/logger.sh

 

-# redirect stdout/stderr to a file

-#exec &> /opt/app/VESCollector/logs/console.txt

+start() {

+    log "Starting application"

+    appPids=`pidof java`

 

-usage() {

-        echo "VESrestfulCollector.sh <start/stop>"

+    if [ ! -z ${appPids} ]; then

+        logWarn "Tried to start an application, but it is already running on PID(s): ${appPids}. Startup aborted."

+        exit 1

+    fi

+

+    ${JAVA_HOME}/bin/java -cp "etc:lib/*" \

+      -Xms256m -Xmx512m \

+      -XX:ErrorFile=logs/java_error%p.log \

+      -XX:+HeapDumpOnOutOfMemoryError \

+      -Dhttps.protocols=TLSv1.1,TLSv1.2 \

+      org.onap.dcae.commonFunction.CommonStartup $* & &>> logs/collector.log

 }

 

+stop() {

+    log "Stopping application"

+    appPids=`pidof java`

 

-BASEDIR=/opt/app/VESCollector/

-

-collector_start() {

-        echo `date +"%Y%m%d.%H%M%S%3N"` - collector_start | tee -a ${BASEDIR}/logs/console.txt

-        collectorPid=`pidof org.onap.dcae.commonFunction`

-

-        if [ ! -z "$collectorPid" ]; then

-                echo  "WARNING: VES Restful Collector already running as PID $collectorPid" | tee -a ${BASEDIR}/logs/console.txt

-                echo  "Startup Aborted!!!" | tee -a ${BASEDIR}/logs/console.txt

-                exit 1

-        fi

-

-

-        # run java. The classpath is the etc dir for config files, and the lib dir

-        # for all the jars.

-        #cd /opt/app/VESCollector/

-        cd ${BASEDIR}

-        #nohup $JAVA -cp "etc${PATHSEP}lib/*" $JAVA_OPTS -Dhttps.protocols=TLSv1.1,TLSv1.2 $MAINCLASS $* &

-		nohup $JAVA -cp "etc${PATHSEP}lib/*" -Xms256m -Xmx512m -XX:ErrorFile=/opt/app/VESCollector/logs/java_error%p.log -XX:+HeapDumpOnOutOfMemoryError -Dhttps.protocols=TLSv1.1,TLSv1.2 $MAINCLASS $* &

-        if [ $? -ne 0 ]; then

-                echo "VES Restful Collector has been started!!!" | tee -a ${BASEDIR}/logs/console.txt

+    if [ ! -z ${appPids} ]; then

+        echo "Killing java PID(s): ${appPids}"

+        kill -9 ${appPids}

+        sleep 5

+        if [ ! $(pidof java) ]; then

+            log "Application stopped"

+        else

+            logWarn "Application did not stop after 5 seconds"

         fi

-

-

-}

-

-collector_stop() {

-         echo `date +"%Y%m%d.%H%M%S%3N"` - collector_stop

-         collectorPid=`pidof org.onap.dcae.commonFunction`

-         if [ ! -z "$collectorPid" ]; then

-                echo "Stopping PID $collectorPid"

-

-                kill -9 $collectorPid

-                sleep 5

-                if [ ! "$(pidof org.onap.dcae.commonFunction)" ]; then

-                         echo "VES Restful Collector has been stopped!!!"

-                else

-                         echo "VES Restful Collector is being stopped!!!"

-                fi

-         else

-                echo  "WARNING: No VES Collector instance is currently running";

-                exit 1

-         fi

-

-

+    else

+        logWarn "Tried to stop an application, but it was not running";

+    fi

 }

 

 collector_configupdate() {

+    if [ -z ${CONSUL_HOST} ] || [ -z ${CONFIG_BINDING_SERVICE} ] || [ -z ${HOSTNAME} ]; then

+        log "Using standard controller configuration (no dynamic configuration done)"

+    else

+        ${JAVA_HOME}/bin/java -cp "etc:lib/*" org.onap.dcae.controller.FetchDynamicConfig $*

 

-        echo `date +"%Y%m%d.%H%M%S%3N"` - collector_configupdate

-        if [ -z "$CONSUL_HOST" ] || [ -z "$CONFIG_BINDING_SERVICE" ] || [ -z "$HOSTNAME" ]; then

-                echo "INFO: USING STANDARD CONTROLLER CONFIGURATION"

+        if [ $? -ne 0 ]; then

+            logWarn "Failed to fetch dynamic configuration from consul into container /opt/app/KV-Configuration.json"

         else

+            log "Dynamic config fetched and written successfully into container /opt/app/KV-Configuration.json"

+        fi

 

-            echo "INFO: DYNAMIC CONFIG INTERFACE SUPPORTED"

-            # move into base directory

-

-            #BASEDIR=`dirname $0`

-            #cd $BASEDIR/..

-            cd /opt/app/VESCollector

-

-            CONFIG_FETCH=org.onap.dcae.controller.FetchDynamicConfig

-            $JAVA -cp "etc${PATHSEP}lib/*"  $CONFIG_FETCH $*

+        if [ -f /opt/app/KV-Configuration.json ]; then

+            ${JAVA_HOME}/bin/java -cp "etc:lib/*" org.onap.dcae.controller.LoadDynamicConfig $*

             if [ $? -ne 0 ]; then

-                echo "ERROR: Failed to fetch dynamic configuration from consul into container /opt/app/KV-Configuration.json"

-            else

-                echo "INFO: Dynamic config fetched and written successfully into container /opt/app/KV-Configuration.json"

-            fi

-

-

-            if [ -f /opt/app/KV-Configuration.json ]; then

-

-                    CONFIG_UPDATER=org.onap.dcae.controller.LoadDynamicConfig

-                    $JAVA -cp "etc${PATHSEP}lib/*"  $CONFIG_UPDATER $*

-                    if [ $? -ne 0 ]; then

-                        echo "ERROR: Failed to update dynamic configuration into Application"

-                    else

-                        echo "INFO: Dynamic config updated successfully into VESCollector configuration!"

-                    fi

-

-					# Identify alias names from keystore and password provided

-

-            		paramName="collector.keystore.alias"

-					localpropertyfile="/opt/app/VESCollector/etc/collector.properties"

-					tmpfile="/opt/app/VESCollector/etc/collector.properties.tmp"

-

-					keystore=`grep collector.keystore.file.location $localpropertyfile | tr -d '[:space:]' | cut -d"=" -f2`

-					keypwdfile=`grep collector.keystore.passwordfile $localpropertyfile | tr -d '[:space:]' | cut -d"=" -f2`

-

-					echo "/usr/bin/keytool -list -keystore $keystore < $keypwdfile | grep "PrivateKeyEntry" | cut -d"," -f1"

-                    tmpalias=`/usr/bin/keytool -list -keystore $keystore < $keypwdfile | grep "PrivateKeyEntry" | cut -d"," -f1`

-                    echo "tmpalias:" $tmpalias

-                    alias=`echo $tmpalias | cut -d":" -f2`

-                    echo "alias:" $alias

-                    sed "s~$paramName=.*~$paramName=$alias~g" $localpropertyfile > $tmpfile

-                    echo `cat $tmpfile > $localpropertyfile`

-                    rm $tmpfile

-                	echo "INFO: Keystore alias updated into configuration"

-

+                echo "ERROR: Failed to update dynamic configuration into Application"

             else

-                echo "ERROR: Configuration file /opt/app/KV-Configuration.json missing"

+                echo "INFO: Dynamic config updated successfully into VESCollector configuration!"

             fi

-

+            paramName="collector.keystore.alias"

+            localpropertyfile="etc/collector.properties"

+            tmpfile="etc/collector.properties.tmp"

+            keystore=`grep collector.keystore.file.location $localpropertyfile | tr -d '[:space:]' | cut -d"=" -f2`

+            keypwdfile=`grep collector.keystore.passwordfile $localpropertyfile | tr -d '[:space:]' | cut -d"=" -f2`

+            echo "/usr/bin/keytool -list -keystore $keystore < $keypwdfile | grep "PrivateKeyEntry" | cut -d"," -f1"

+            tmpalias=`/usr/bin/keytool -list -keystore $keystore < $keypwdfile | grep "PrivateKeyEntry" | cut -d"," -f1`

+            alias=`echo $tmpalias | cut -d":" -f2`

+            sed "s~$paramName=.*~$paramName=$alias~g" $localpropertyfile > $tmpfile

+            echo `cat $tmpfile > $localpropertyfile`

+            rm $tmpfile

+            log "Keystore alias updated"

+        else

+            logWarn "Configuration file /opt/app/KV-Configuration.json missing"

         fi

+    fi

 }

 

-

-## Check usage

-if [ $# -ne 1 ]; then

-        usage

-        exit

-fi

-

-

-## Pre-setting

-

-# use JAVA_HOME if provided

-if [ -z "$JAVA_HOME" ]; then

-        echo "ERROR: JAVA_HOME not setup"

-        echo "Startup Aborted!!"

-        exit 1

-        #JAVA=java

-else

-        JAVA=$JAVA_HOME/bin/java

-fi

-

-

-MAINCLASS=org.onap.dcae.commonFunction.CommonStartup

-

-# determine a path separator that works for this platform

-PATHSEP=":"

-case "$(uname -s)" in

-

-        Darwin)

-                ;;

-

-         Linux)

-                ;;

-

-         CYGWIN*|MINGW32*|MSYS*)

-                PATHSEP=";"

-                ;;

-

-        *)

-                ;;

-esac

-

-

-

-

 case $1 in

-        "start")

-                collector_configupdate | tee -a ${BASEDIR}/logs/console.txt

-                collector_start

-                ;;

-        "stop")

-                collector_stop | tee -a ${BASEDIR}/logs/console.txt

-                ;;

-        *)

-                usage

-                ;;

+  "start") collector_configupdate; start ;;

+  "stop")  stop ;;

+  *)       echo "Bad usage. Should be: /bin/bash <this> start/stop"

 esac

 

diff --git a/src/main/scripts/VESrestfulCollector_Status.sh b/src/main/scripts/VESrestfulCollector_Status.sh
deleted file mode 100644
index 47be7a14..00000000
--- a/src/main/scripts/VESrestfulCollector_Status.sh
+++ /dev/null
@@ -1,41 +0,0 @@
-###
-# ============LICENSE_START=======================================================
-# PROJECT
-# ================================================================================
-# Copyright (C) 2017 AT&T Intellectual Property. All rights reserved.
-# ================================================================================
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-# 
-#      http://www.apache.org/licenses/LICENSE-2.0
-# 
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# ============LICENSE_END=========================================================
-###
-
-#!/bin/sh
-
-#secPid=`pgrep -f com.att.dcae.commonFunction.CommonStartup` --> master
-secPid=`pgrep -f org.onap.dcae.commonFunction.CommonStartup`
-
-
-if [ "${secPid}" ]
-then
-        #errorcnt = `grep -c "CambriaSimplerBatchPublisher - Send failed" ../logs/collector.log`
-        errorcnt=`tail -1000 ../logs/collector.log | grep -c "CambriaSimplerBatchPublisher - Send failed"`
-        
-        if [ $errorcnt -gt 10 ]
-        then
-                echo "VESCollecter_Is_HavingError to publish"
-        else
-                echo "VESCollecter_Is_Running as PID $secPid"
-        fi
-else
-   echo "VESCollecter_Is_Not_Running"
-fi
-exit
diff --git a/src/main/scripts/docker-entry.sh b/src/main/scripts/docker-entry.sh
index 96cb088c..0aad7584 100644
--- a/src/main/scripts/docker-entry.sh
+++ b/src/main/scripts/docker-entry.sh
@@ -1,9 +1,10 @@
-#!/bin/sh

+#!/bin/bash

 ###

 # ============LICENSE_START=======================================================

 # PROJECT

 # ================================================================================

 # Copyright (C) 2017-2018 AT&T Intellectual Property. All rights reserved.

+# Copyright (C) 2018 Nokia Networks Intellectual Property. All rights reserved.

 # ================================================================================

 # Licensed under the Apache License, Version 2.0 (the "License");

 # you may not use this file except in compliance with the License.

@@ -18,36 +19,48 @@
 # limitations under the License.

 # ============LICENSE_END=========================================================

 ###

+source bin/logger.sh

 

-if [ ! -z "$COLLECTOR_IP" ]; then

-        echo $COLLECTOR_IP  $(hostname).dcae.simpledemo.onap.org >> /etc/hosts

-fi

+# Redirect all stdout & stderr to a main log file, but also let it print into the console

+# At the time this script is invoked, these directories and files do not exist yet, so we need to create them

+mkdir -p logs

+touch logs/collector.log

+exec &> >(tee -a logs/collector.log)

 

-if [ ! -z "$DMAAPHOST" ]; then

-        if [ -z "$(echo $DMAAPHOST | sed -e 's/[0-9\.]//g')" ]; then 

-                echo "$DMAAPHOST  onap-dmaap" >> /etc/hosts

-        else

-                echo "onap-dmaap $DMAAPHOST" >> /etc/host.aliases

-        fi

-else

-	echo "DMAAPHOST ENV NOT SET!! PUBLISH WILL NOT BE SUPPORTED"

+log "Enabling log rotation for collector.log"

+loggedCommand "cp etc/logrotate.conf /etc/logrotate.d"

+echo "* *	* * *	root    logrotate /etc/logrotate.conf" >> /etc/crontab

+log "Restarting cron"

+loggedCommand "service cron reload"

+loggedCommand "service cron start"

+

+log "Main application entry-point invoked"

+

+if [ ! -z ${COLLECTOR_IP} ]; then

+    log "Collector ip (${COLLECTOR_IP}) (env var 'COLLECTOR_IP') found, adding entry to /etc/hosts"

+    echo ${COLLECTOR_IP}  $(hostname).dcae.simpledemo.onap.org >> /etc/hosts

 fi

 

-if [ -z "$CONSUL_HOST" ] || [ -z "$CONFIG_BINDING_SERVICE" ] || [ -z "$HOSTNAME" ]; then

-                echo "INFO: USING STANDARD ALONE CONFIGURATION SETUP"

-		## For Container supporting both classic and GEN2 controller - below line should be uncommented, provided service manager package is included

-		#/opt/app/manager/start-manager.sh

+if [ ! -z ${DMAAPHOST} ]; then

+    if [ -z "$(echo ${DMAAPHOST} | sed -e 's/[0-9\.]//g')" ]; then

+        log "DMaaP host (${DMAAPHOST}) (env var 'DMAAPHOST') found, adding entry to /etc/hosts"

+        echo "${DMAAPHOST}  onap-dmaap" >> /etc/hosts

+    else

+        log "DMaaP host (${DMAAPHOST}) (env var 'DMAAPHOST') found, adding entry to /etc/host.aliases"

+        echo "onap-dmaap ${DMAAPHOST}" >> /etc/host.aliases

+    fi

 else

-                echo "INFO: USING DCAEGEN2 CONTROLLER"

+	logWarn "DMaaP host (env var 'DMAAPHOST') is missing. Events will not be published to DMaaP"

 fi

 

-/opt/app/VESCollector/bin/VESrestfulCollector.sh stop

-/opt/app/VESCollector/bin/VESrestfulCollector.sh start &

+log "Scheduling application to be started, looping indefinitely to hold the docker process"

+bin/VESrestfulCollector.sh stop

+bin/VESrestfulCollector.sh start &

 

-# Add below if config polling should be enabled

-# More specific to K8 deployment in ONAP

-if [ ! -z "$CBSPOLLTIMER" ]; then

-	/opt/app/VESCollector/bin/VESConfigPoller.sh &

+# Add below if config polling should be enabled. More specific to K8 deployment in ONAP

+if [ ! -z ${CBSPOLLTIMER} ]; then

+    log "Configuration poll time (${CBSPOLLTIMER}) (env var 'CBSPOLLTIMER') found, enabling configuration polling from CBS"

+    bin/VESConfigPoller.sh &

 fi

 

 while true; do sleep 1000; done

diff --git a/src/main/scripts/logger.sh b/src/main/scripts/logger.sh
new file mode 100644
index 00000000..0c56aef0
--- /dev/null
+++ b/src/main/scripts/logger.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+###
+# ============LICENSE_START=======================================================
+# PROJECT
+# ================================================================================
+# Copyright (C) 2018 Nokia Networks Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+###
+
+log() {
+  logMessage "INFO " "$1"
+}
+
+logWarn() {
+  logMessage "WARN " "$1"
+}
+
+logError() {
+  logMessage "ERROR" "$1"
+}
+
+# Mimics log4j formatter so log files are consistent
+logMessage() {
+	echo "[$(date -u +'%Y-%m-%d %H:%M:%S,%3N')][$1][$(printf "%-9s %s\n" "PID $$")][$0] - $2"
+}
+
+# Run command, catch all the stdout and stderr and based on whether it succeeded, take the output,
+# and log them using common formatter.
+# It is done, so that the log files could be consistent and not look like swiss cheese having
+# nicely formatted lines surrounded with raw command outputs
+# All log lines that are logged by those external comments are prepended with (ext process) so they
+# can be distinguished from hand-rolled messages
+loggedCommand() {
+  output=$($1 2>&1)
+  if [ ! -z "${output}" ]; then
+    if [ $? -eq 0 ]; then
+        while read -r line; do
+            log "(ext process) $line"
+        done <<< "$output"
+    else
+        while read -r line; do
+            logError "(ext process) $line"
+        done <<< "$output"
+    fi
+  fi
+}
\ No newline at end of file
diff --git a/src/main/scripts/reconfigure.sh b/src/main/scripts/reconfigure.sh
index e8766d6f..b68ad000 100644
--- a/src/main/scripts/reconfigure.sh
+++ b/src/main/scripts/reconfigure.sh
@@ -5,6 +5,7 @@
 # PROJECT
 # ================================================================================
 # Copyright (C) 2017 AT&T Intellectual Property. All rights reserved.
+# Copyright (C) 2018 Nokia Networks Intellectual Property. All rights reserved.
 # ================================================================================
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,14 +20,13 @@
 # limitations under the License.
 # ============LICENSE_END=========================================================
 ###
+source bin/logger.sh
 
-
-if [ -z "$CONSUL_HOST" ] || [ -z "$CONFIG_BINDING_SERVICE" ] || [ -z "$HOSTNAME" ]; then
-                echo "INFO: USING STANDARD CONTROLLER"
-                /opt/app/manager/start-manager.sh
+if [ -z ${CONSUL_HOST} ] || [ -z ${CONFIG_BINDING_SERVICE} ] || [ -z ${HOSTNAME} ]; then
+    log "Using standard controller (start-manager.sh)"
+    /opt/app/manager/start-manager.sh
 else
-                echo "INFO: USING DCAEGEN2 CONTROLLER"
-                /opt/app/VESCollector/bin/VESrestfulCollector.sh stop
-                /opt/app/VESCollector/bin/VESrestfulCollector.sh start &
-fi
-#while true; do sleep 1000; done
+    log "Using DCAEGEN2 controller (VESrestfulCollector.sh)"
+    bin/VESrestfulCollector.sh stop
+    bin/VESrestfulCollector.sh start &
+fi
\ No newline at end of file
diff --git a/src/main/scripts/run-dcae-controller-ves-collector-daemon.sh b/src/main/scripts/run-dcae-controller-ves-collector-daemon.sh
deleted file mode 100644
index 544bf3f6..00000000
--- a/src/main/scripts/run-dcae-controller-ves-collector-daemon.sh
+++ /dev/null
@@ -1,39 +0,0 @@
-#!/bin/bash
-
-###
-# ============LICENSE_START=======================================================
-# PROJECT
-# ================================================================================
-# Copyright (C) 2017 AT&T Intellectual Property. All rights reserved.
-# ================================================================================
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# ============LICENSE_END=========================================================
-###
-
-
-IMAGE="dcae-controller-ves-collector"
-VER="latest"
-HOST=$(hostname)
-NAME="ves-collector-$HOST"
-HOST=$(hostname -f)
-
-HOST_VM_LOGDIR="/var/log/${HOST}-docker"
-
-CMD="/bin/bash"
-# remove the imate, interactive terminal, map exposed port
-set -x
-#docker run -d -p 8080:8080/tcp -p 8443:8443/tcp -P --name ${NAME} ${IMAGE}:${VER}
-#docker run -td --name ${NAME} ${IMAGE}:${VER} ${CMD}
-#docker run -td --name ${NAME} ${IMAGE}:${VER}
-docker run -td -p 8080:8080/tcp -p 8443:8443/tcp -P --name ${NAME} ${IMAGE}:${VER} ${CMD}
-
diff --git a/src/main/scripts/run-dcae-controller-ves-collector-interactive.sh b/src/main/scripts/run-dcae-controller-ves-collector-interactive.sh
deleted file mode 100644
index e5759b10..00000000
--- a/src/main/scripts/run-dcae-controller-ves-collector-interactive.sh
+++ /dev/null
@@ -1,39 +0,0 @@
-#!/bin/bash
-
-###
-# ============LICENSE_START=======================================================
-# PROJECT
-# ================================================================================
-# Copyright (C) 2017 AT&T Intellectual Property. All rights reserved.
-# ================================================================================
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# ============LICENSE_END=========================================================
-###
-
-
-IMAGE="dcae-controller-ves-collector"
-VER="latest"
-HOST=$(hostname)
-NAME="ves-collector-$HOST"
-HOST=$(hostname -f)
-CMD="/bin/bash"
-
-HOST_VM_LOGDIR="/var/log/${HOST}-docker-${IMAGE}"
-
-# remove the imate, interactive terminal, map exposed port
-set -x
-docker run --rm -it  -v ${HOST_VM_LOGDIR}/manager_ves-collector:/opt/app/manager/logs \
-    -v ${HOST_VM_LOGDIR}/VEScollector:/opt/app/VEScollector/logs \
-    -v /etc/dcae:/etc/dcae \
-    --name ${NAME} ${IMAGE}:${VER} \
-   ${CMD}