diff options
| author |   Nelson,Thomas(tn1381)(arthurdent3) <tn1381@att.com> | 2018-09-19 16:52:36 -0400 |
|---|---|---|
| committer |   Tschaen, Brendan <ctschaen@att.com> | 2018-10-26 19:08:42 +0000 |
| commit | 4807fdfc7adf7945c31368e3b420dc853a29f4cd (patch) | |
| tree | 5203c5f243ed1483f7ce314d3b680d803cdf9dbb /kubernetes | |
| parent | a3a690dc96c6e70ff84f7e22ba23ebd8349daa24 (diff) | |
Adding MUSIC Common charts
Includes:
Cassandra 3.11 - Current Common cassandra is lacking security.
Tomcat 8.5 - Docker Hub
Zookeeper 3.4 - Chart from Incubator/Zookeeper
- https://github.com/helm/charts/tree/master/incubator/zookeeper
Fixed Issue with Zookeeper not starting. Was unable to write to
pvc location.
add copyright header to the files where it's missing
remove all the trailing whitespaces
bump charts version to 3.0.0 and also requirement chart to 3.0.0
Adding resource limits
Update Music Chart.yaml to 3.0.0
Make recomended fixes due to resources bug.
Updated port to 76
Update cassandra values.
Add nodePortPrefixExt
Fix Tomcat sevice Name.
Update Names
Add Readiness Image
Update Service names
Fix cassandra service
Update tomcat chart
Update job chart to incluede timeout.
Update job chart to include delay for run
Remove some unneded files and
update tomcat helm readiness
Replace removed files that were braking the jenkins build
Made suggested changes
Update Replication factor from 1 to 3,3 is default, 1 was set for testing.
Move tag:version into image:
Remove commented out values and aafAdminUrl
Change-Id: I47eafae052cbe7355468655e5f8fcda8402bafd6
Issue-ID: MUSIC-99
Signed-off-by: Nelson,Thomas(tn1381)(arthurdent3) <tn1381@att.com>
Signed-off-by: Thomas Nelson Jr arthuerdent3 <nelson24@att.com>
Diffstat (limited to 'kubernetes')
41 files changed, 2025 insertions, 0 deletions
diff --git a/kubernetes/common/music/Chart.yaml b/kubernetes/common/music/Chart.yaml new file mode 100644 index 0000000000..680216389a --- /dev/null +++ b/kubernetes/common/music/Chart.yaml @@ -0,0 +1,18 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +description: MUSIC - Multi-site State Coordination Service +name: music +version: 3.0.0 diff --git a/kubernetes/common/music/charts/music-cassandra-job/.helmignore b/kubernetes/common/music/charts/music-cassandra-job/.helmignore new file mode 100644 index 0000000000..f0c1319444 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/.helmignore @@ -0,0 +1,21 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*~ +# Various IDEs +.project +.idea/ +*.tmproj diff --git a/kubernetes/common/music/charts/music-cassandra-job/Chart.yaml b/kubernetes/common/music/charts/music-cassandra-job/Chart.yaml new file mode 100644 index 0000000000..fd421764fa --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/Chart.yaml @@ -0,0 +1,19 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +description: Cassandra Job - Run CQL Scripts after Cassandra Starts. +name: music-cassandra-job +version: 3.0.0 + diff --git a/kubernetes/common/music/charts/music-cassandra-job/resources/LICENSE.txt b/kubernetes/common/music/charts/music-cassandra-job/resources/LICENSE.txt new file mode 100644 index 0000000000..7f60913d26 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/resources/LICENSE.txt @@ -0,0 +1,13 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.
\ No newline at end of file diff --git a/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin.cql b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin.cql new file mode 100644 index 0000000000..a76d774bd3 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin.cql @@ -0,0 +1,19 @@ +CREATE KEYSPACE IF NOT EXISTS admin + WITH REPLICATION = { + 'class' : '{{.Values.cql.keyspace.replicationClass}}', + 'replication_factor': {{.Values.cql.keyspace.replicationFactor}} + } + AND DURABLE_WRITES = true; + +CREATE TABLE IF NOT EXISTS admin.keyspace_master ( + uuid uuid, + keyspace_name text, + application_name text, + is_api boolean, + password text, + username text, + is_aaf boolean, + PRIMARY KEY (uuid) +); + +describe keyspaces; diff --git a/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin_pw.cql b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin_pw.cql new file mode 100644 index 0000000000..24f2ad77f7 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/admin_pw.cql @@ -0,0 +1,8 @@ +CREATE ROLE IF NOT EXISTS {{.Values.cql.adminUser.username}} +WITH PASSWORD = '{{.Values.cql.adminUser.password}}' +AND SUPERUSER = true +AND LOGIN = true; + +ALTER ROLE cassandra +WITH PASSWORD = '{{.Values.cql.adminUser.passwordReplace}}'; + diff --git a/kubernetes/common/music/charts/music-cassandra-job/resources/cql/extra/check.cql b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/extra/check.cql new file mode 100644 index 0000000000..a516be857b --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/resources/cql/extra/check.cql @@ -0,0 +1,23 @@ +CREATE KEYSPACE testks + WITH REPLICATION = { + 'class' : '{{.Values.cql.keyspace.replicationClass}}', + 'replication_factor': {{.Values.cql.keyspace.replicationFactor}} + } + AND DURABLE_WRITES = true; + +CREATE TABLE testks.keyspace_master_table ( + uuid uuid, + keyspace_name text, + application_name text, + is_api boolean, + password text, + username text, + is_aaf boolean, + PRIMARY KEY (uuid) +); + +DESCRIBE KEYSPACES; +DESCRIBE keyspace testks; +SELECT * FROM system_auth.roles; +DROP keyspace testks; + diff --git a/kubernetes/common/music/charts/music-cassandra-job/templates/configmap.yaml b/kubernetes/common/music/charts/music-cassandra-job/templates/configmap.yaml new file mode 100755 index 0000000000..011dccda25 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/templates/configmap.yaml @@ -0,0 +1,24 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "common.fullname" . }}-cql + namespace: {{ include "common.namespace" . }} +data: +{{ tpl (.Files.Glob "resources/cql/*").AsConfig . | indent 2 }} + diff --git a/kubernetes/common/music/charts/music-cassandra-job/templates/configmap_extra.yaml b/kubernetes/common/music/charts/music-cassandra-job/templates/configmap_extra.yaml new file mode 100755 index 0000000000..72733b3088 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/templates/configmap_extra.yaml @@ -0,0 +1,24 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "common.fullname" . }}-extra-cql + namespace: {{ include "common.namespace" . }} +data: +{{ tpl (.Files.Glob "resources/cql/extra/*").AsConfig . | indent 2 }} + diff --git a/kubernetes/common/music/charts/music-cassandra-job/templates/job.yaml b/kubernetes/common/music/charts/music-cassandra-job/templates/job.yaml new file mode 100644 index 0000000000..6ec282e44e --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/templates/job.yaml @@ -0,0 +1,88 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "common.fullname" . }}-config + namespace: {{ include "common.namespace" . }} + labels: + app: {{ include "common.name" . }}-job + chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + template: + metadata: + labels: + app: {{ include "common.name" . }}-job + release: {{ .Release.Name }} + spec: + restartPolicy: Never + initContainers: + - name: {{ include "common.name" . }}-readiness + image: "{{ .Values.global.readinessRepository }}/{{ .Values.global.readinessImage }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + command: + - /root/ready.py + args: + - --container-name + - music-cassandra + env: + - name: NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + containers: + - name: {{ include "common.name" . }}-update-job + image: "{{ .Values.global.repository }}/{{ .Values.job.cassandra.image }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + env: + - name: CASS_HOSTNAME + value: "{{ .Values.job.host }}" + - name: USERNAME + value: "{{ .Values.cql.adminUser.username }}" + - name: PORT + value: "{{ .Values.job.port }}" + - name: PASSWORD + value: "{{ .Values.cql.adminUser.password }}" + - name: TIMEOUT + value: "{{ .Values.job.timeout }}" + - name: DELAY + value: "{{ .Values.job.delay }}" + volumeMounts: + # Admin cql Files that setup Admin Keyspace and Change Admin user. + - name: {{ include "common.name" . }}-cql + mountPath: /cql/admin.cql + subPath: admin.cql + - name: {{ include "common.name" . }}-cql + mountPath: /cql/admin_pw.cql + subPath: admin_pw.cql + # This is where Apps or MISC will put any of their own startup cql scripts. + - name: {{ include "common.name" . }}-extra-cql + mountPath: /cql/extra + volumes: + - name: {{ include "common.name" . }}-cql + configMap: + name: {{ include "common.fullname" . }}-cql + - name: {{ include "common.name" . }}-extra-cql + configMap: + name: {{ include "common.fullname" . }}-extra-cql + restartPolicy: Never + imagePullSecrets: + - name: "{{ include "common.namespace" . }}-docker-registry-key" + diff --git a/kubernetes/common/music/charts/music-cassandra-job/values.yaml b/kubernetes/common/music/charts/music-cassandra-job/values.yaml new file mode 100644 index 0000000000..65883d9656 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra-job/values.yaml @@ -0,0 +1,70 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Default values for cassandra. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. +global: + pullPolicy: Always + repository: nexus3.onap.org:10001 + + # readiness check + readinessRepository: oomk8s + readinessImage: readiness-check:2.0.0 + + # logging agent + loggingRepository: docker.elastic.co + loggingImage: beats/filebeat:5.5.0 + + replicaCount: 3 + +job: + host: music-cassandra + port: 9042 + busybox: + image: library/busybox:latest + cassandra: + image: onap/music/cassandra_job:latest + timeout: 30 + delay: 120 +cql: + keyspace: + replicationClass: "SimpleStrategy" + replicationFactor: 3 + adminUser: + username: nelson24 + password: nelson24 + passwordReplace: A2C4E6G8I0J2L4O6Q8S0U2W4Y6 + +podManagementPolicy: OrderedReady +updateStrategy: + type: OnDelete + +ingress: + enabled: false + +tolerations: [] + +affinity: {} + +persistence: + enabled: true + +resources: + limits: + cpu: 1 + memory: 1Gi + requests: + cpu: 1 + memory: 1Gi diff --git a/kubernetes/common/music/charts/music-cassandra/.helmignore b/kubernetes/common/music/charts/music-cassandra/.helmignore new file mode 100644 index 0000000000..f0c1319444 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/.helmignore @@ -0,0 +1,21 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*~ +# Various IDEs +.project +.idea/ +*.tmproj diff --git a/kubernetes/common/music/charts/music-cassandra/Chart.yaml b/kubernetes/common/music/charts/music-cassandra/Chart.yaml new file mode 100644 index 0000000000..7aae51ac98 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/Chart.yaml @@ -0,0 +1,19 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +description: ONAP - Cassandra Database +name: music-cassandra +version: 3.0.0 + diff --git a/kubernetes/common/music/charts/music-cassandra/requirements.yaml b/kubernetes/common/music/charts/music-cassandra/requirements.yaml new file mode 100644 index 0000000000..a22007c38a --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/requirements.yaml @@ -0,0 +1,18 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +dependencies: + - name: common + version: ~3.0.0 + repository: '@local'
\ No newline at end of file diff --git a/kubernetes/common/music/charts/music-cassandra/templates/service.yaml b/kubernetes/common/music/charts/music-cassandra/templates/service.yaml new file mode 100644 index 0000000000..818c70e8f3 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/templates/service.yaml @@ -0,0 +1,54 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "common.servicename" . }} + namespace: {{ include "common.namespace" . }} + labels: + app: {{ include "common.name" . }} + chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + annotations: + service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" +spec: + type: {{ .Values.service.type }} + # Not working, open k8s bug: https://github.com/kubernetes/kubernetes/issues/58662 + publishNotReadyAddresses: true + ports: + {{if eq .Values.service.type "NodePort" -}} + - port: {{ .Values.service.internalPort }} + nodePort: {{ .Values.global.nodePortPrefix | default .Values.nodePortPrefix }}{{ .Values.service.nodePort }} + name: {{ .Values.service.portName }} + - port: {{ .Values.service.internalPort2 }} + nodePort: {{ .Values.global.nodePortPrefix | default .Values.nodePortPrefix }}{{ .Values.service.nodePort2 }} + name: {{ .Values.service.portName3 }} + - port: {{ .Values.service.internalPort3 }} + nodePort: {{ .Values.global.nodePortPrefix | default .Values.nodePortPrefix }}{{ .Values.service.nodePort3 }} + name: {{ .Values.service.portName3 }} + {{- else -}} + - port: {{ .Values.service.internalPort }} + name: {{ .Values.service.portName }} + - port: {{ .Values.service.internalPort2 }} + name: {{ .Values.service.portName2 }} + - port: {{ .Values.service.internalPort3 }} + name: {{ .Values.service.portName3 }} + {{- end}} + selector: + app: {{ include "common.name" . }} + release: {{ .Release.Name }} + clusterIP: None diff --git a/kubernetes/common/music/charts/music-cassandra/templates/statefulset.yaml b/kubernetes/common/music/charts/music-cassandra/templates/statefulset.yaml new file mode 100644 index 0000000000..103b1df1c1 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/templates/statefulset.yaml @@ -0,0 +1,135 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + + +apiVersion: apps/v1beta1 +kind: StatefulSet +metadata: + name: {{ include "common.fullname" . }} + namespace: {{ include "common.namespace" . }} + labels: + app: {{ include "common.name" . }} + chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 1 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - {{ .Chart.Name }} + topologyKey: kubernetes.io/hostname + serviceName: {{ include "common.servicename" . }} + replicas: {{ .Values.replicaCount }} + podManagementPolicy: {{ .Values.podManagementPolicy }} + updateStrategy: + type: {{ .Values.updateStrategy.type }} + template: + metadata: + labels: + app: {{ include "common.name" . }} + release: {{ .Release.Name }} + name: {{ include "common.name" . }} + spec: + containers: + - name: {{ include "common.name" . }} + image: "{{ .Values.global.repository }}/{{ .Values.image.image }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + ports: + - containerPort: {{ .Values.service.internalPort }} + - containerPort: {{ .Values.service.internalPort2 }} + - containerPort: {{ .Values.service.internalPort3 }} + {{- if eq .Values.liveness.enabled true }} + livenessProbe: + exec: + command: + - /bin/bash + - -c + - nodetool status | grep $POD_IP | awk '$1!="UN" { exit 1; }' + initialDelaySeconds: {{ .Values.liveness.initialDelaySeconds }} + periodSeconds: {{ .Values.liveness.periodSeconds }} + {{ end -}} + readinessProbe: + exec: + command: + - /bin/bash + - -c + - nodetool status | grep $POD_IP | awk '$1!="UN" { exit 1; }' + initialDelaySeconds: {{ .Values.readiness.initialDelaySeconds }} + timeoutSeconds: {{ .Values.liveness.periodSeconds }} + lifecycle: + preStop: + exec: + command: ["/bin/sh", "-c", "PID=$(pidof java) && kill $PID && while ps -p $PID > /dev/null; do sleep 1; done"] + env: + {{- $seed_size := default 1 .Values.replicaCount | int -}} + {{- $global := . }} + - name: MAX_HEAP_SIZE + value: {{ .Values.config.heap.max }} + - name: HEAP_NEWSIZE + value: {{ .Values.config.heap.min }} + - name: CASSANDRA_SEEDS + value: "{{- range $i, $e := until $seed_size }}{{ template "common.fullname" $global }}-{{ $i }}.{{ include "common.servicename" $global }}{{- if (lt ( add 1 $i ) $seed_size ) }},{{- end }}{{- end }}" + - name: JVM_OPTS + value: {{ .Values.config.jvmOpts | quote }} + - name: CASSANDRA_CLUSTER_NAME + value: {{ .Values.config.clusterName | quote }} + - name: CASSANDRA_DC + value: {{ .Values.config.dataCenter | quote }} + - name: CASSANDRA_RACK + value: {{ .Values.config.rackName | quote }} + - name: CASSANDRA_AUTO_BOOTSTRAP + value: {{ .Values.config.autoBootstrap | quote }} + - name: POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + volumeMounts: + - name: cassandra-data + mountPath: /var/lib/cassandra + resources: +{{ toYaml .Values.resources | indent 10 }} + volumes: + - name: localtime + hostPath: + path: /etc/localtime + {{- if not .Values.persistence.enabled }} + - name: cassandra-data + emptyDir: {} + {{- else }} + volumeClaimTemplates: + - metadata: + name: cassandra-data + labels: + app: {{ template "common.fullname" . }} + chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" + annotations: + volume.beta.kubernetes.io/storage-class: {{ .Values.persistence.storageClass }} + spec: + storageClassName: {{ .Values.persistence.storageClass }} + accessModes: + - {{ .Values.persistence.accessMode | quote }} + resources: + requests: + storage: {{ .Values.persistence.size | quote }} + {{- end }} diff --git a/kubernetes/common/music/charts/music-cassandra/templates/volumes.yaml b/kubernetes/common/music/charts/music-cassandra/templates/volumes.yaml new file mode 100644 index 0000000000..83e7e189b5 --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/templates/volumes.yaml @@ -0,0 +1,52 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +{{ if .Values.persistence.enabled }} +{{- $root := . -}} +{{ range $i, $e := until (atoi (quote $root.Values.replicaCount) | default 3) }} +--- +apiVersion: v1 +kind: PersistentVolume +metadata: + name: {{ $root.Release.Name }}-{{ $root.Values.service.name }}-{{ $i }} + namespace: {{ $root.Release.Namespace }} + labels: + type: {{ $root.Values.persistence.storageType }} + app: {{ $root.Values.service.name }} + chart: {{ $root.Chart.Name }}-{{ $root.Chart.Version | replace "+" "_" }} + release: {{ $root.Release.Name }} + heritage: {{ $root.Release.Service }} +spec: + capacity: + storage: {{ $root.Values.persistence.size }} + accessModes: + - {{ $root.Values.persistence.accessMode }} + hostPath: + path: {{ $root.Values.persistence.mountPath }}/{{ $root.Release.Name }}/{{ $root.Values.persistence.mountSubPath }}-{{$i}} + persistentVolumeReclaimPolicy: {{ $root.Values.persistence.volumeReclaimPolicy }} + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 1 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - "{{ $root.Chart.Name }}" + topologyKey: kubernetes.io/hostname +{{ end }} +{{ end }} diff --git a/kubernetes/common/music/charts/music-cassandra/values.yaml b/kubernetes/common/music/charts/music-cassandra/values.yaml new file mode 100644 index 0000000000..603e886c0a --- /dev/null +++ b/kubernetes/common/music/charts/music-cassandra/values.yaml @@ -0,0 +1,119 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. +global: + nodePortPrefix: 302 + + pullPolicy: Always + repository: nexus3.onap.org:10001 + + # readiness check + readinessRepository: oomk8s + readinessImage: readiness-check:2.0.0 + + # logging agent + loggingRepository: docker.elastic.co + loggingImage: beats/filebeat:5.5.0 + +replicaCount: 3 + +# Cassandra Image - This image is modified from the original on +# Docker Hub where the Security has been turned on. +# When logging into DB the default username and password are 'cassandra' +# kubectl exec -it <cassandra-n> -n <namespace> cqlsh -u cassandra -p cassandra +image: + image: onap/music/cassandra_3_11:latest + pullPolicy: Always + +# Cassandra ENV configuration +config: + heap: + max: 512M + min: 100M + jvmOpts: -Dcassandra.consistent.rangemovement=false + clusterName: music-cluster + dataCenter: onap-1 + rackName: Rack1 + autoBootstrap: true + ports: + cql: 9042 + thrift: 9160 + # If a JVM Agent is in place + # agent: 61621 + +service: + expose: true + type: ClusterIP + name: music-cassandra + internalPort: 9042 + portName: cql + internalPort2: 9160 + portName2: thrift + internalPort3: 61621 + portName3: agent + +# probe configuration parameters +liveness: + initialDelaySeconds: 120 + periodSeconds: 10 + # necessary to disable liveness probe when setting breakpoints + # in debugger so K8s doesn't restart unresponsive container + enabled: true + +readiness: + initialDelaySeconds: 10 + periodSeconds: 10 + +podManagementPolicy: OrderedReady +updateStrategy: + type: OnDelete + +ingress: + enabled: false + +tolerations: [] + +persistence: + enabled: true + + ## A manually managed Persistent Volume and Claim + ## Requires persistence.enabled: true + ## If defined, PVC must be created manually before volume will be bound + # existingClaim: + volumeReclaimPolicy: Retain + + ## database data Persistent Volume Storage Class + ## If defined, storageClassName: <storageClass> + ## If set to "-", storageClassName: "", which disables dynamic provisioning + ## If undefined (the default) or set to null, no storageClassName spec is + ## set, choosing the default provisioner. (gp2 on AWS, standard on + ## GKE, AWS & OpenStack) + ## + ## storageClass: "-" + accessMode: ReadWriteOnce + size: 2Gi + mountPath: /dockerdata-nfs/ + mountSubPath: common/cassandra/data + storageType: local + storageClass: "" + +resources: + limits: + cpu: 4 + memory: 8Gi + requests: + cpu: 2 + memory: 4Gi diff --git a/kubernetes/common/music/charts/music-tomcat/Chart.yaml b/kubernetes/common/music/charts/music-tomcat/Chart.yaml new file mode 100755 index 0000000000..d0fdc6088c --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/Chart.yaml @@ -0,0 +1,18 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +description: ONAP - MUSIC Tomcat Container +name: music-tomcat +version: 3.0.0 diff --git a/kubernetes/common/music/charts/music-tomcat/requirements.yaml b/kubernetes/common/music/charts/music-tomcat/requirements.yaml new file mode 100755 index 0000000000..28352e407c --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/requirements.yaml @@ -0,0 +1,19 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +dependencies: + - name: common + version: ~3.0.0 + repository: '@local'
\ No newline at end of file diff --git a/kubernetes/common/music/charts/music-tomcat/resources/config/music.properties b/kubernetes/common/music/charts/music-tomcat/resources/config/music.properties new file mode 100755 index 0000000000..b977ca58ee --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/resources/config/music.properties @@ -0,0 +1,32 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +my.public.ip=localhost +all.public.ips=localhost +my.id=0 +all.ids=0 +### Host Info ### +zookeeper.host={{.Values.properties.zookeeperHost}} +cassandra.host={{.Values.properties.cassandraHost}} +### User Info ### +cassandra.user={{.Values.properties.cassandraUser}} +cassandra.password={{.Values.properties.cassandraPassword}} +### AAF Endpoint ### +aaf.endpoint.url={{.Values.properties.aafEndpointUrl}} +### Admin API ### +# AAF UAT +aaf.admin.url={{.Values.properties.aafAdminUrl}} +# AAF PROD +admin.aaf.role={{.Values.properties.adminAafRole}} +music.namespace={{.Values.properties.musicNamespace}} diff --git a/kubernetes/common/music/charts/music-tomcat/templates/configmap.yaml b/kubernetes/common/music/charts/music-tomcat/templates/configmap.yaml new file mode 100755 index 0000000000..15859345e8 --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/templates/configmap.yaml @@ -0,0 +1,23 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "common.fullname" . }}-configmap + namespace: {{ include "common.namespace" . }} +data: +{{ tpl (.Files.Glob "resources/config/*").AsConfig . | indent 2 }} diff --git a/kubernetes/common/music/charts/music-tomcat/templates/deployment.yaml b/kubernetes/common/music/charts/music-tomcat/templates/deployment.yaml new file mode 100755 index 0000000000..1fec55caca --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/templates/deployment.yaml @@ -0,0 +1,115 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: {{ include "common.fullname" . }} + namespace: {{ include "common.namespace" . }} + labels: + app: {{ include "common.name" . }} + chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + replicas: {{ .Values.replicaCount }} + template: + metadata: + labels: + app: {{ include "common.name" . }} + release: {{ .Release.Name }} + spec: + initContainers: + - name: {{ include "common.name" . }}-zookeeper-readiness + image: "{{ .Values.global.readinessRepository }}/{{ .Values.global.readinessImage }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + command: + - /root/ready.py + args: + - --container-name + - zookeeper + env: + - name: NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: {{ include "common.name" . }}-cassandra-readiness + image: "{{ .Values.global.readinessRepository }}/{{ .Values.global.readinessImage }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + command: + - /root/job_complete.py + args: + - -j + - "{{ .Release.Name }}-music-cassandra-job-config" + env: + - name: NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace +# War Container + - name: "{{ .Chart.Name }}-war" + image: "{{ .Values.repository }}/{{ .Values.warImage }}" + command: ["cp","/app/MUSIC.war","/webapps"] + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + ports: + volumeMounts: + - mountPath: /webapps + name: shared-data + containers: + # Tomcat Container + - name: "{{ include "common.name" . }}" + image: "{{ .Values.repository }}/{{ .Values.image }}" + imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} + ports: + - containerPort: {{ .Values.service.internalPort }} + # disable liveness probe when breakpoints set in debugger + # so K8s doesn't restart unresponsive container + {{- if eq .Values.liveness.enabled true }} + livenessProbe: + tcpSocket: + port: {{ .Values.service.internalPort }} + initialDelaySeconds: {{ .Values.liveness.initialDelaySeconds }} + periodSeconds: {{ .Values.liveness.periodSeconds }} + {{ end -}} + readinessProbe: + tcpSocket: + port: {{ .Values.service.internalPort }} + initialDelaySeconds: {{ .Values.readiness.initialDelaySeconds }} + periodSeconds: {{ .Values.readiness.periodSeconds }} + volumeMounts: + - mountPath: /etc/localtime + name: localtime + readOnly: true + - mountPath: /usr/local/tomcat/webapps + name: shared-data + - name: properties-music + mountPath: /opt/app/music/etc/music.properties + subPath: music.properties + resources: +{{ toYaml .Values.resources | indent 10 }} + volumes: + - name: shared-data + emptyDir: {} + - name: localtime + hostPath: + path: /etc/localtime + - name: properties-music + configMap: + name: {{ include "common.fullname" . }}-configmap + imagePullSecrets: + - name: "{{ include "common.namespace" . }}-docker-registry-key" diff --git a/kubernetes/common/music/charts/music-tomcat/templates/service.yaml b/kubernetes/common/music/charts/music-tomcat/templates/service.yaml new file mode 100755 index 0000000000..f6a086cc99 --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/templates/service.yaml @@ -0,0 +1,42 @@ +{{/* +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*/}} + +apiVersion: v1 +kind: Service +metadata: + name: {{ include "common.servicename" . }} + namespace: {{ include "common.namespace" . }} + labels: + app: {{ include "common.name" . }} + chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + annotations: +spec: + type: {{ .Values.service.type }} + ports: + {{if eq .Values.service.type "NodePort" -}} + - port: {{ .Values.service.externalPort }} + nodePort: {{ .Values.global.nodePortPrefixExt | default .Values.nodePortPrefixExt }}{{ .Values.service.nodePort }} + name: {{ .Values.service.portName }} + {{- else -}} + - port: {{ .Values.service.externalPort }} + targetPort: {{ .Values.service.internalPort }} + name: {{ .Values.service.portName }} + {{- end}} + selector: + app: {{ include "common.name" . }} + release: {{ .Release.Name }} diff --git a/kubernetes/common/music/charts/music-tomcat/values.yaml b/kubernetes/common/music/charts/music-tomcat/values.yaml new file mode 100755 index 0000000000..67a47ae7a6 --- /dev/null +++ b/kubernetes/common/music/charts/music-tomcat/values.yaml @@ -0,0 +1,103 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +################################################################# +# Global configuration defaults. +################################################################# +global: + nodePortPrefix: 302 + nodePortPrefixExt: 304 + repository: nexus3.onap.org:10001 + + # readiness check + readinessRepository: oomk8s + readinessImage: readiness-check:2.0.0 + + # logging agent + loggingRepository: docker.elastic.co + loggingImage: beats/filebeat:5.5.0 + +################################################################# +# Application configuration defaults. +################################################################# +# application image +repository: nexus3.onap.org:10001 +image: library/tomcat:8.5 +pullPolicy: Always +warImage: onap/music/music:latest + +# flag to enable debugging - application support required +debugEnabled: false + +# application configuration +config: + usernameCassandra: cassandra1 + passwordCassandra: cassandra1 + +# default number of instances +replicaCount: 3 + +job: + host: cassandra + port: 9042 + busybox: + image: library/busybox:latest + +nodeSelector: {} + +affinity: {} + +# probe configuration parameters +liveness: + initialDelaySeconds: 10 + periodSeconds: 10 + # necessary to disable liveness probe when setting breakpoints + # in debugger so K8s doesn't restart unresponsive container + enabled: true + +readiness: + initialDelaySeconds: 10 + periodSeconds: 10 + +service: + type: NodePort + name: music-tomcat + externalPort: 8080 + internalPort: 8080 + nodePort: 76 + portName: tomcat +ingress: + enabled: false + +#resources: {} +resources: + limits: + cpu: 2 + memory: 2Gi + requests: + cpu: 2 + memory: 1Gi + + +properties: + zookeeperHost: zookeeper + cassandraHost: music-cassandra + cassandraUser: nelson24 + cassandraPassword: nelson24 + + # Admin API + # ONAP AAF + aafAdminUrl: + diff --git a/kubernetes/common/music/charts/zookeeper/.helmignore b/kubernetes/common/music/charts/zookeeper/.helmignore new file mode 100644 index 0000000000..f0c1319444 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/.helmignore @@ -0,0 +1,21 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*~ +# Various IDEs +.project +.idea/ +*.tmproj diff --git a/kubernetes/common/music/charts/zookeeper/Chart.yaml b/kubernetes/common/music/charts/zookeeper/Chart.yaml new file mode 100644 index 0000000000..01e81736f6 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/Chart.yaml @@ -0,0 +1,15 @@ +name: zookeeper +home: https://zookeeper.apache.org/ +version: 1.0.2 +appVersion: 3.4.10 +description: Centralized service for maintaining configuration information, naming, + providing distributed synchronization, and providing group services. +icon: https://zookeeper.apache.org/images/zookeeper_small.gif +sources: +- https://github.com/apache/zookeeper +- https://github.com/kubernetes/contrib/tree/master/statefulsets/zookeeper +maintainers: +- name: lachie83 + email: lachlan.evenson@microsoft.com +- name: kow3ns + email: owensk@google.com diff --git a/kubernetes/common/music/charts/zookeeper/OWNERS b/kubernetes/common/music/charts/zookeeper/OWNERS new file mode 100644 index 0000000000..dd9facde2a --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/OWNERS @@ -0,0 +1,6 @@ +approvers: +- lachie83 +- kow3ns +reviewers: +- lachie83 +- kow3ns diff --git a/kubernetes/common/music/charts/zookeeper/README.md b/kubernetes/common/music/charts/zookeeper/README.md new file mode 100644 index 0000000000..22bbac49dc --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/README.md @@ -0,0 +1,140 @@ +# incubator/zookeeper + +This helm chart provides an implementation of the ZooKeeper [StatefulSet](http://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/) found in Kubernetes Contrib [Zookeeper StatefulSet](https://github.com/kubernetes/contrib/tree/master/statefulsets/zookeeper). + +## Prerequisites +* Kubernetes 1.6+ +* PersistentVolume support on the underlying infrastructure +* A dynamic provisioner for the PersistentVolumes +* A familiarity with [Apache ZooKeeper 3.4.x](https://zookeeper.apache.org/doc/current/) + +## Chart Components +This chart will do the following: + +* Create a fixed size ZooKeeper ensemble using a [StatefulSet](http://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/). +* Create a [PodDisruptionBudget](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-disruption-budget/) so kubectl drain will respect the Quorum size of the ensemble. +* Create a [Headless Service](https://kubernetes.io/docs/concepts/services-networking/service/) to control the domain of the ZooKeeper ensemble. +* Create a Service configured to connect to the available ZooKeeper instance on the configured client port. +* Optionally apply a [Pod Anti-Affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature) to spread the ZooKeeper ensemble across nodes. +* Optionally start JMX Exporter and Zookeeper Exporter containers inside Zookeeper pods. +* Optionally create a job which creates Zookeeper chroots (e.g. `/kafka1`). + +## Installing the Chart +You can install the chart with the release name `zookeeper` as below. + +```console +$ helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator +$ helm install --name zookeeper incubator/zookeeper +``` + +If you do not specify a name, helm will select a name for you. + +### Installed Components +You can use `kubectl get` to view all of the installed components. + +```console{%raw} +$ kubectl get all -l app=zookeeper +NAME: zookeeper +LAST DEPLOYED: Wed Apr 11 17:09:48 2018 +NAMESPACE: default +STATUS: DEPLOYED + +RESOURCES: +==> v1beta1/PodDisruptionBudget +NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE +zookeeper N/A 1 1 2m + +==> v1/Service +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +zookeeper-headless ClusterIP None <none> 2181/TCP,3888/TCP,2888/TCP 2m +zookeeper ClusterIP 10.98.179.165 <none> 2181/TCP 2m + +==> v1beta1/StatefulSet +NAME DESIRED CURRENT AGE +zookeeper 3 3 2m +``` + +1. `statefulsets/zookeeper` is the StatefulSet created by the chart. +1. `po/zookeeper-<0|1|2>` are the Pods created by the StatefulSet. Each Pod has a single container running a ZooKeeper server. +1. `svc/zookeeper-headless` is the Headless Service used to control the network domain of the ZooKeeper ensemble. +1. `svc/zookeeper` is a Service that can be used by clients to connect to an available ZooKeeper server. + +## Configuration +You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. + +Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, + +```console +$ helm install --name my-release -f values.yaml incubator/zookeeper +``` + +## Default Values + +- You can find all user-configurable settings, their defaults and commentary about them in [values.yaml](values.yaml). + +## Deep Dive + +## Image Details +The image used for this chart is based on Ubuntu 16.04 LTS. This image is larger than Alpine or BusyBox, but it provides glibc, rather than ulibc or mucl, and a JVM release that is built against it. You can easily convert this chart to run against a smaller image with a JVM that is built against that image's libc. However, as far as we know, no Hadoop vendor supports, or has verified, ZooKeeper running on such a JVM. + +## JVM Details +The Java Virtual Machine used for this chart is the OpenJDK JVM 8u111 JRE (headless). + +## ZooKeeper Details +The ZooKeeper version is the latest stable version (3.4.10). The distribution is installed into /opt/zookeeper-3.4.10. This directory is symbolically linked to /opt/zookeeper. Symlinks are created to simulate a rpm installation into /usr. + +## Failover +You can test failover by killing the leader. Insert a key: +```console +$ kubectl exec zookeeper-0 -- /opt/zookeeper/bin/zkCli.sh create /foo bar; +$ kubectl exec zookeeper-2 -- /opt/zookeeper/bin/zkCli.sh get /foo; +``` + +Watch existing members: +```console +$ kubectl run --attach bbox --image=busybox --restart=Never -- sh -c 'while true; do for i in 0 1 2; do echo zk-${i} $(echo stats | nc <pod-name>-${i}.<headless-service-name>:2181 | grep Mode); sleep 1; done; done'; + +zk-2 Mode: follower +zk-0 Mode: follower +zk-1 Mode: leader +zk-2 Mode: follower +``` + +Delete Pods and wait for the StatefulSet controller to bring them back up: +```console +$ kubectl delete po -l app=zookeeper +$ kubectl get po --watch-only +NAME READY STATUS RESTARTS AGE +zookeeper-0 0/1 Running 0 35s +zookeeper-0 1/1 Running 0 50s +zookeeper-1 0/1 Pending 0 0s +zookeeper-1 0/1 Pending 0 0s +zookeeper-1 0/1 ContainerCreating 0 0s +zookeeper-1 0/1 Running 0 19s +zookeeper-1 1/1 Running 0 40s +zookeeper-2 0/1 Pending 0 0s +zookeeper-2 0/1 Pending 0 0s +zookeeper-2 0/1 ContainerCreating 0 0s +zookeeper-2 0/1 Running 0 19s +zookeeper-2 1/1 Running 0 41s +``` + +Check the previously inserted key: +```console +$ kubectl exec zookeeper-1 -- /opt/zookeeper/bin/zkCli.sh get /foo +ionid = 0x354887858e80035, negotiated timeout = 30000 + +WATCHER:: + +WatchedEvent state:SyncConnected type:None path:null +bar +``` + +## Scaling +ZooKeeper can not be safely scaled in versions prior to 3.5.x. This chart currently uses 3.4.x. There are manual procedures for scaling a 3.4.x ensemble, but as noted in the [ZooKeeper 3.5.2 documentation](https://zookeeper.apache.org/doc/r3.5.2-alpha/zookeeperReconfig.html) these procedures require a rolling restart, are known to be error prone, and often result in a data loss. + +While ZooKeeper 3.5.x does allow for dynamic ensemble reconfiguration (including scaling membership), the current status of the release is still alpha, and 3.5.x is therefore not recommended for production use. + +## Limitations +* StatefulSet and PodDisruptionBudget are beta resources. +* Only supports storage options that have backends for persistent volume claims. diff --git a/kubernetes/common/music/charts/zookeeper/templates/NOTES.txt b/kubernetes/common/music/charts/zookeeper/templates/NOTES.txt new file mode 100644 index 0000000000..4f7a27bd99 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/NOTES.txt @@ -0,0 +1,7 @@ +Thank you for installing ZooKeeper on your Kubernetes cluster. More information +about ZooKeeper can be found at https://zookeeper.apache.org/doc/current/ + +Your connection string should look like: + {{ template "common.fullname" . }}-0.{{ template "common.fullname" . }}-headless:{{ .Values.service.ports.client.port }},{{ template "common.fullname" . }}-1.{{ template "common.fullname" . }}-headless:{{ .Values.service.ports.client.port }},... + +You can also use the client service {{ template "common.fullname" . }}:{{ .Values.service.ports.client.port }} to connect to an available ZooKeeper server. diff --git a/kubernetes/common/music/charts/zookeeper/templates/_helpers.tpl b/kubernetes/common/music/charts/zookeeper/templates/_helpers.tpl new file mode 100644 index 0000000000..1ef5353fa1 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/_helpers.tpl @@ -0,0 +1,32 @@ +{{/* vim: set filetype=mustache: */}} +{{/* +Expand the name of the chart. +*/}} +{{- define "common.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "common.fullname" -}} +{{- if .Values.fullnameOverride -}} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- $name := default .Chart.Name .Values.nameOverride -}} +{{- if contains $name .Release.Name -}} +{{- .Release.Name | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "zookeeper.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} +{{- end -}} diff --git a/kubernetes/common/music/charts/zookeeper/templates/config-jmx-exporter.yaml b/kubernetes/common/music/charts/zookeeper/templates/config-jmx-exporter.yaml new file mode 100644 index 0000000000..aeb9a2c031 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/config-jmx-exporter.yaml @@ -0,0 +1,19 @@ +{{- if .Values.exporters.jmx.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ .Release.Name }}-jmx-exporter + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +data: + config.yml: |- + hostPort: 127.0.0.1:{{ .Values.env.JMXPORT }} + lowercaseOutputName: {{ .Values.exporters.jmx.config.lowercaseOutputName }} + rules: +{{ .Values.exporters.jmx.config.rules | toYaml | indent 6 }} + ssl: false + startDelaySeconds: {{ .Values.exporters.jmx.config.startDelaySeconds }} +{{- end }} diff --git a/kubernetes/common/music/charts/zookeeper/templates/job-chroots.yaml b/kubernetes/common/music/charts/zookeeper/templates/job-chroots.yaml new file mode 100644 index 0000000000..38592ddeac --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/job-chroots.yaml @@ -0,0 +1,62 @@ +{{- if .Values.jobs.chroots.enabled }} +{{- $root := . }} +{{- $job := .Values.jobs.chroots }} +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ template "common.fullname" . }}-chroots + annotations: + "helm.sh/hook": post-install,post-upgrade + "helm.sh/hook-weight": "-5" + "helm.sh/hook-delete-policy": hook-succeeded + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + component: jobs + job: chroots +spec: + activeDeadlineSeconds: {{ $job.activeDeadlineSeconds }} + backoffLimit: {{ $job.backoffLimit }} + completions: {{ $job.completions }} + parallelism: {{ $job.parallelism }} + template: + metadata: + labels: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} + component: jobs + job: chroots + spec: + restartPolicy: {{ $job.restartPolicy }} + containers: + - name: main + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: + - /bin/bash + - -o + - pipefail + - -euc + {{- $port := .Values.service.ports.client.port }} + - > + sleep 15; + export SERVER={{ template "common.fullname" $root }}:{{ $port }}; + {{- range $job.config.create }} + echo '==> {{ . }}'; + echo '====> Create chroot if does not exist.'; + zkCli.sh -server {{ template "common.fullname" $root }}:{{ $port }} get {{ . }} 2>&1 >/dev/null | grep 'cZxid' + || zkCli.sh -server {{ template "common.fullname" $root }}:{{ $port }} create {{ . }} ""; + echo '====> Confirm chroot exists.'; + zkCli.sh -server {{ template "common.fullname" $root }}:{{ $port }} get {{ . }} 2>&1 >/dev/null | grep 'cZxid'; + echo '====> Chroot exists.'; + {{- end }} + env: + {{- range $key, $value := $job.env }} + - name: {{ $key | upper | replace "." "_" }} + value: {{ $value | quote }} + {{- end }} + resources: +{{ toYaml $job.resources | indent 12 }} +{{- end -}} diff --git a/kubernetes/common/music/charts/zookeeper/templates/poddisruptionbudget.yaml b/kubernetes/common/music/charts/zookeeper/templates/poddisruptionbudget.yaml new file mode 100644 index 0000000000..7a9f3fe105 --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/poddisruptionbudget.yaml @@ -0,0 +1,17 @@ +apiVersion: policy/v1beta1 +kind: PodDisruptionBudget +metadata: + name: {{ template "common.fullname" . }} + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + component: server +spec: + selector: + matchLabels: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} + component: server +{{ toYaml .Values.podDisruptionBudget | indent 2 }} diff --git a/kubernetes/common/music/charts/zookeeper/templates/service-headless.yaml b/kubernetes/common/music/charts/zookeeper/templates/service-headless.yaml new file mode 100644 index 0000000000..70ebf6013e --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/service-headless.yaml @@ -0,0 +1,21 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "common.fullname" . }}-headless + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + clusterIP: None + ports: +{{- range $key, $port := .Values.ports }} + - name: {{ $key }} + port: {{ $port.containerPort }} + targetPort: {{ $port.name }} + protocol: {{ $port.protocol }} +{{- end }} + selector: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} diff --git a/kubernetes/common/music/charts/zookeeper/templates/service.yaml b/kubernetes/common/music/charts/zookeeper/templates/service.yaml new file mode 100644 index 0000000000..6ac3066ecf --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/service.yaml @@ -0,0 +1,23 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ .Values.service.name }} + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + annotations: +{{- with .Values.service.annotations }} +{{ toYaml . | indent 4 }} +{{- end }} +spec: + type: {{ .Values.service.type }} + ports: + {{- range $key, $value := .Values.service.ports }} + - name: {{ $key }} +{{ toYaml $value | indent 6 }} + {{- end }} + selector: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} diff --git a/kubernetes/common/music/charts/zookeeper/templates/statefulset.yaml b/kubernetes/common/music/charts/zookeeper/templates/statefulset.yaml new file mode 100644 index 0000000000..1efd46657c --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/statefulset.yaml @@ -0,0 +1,188 @@ +apiVersion: apps/v1beta1 +kind: StatefulSet +metadata: + name: {{ template "common.fullname" . }} + labels: + app: {{ template "common.name" . }} + chart: {{ .Chart.Name }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + component: server +spec: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 1 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - "{{ .Chart.Name }}" + serviceName: {{ template "common.fullname" . }}-headless + replicas: {{ .Values.replicaCount }} + terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }} + selector: + matchLabels: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} + component: server + updateStrategy: +{{ toYaml .Values.updateStrategy | indent 4 }} + template: + metadata: + labels: + app: {{ template "common.name" . }} + release: {{ .Release.Name }} + component: server + {{- if .Values.podLabels }} + ## Custom pod labels + {{- range $key, $value := .Values.podLabels }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + annotations: + {{- if .Values.podAnnotations }} + ## Custom pod annotations + {{- range $key, $value := .Values.podAnnotations }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + spec: +{{- if .Values.schedulerName }} + schedulerName: "{{ .Values.schedulerName }}" +{{- end }} + securityContext: +{{ toYaml .Values.securityContext | indent 8 }} + containers: + + - name: zookeeper + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: + - /bin/bash + - -xec + - zkGenConfig.sh && exec zkServer.sh start-foreground + ports: +{{- range $key, $port := .Values.ports }} + - name: {{ $key }} +{{ toYaml $port | indent 14 }} +{{- end }} + livenessProbe: +{{ toYaml .Values.livenessProbe | indent 12 }} + readinessProbe: +{{ toYaml .Values.readinessProbe | indent 12 }} + env: + - name: ZK_REPLICAS + value: {{ .Values.replicaCount | quote }} + {{- range $key, $value := .Values.env }} + - name: {{ $key | upper | replace "." "_" }} + value: {{ $value | quote }} + {{- end }} + resources: +{{ include "common.resources" . }} + volumeMounts: + - name: zookeeper-data + mountPath: /var/lib/zookeeper + +{{- if .Values.exporters.jmx.enabled }} + - name: jmx-exporter + image: "{{ .Values.exporters.jmx.image.repository }}:{{ .Values.exporters.jmx.image.tag }}" + imagePullPolicy: {{ .Values.exporters.jmx.image.pullPolicy }} + ports: + {{- range $key, $port := .Values.exporters.jmx.ports }} + - name: {{ $key }} +{{ toYaml $port | indent 14 }} + {{- end }} + livenessProbe: +{{ toYaml .Values.exporters.jmx.livenessProbe | indent 12 }} + readinessProbe: +{{ toYaml .Values.exporters.jmx.readinessProbe | indent 12 }} + env: + - name: SERVICE_PORT + value: {{ .Values.exporters.jmx.ports.jmxxp.containerPort | quote }} + {{- with .Values.exporters.jmx.env }} + {{- range $key, $value := . }} + - name: {{ $key | upper | replace "." "_" }} + value: {{ $value | quote }} + {{- end }} + {{- end }} + resources: +{{ toYaml .Values.exporters.jmx.resources | indent 12 }} + volumeMounts: + - name: config-jmx-exporter + mountPath: /opt/jmx_exporter/config.yml + subPath: config.yml +{{- end }} + +{{- if .Values.exporters.zookeeper.enabled }} + - name: zookeeper-exporter + image: "{{ .Values.exporters.zookeeper.image.repository }}:{{ .Values.exporters.zookeeper.image.tag }}" + imagePullPolicy: {{ .Values.exporters.zookeeper.image.pullPolicy }} + args: + - -bind-addr=:{{ .Values.exporters.zookeeper.ports.zookeeperxp.containerPort }} + - -metrics-path={{ .Values.exporters.zookeeper.path }} + - -zookeeper=localhost:{{ .Values.ports.client.containerPort }} + - -log-level={{ .Values.exporters.zookeeper.config.logLevel }} + - -reset-on-scrape={{ .Values.exporters.zookeeper.config.resetOnScrape }} + ports: + {{- range $key, $port := .Values.exporters.zookeeper.ports }} + - name: {{ $key }} +{{ toYaml $port | indent 14 }} + {{- end }} + livenessProbe: +{{ toYaml .Values.exporters.zookeeper.livenessProbe | indent 12 }} + readinessProbe: +{{ toYaml .Values.exporters.zookeeper.readinessProbe | indent 12 }} + env: + {{- range $key, $value := .Values.exporters.zookeeper.env }} + - name: {{ $key | upper | replace "." "_" }} + value: {{ $value | quote }} + {{- end }} + resources: +{{ toYaml .Values.exporters.zookeeper.resources | indent 12 }} +{{- end }} + + {{- with .Values.nodeSelector }} + nodeSelector: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: +{{ toYaml . | indent 8 }} + {{- end }} + {{- if (or .Values.exporters.jmx.enabled (not .Values.persistence.enabled)) }} + volumes: + {{- if .Values.exporters.jmx.enabled }} + - name: config-jmx-exporter + configMap: + name: {{ .Release.Name }}-jmx-exporter + {{- end }} + {{- if not .Values.persistence.enabled }} + - name: zookeeper-data + emptyDir: {} + {{- end }} + {{- end }} + {{- if .Values.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: zookeeper-data + labels: + app: {{ .Chart.Name }} + chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" + annotations: + volume.beta.kubernetes.io/storage-class: {{ .Values.persistence.storageClass }} + spec: + storageClassName: {{ .Values.persistence.storageClass }} + accessModes: + - {{ .Values.persistence.accessMode | quote }} + resources: + requests: + storage: {{ .Values.persistence.size | quote }} + {{- end }} diff --git a/kubernetes/common/music/charts/zookeeper/templates/volumes.yaml b/kubernetes/common/music/charts/zookeeper/templates/volumes.yaml new file mode 100644 index 0000000000..b0c05fdbfd --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/templates/volumes.yaml @@ -0,0 +1,25 @@ +{{ if .Values.persistence.enabled }} +{{- $root := . -}} +{{ range $i, $e := until (atoi (quote $root.Values.replicaCount) | default 3) }} +--- +apiVersion: v1 +kind: PersistentVolume +metadata: + name: {{ $root.Release.Name }}-{{ $root.Values.service.name }}-{{ $i }} + namespace: {{ $root.Release.Namespace }} + labels: + type: {{ $root.Values.persistence.storageType }} + app: {{ $root.Values.service.name }} + chart: {{ $root.Chart.Name }}-{{ $root.Chart.Version | replace "+" "_" }} + release: {{ $root.Release.Name }} + heritage: {{ $root.Release.Service }} +spec: + capacity: + storage: {{ $root.Values.persistence.size }} + accessModes: + - {{ $root.Values.persistence.accessMode }} + hostPath: + path: {{ $root.Values.persistence.mountPath }}/{{ $root.Release.Name }}/{{ $root.Values.persistence.mountSubPath }}-{{$i}} + persistentVolumeReclaimPolicy: {{ $root.Values.persistence.volumeReclaimPolicy }} +{{ end }} +{{ end }} diff --git a/kubernetes/common/music/charts/zookeeper/values.yaml b/kubernetes/common/music/charts/zookeeper/values.yaml new file mode 100644 index 0000000000..ea02e6151e --- /dev/null +++ b/kubernetes/common/music/charts/zookeeper/values.yaml @@ -0,0 +1,284 @@ +## As weighted quorums are not supported, it is imperative that an odd number of replicas +## be chosen. Moreover, the number of replicas should be either 1, 3, 5, or 7. +## +## ref: https://github.com/kubernetes/contrib/tree/master/statefulsets/zookeeper#stateful-set +replicaCount: 3 # Desired quantity of ZooKeeper pods. This should always be (1,3,5, or 7) + +podDisruptionBudget: + maxUnavailable: 1 # Limits how many Zokeeper pods may be unavailable due to voluntary disruptions. + +terminationGracePeriodSeconds: 1800 # Duration in seconds a Zokeeper pod needs to terminate gracefully. + +## OnDelete requires you to manually delete each pod when making updates. +## This approach is at the moment safer than RollingUpdate because replication +## may be incomplete when replication source pod is killed. +## +## ref: http://blog.kubernetes.io/2017/09/kubernetes-statefulsets-daemonsets.html +updateStrategy: + type: OnDelete # Pods will only be created when you manually delete old pods. + +## refs: +## - https://github.com/kubernetes/contrib/tree/master/statefulsets/zookeeper +## - https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/Makefile#L1 +image: + #repository: nexus3.onap.org:10001/library/zookeeper + #tag: 3.3 + repository: gcr.io/google_samples/k8szk # Container image repository for zookeeper container. + tag: v3 # Container image tag for zookeeper container. + pullPolicy: IfNotPresent # Image pull criteria for zookeeper container. + +service: + name: zookeeper + type: ClusterIP # Exposes zookeeper on a cluster-internal IP. + annotations: {} # Arbitrary non-identifying metadata for zookeeper service. + ## AWS example for use with LoadBalancer service type. + # external-dns.alpha.kubernetes.io/hostname: zookeeper.cluster.local + # service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true" + # service.beta.kubernetes.io/aws-load-balancer-internal: "true" + ports: + client: + port: 2181 # Service port number for client port. + targetPort: client # Service target port for client port. + protocol: TCP # Service port protocol for client port. + + +ports: + client: + containerPort: 2181 # Port number for zookeeper container client port. + protocol: TCP # Protocol for zookeeper container client port. + election: + containerPort: 3888 # Port number for zookeeper container election port. + protocol: TCP # Protocol for zookeeper container election port. + server: + containerPort: 2888 # Port number for zookeeper container server port. + protocol: TCP # Protocol for zookeeper container server port. + +# Resource Limit flavor -By Default using small +flavor: large +# Segregation for Different environment (Small and Large) +resources: + small: + limits: + cpu: 1 + memory: 1Gi + requests: + cpu: 500m + memory: 500Mi + large: + limits: + cpu: 3 + memory: 2Gi + requests: + cpu: 2 + memory: 1Gi + unlimited: {} + +nodeSelector: {} # Node label-values required to run zookeeper pods. + +tolerations: [] # Node taint overrides for zookeeper pods. + +affinity: {} # Criteria by which pod label-values influence scheduling for zookeeper pods. +affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - topologyKey: "kubernetes.io/hostname" + labelSelector: + matchLabels: + release: zookeeper + +podAnnotations: {} # Arbitrary non-identifying metadata for zookeeper pods. + +podLabels: {} # Key/value pairs that are attached to zookeeper pods. + +livenessProbe: + exec: + command: + - zkOk.sh + initialDelaySeconds: 20 + +readinessProbe: + exec: + command: + - zkOk.sh + initialDelaySeconds: 20 + +securityContext: + fsGroup: 1000 + #runAsUser: 1000 + +persistence: + enabled: true + ## zookeeper data Persistent Volume Storage Class + ## If defined, storageClassName: <storageClass> + ## If set to "-", storageClassName: "", which disables dynamic provisioning + ## If undefined (the default) or set to null, no storageClassName spec is + ## set, choosing the default provisioner. (gp2 on AWS, standard on + ## GKE, AWS & OpenStack) + ## + volumeReclaimPolicy: Retain + accessMode: ReadWriteOnce + mountPath: /dockerdata-nfs + mountSubPath: music/zookeeper + storageType: local + storageClass: "" + size: 4Gi + +## Exporters query apps for metrics and make those metrics available for +## Prometheus to scrape. +exporters: + + jmx: + enabled: false + image: + repository: sscaling/jmx-prometheus-exporter + tag: 0.3.0 + pullPolicy: IfNotPresent + config: + lowercaseOutputName: false + rules: + - pattern: "org.apache.ZooKeeperService<name0=ReplicatedServer_id(\\d+)><>(\\w+)" + name: "zookeeper_$2" + - pattern: "org.apache.ZooKeeperService<name0=ReplicatedServer_id(\\d+), name1=replica.(\\d+)><>(\\w+)" + name: "zookeeper_$3" + labels: + replicaId: "$2" + - pattern: "org.apache.ZooKeeperService<name0=ReplicatedServer_id(\\d+), name1=replica.(\\d+), name2=(\\w+)><>(\\w+)" + name: "zookeeper_$4" + labels: + replicaId: "$2" + memberType: "$3" + - pattern: "org.apache.ZooKeeperService<name0=ReplicatedServer_id(\\d+), name1=replica.(\\d+), name2=(\\w+), name3=(\\w+)><>(\\w+)" + name: "zookeeper_$4_$5" + labels: + replicaId: "$2" + memberType: "$3" + startDelaySeconds: 30 + env: {} + resources: {} + path: /metrics + ports: + jmxxp: + containerPort: 9404 + protocol: TCP + livenessProbe: + httpGet: + path: /metrics + port: jmxxp + initialDelaySeconds: 30 + periodSeconds: 15 + timeoutSeconds: 60 + failureThreshold: 8 + successThreshold: 1 + readinessProbe: + httpGet: + path: /metrics + port: jmxxp + initialDelaySeconds: 30 + periodSeconds: 15 + timeoutSeconds: 60 + failureThreshold: 8 + successThreshold: 1 + + zookeeper: + enabled: false + image: + repository: josdotso/zookeeper-exporter + tag: v1.1.2 + pullPolicy: IfNotPresent + config: + logLevel: info + resetOnScrape: "true" + env: {} + resources: {} + path: /metrics + ports: + zookeeperxp: + containerPort: 9141 + protocol: TCP + livenessProbe: + httpGet: + path: /metrics + port: zookeeperxp + initialDelaySeconds: 30 + periodSeconds: 15 + timeoutSeconds: 60 + failureThreshold: 8 + successThreshold: 1 + readinessProbe: + httpGet: + path: /metrics + port: zookeeperxp + initialDelaySeconds: 30 + periodSeconds: 15 + timeoutSeconds: 60 + failureThreshold: 8 + successThreshold: 1 + +env: + + ## Options related to JMX exporter. + JMXAUTH: "false" + JMXDISABLE: "false" + JMXPORT: 1099 + JMXSSL: "false" + + ## The port on which the server will accept client requests. + ZK_CLIENT_PORT: 2181 + + ## The port on which the ensemble performs leader election. + ZK_ELECTION_PORT: 3888 + + ## The JVM heap size. + ZK_HEAP_SIZE: 2G + + ## The number of Ticks that an ensemble member is allowed to perform leader + ## election. + ZK_INIT_LIMIT: 5 + + ## The Log Level that for the ZooKeeper processes logger. + ## Choices are `TRACE,DEBUG,INFO,WARN,ERROR,FATAL`. + ZK_LOG_LEVEL: INFO + + ## The maximum number of concurrent client connections that + ## a server in the ensemble will accept. + ZK_MAX_CLIENT_CNXNS: 60 + + ## The maximum session timeout that the ensemble will allow a client to request. + ## Upstream default is `20 * ZK_TICK_TIME` + ZK_MAX_SESSION_TIMEOUT: 40000 + + ## The minimum session timeout that the ensemble will allow a client to request. + ## Upstream default is `2 * ZK_TICK_TIME`. + ZK_MIN_SESSION_TIMEOUT: 4000 + + ## The delay, in hours, between ZooKeeper log and snapshot cleanups. + ZK_PURGE_INTERVAL: 0 + + ## The port on which the leader will send events to followers. + ZK_SERVER_PORT: 2888 + + ## The number of snapshots that the ZooKeeper process will retain if + ## `ZK_PURGE_INTERVAL` is set to a value greater than `0`. + ZK_SNAP_RETAIN_COUNT: 3 + + ## The number of Tick by which a follower may lag behind the ensembles leader. + ZK_SYNC_LIMIT: 10 + + ## The number of wall clock ms that corresponds to a Tick for the ensembles + ## internal time. + ZK_TICK_TIME: 2000 + +jobs: + chroots: + enabled: false + activeDeadlineSeconds: 300 + backoffLimit: 5 + completions: 1 + config: + create: [] + # - /kafka + # - /ureplicator + env: [] + parallelism: 1 + resources: {} + restartPolicy: Never diff --git a/kubernetes/common/music/requirements.yaml b/kubernetes/common/music/requirements.yaml new file mode 100644 index 0000000000..220dc4619b --- /dev/null +++ b/kubernetes/common/music/requirements.yaml @@ -0,0 +1,18 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +dependencies: + - name: common + version: ~3.0.0 + repository: '@local' diff --git a/kubernetes/common/music/values.yaml b/kubernetes/common/music/values.yaml new file mode 100644 index 0000000000..51c467cf2f --- /dev/null +++ b/kubernetes/common/music/values.yaml @@ -0,0 +1,50 @@ +# Copyright © 2018 AT&T, Amdocs, Bell Canada Intellectual Property. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +################################################################# +# Global configuration defaults. +################################################################# +global: + nodePortPrefix: 302 + repository: nexus3.onap.org:10001 + + readinessRepository: oomk8s + readinessImage: readiness-check:2.0.0 + loggingRepository: docker.elastic.co + loggingImage: beats/filebeat:5.5.0 + +# flag to enable debugging - application support required +debugEnabled: false + +# default number of instances +replicaCount: 3 + +nodeSelector: {} + +affinity: {} + +# probe configuration parameters +liveness: + initialDelaySeconds: 10 + periodSeconds: 10 + # necessary to disable liveness probe when setting breakpoints + # in debugger so K8s doesn't restart unresponsive container + enabled: true + +readiness: + initialDelaySeconds: 10 + periodSeconds: 10 + +resources: {} + |