1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
|
.. ============LICENSE_START==========================================
.. ===================================================================
.. Copyright © 2017 AT&T Intellectual Property. All rights reserved.
.. ===================================================================
.. Licensed under the Creative Commons License, Attribution 4.0 Intl. (the "License");
.. you may not use this documentation except in compliance with the License.
.. You may obtain a copy of the License at
..
.. https://creativecommons.org/licenses/by/4.0/
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
.. ============LICENSE_END============================================
.. ECOMP is a trademark and service mark of AT&T Intellectual Property.
APPC Logging Guide
==================
The APPC modules manage logging functionality according to the logging
configuration file ``org.ops4j.pax.logging.cfg``. APPC configuration defines
the location of the logging configuration file. Each APPC module
configured to generate logs reads the configuration file periodically
and generates logs according to the current logging level.
Logging Architecture
---------------------
The following diagram illustrates the APPC logging architecture.
|image0|
Log Types
---------
APPC generates several types of logs to capture information needed to
operate, troubleshoot, and report on performance:
- **Audit Log**: The Audit Log provides a summary view of the
processing of a request – a transaction – within an application. It
captures activity requests received by APPC and includes such
information as the time the activity initiated, when it finishes, and
the API invoked at the component.
- **Metric Log**: The Metrics Log provides a more detailed view into
the processing of a transaction within APP-C. It captures the
beginning and ending of activities necessary to complete the
transaction within APPC modules, for example, the Dispatcher module,
and other ONAP components such as A&AI.
- **Error Log**: The Error Log captures INFO, WARN, ERROR, and FATAL
conditions sensed (“exception handled”) by the APPC software
components.
- **Debug Log**: The optional Debug Log captures data that may be required to debug and correct abnormal conditions of the application.
The APPC modules manage logging functionality according to the logging
configuration file ``org.ops4j.pax.logging.cfg``. APPC configuration defines
the location of the logging configuration file. Each APPC module
configured to generate logs reads the configuration file periodically
and generates logs according to the current logging level.
Logging Levels
~~~~~~~~~~~~~~
+-------------+----------------+---------------+-------------+-------------+------------+
| **Level** | **Log type** |
+=============+================+===============+=============+=============+============+
| | | **Errors** |
+-------------+----------------+---------------+-------------+-------------+------------+
| | **Audit** | **Metrics** | **FATAL** | **ERROR** | **WARN** |
+-------------+----------------+---------------+-------------+-------------+------------+
| OFF | No | No | No | No | No |
+-------------+----------------+---------------+-------------+-------------+------------+
| FATAL | Yes | Yes | Yes | No | No |
+-------------+----------------+---------------+-------------+-------------+------------+
| ERROR | Yes | Yes | Yes | Yes | No |
+-------------+----------------+---------------+-------------+-------------+------------+
| WARN | Yes | Yes | Yes | Yes | Yes |
+-------------+----------------+---------------+-------------+-------------+------------+
Response Codes
~~~~~~~~~~~~~~
+----------------------------+--------------------------------+
| **Response code ranges** | **Error type** |
+============================+================================+
| 100-199 | Permission errors |
+----------------------------+--------------------------------+
| 200-299 | Availability errors/Timeouts |
+----------------------------+--------------------------------+
| 300-399 | Data errors |
+----------------------------+--------------------------------+
| 400-499 | Schema errors |
+----------------------------+--------------------------------+
| 500-599 | Business process errors |
+----------------------------+--------------------------------+
| 900-999 | Unknown errors |
+----------------------------+--------------------------------+
Logging Output
---------------
APPC generates logging output according to Event and Error Logging Framework(EELF)requirements in the
following format:
Error Log
~~~~~~~~~~
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| **Field** | **Description** |
+=========================+=================================================================================================================================================================================================================================================+
| Timestamp | Date-time when error occurs in UTC. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| RequestID | Universally unique transaction requestID (UUID). |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ThreadId | Blank |
| | |
| | This traces processing of a request over a number of threads of a single ONAP component. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ServiceName | Externally advertised API invoked by clients of this component, for example, "Audit", "HealthCheck". |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PartnerName | Client or user invoking the API. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TargetEntity | ONAP component/subcomponent or non-ONAP entity invoked for this suboperation’s activities, for example, A&AI, VF, VM, MySql. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TargetServiceName | Known name of External API/operation activities invoked on TargetEntity (ONAP component/subcomponent or non-ONAP entity), for example, VM UUID, VF UUID. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ErrorCategory | WARN or ERROR |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ErrorCode | Code representing the error condition according to the error categories. |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ErrorDescription | Human readable description of the error - Error name, for example: "CONFIGURATION\_STARTED". |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| detailMessage | This field may contain any additional information useful in describing the error condition, for example: |
| | |
| | ``Application Controller (APP-C) initialization started at {0}|\`` |
| | ``No resolution is required, this is an informational message|\`` |
| | ``The APP-C component configuration has been started because the component is being initialized or loaded for the first time, or a new instance of the component is being created. This message indicates that the component is starting.`` |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Debug Log
~~~~~~~~~
+-----------------+------------------------------------------------------------+
| **Field** | **Description** |
+=================+============================================================+
| Timestamp | Date-time of the log record. |
+-----------------+------------------------------------------------------------+
| RequestID | Universally unique transaction requestID (UUID). |
+-----------------+------------------------------------------------------------+
| DebugInfo | Debug Information |
+-----------------+------------------------------------------------------------+
| End of Record | Designates the logical end of a multi-line debug record. |
+-----------------+------------------------------------------------------------+
Audit Log
~~~~~~~~~
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| **Field** | **Description** |
+======================================+===========================================================================================================================================================================================================+
| BeginTimestamp | Date-time of the start of a request activity. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EndTimestamp | Date-time of the end of a request activity. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| RequestID | Universally unique transaction request ID (UUID). |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| serviceInstanceID | Uniquely identifies a service instance, for example, “service graph”. The primary key, for example, in A&AI, to reference or manage the service instance as a unit. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| threadId | Empty |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| physical/virtual server name | Empty (the value added by the log files collecting agent). |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| serviceName | Externally advertised API invoked by clients of this component, for example, "Audit", "HealthCheck". |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PartnerName | Client or user invoking the API. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| StatusCode | High-level success or failure of the request (COMPLETE or ERROR). |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ResponseCode | Application specific response code - LCM API error codes categorized according to the logging categories. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ResponseDescription | Human readable description of the application specific response code, for example, "INVALID INPUT PARAMETER - ${detailedErrorMsg}". |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| instanceUUID | Universally unique identifier to differentiate between multiple instances of the same (named), log writing component - the specific APPC instance UUID. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Category log level | Enum: “INFO” \| “WARN” \|”DEBUG” \| “ERROR” \| “FATAL”. Current log level for the entire APP-C. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Severity | Blank |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Server IP address | Blank |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ElapsedTime | Elapsed time to complete processing of an API or request at the granularity available to the component system. This value should be the difference between BeginTimestamp and EndTimestamp fields. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Server | VM FQDN if virtualized, otherwise the host name of the logging component. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ClientIPaddress | Requesting remote client application’s IP address if known, otherwise empty. |
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Metrics Log
~~~~~~~~~~~
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| **Field** | **Description** |
+===============================================+==================================================================================================================================================================================================================+
| BeginTimestamp | Date-time when a suboperation activity is begun in UTC |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EndTimestamp | Date-time when a supoperation activity is completed in UTC |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| RequestID | Universally unique transaction request ID (UUID) |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| serviceInstanceID | VMUUID, VFUUID |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| threadId | Optional |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| physical/virtual server name | Empty if its value can be added by the log files collecting agent. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| serviceName | For example: "Audit", "HealthCheck" etc |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PartnerName Client or user invoking the API | |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TargetEntity | APPC internal subcomponent, for example, MD-SAL, or external component, for example, A&AI, SSH, Netconf, invoked for this suboperation. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TargetServiceName | Operation activities invoked on TargetEntity e.g. A&AI GET generic- vnf |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| StatusCode | High level success or failure of the suboperation activities (COMPLETE or ERROR) |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ResponseCode | Specific response code returned by the suboperation activities. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ResponseDescription | Human readable description of the response code. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| instanceUUID | APPC instance ID. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Category log level | Enumerated values: “INFO” \| “WARN” \|”DEBUG” \| “ERROR” \| “FATAL”. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Severity | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Server IP address | The logging component host server’s IP address. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ElapsedTime | Elapsed time to complete processing of the sub operation activities at the granularity available to the component system. This value should be the difference between EndTimestamp and BeginTimestamp fields. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Server | VM FQDN if virtualized, otherwise the host name of the logging component. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ClientIP | Requesting remote client application’s IP address. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| class name | Optional. The name of the class that has caused the log record creation. For OO programing languages that support this concept. |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Unused | |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ProcessKey | Optional |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TargetVirtualEntity | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CustomField1 | Empty (specific attributes exposed by developers) |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CustomField2 | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CustomField3 | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CustomField4 | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| detailMessage | Empty |
+-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Log File Locations
------------------
The logging configuration file, ``org.ops4j.pax.logging.cfg`` are located in
appc Git repository:
``/appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg``
The logs are stored at the location defined by the appropriate appender:
``log4j.appender.error.File=${karaf.data}/log/APPC/appc-error.log``
``log4j.appender.debug.file=${karaf.data}/log/APPC/appc-debug.log``
``log4j.appender.metric.File=${karaf.data}/log/APPC/appc-metric.log``
``log4j.appender.audit.File=${karaf.data}/log/APPC/appc-audit.log``
Enabling APPC Logging
----------------------
APPC uses Event and Error Logging Framework (EELF) for application logs.
To enable EELF logging:
1. Replace the default configuration file located at
``/opt/opendaylight/current/etc/org.ops4j.pax.logging.cfg``
with the configuration file that is checked into git:
``/appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg``
2. Stop and restart ODL controller for the configuration changes to take
effect.
3. Verify logging changes at the following log paths:
- ``/opt/opendaylight/current/data/log/eelf/karaf.log``
This log contains the regular karaf.log output reformatted to use
the EELF MDC properties and the pattern that is configured in the
``org.ops4j.pax.logging.cfg`` file.
- ``/opt/opendaylight/current/data/log/APPC/<package-name>``
This directory contains the audit, metric, error, and debug logs that are configured in the ``org.ops4j.pax.logging.cfg`` file.
**Note:**
``/opt/opendaylight/current/data/log/APPC/controller`` contains the logs generated from the package ``org.openecomp.\*`` (all APPC logs)
- Error.log: alarms –ERROR level logs and above
- Info.log: INFO level logs only
- Debug.log: debugging – DEBUG level and above
- Audit – AUDIT level and above
Log Rotation
------------
Log rotation is performed after every 100 MB size limit is reached. The
log rotation interval is defined as part of the EELF framework.
.. |image0| image:: APPCLoggingArchitecturediagram.png
:width: 6.49097in
:height: 3.46181in
|
