diff options
| author |   rn509j <rn509j@att.com> | 2017-10-10 16:02:15 -0400 |
|---|---|---|
| committer | rn509j <rn509j@att.com> | 2017-10-10 16:04:11 -0400 |
| commit | a593b099ae364963722c47399f5050a19c70ea63 (patch) | |
| tree | 9a894b88f57188221708db8dfa0e32f318aadf9e | |
| parent | cf86a1ca9b936ee3313614b5ff3b465971dff899 (diff) | |
adding filter document
DMAAP-149
Signed-off-by: rn509j <rn509j@att.com>
Change-Id: Ie52a0976b16ac797c40aeb5691d8d46b6247c96d
| -rw-r--r-- | docs/Filter/Filter.rst | 582 | ||||
| -rw-r--r-- | docs/Filter/images/image1.png | bin | 0 -> 107880 bytes | |||
| -rw-r--r-- | docs/Filter/images/image2.png | bin | 0 -> 159561 bytes | |||
| -rw-r--r-- | docs/Installation/Installation.rst | 57 | ||||
| -rw-r--r-- | docs/Installation/docker.png | bin | 0 -> 6187 bytes | |||
| -rw-r--r-- | docs/index.rst | 2 | ||||
| -rw-r--r-- | docs/message-router/message router.jpg (renamed from docs/message router.jpg) | bin | 38995 -> 38995 bytes | |||
| -rw-r--r-- | docs/message-router/message-router.rst | 40 |
8 files changed, 660 insertions, 21 deletions
diff --git a/docs/Filter/Filter.rst b/docs/Filter/Filter.rst new file mode 100644 index 0000000..b9f8d07 --- /dev/null +++ b/docs/Filter/Filter.rst @@ -0,0 +1,582 @@ +Filter
+
+A filter examine an event and decides if it matches or doesn't. Filters
+are mainly used in rules to decide if the processing entries should be
+executed on the given event. They're also used for settings, and systems
+like the Graph Correlator re-use Highland Park's filter mechanism to
+specify which alarms fit in a correlation. Some publishers may produce
+topics with a lot of volume and a subscriber may want only a portion of
+those messages. The subscriber can certainly filter messages after
+receiving them, but it may be substantially more efficient to ask the
+API server to filter the messages before sending them to the
+subscriber.The standard library includes a number of simple filters. The
+Equals filter, for example, compares a field's value with some other
+value and returns true if the values match. The standard library also
+includes filter classes called And, Or, and Not, so you can compose more
+complex filters. For example, written in the standard JSON config
+format:
+
++-----------------------------------------+
+| "filter":{ |
+| "class":"And", |
+| "filters": |
+| [ |
+| { "class":"Equals", "foo":"abc" }, |
+| { "class":"Assigned", "field":"bar" } |
+| ] |
+| } |
++-----------------------------------------+
+
+| This filter matches events in which the field "foo" has value "abc"
+ and the field "bar" is assigned to some value (as opposed to not being
+ present on the event). Filters are used by the consumers to filter out
+ data and consume only specific set of data which matches the
+ conditions mentioned in filter. Filters can be passed as a query
+ parameter by the consumer in consume request as mentioned below:
+ **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",
+ "field":"email", "value":"test@abc.com" }**
+| Filters can be applied only on data in JSON format i.e. if applied,
+ filters will automatically ignore any non-json data. While consuming,
+ request CONTENT\_TYPE is not relevant to filter.
+
+All the supported filter can be found below.
+
+`Types of
+Filters <https://wiki.web.att.com/display/MessageRouter/Types+of+Filters>`__
+
+| DMaaP Message Router supports all the filters which were supported by
+ DMaaP Message Router and are mentioned below:-
+| **All Alarms:**
+| Match all alarms.
+| **And:**
+| Create a set of filters. This filter matches when all of them matches.
+
++-------------+--------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+====================+============+=====================+
+| filters | Combined Filters | LIST | A list of filters |
++-------------+--------------------+------------+---------------------+
+| | | | |
++-------------+--------------------+------------+---------------------+
+
+| **Assigned:**
+| Choose a field from the event to check for assignment. This filter
+ matches when the field is assigned.
+
++-------------------+--------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++===================+========================================================+============+=================+
+| field | The field to check for on the event. | STRING | A field name |
++-------------------+--------------------------------------------------------+------------+-----------------+
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |
++-------------------+--------------------------------------------------------+------------+-----------------+
+
+| **Contains:**
+| Check if a search string contains another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| String | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| | | | |
++-------------+---------------------------------------------------+------------+--------------+
+
+| **EndsWith**:
+| Check if a search string ends with another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| string | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+
+| **Equals:**
+| Choose a field from the event and a value to check for equality.
+
++-------------+----------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+==============================================+============+==============+
+| field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+| value | The value to match. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+
+| **FlatironObjectExists**
+| Matches when the given object exists in the given Flatiron instance.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| oid | The OID of the object to look for. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+| flatiron | The name of the Flatiron client instance. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+| **IsAging**
+| Choose a field to test. This filter matches if the expression is
+ numeric.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+| **IsNumeric**
+| Choose a field to test. This filter matches if the expression is
+ numeric.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+| **MathCondition**
+| Choose a field from the event and a value for logical math conditions.
+
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=================================================+============+===================================+
+| Field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| Value | The value to consider. Supports ${} notation. | STRING | Any string |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| operator | The operation. | STRING | One of { "<=", ">=", ">", "<" } |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| | | | |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+
+| **NoAlarms**
+| Don't match any alarms.
+| **Not**
+| Negate the configured filter.
+
++-------------+-------------------------+------------+-------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=========================+============+=============+
+| filter | The filter to negate. | FILTER | A filter |
++-------------+-------------------------+------------+-------------+
+
+| **NotEqual**
+| Choose a field from the event and a value to check for inequality.
+
++-------------+----------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+==============================================+============+==============+
+| field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+| value | The value to match. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+
+| **NotOneOf**
+| Match when the specified field does not have a value from the given
+ list.
+
++-------------+---------------------------------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+=====================+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+---------------------+
+| values | The matching values. | LIST | A list of strings |
++-------------+---------------------------------------------+------------+---------------------+
+
+| **OneOf**
+| Match when the specified field has a value from the given list.
+
++-------------+---------------------------------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+=====================+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+---------------------+
+| values | The matching values. | LIST | A list of strings |
++-------------+---------------------------------------------+------------+---------------------+
+
+| **Or**
+| Create a set of filters. This filter matches when any one of them
+ matches.
+
++-------------+--------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+====================+============+=====================+
+| filters | Combined Filters | LIST | A list of filters |
++-------------+--------------------+------------+---------------------+
+
+| **RegEx**
+| Choose a field from the event to match against the regular expression
+ you provide.
+
++-------------+---------------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=========================================================+============+==============+
+| field | The text to check for a match. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------------+------------+--------------+
+| value | The regular expression (pattern) to match. | STRING | Any string |
++-------------+---------------------------------------------------------+------------+--------------+
+
+| **StartsWith**
+| Check if a search string starts with another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| string | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+
+| **Unassigned**
+| Choose a field from the event to check for assignment. This filter
+ matches when the field is not assigned.
+
++-------------------+--------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++===================+========================================================+============+=================+
+| field | The field to check for on the event. | STRING | A field name |
++-------------------+--------------------------------------------------------+------------+-----------------+
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |
++-------------------+--------------------------------------------------------+------------+-----------------+
+
+| **WithinSecondsFrom**
+| This filter matches when the specified epoch time value is within the
+ given number of seconds from the baseline time value. Both time values
+ are assumed to be in seconds. If a value is in milliseconds, set
+ baselineTimeInMillis and/or eventTimeInMillis to true.
+
++------------------------+------------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++========================+============================================================+============+=================+
+| field | The time value to test. Supports ${} | STRING | A field name |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| seconds | The number of seconds. | NUMBER | A number |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| baselineTimeInMillis | Whether to convert the baseline value from milliseconds. | BOOLEAN | True or False |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| baseline | The baseline time value. Supports ${}. | STRING | Any string |
++------------------------+------------------------------------------------------------+------------+-----------------+
+
+| **WithinTimeFromNow**
+| This filter matches when the named field has an epoch time value
+ within the given number of seconds from the current time. The event's
+ time value is assumed to be in seconds. If it's in milliseconds, set
+ eventTimeInMillis to true.
+
++---------------------+---------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=====================+=========================================================+============+=================+
+| field | The field to check on the event. | STRING | A field name |
++---------------------+---------------------------------------------------------+------------+-----------------+
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |
++---------------------+---------------------------------------------------------+------------+-----------------+
+| seconds | The number of seconds. | NUMBER | A number |
++---------------------+---------------------------------------------------------+------------+-----------------+
+
+**Limit:**
+
+- Limit is the integer value and DMaaP Message Router will consumes
+ only that set of message which are specified in limit.
+
+| Suppose if we set limit=2, then only 2 sets of data will be consumed.
+| *Get \ **<<topicName>>/group/2?limit=4***
+| Let us suppose if
+| **No of data available** = 4
+| **Set limit** = 6
+| i.e. limit>no of data
+| In this scenario all 4 sets of data will be consumed.
+
+- If limit is not passed with the url then by default limit is set to
+ 4096.
+
+| i.e. 4096 sets of data will be consumed.
+| **Timeout and Long Poll:**
+
+- Timeout is the integer value which will be treated by DMaaP Message
+ Router as time in millisecond.
+
+
+
+- Get
+
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| `***http://localhost/DMaaP/dmaaprest/events/<<topicName>>/group/2?timeout=20000*** <http://localhost/DMaaP/dmaaprest/events/%3c%3ctopicName%3e%3e/group/2?timeout=20000>`__ |
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+- If there is no data available to be consumed, then DMaaP Message
+ Router will poll for the particular period of time specified in
+ timeout this mechanism is known as Long Poll.
+
+- If timeout is not passed with url then DMaaP Message Router will set
+ the value of timeout =10000
+
+- i.e. if no set of data are available then DMaaP Message Router will
+ poll for 10000 ms.
+
+***Meta:***
+
+- Meta is a Boolean value.
+
+- DMaaP Message Router reads the value of meta from
+ MRConfiguration.properties file at the time of startup.
+
+- If the value of meta is not null and if value of meta is one of these
+ values true, yes, on, 1, y, checked then DMaaP Message Router will
+ take meta flag as true, else it will be false.
+
+- If meta is set to true then consumer will get the value of message
+ offset along with message.
+
+|image1|
+| **Pretty**:
+
+- Pretty is a Boolean value.
+
+- DMaaP Message Router reads the value of pretty from
+ MRConfiguration.properties file at the time of startup.
+
+- If the value of pretty is not null and if value of pretty is one of
+ these values true, yes, on, 1, y, checked then DMaaP Message Router
+ will take pretty flag as true, else it will be false.
+
+- If pretty is set to true then different sets of messages will be
+ printed in next line separated by comma (,).
+
+|image2|
+| **Filter**
+| A filter examine an event and decides if it matches or doesn't.
+| Filters are mainly used in rules to decide if the processing entries
+ should be executed on the given event. They're also used for settings,
+ and systems like the Graph Correlator re-use Highland Park's filter
+ mechanism to specify which alarms fit in a correlation.
+| The standard library includes a number of simple filters. The Equals
+ filter, for example, compares a field's value with some other value
+ and returns true if the values match.
+| The standard library also includes filter classes called And, Or, and
+ Not, so you can compose more complex filters. For example, written in
+ the standard JSON config format:
+
++-----------------------------------------+
+| "filter":{ |
+| "class":"And", |
+| "filters": |
+| [ |
+| { "class":"Equals", "foo":"abc" }, |
+| { "class":"Assigned", "field":"bar" } |
+| ] |
+| } |
++-----------------------------------------+
+
+| This filter matches events in which the field "foo" has value "abc"
+ and the field "bar" is assigned to some value (as opposed to not being
+ present on the event).
+| Filters are used by the consumers to filter out data and consume only
+ specific set of data which matches the conditions mentioned in filter.
+| Filters can be passed as a query parameter by the consumer in consume
+ request as mentioned below:
+| **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",
+ "field":"email", "value":"`test@abc.com <mailto:test@att.com>`__" }**
+| Filters can be applied only on data in JSON format i.e. if applied,
+ filters will automatically ignore any non-json data.
+| While consuming, request CONTENT\_TYPE is not relevant to filter.
+
+The MR API allows a subscriber pass a Highland Park filter as part of
+the GET request. This will filter the stream of messages sent back to
+the subscriber, but for this to work, there are some requirements:
+
+- The message payload must be JSON
+
+- Only a filter built from Highland Park's Standard Library may be
+ used. (The Cambria API server doesn't have access to plugged in
+ filters.)
+
+- The filter must be encoded properly in the URL path.
+
+ Server-side filtering can also be setup in the Java client as
+illustrated below
+
+**Filtering Consumer**
+
+You can also provide a Highland Park filter to your consumer instance,
+and this filter is passed on to the server in the GET request. One way
+to create the filter is programmatically. In your code, instantiate a
+filter from the Highland Park Standard Library Then create a String
+representation of the filter using the FilterIo.write utility. This
+String can then be passed to the Cambria client instance for use on the
+server.
+
+Remember, only Highland Park standard library filter components can be
+used -- no plug-ins are available in the Cambria server context.
+
+package org.onap.sa.highlandPark.integration;
+
+import java.io.IOException;
+
+import java.util.UUID;
+
+import org.onap.nsa.cambria.client.CambriaClientFactory;
+
+import org.onap.nsa.cambria.client.CambriaConsumer;
+
+import org.onap.sa.highlandPark.processor.HpEvent;
+
+import org.onap.sa.highlandPark.stdlib.filters.FilterIo;
+
+import org.onap.sa.highlandPark.stdlib.filters.OneOf;
+
+public class ExampleFilteringConsumer
+
+{
+
+public static void main ( String[] args ) throws IOException,
+InterruptedException
+
+{
+
+// Cambria clients take a set of 1 or more servers to use in round-robin
+fashion.
+
+// If a server becomes unreachable, another in the group is used.
+
+final String
+serverGroup="ueb01hydc.it.att.com,ueb02hydc.it.att.com,ueb03hydc.it.att.com";
+
+// choose a topic
+
+final String topic = "TEST-TOPIC";
+
+// Cambria clients can run in a cooperative group to handle high-volume
+topics.
+
+// Here, we create a random group name, which means this client is not
+re-startable.
+
+final String consumerGroup = UUID.randomUUID ().toString ();
+
+final String consumerId = "0";
+
+// Cambria clients can sit in a tight loop on the client side, using a
+long-poll
+
+// to wait for messages, and a limit to tell the server the most to send
+at a time.
+
+final int longPollMs = 30\*1000;
+
+final int limit = -1;
+
+// The Cambria server can filter the returned message stream using
+filters from the
+
+// Highland Park system. Here, we create a simple filter to test for the
+AlarmID
+
+// value being one of the Mobility power alarms.
+
+final OneOf oneOf = new OneOf ( "AlarmId", kPowerAlarms );
+
+// create the consumer
+
+final CambriaConsumer cc = CambriaClientFactory.createConsumer (
+serverGroup, topic,
+
+consumerGroup, consumerId, longPollMs, limit, FilterIo.write ( oneOf )
+);
+
+// now loop reading messages. Note that cc.fetch() will wait in its HTTP
+receive
+
+// method for up to 30 seconds (longPollMs) when nothing's available at
+the server.
+
+long count = 0;
+
+while ( true )
+
+{
+
+for ( String msg : cc.fetch () )
+
+{
+
+System.out.println ( "" + (++count) + ": " + msg );
+
+}
+
+}
+
+}
+
+private static final String[] kPowerAlarms =
+
+{
+
+"HUB COMMERCIAL POWER FAIL\_FWD",
+
+"HUB COMMERCIAL POWER FAIL",
+
+"RBS COMMERCIAL POWER FAIL - Fixed\_FWD",
+
+"RBS COMMERCIAL POWER FAIL\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - No Generator\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - Portable\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - Shared\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - Yes\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - YES\_FWD",
+
+"RBS COMMERCIAL POWER FAIL - Fixed",
+
+"RBS COMMERCIAL POWER FAIL - No Generator",
+
+"RBS COMMERCIAL POWER FAIL - Portable",
+
+"RBS COMMERCIAL POWER FAIL - Shared",
+
+"RBS COMMERCIAL POWER FAIL - YES",
+
+"RBS COMMERCIAL POWER FAIL - Yes",
+
+"RBS COMMERCIAL POWER FAIL",
+
+"HUB COMMERCIAL POWER FAIL - Fixed",
+
+"HUB COMMERCIAL POWER FAIL - No Generator",
+
+"HUB COMMERCIAL POWER FAIL - Portable",
+
+"HUB COMMERCIAL POWER FAIL - Shared",
+
+"HUB COMMERCIAL POWER FAIL - Fixed\_FWD",
+
+"HUB COMMERCIAL POWER FAIL - No Generator\_FWD",
+
+"HUB COMMERCIAL POWER FAIL - Portable\_FWD",
+
+"HUB COMMERCIAL POWER FAIL - Shared\_FWD",
+
+};
+
+}
+
+
+
+** Filter Builder**
+
+ MR server-side filtering allows a consumer to filter the stream of
+messages returned from the GET call. The following link provide details
+of building some of the filter to illustrate Filter Builder. It is not
+meant to cover and provide examples of every filter
+
+.. |image1| image:: images/image1.png
+.. |image2| image:: images/image2.png
+
diff --git a/docs/Filter/images/image1.png b/docs/Filter/images/image1.png Binary files differnew file mode 100644 index 0000000..d424f15 --- /dev/null +++ b/docs/Filter/images/image1.png diff --git a/docs/Filter/images/image2.png b/docs/Filter/images/image2.png Binary files differnew file mode 100644 index 0000000..faddcfe --- /dev/null +++ b/docs/Filter/images/image2.png diff --git a/docs/Installation/Installation.rst b/docs/Installation/Installation.rst new file mode 100644 index 0000000..8026928 --- /dev/null +++ b/docs/Installation/Installation.rst @@ -0,0 +1,57 @@ +=================================
+DMAAP MESSAGE ROUTER INSTALLATION
+=================================
+This document describes how to install and access DMaaP Message Router.
+Message Router has 3 docker containers. Dmaap\_container,
+kafka\_contaienr and zookeeper\_container. Zookeeper runs on 172.18.0.3,
+kafka runs on 172.18.0.2 and dmaap on 172.18.0.4.
+
+1) Clone message service repo
+
+ git clone http://gerrit.onap.org/r/dmaap/messagerouter/messageservice
+
+2) copy
+ messageservice/bundleconfig-local/etc/appprops/MsgRtrApi.properties
+ to /var/tmp directory
+
+3) In /var/tmp/MsgRtrApi.properties, change value of below variables as
+ shown below:
+
+ config.zk.servers=172.18.0.3
+
+ kafka.metadata.broker.list=172.18.0.2:9092
+
+4) Install docker and docker-compose
+
+5) Go to messageservice/src/main/resources/docker-compose and run:
+
+ docker-compose up –d
+
+ This should start 3 containers.
+
+6) Run ‘docker ps’. It should show 3 containers.
+
+ |image0|
+
+ .. |image0| image:: docker.png
+ :width: 5.84375in
+ :height: 0.76042in
+
+Testing
+-------
+
+- For publishing, create a sample.txt file with some content in the
+ directory where you will run below rest api. Run below rest api:
+
+ curl -H "Content-Type:text/plain" -X POST -d @sample.txt
+ http://172.18.0.4:3904/events/TestTopic1
+
+- For subscribing, run below rest api:
+
+ curl -H "Content-Type:text/plain" -XGET
+ http://172.18.0.4:3904/events/TestTopic1/CG1/C1?timeout=1000
+
+ Note: You will only receive messages which have been published after
+ you have subscribed to a topic.
+
+
diff --git a/docs/Installation/docker.png b/docs/Installation/docker.png Binary files differnew file mode 100644 index 0000000..ec26ef5 --- /dev/null +++ b/docs/Installation/docker.png diff --git a/docs/index.rst b/docs/index.rst index be660db..4eda778 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -11,5 +11,7 @@ Message Router (MR) :maxdepth: 2 message-router/message-router + Installation/Installation + Filter/Filter diff --git a/docs/message router.jpg b/docs/message-router/message router.jpg Binary files differindex 100973d..100973d 100644 --- a/docs/message router.jpg +++ b/docs/message-router/message router.jpg diff --git a/docs/message-router/message-router.rst b/docs/message-router/message-router.rst index fbd7015..983603a 100644 --- a/docs/message-router/message-router.rst +++ b/docs/message-router/message-router.rst @@ -65,8 +65,8 @@ are not inspected for content. | Content-Type | Description |
+=========================+================================================================================================================+
| text/plain | Each line in the POST body is treated as a separate message. No partition key is specified, and therefore no |
-| | order is guaranteed. This format is mainly for test, as messages are highly likely to be re-ordered when |
-| | delivered through the Kafka cluster. |
+| | order is guaranteed. This format is mainly for test, as messages are highly likely to be re-ordered when |
+| | delivered through the Kafka cluster. |
+-------------------------+----------------------------------------------------------------------------------------------------------------+
| application/json | The payload maybe a single JSON object or a JSON array of JSON objects. Each object is handled as an individual|
| | message..Note that use of this format may result in equivalent but altered JSON objects sent to consumers. |
@@ -112,7 +112,7 @@ Request Parameters | partitionKey | | QueryParam | String | | N | String value |?Partitionkey=123 |
+--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+-----------------------------+
-**NOTE:** Publishers/user should have access on the topics. The user (id) and
+**NOTE **: Publishers/user should have access on the topics. The user (id) and
permissions details needs to be in AAF.
Response Parameters
@@ -210,14 +210,13 @@ Subscribers Request URL:
============
-GET http(s)://{HOST:PORT}}/events/{topicname}/{consumegroup}/{consumerid}?{timeout=x}
+GET http(s)://{HOST:PORT}}/events/{topicname}/{consumegroup}/{consumerid}/{timeout=x}
Request Parameters:
==================
+-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
| Name | Description | Param Type | data type | MaxLen | Req’d | Format | Valid/Example Values |
-| | | | | | | | |
+=============+=================================+==================+============+==============+=============+=============+=================================================+
| Topicname | topic name to be posted | Path | String | 40 | Y | namespace. | |
| | | | | | | String | |
@@ -237,7 +236,7 @@ Request Parameters: | Password | | Header | String | 1 | N | | |
+-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
-**NOTE1:**Subscribers /user should have access on the topics. The user () and
+**NOTE1**:Subscribers /user should have access on the topics. The user () and
permissions details needs to be in AAF.
Response Parameters:
@@ -258,7 +257,7 @@ Response Parameters: +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
| ResponseBody | Messages consumed from topic | Json | Json | |
+------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
-|
+
+---------------------------+------------------------------------+
| Response statusCode | Response statusMessage |
+===========================+====================================+
@@ -268,7 +267,7 @@ Response Parameters: +---------------------------+------------------------------------+
| 500-599 | the DMaaP service has a problem |
+---------------------------+------------------------------------+
-|
+
+-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
| Error code | HTTP Code | Description |Issue reason |
+=========================+=================+============================+====================================================================================================+
@@ -306,18 +305,17 @@ Request Parameters: +----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
| Name | Description | Param Type | data type | MaxLen | Req’d | Format | Valid/Example Values |
-| | | | | | | | |
+================+=================================+==================+============+==============+=============+=============+===================================+
| Topicname | topicname to be created in MR | Body | String | 20 | Y | Json | com.att.dmaap.mr.metrics |
+----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
-|topicDescription| description for topic | Body | String | 15 | Y | | |
+|topicDescription| description for topic | Body | String | 15 | Y | | |
+----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
|partitionCount | Kafka topic partition | Body | String | 1 | Y | | |
+----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
|replicationCount| Kafka topic replication | Body | String | 1 | Y | | 3 (Default -for 3 node Kafka ) |
+----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
|transaction |to create transaction id for | Body | Boolean | | | | |
-| | each message transaction | | | 1 | N | | true |
+| | each message transaction | | | 1 | N | | true |
| Enabled | | | | | | | |
+----------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+
|Content-Type | application/json | Header | String | | | | application/json |
@@ -337,7 +335,7 @@ Request Parameters: | Error code | HTTP Code | Description |
+=========================+=================+==================================================+
| DMaaP\_MR\_ERR\_5001 | 500 | Failed to retrieve list of all topics | +-------------------------+-----------------+--------------------------------------------------+
-| DMaaP\_MR\_ERR\_5002 | 500 | Failed to retrieve details of topic:<topicName> | |+-------------------------+----------------+--------------------------------------------------+
+| DMaaP\_MR\_ERR\_5002 | 500 | Failed to retrieve details of topic:<topicName> | +-------------------------+-----------------+--------------------------------------------------+
| DMaaP\_MR\_ERR\_5003 | 500 |Failed to create topic:<topicName> | +-------------------------+-----------------+--------------------------------------------------+
| DMaaP\_MR\_ERR\_5004 | 500 | Failed to delete topic:<topicName> | +-------------------------+-----------------+--------------------------------------------------+
@@ -349,7 +347,7 @@ Response Parameters +==================+================================+============+==============+===========================================================+
| httpStatusCode | | | | 200, 201 etc. |
+------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
-| mrErrorCode | Numeric error code | | |5005 |
+| mrErrorCode | Numeric error code | | | 5005 |
+------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
| errorMessage | | | | SUCCESS, or error message. |
+------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
@@ -366,19 +364,19 @@ Sample Request: +-----------------------------------------------------------------------------------+
| POST http://<hostname>:3904/topic/create |
|Request Body |
-|{"topicName":"com.att.dmaap.mr.topicname","description":"This is a SAPTopic ", |
+|{"topicName":"com.abc.dmaap.mr.topicname","description":"This is a SAPTopic ", |
| "partitionCount":"1","replicationCount":"3","transactionEnabled":"true"} |
| Content-Type: application/json |
|Example: |
|curl -u XXXc@csp.abc.com:xxxxx$ -H 'Content-Type:application/json' -X POST -d |
-|@topicname.txt http://mrlocal00.dcae.proto.research.att.com:3904/topics/create |
+|@topicname.txt http://mrlocal00.dcae.proto.research.abc.com:3904/topics/create |
|{ |
| "writerAcl": { |
| "enabled": false, |
| "users": [] |
| }, |
| "description": "This is a TestTopic", |
-| "name": "com.att.ecomp_test.crm.Load9", |
+| "name": "com.abc.ecomp_test.crm.Load9", |
| "readerAcl": { |
| "enabled": false, |
| "users": [] |
@@ -400,7 +398,7 @@ Request Parameters +--------------------------+---------------------------------+------------------+------------+-----------+-------------+-----------------+-----------------------------+
| Name | Description | Param Type | Data type | Max Len | Req’d | Format | Valid/EXample values |
+==========================+=================================+==================+============+===========+=============+=================+=============================+
-| Topicname | topic name details | Body | String | 20 | Y | Json | com.att.dmaap.mr.metrics |
+| Topicname | topic name details | Body | String | 20 | Y | Json | com.abc.dmaap.mr.metrics |
+--------------------------+---------------------------------+------------------+------------+-----------+-------------+-----------------+-----------------------------+
@@ -410,7 +408,7 @@ Response Parameters +------------------+------------------------+------------+----------+---------+--------------------------+
| Name | Description | ParamType | datatype |Format | Valid/Example Values |
+==================+========================+============+==========+=========+==========================+
-| topicname | topic name details | Body | String | Json | com.att.dmaap.mr.metrics |
+| topicname | topic name details | Body | String | Json | com.abc.dmaap.mr.metrics |
+------------------+------------------------+------------+----------+---------+--------------------------+
| description | | | String | | |
+------------------+------------------------+------------+----------+---------+--------------------------+
@@ -442,14 +440,14 @@ Sample Request: | { |
| "txenabled": true, |
| "description": "This is a TestTopic", |
-| "owner": "rs857c@csp.att.com", |
-| "topicName": "com.att.ecomp_test.crm.Load9" |
+| "owner": "XXXX@csp.abc.com", |
+| "topicName": "com.abc.ecomp_test.crm.Load9" |
| }, |
| { |
| "txenabled": false, |
| "description": "", |
| "owner": "", |
-| "topicName": "com.att.ecomp_test.crm.Load1" |
+| "topicName": "com.abc.ecomp_test.crm.Load1" |
| }, |
+-----------------------------------------------------------------------------------+
|
