1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
|
# Streams Formatting Quick Reference
Each of the following tables represents an example of a publisher and its subscriber, which are of course, different components. This focuses on the fields that are ‘different’ for each of these TYPEs, to illustrate the relationship between `config_key`, dmaap connection object, and the generated configuration. Some notes on specific properties:
* `config_key` is an arbitrary string, chosen by the component developer. It is returned in the generated configuration where it contains specific values for the target connection
* `format`, `version`, and `type` properties in the subscriber would match these properties in the publisher
* `aaf_username` and `aaf_password` may be different between the publisher and the subscriber
### Using http
#### *Publishing Component*
| component spec | runtime platform generated config |
|----------------|-----------------------------------|
|"streams":{<br> "publishes":[{<br> "config_key":"prediction",<br> "format":"some-format",<br> "type":"http",<br> "version":"0.1.0"<br> }]<br>}<br>|"streams_publishes":{<br> "prediction":["10.100.1.100:32567/data"] |
#### *Subscribing Component*
| component spec | runtime platform generated config |
|----------------|-----------------------------------|
|"streams":{<br> "subscribes":[{<br> "route":"/data",<br> "format":"some-format",<br> "type":"http",<br> "version":"0.1.0"<br> }]<br>}<br>|"N/A"|
### Using Message Router
#### *Publishing Component*
Note: When deploying, this component should be deployed first so satisfy downstream dependencies. Refer to the –force option in component ‘run’ command for more information.
| component spec | Dmaap Connection Object | runtime platform generated config |
|----------------|-------------------------| --------------------------------- |
|"streams":{<br> "publishes":[{<br> "config_key":"mr_output",<br> "format":"some-format",<br> "type":"message_router",<br> "version":"0.1.0"<br> }]<br>} | {<br> "type":"message_router",<br> "dmaap_info": {<br> "topic_url": "https://we-are-message-router.us:3905/events/some-topic" }<br>} <br><br>*Note: For message router, this object is identical for the publisher and the subscriber* | "streams_publishes":{<br> "mr_output":{<br> "aaf_username":"pub-user",<br> "aaf_password":"pub-pwd",<br> "type":"message_router",<br> "dmaap_info":{<br> "topic_url":"https://we-are-message-router.us:3905/events/some-topic"}<br> }<br>},<br> "streams_subscribes":{<br>…<br>}
#### *Subscribing Component*
| component spec | Dmaap Connection Object | runtime platform generated config |
|----------------|-------------------------| --------------------------------- |
|"streams":{<br> "subscribes":[{<br> "config_key":"mr_input",<br> "format":"some-format",<br> "type":"message_router",<br> "version":"0.1.0"<br> }]<br>} | {<br> "type":"message_router",<br> "dmaap_info": {<br> "topic_url": "https://we-are-message-router.us:3905/events/some-topic" }<br>} <br><br>*Note: For message router, this object is identical for the publisher and the subscriber* | "streams_publishes":{<br>…<br>},<br> "streams_subscribes":{<br> "mr_input":{<br> "aaf_username":"sub-user",<br> "aaf_password":"sub-pwd",<br> "type":"message_router",<br> "dmaap_info":{<br> "topic_url":"https://we-are-message-router.us:3905/events/some-topic"}<br> }<br>}
### Using Data Router
#### *Publishing Component*
| component spec | Dmaap Connection Object | runtime platform generated config |
|----------------|-------------------------| --------------------------------- |
|"streams":{<br> "publishes":[{<br> "config_key":"dr_output",<br> "format":"some-format",<br> "type":"data_router",<br> "version":"0.1.0"<br> }]<br>} | {<br> "type":"data_router",<br> "dmaap_info": {<br> "location": "mtc00",<br> "publish_url": "https://we-are-data-router.us/feed/xyz", <br> "log_url": "https://we-are-data-router.us/feed/xyz/logs",<br> "username": "pub-user",<br> "password": "pub-password",<br> "publisher_id": "123456"}<br>} | streams_publishes":{<br> "dr_output":{<br> "type":"data_router",<br> "dmaap_info":{<br> "location":"mtc00",<br> "publish_url":"https://we-are-data-router.us/feed/xyz",<br> "log_url":"https://we-are-data-router.us/feed/xyz/logs",<br> "username":"pub-user",<br> "password":"pub-password",<br> "publisher_id":"123456"}<br> }<br>},<br> "streams_subscribes":{<br> …<br> }
#### *Subscribing Component*
| component spec | Dmaap Connection Object | runtime platform generated config |
|----------------|-------------------------| --------------------------------- |
|"streams":{<br> "subscribes":[{<br> "config_key":"dr_input",<br> "format":"some-format",<br> "type":"data_router",<br> "version":"0.1.0",<br> "route":"/target-path"<br> }]<br>} | {<br> "type":"data_router",<br> "dmaap_info": {<br> "location": "mtc00",<br> "delivery_url": "https://my-subscriber-app.dcae:8080/target-path", <br> "username": "sub-user",<br> "password": "sub-password",<br> "subscriber_id": "789012"}<br>} | "streams_publishes":{<br> …<br> },<br> "streams_subscribes":{<br> "dr_input":{<br> "type":"data_router",<br> "dmaap_info":{<br> "location":"mtc00",<br> "delivery_url":"https://my-subscriber-app.dcae:8080/target-path",<br> "username":"sub-user",<br> "password":"sub-password",<br> "subscriber_id":"789012"}<br> }<br>}
|
