1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
|
# CDS Artifact Manager
Artifact Manager is a very simple gRPC service that lets you upload, download and delete CBA archives. It can be ran as a standalone micro service (using `server.py`) or you can include it's methods in a service like `py-executor`.
## Configuration
Configuration is stored in `.ini` file, you can specify a path and name of a file using `CONFIGURATION` env variable.
For possible variables please see example below (with inline comments):
```
[artifactManagerServer]
port=50052 # A port on which the server will be listening
logFile=server.log # Path to a log file
maxWorkers=20 # Max number of concurent workers
debug=true # Debug flag
logConfig=logging.yaml # A special MDC logger config
fileRepositoryBasePath=/tmp/ # A FS path where we should store CBA files
```
## Methods
Below is a list of gRPC methods handled by the service. The `proto` files are available in `artifact-manager/manager/proto` directory.
All methods expect `CommonHeader` with:
* `timestamp` - datetime in UTC with this: `%Y-%m-%dT%H:%M:%S.%fZ` format
* `originatorId` - name of the component (eg. `CDS`)
* `requestId` - ID of the request
* `subRequestId` - Sub ID of the request
* `flag` - TBD
and an `ActionIdentifiers` with following fields:
* `blueprintName` - name of the blueprint
* `blueprintVersion` - version number of blueprint (as string)
* `actionName` - TBD
* `mode` - TBD
### Upload
Upload `CBA.zip` file for storage in artifact manager. File needs to be sent as a binary data in `fileChunk` field.
#### Example
```
stub: BlueprintManagementServiceStub = BlueprintManagementServiceStub(channel)
with open(file_path, "rb") as cba_file:
msg: BlueprintUploadInput = BlueprintUploadInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
msg.fileChunk.chunk = cba_file.read()
return stub.uploadBlueprint(msg)
```
### Download
Download existing `CBA.zip` file.
#### Example
```
stub: BlueprintManagementServiceStub = BlueprintManagementServiceStub(channel)
msg: BlueprintDownloadInput = BlueprintDownloadInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
return stub.downloadBlueprint(msg)
```
### Remove
Delete existing `CBA.zip` file.
#### Example
```
stub: BlueprintManagementServiceStub = BlueprintManagementServiceStub(channel)
msg: BlueprintRemoveInput = BlueprintRemoveInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
return stub.removeBlueprint(msg)
```
## Full gRPC Client Example
```
import logging
import sys
from argparse import ArgumentParser, FileType, Namespace
from configparser import ConfigParser
from datetime import datetime
from pathlib import Path
import zipfile
from grpc import Channel, ChannelCredentials, insecure_channel, secure_channel, ssl_channel_credentials
from proto.BlueprintManagement_pb2 import (
BlueprintDownloadInput,
BlueprintRemoveInput,
BlueprintUploadInput,
BlueprintManagementOutput,
)
from proto.BlueprintManagement_pb2_grpc import BlueprintManagementServiceStub
logging.basicConfig(level=logging.DEBUG)
class ClientArgumentParser(ArgumentParser):
"""Client argument parser.
It has two arguments:
- config_file - provide a path to configuration file. Default is ./configuration-local.ini
- actions - list of actions to do by client. It have to be a list of given values: upload, download, remove.
"""
DEFAULT_CONFIG_PATH: str = str(Path(__file__).resolve().with_name("configuration-local.ini"))
def __init__(self, *args, **kwargs):
"""Initialize argument parser."""
super().__init__(*args, **kwargs)
self.description: str = "Artifact Manager client example"
self.add_argument(
"--config_file",
type=FileType("r"),
default=self.DEFAULT_CONFIG_PATH,
help="Path to the client configuration file. By default it's `configuration-local.ini` file from Artifact Manager directory",
)
self.add_argument(
"--actions", nargs="+", default=["upload", "download", "remove"], choices=["upload", "download", "remove"]
)
class Client:
"""Client class.
Implements methods which can be called to server.
"""
def __init__(self, channel: Channel, config: ConfigParser) -> None:
"""Initialize client class.
:param channel: gprc channel object
:param config: ConfigParser object with "client" section
"""
self.channel: Channel = channel
self.stub: BlueprintManagementServiceStub = BlueprintManagementServiceStub(self.channel)
self.config = config
def upload(self) -> BlueprintManagementOutput:
"""Prepare upload message and send it to server."""
logging.info("Call upload client method")
with open(self.config.get("client", "cba_file"), "rb") as cba_file:
msg: BlueprintUploadInput = BlueprintUploadInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
msg.fileChunk.chunk = cba_file.read()
return self.stub.uploadBlueprint(msg)
def download(self) -> BlueprintManagementOutput:
"""Prepare download message and send it to server."""
logging.info("Call download client method")
msg: BlueprintDownloadInput = BlueprintDownloadInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
return self.stub.downloadBlueprint(msg)
def remove(self) -> BlueprintManagementOutput:
"""Prepare remove message and send it to server."""
logging.info("Call remove client method")
msg: BlueprintRemoveInput = BlueprintRemoveInput()
msg.actionIdentifiers.blueprintName = "Test"
msg.actionIdentifiers.blueprintVersion = "0.0.1"
return self.stub.removeBlueprint(msg)
if __name__ == "__main__":
arg_parser: ClientArgumentParser = ClientArgumentParser()
args: Namespace = arg_parser.parse_args()
config_parser: ConfigParser = ConfigParser()
config_parser.read_file(args.config_file)
server_address: str = f"{config_parser.get('client', 'address')}:{config_parser.get('client', 'port')}"
if config_parser.getboolean("client", "use_ssl", fallback=False):
logging.info(f"Create secure connection on {server_address}")
with open(config_parser.get("client", "private_key_file"), "rb") as private_key_file, open(
config_parser.get("client", "certificate_chain_file"), "rb"
) as certificate_chain_file:
ssl_credentials: ChannelCredentials = ssl_channel_credentials(
private_key=private_key_file.read(), certificate_chain=certificate_chain_file.read()
)
channel: Channel = secure_channel(server_address, ssl_credentials)
else:
logging.info(f"Create insecure connection on {server_address}")
channel: Channel = insecure_channel(server_address)
with channel:
client: Client = Client(channel, config_parser)
for action in args.actions:
logging.info("Get response")
logging.info(getattr(client, action)())
```
|
