diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 273 |
1 files changed, 273 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..97831cc --- /dev/null +++ b/README.md @@ -0,0 +1,273 @@ +# Search Engine Micro Service + +The _Search Engine_ micro service exposes APIs via REST which allow clients to interact with the search database back end without requiring direct knowledge of or interaction with the underlying technology. + +## High Level Concepts +This section establishes some of the terminology and concepts that relate to interacting with the _Search Engine_ service. +A much more detailed examination of these concepts can be found on the [Search Engine Design Share](http://d2athenaconf:8090/confluence/display/AAI/AAI-4633%3A+Search+DB+Abstraction%3A+Expose+REST+Interface) Confluence page. + +### Documents +_Documents_ are the _things_ that we want to put into our document store. At its most basic, a _document_ is a collection of _fields_ which contain the data that we want to be able to store and query. + +_Fields_ are defined as having a name, a type, and optional parameters indicating whether or not the field is intended to be searchable, and if so, how it should be indexed. + +### Indexes +An _index_ is essentially a collection of _documents_. It is the top-level container into which we will store our documents. A single data store may have multiple _indexes_ for the purposes of segregating different types of data (most queries are performed across all documents within *single* instance). + +--- +## Getting Started + +### Building The Micro Service + +After checking out the project, execute the following Maven command from the project's top level directory: + + > mvn clean install + +### Running The Micro Service Locally +To run the microservice in your local environment, execute the following Maven command from the project's top level directory: + + > mvn -P runAjsc + +### Running The Micro Service Within An Eclipse Environment +It is often extremely useful to be able to run a micro service from within Eclipse in order to set breakpoints and perform general debugging activities. + +For a good reference on how to launch any of the D2 micro services from within an Eclipse environment, refer to the following Confluence page: [Running An AJSC Container Within Eclipse](http://d2athenaconf:8090/confluence/pages/viewpage.action?pageId=1840887#DevelopingMicroserviceswithAT&T-RunninganAJSCContainerwithinEclipse) + +--- + +## Public Interfaces + +### Echo Service +The _Search Database Abstraction_ micro service supports the standard echo service to allow it to be 'pinged' to verify that the service is up and responding. + +The echo service is reachable via the following REST end point: + + http://{host}:9509/services/search-data-service/v1/jaxrsExample/jaxrs-services/echo/{input} + +### Indexes +The _Search Engine_ service supports simple creation and deletion of document indexes via the following REST API calls: + +##### Create Index + Method : POST + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/ + URL Params : index - The name of the index to be created. + Request Payload: + A document structure expressed as json. + + Response Payload: + {"url": "< resource location of the index >" + +##### Delete Index + Method : DELETE + URL : http://<host>:9509/services/search-data-service/v1/search/indexes/<index>/ + URL Params : index - The name of the index to be deleted. + Request Payload: + None + + +### Documents + +##### Create Document Without Specifying a Document Identifier +Documents can be created via a POST request with no document identifier specified. In this case the document store will generate an identifier to associate with the document. + + Method : POST + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/documents/ + URL Params : index - The name of the index to create the document in. + Request Payload: + Document contents expressed as a JSON object containing key/value pairs. + + Response Payload: + { "etag": "string", "url": "string" } + +##### Create or Update Document With a Specified Document Identifier +Documents can also be created via a PUT request which includes an identifier to associate with the document. The put endpoint is actually used for both creates and updates, where this is distinguished as follows: +* If the request header DOES NOT include a value in the If-Match field, then the request is assumed to be a document create. +* If the request header DOES contain a value in the If-Match field, then the request is assumed to be a document update. + + Method : PUT + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/documents<document id> + URL Params : index - The name of the index to create or update the document in. + document id - The identifier of the document to be created or updated. + Request Payload: + Document contents expressed as a JSON object containing key/value pairs. + + Response Payload: + { "etag": "string", "url": "string"} + +##### Delete a Document + + Method: : DELETE + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/documents<document id> + URL Params : index - The name of the index to remove the document from. + document id - the identifier of the document to be deleted. + Request Payload: + None. + +##### Retrieve a Document + + Method: : GET + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/documents<document id> + URL Params : index - The name of the index to retrieve the document from. + document id - the identifier of the document to be retrieved. + Request Payload: + None. + + +### Searching the Document Store +Search statements are passed to the _Search Data Service_ as a JSON object which is structured as follows: + +_Filters_ +* A "filter" stanza defines a set of queries to be run in _non-scoring-mode_ to reduce the document set to a smaller subset to be searched. +* The filter stanza is optional - omitting it implies that the query is _unfiltered_. +* This stanza is represented as a JSON object with the following structure: + + "filter": { + "all": [ { query }, { query },....{ query }], + "any": [ { query }, { query },....{ query }] + }, + +Where: +* the _all_ list defines a set of queryies such that ALL queries in the list must be satisfied for the document to pass the filter. +* the _any_ list defines a set of queryies such that ANY single query in the list must be satisfied for the document to pass the filter. + +_Queries_ +The following types of query statements are supported by the _Search Data Service_: + +_Term Query_: + +A term query attempts to match the literal value of a field, with no advanced parsing or analysis of the query string. This type of query is most appropriate for structured data like numbers, dates and enums, rather than full text fields. + + // Find documents where the specified field contains the supplied value + "match": { + "field": "value" + } + + // Find documents where the specified field DOES NOT contain the supplied value + "not-match": { + "field": "value" + } + +_Parsed Query_: + +Parsed queries apply a query parser to the supplied query string in order to determine the exact query to apply to the specified field. +The query string is parsed into a series of terms and operators, as described below: + +Terms may be any of the following: +* single words +* exact phrases, as denoted by enclosing the phrase in open and close quotations. Example: "this is my exact phrase" +* regular expressions, as denoted by wrapping the expressing in forward slash ( / ) character. Example: /joh?n(ath[oa]n)/ + +The supported operators are as follows: +* AND - Both terms to the left or right of the operator MUST be present +* OR - Either the term to the left or right of the operator MUST be present +* NOT - The term to the right of the operator MUST NOT be present. + + "parsed-query": { + "field": "fieldname", + "query-string": "string" + } + +_Range Query_: + + Range queries match fields whose term value falls within the specified numeric or date range. + Supported bounds operators include: + * gt - Greater than + * gte - Greater than or equal to + * lt - Less than + * lte - Less than or equal to + + "range": { + "field": "fieldname", + "operator": "value", + "operator": "value" + } + +##### Examples +The following snippet illustrates a search statement describing a filtered query which uses examples of all of the supported query types: + + { + "filter": { + "all": [{"range": {"field": "timestamp", "lte": "2016-12-01T00:00:00.558+03:00"}}], + "any": [ ] + }, + + "queries": [ + {"match": {"field": "name", "value": "Bob"}}, + {"parsed-query": {"field": "street-name", "query-string": "Main OR First"}}, + {"range": {"field": "street-number", "gt": 10, "lt": 50}} + ] + } + +##### REST Endpoint + + Method: : POST + URL : https://<host>:9509/services/search-data-service/v1/search/indexes/<index>/query + URL Params : index - The name of the index to apply the query to. + + Request Payload: + { + "filter": { + "all": [ { query }, { query },....{ query }], + "any": [ { query }, { query },....{ query }] + }, + + "queries": [ + { query }, + . + . + { query } + ] + } + +### Bulk Operations +Bulk operations allow the client to bundle a number of actions into a single REST request. +It is important to note that individual operations bundled into a bulk request are considered by the _Search Service_ to be completely independent operations. This has a few important consequences: +* No guarantees are made with respect to the order in which the individual operations will be processed by the document store. +* There is no implied transactionality between the operations. Individual operations my succeed or fail independently of one another, and it is entirely possible for the client to receive back a result set indicating a mix of success and failure results for the individual operations. + +##### Submit Bulk Request + Method : POST + URL : http://<host>:9509/services/search-data-service/v1/search/bulk/ + URL Params : NONE + Request Payload: + A json structure containing all of the bundled actions to be performed. + It must correspond to the following format: + [ + { "operation": {{<metaData>}, {<document>}, + { "operation": {{<metaData>}, {<document>}, + . + . + { "operation": {{<metaData>}, {<document>}, + ] + + Where, + operation - Is one of: "create", "update", or "delete" + + metaData - A structure containing meta-data associated with the individual operation to be performed. Valid fields include: + "url" - The resource identifier of the document to be operated on. + "etag" - Identifies the version of the document to be acted on. Required for "update" and "delete" operations. + + document - The document contents for "create" and "update" operations. + + Example Payload: + [ + {"create": {"metaData": {"url": "/services/search-data-service/v1/indexes/the-index/documents/1"}, "document": {"f1": "v1", "f2": "v2"}}}, + {"create": {"metaData": {"url": "/services/search-data-service/indexes/the-index/documents/2"}, "document": {"f1": "v1", "f2": "v2"}}}, + {"update": {"metaData": {"url": "/services/search-data-service/v1/search/indexes/the-index/documents/8", "etag": "1"}, "document": {"f1": "v1a", "f2": "v2a"}}}, + {"delete": {"metaData": {"url": "/services/search-data-service/v1/search/indexes/the-index/documents/99", "etag": "3"}}} + ] + + Response Payload: + The response body will contain an aggregation of the collective results as well as separate status codes for each of the operations in the request. + Example: + { + "total_operations": 4, + "total_success": 1, + "total_fails": 3, + "results": [ + {"operation": "create", "url": "/services/search-data-service/v1/search/indexes/the-index/documents/1", "etag": "1", "status-code": "201", "status-message": "OK"}, + {"operation": "create", "url": "/services/search-data-service/v1/search/indexes/the-index/documents/2", "etag": "1", "status-code": "201", "status-message": "OK"}, + {"operation": "update", "url": "/services/search-data-service/v1/search/indexes/the-index/documents/8", "etag": "2", "status-code": "200", "status-message": "OK"}, + {"operation": "delete", "url": "/services/search-data-service/v1/search/indexes/the-index/documents/2", "status-code": "200", "status-message": "OK"} + ] +}
\ No newline at end of file |