Age | Commit message (Collapse) | Author | Files | Lines |
|
and dataProducerIdentifier
- Created a new repo. service for fragment table that executes a native
sql query to find first ready cm handle id based on moduleset tag and
then returns list of module references.
- Exposed a new interface into module service that is used by
module sync service to get list of midule refs by module set tag.
Issue-ID: CPS-2353
Change-Id: I438dbd1ed37c1ff4e15f792e93a095aa604120bc
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
|
|
There is an issue in a method deleteDataNodes(dataspace, anchors)
where it doesn't send data update events. This is because it fetches
anchors using a method getAnchors(dataspace, schemaSetNames), when it
needs to fetch anchors by name: getAnchors(dataspace, anchorNames)
Changes:
- Rename getAnchors method using schemasets to getAnchorsBySchemaSetNames
- Add a method getAnchors(dataspace, anchorNames)
- Update test of deleteDataNodes method using getAnchors so it checks
that data update events are sent
Issue-ID: CPS-2254
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I55fe853f0a9278a66a5724bf4cf2723b0e2fbc8b
|
|
This greatly improves performance of module searches by eliminating
unneeded SQL queries via Hibernate lazy fetching.
Issue-ID: CPS-2190
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie9e65017d0027366456f1741cc37b10679317b25
|
|
The check for existing modules is extremely slow, being dependent on
the file sizes of the Yang modules being queries.
The validation check is also completely unnecessary, since it simply
ignores any exceptions and returns an empty set, which the main logic
does implicitly. (The Dataspace check is also not needed, as the NCMP
dataspace is guaranteed to exist.)
Issue-ID: CPS-2190
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I329c1f8aac3f50bda0333e6c9c686f47af2e009f
|
|
- uplifted spring boot from 3.1.2 to 3.2.4
- removed redundant dependencies
- upgraded relevant dependencies
- use of @Bean(some_name) is give a specific bean name
- using @Bean(some_name) with @Qualifier so on the injection point to specify which bean to actually inject, this is used as we deal with multiple beans of the same type.
Issue-ID: CPS-2101
Change-Id: I64d5bf567b9eb03fb39034a46528d0160b332715
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
|
|
- reverting back to spring 3.1.2
- manual changes to revert as other apps not fully ready for the
delivery
Issue-ID: CPS-2133
Change-Id: I670b67916fa4a5d4e5bced6548b7fcd5ee12e855
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
|
|
- Upgraded hypersistence-utils-hibernate-60 to hypersistence-utils-hibernate-63
- Introduced @Qualifier annotations where needed
Issue-ID: CPS-2101
Change-Id: I8303592ed91584deb6ca5cd4e49800210b6ea989
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
|
|
Issue-ID: CPS-2087
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I62b452f34f08a32a6d80671c7af560fbceec1285
|
|
- add query parameters: module-name and revision to OpenAPI
- extend the controller method to hande the new parameters
- add the new method stack to the service layer
- extend the SQL query to support the new parameters
- add unit and integration testwares
Issue-ID: CPS-1135
Change-Id: I089ad2ad71effb58ac0ba809e9f441d6cdb59c4f
Signed-off-by: halil.cakal <halil.cakal@est.tech>
|
|
-Removed Lambda from CompositeStateUtils
-Restructured CpsDataPersistenceServiceImpl to more logical layout
Issue-ID: CPS-1238
Change-Id: I09096dab474230d04b15c204cbea95a2f1285366
Signed-off-by: seanbeirne <sean.beirne@est.tech>
|
|
|
|
- Implement fragment ID sequence generator with allocation size of 100
- Enable write batching for insert and update, with batch size of 100
- Write performance is over 2x faster
Issue-ID: CPS-1795
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7ebd0f834abb57c593603e2cd3fafc7e3425ab1e
|
|
CpsDataService::saveListElementsBatch method is not needed as
saveListElements supports saving multiple list elements in a single
operation. This both simplifies implementation and greatly improves
performance when saving list elements, as the Yang parser need only
run once for the whole batch.
- Change InventoryPersistence to save CM-handles in batches of 100
using existing CpsDataService::saveListElements method.
- Remove not needed CpsDataService::saveListElementBatch.
Issue-ID: CPS-2019
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8b74dda2917e094d064b42f2c0e4d57029b90395
|
|
This change fixes a few issues related to JSON encoding of
FragmentEntity attributes (data leaves). This significantly
improves update performance in cases of partial updates.
- Normalize JSON and order leaves by name when comparing
data leaves during update operations.
- Update performance test timings.
Issue-ID: CPS-2018
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia764a353bf96c05758827845e1358745247ee237
|
|
- Incl. integration test
- Prod code is changed to use new method that update yand module.
- updated module resource to return multiple modules.
Issue-ID: CPS-1806
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I751d9393ce78a3be9daeaff6d0252738c02115e0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
|
|
- Added new schema to upgrade model into component.
- Modified Ncmp rest input mapper to add upgradedCmHandles attributes.
- Modified cm handle state mapper to add new lock reason.
- Added new method to parse and upgrade Cm handles in DmiRegistration.
- YANG data converter is modified to add "module-set-tag" atribute.
- Cm handle new query method is added for cps path without appending
ancestor.
- Modified setCompositeStateForRetry to add lock reason.
- New lock reason category is added.
- Existing module sync service is modified to upgrade the model
"syncAndCreateOrUpgradeSchemaSetAndAnchor".
- Sync util method "getModuleSyncFailedCmHandles" to modified to add
another lock reason "LOCKED_MISBEHAVING".
- Added new attribute "UpgradedCmHandles" into DmiPluginRegistration and DmiPluginRegistrationResponse.
- New attribute "moduleSetTag" is added into NcmpServiceCmHandle.
- New model "UpgradedCmHandles" is added.
- New method "updateSchemaSetWithExistingModules" is added into cps
module service to update cm handle with exsting model.
- Code coverage is reducced to 96 percentage that would be addressed and
pushed into new patch.
Issue-ID: CPS-1798
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I540acb404e38fc434de87a0d959bfde710a18b03
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
|
|
- convert normal java strings to text block - feature from java 15
- handling callback for legacy events. Once the deprecated methods are
removed then this would not be needed.
Issue-ID: CPS-89
Change-Id: I9b537ebee2c284c8109e6cc6c8961a430ee9599d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
|
|
Issue-ID: CPS-1789
Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Iee5c514ccdba36a387f83948d8a3ba26e6b1de5b
|
|
This reverts commit 9693ec51cf6526082f0ad0c3ad208d144cbbb163.
Reason for revert: Bug fix delivery
Change-Id: I73bdc1528192c662983b0bbef73b10b6d612a719
Signed-off-by: egernug <gerard.nugent@est.tech>
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
|
|
Issue-ID: CPS-1789
Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I932fca8a50cedf2ae0604401c5e2b27896e4d449
|
|
- Exposed an interface to update anchor by schema set name.
- New interface is implemented into RI model.
- New native query is exposed to update id with given schema set name.
- A new integration test is written to test new interface into cps core.
Issue-ID: CPS-1800
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ibf44712e11b53cb6673b04b9e3fd864321c90839
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
|
|
The error is caused by the fetch-descendants code using
ResultSet.getLong which returns a 'long' primitive instead
of a 'Long' object. Thus a parent ID of 'NULL' becomes '0',
which causes an error during update. To preserve the NULL
value, ResultSet.getObject must be used.
Issue-ID: CPS-1841
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ida6837a04954cd3c23f0f2faabd7d0712d8ee19d
|
|
Issue-ID: CPS-475
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7623a8301fd4567bf96c605eb26bf6c02b1d3c76
|
|
|
|
Add pagination query parameters in query across all anchors API
pagination parameters (pageIndex and pageSize) are optional
default is to query all fragments
each pageSize represents number of records(number of anchors)
TotalRecords is returned in response header to find number of pages.
- If pagination option is provided in request then query number of
anchors equal to pageSize. pageIndex is used for setting offset.
- return number of records(one anchor per record) as per pagesize
and pageSize
Issue-ID: CPS-1605
Change-ID: I73f97f986a817d423f93a8d922dcd9647b2504bc
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
|
|
- getDataNodes can now retrieve list data nodes.
Issue-ID: CPS-1696
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I320a368d6cb73599d3f7c13fe9b8ab7d0cc69470
|
|
Apostrophe is not currently handled correctly, and having apostrophe in
the xpath will lead to various errors.
For example, normalizing this xpath works:
/path[@name="I'm quoted"] -> /path[@name='I\'m quoted']
However the resulting xpath will throw a PathParsingException if parsed!
(Thus path normalization is not idempotent.)
- Use '' for escaping apostrophe in single quoted leaf value,
to comply with XPath standard (and use "" for escaping in ").
- Use Liquibase to make existing data comply with new rules.
- Leaf values in data leaves are now unescaped, e.g. "I'm quoted"
- Quoting is now consistent for leaf/text/contains conditions.
Issue-ID: CPS-1769
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0
|
|
Presently, a query using the same leaf name twice will fail:
//books[@price > 10 and @price < 20]
It is caused by storing data leaves in a Map.
This is fixed by storing data leaves in a List<DataLeaf>.
Issue-ID: CPS-1779
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie6990ea5e622cf37e986b720a0a07fb69ce4f03b
|
|
Issue-ID: CPS-1774
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10
|
|
This fixes issues with special characters for CPS-500, CPS-1756,
CPS-1758, and CPS-1760. It also improves query performance.
- use SQL LIKE instead of regex in Cps Path queries
Issue-ID: CPS-1763
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5c179882bfba71d3b009c60059e9073f46227e7d
|
|
If '%' and '_' are used in the contains-condition of a CpsPath query,
incorrect results will be returned. For example:
/bookstore/categories[contains(@code, "%")]
Special characters in the contains-condition value must be escaped.
Issue-ID: CPS-1762
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2fdd5a26433d510cd7d6af5b734a6779b537d63d
|
|
Avoid fetching descendants during batchUpdateDataLeaves,
as descendants are not needed:
- Remove prefetch descendants step from getFragmentEntities;
- Explicitly prefetch descendants in operations requiring it;
- Update performance tests (5x faster for batch update).
Issue-ID: CPS-1675
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7
|
|
Avoid using Spring Data "interface projection" in FragmentRepository.
The use of FragmentExtract in FragmentRepository is causing an
overhead of around 5 kilobytes per fragment, which is leading to
abnormally high memory usage when queries return a large number of
nodes. For example, around 250MB of additional memory is needlessly
used when fetching 50,000 datanodes.
- Remove FragmentExtract interface and FragmentEntityArranger class.
- Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper
to fetch FragmentEntity descendants in a single SQL query.
- Many CpsDataService operations have memory reductions:
- queryDataNodes
- getDataNodesForMultipleXpaths
- updateDataNodesAndDescendants
- updateNodeLeaves
- and any NCMP methods using the above.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
|
|
|
|
Recursive SQL queries are incorrectly returning maxDepth + 1 levels
of fragments due to incorrect comparison in the WHERE clause.
This code is used by getDataNodes and queryDataNodes to
prefetch the descendants before converting to datanodes.
Prefetching extra descendants causes high memory usage.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
|
|
- Moved all Data Intergation Test
- Added/improved some module Moudle Servcie test too
- Removed some unused (impossible) code-paths in prod code
- Replaced null's with meaningful strings (in edge case scenarios)
- Replaced @Shared with static (shorter) in some test
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
|
|
Issue-ID: CPS-1273
Change-Id: I5d562463b9a49abfe0436047a637857d10596fff
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
|
|
As Liquibase is used, JPA does not manage database definition.
Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long)
IDs. Other entities have INTEGER IDs.
- Change AnchorEntity/AnchorRepository ID type to Long
- Change YangResourceEntity/YangResourceRepository and
ModuleReferenceRepository ID to Integer
- Modify datatypes of foreign key references in liquibase changelog to
use correct types for dataspace_id, schema_set_id, yang_resource_id
- Update test liquibase changelog
Issue-ID: CPS-1676
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
|
|
- finalized moving of admin service interation test
(just some scenarios were missing)
- deleted old test class (code coverage in RI module dropped)
- line/branch coverage now 100/100 (was 97/91)
- small refactoring; renaming and removed unnecessary check
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
|
|
- Remove dataspace_id column from fragment table and drop associated
index, foreign key constraint and uniqueness constaint.
- Add uniqueness constraint using only fragment xpath and anchor_id.
- Add not-null constraint to anchor_id.
- Update code and test data to remove references to dataspace_id.
- Repopulate the dataspace_id in Liquibase rollback
(verified with manual testing of update/rollback).
Issue-ID: CPS-1677
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d
|
|
- all modules service interation test moved
- added some aditional scenarios to increase coverage
- removed unused method(s)
- remove javdoc on private methods
- some minor refactoring like replace 'var' with correct type
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2
|
|
- dropped ri module coverage to alow for move
- removed performance tests (already better tests in intergration module)
- added 'save' iue cases in new integration-test
- removed no longer used version for save single data node
(todo: remove old tests for similar functionality, will wait until new test cover same)
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4
|
|
- Added new method updateMultipleDataLeaves to perform Update on
multiple data nodes
- Deprecated singular method of update data node(updateDataLeaves)
- Refactored code where singular version was used
- Updated release notes
Issue-ID: CPS-1006
Signed-off-by: <as00745003@techmahindra.com>
Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc
|
|
|
|
|
|
'.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."
|
|
|
|
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex
# Updated the groovy files to add 'assert' and to replace deprecated method
# Removed deprecated 'storeDataNode' method.
Issue-ID: CPS-1594
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
|
|
FragmentEntity incorrectly has OneToOne relationship to AnchorEntity,
instead of ManyToOne. The reason this has not caused errors so far is
that Liquibase is being used to generate database definition, not JPA.
Issue-ID: CPS-863
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8d3c5dc19193a84fa59526de7f2cbab7ab20a8fc
|
|
- Add recursive SQL method to fetch descendants from queries.
This changes worst-case complexity from quadratic to linear,
resulting in extremely large performance increase for large
number of datanodes.
- Remove RegexQuickFind algorithm as it is no longer faster.
- Updated query performance test timings
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If5f0b54a88af1cb681006bbeca7043345dcdc8da
|