Age | Commit message (Collapse) | Author | Files | Lines |
|
Recursive SQL queries are incorrectly returning maxDepth + 1 levels
of fragments due to incorrect comparison in the WHERE clause.
This code is used by getDataNodes and queryDataNodes to
prefetch the descendants before converting to datanodes.
Prefetching extra descendants causes high memory usage.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
|
|
Issue-ID: CPS-1273
Change-Id: I5d562463b9a49abfe0436047a637857d10596fff
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
|
|
As Liquibase is used, JPA does not manage database definition.
Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long)
IDs. Other entities have INTEGER IDs.
- Change AnchorEntity/AnchorRepository ID type to Long
- Change YangResourceEntity/YangResourceRepository and
ModuleReferenceRepository ID to Integer
- Modify datatypes of foreign key references in liquibase changelog to
use correct types for dataspace_id, schema_set_id, yang_resource_id
- Update test liquibase changelog
Issue-ID: CPS-1676
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
|
|
- finalized moving of admin service interation test
(just some scenarios were missing)
- deleted old test class (code coverage in RI module dropped)
- line/branch coverage now 100/100 (was 97/91)
- small refactoring; renaming and removed unnecessary check
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
|
|
- Remove dataspace_id column from fragment table and drop associated
index, foreign key constraint and uniqueness constaint.
- Add uniqueness constraint using only fragment xpath and anchor_id.
- Add not-null constraint to anchor_id.
- Update code and test data to remove references to dataspace_id.
- Repopulate the dataspace_id in Liquibase rollback
(verified with manual testing of update/rollback).
Issue-ID: CPS-1677
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d
|
|
- all modules service interation test moved
- added some aditional scenarios to increase coverage
- removed unused method(s)
- remove javdoc on private methods
- some minor refactoring like replace 'var' with correct type
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2
|
|
- dropped ri module coverage to alow for move
- removed performance tests (already better tests in intergration module)
- added 'save' iue cases in new integration-test
- removed no longer used version for save single data node
(todo: remove old tests for similar functionality, will wait until new test cover same)
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4
|
|
- Added new method updateMultipleDataLeaves to perform Update on
multiple data nodes
- Deprecated singular method of update data node(updateDataLeaves)
- Refactored code where singular version was used
- Updated release notes
Issue-ID: CPS-1006
Signed-off-by: <as00745003@techmahindra.com>
Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc
|
|
|
|
|
|
'.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."
|
|
|
|
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex
# Updated the groovy files to add 'assert' and to replace deprecated method
# Removed deprecated 'storeDataNode' method.
Issue-ID: CPS-1594
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
|
|
FragmentEntity incorrectly has OneToOne relationship to AnchorEntity,
instead of ManyToOne. The reason this has not caused errors so far is
that Liquibase is being used to generate database definition, not JPA.
Issue-ID: CPS-863
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8d3c5dc19193a84fa59526de7f2cbab7ab20a8fc
|
|
- Add recursive SQL method to fetch descendants from queries.
This changes worst-case complexity from quadratic to linear,
resulting in extremely large performance increase for large
number of datanodes.
- Remove RegexQuickFind algorithm as it is no longer faster.
- Updated query performance test timings
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If5f0b54a88af1cb681006bbeca7043345dcdc8da
|
|
- Fix CPS-1580: Query Across All Anchors Does NOT Filter on Dataspace
- Fix CPS-1582: NullPointerException in queryDataNodesAcrossAnchors
- Improve performance of queryDataNodesAcrossAnchors
- Refactor queryDataNodes
- Refactor FragmentQueryBuilder
Issue-ID: CPS-1580
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8b83d4c580280d8719ab2ac33f3914f53e798774
|
|
This is a preparatory patch to make refactoring easier in CPS-1664 #3
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1f94fef137ae41f7dedc585d8314f13493885d1b
|
|
|
|
- add specialized regex
- fixed legacy issues with absolute paths (added ^ in regex's)
- split into 2 methods for normal queries and quickfind
- fixed order of private methods
Issue-ID: CPS-1671
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I785a230cd0e3e0359e3cbb0fa754a21b6e2be7ff
|
|
Include Anchor and Xpath in equals and hashCode methods for
FragmentEntity. (This also requires adding equals and hashCode for
AnchorEntity and DataspaceEntity.) The combination of dataspace,
anchor, and xpath uniquely identify a fragment/datanode.
This allows FragmentEntity objects returned from query across anchors
to be stored in Set collections.
Performance was observed to be unaffected by the change.
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2c7e3957e392af36f5230d08c9bbd550f44e7444
|
|
This bug causes queries using contains-condition to return wrong
results, when INCLUDE_ALL_DESCENDANTS option is set.
Issue-ID: CPS-1663
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If323e0e3b54d8c9a213dffda46897b2efb01fe2a
|
|
# Remove unused method "updateDataNodeAndDescendants"
# Remove unused production code
Issue-ID: CPS-1594
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Id909e9da3e9181c181dfdfe62f051451d8e488a2
|
|
SQL queries taking collection parameters currently create a seperate
query parameter for each collection element. There is a limit of
around 2^15 (32,768) query parameters.
Postgres DB natively supports array types, in which the whole array
is transmitted in binary as a single parameter. Changing queries to
use arrays removes the 32K limit on queries.
- Add support for Postgres arrays to queries
- Change repository methods to use arrays, and provide overloaded
versions taking collection parameters
- Update tests to reflect 32K limit being lifted
Issue-ID: CPS-1573
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I64f2aeaedbe54bfe12e3079cba0f2216759142c3
|
|
|
|
Issue-ID: CPS-1272
Change-Id: Ic81d1322cacc64a8752916324b801d02be47d34f
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
|
|
There is newer release of the dependency that provides JSONB support.
The newer version provides additional features, such as Postgres
arrays. It was renamed from hibernate-types to hypersistence-utils.
Issue-ID: CPS-1573
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iab556850041b592a8c4d515270dfe4bd9547957a
|
|
Remove dependency creating pipeline issue
Issue-ID: CPS-1626
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I64a6019a8196aa98dd3874d08f373c4fb9e3b8c9
|
|
- Add Liquibase change to cascade delete fragments
- Remove cascade delete logic from FragmentNativeRepository
- Remove unused method deleteFragmentEntity
Issue-ID: CPS-1617
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iecf4f3022b402e0328609ddc11a879c83e5d7f30
|
|
Issue-ID: CPS-1573
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I55312bdc5454ed9ca60441b3968d19a61028eb66
|
|
Issue-ID: CPS-1215
Change-Id: I91fdf5bddcc4fc12a8cf9dbce75f77c832c55871
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
|
|
Issue-ID: CPS-1603
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I85aca8bccc28d36aa4cefa0a32cb9bdf2156618a
|
|
Issue-ID: CPS-1396
Change-ID: I73f97f986a817d423f93a8d922dcd9647b1412ab
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
|
|
- Use Anchor object instead of names in processDataUpdatedEventAsync
- Use Anchor object instead of names in buildDataNodes
- Avoid unnecessary ArrayList copy in updateDataNodesAndDescendants
Issue-ID: CPS-1536
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I46936a655a3d151357e55b68e1c4161e07100e84
|
|
- Add SQL query that can fetch needed fragments to any given depth
- Update getFragmentEntities to use new query
- Remove now unused FragmentRepositoryMultiPathQuery
- Remove unused TempTableCreator method
- Result: getDataNodesForMultipleXpaths is up to 10 times faster
Issue-ID: CPS-1525
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I07cbc9da5ab994ce7e0c2b02d7ca05089f05dab0
|
|
|
|
|
|
- Remove unneeded calls to DataspaceRepository::getByName
- Remove unneeded calls to AnchorRepository::getByDataspaceAndName
- Refactor FragmentRepository
Issue-ID: CPS-1536
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2121f6247070ee4a7c000e06ec66a6278b758540
|
|
- Make deleteDataNode and deleteListDataNode call deleteDataNodes
- Add onlySupportListDeletion option to deleteDataNodes to support
original deleteListDataNode behaviour
- Allow delete root xpath in deleteDataNodes
- Fix incorrect use of PathParsingException in deleteDataNode
- Update performance tests timings
Issue-ID: CPS-1523
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I92c3c3ce606a5ab2cb8e6779d1ee0f9853529982
|
|
Issue-ID: CPS-1509
Change-Id: I2eded9a23b56f5263591c710c7a707cf281f93d7
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
|
|
Currently, the database is sequentially scanning when deleting from
schema_set_yang_resources table as there are no indexes on the table.
Adding an index doubles performance of NCMP CM-handle deregistration.
Issue-ID: CPS-1535
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia6ac7ef95d1662f49a77fc72112f1e29a67079ff
|
|
By default, temp tables are persisted until the end of the session.
This causes "Out of shared memory" errors in the database, which then
auto-vacuums the orphaned temp tables to free memory. To fix this, we
drop temp tables when the transaction finishes, using ON COMMIT DROP.
Issue-ID: CPS-1533
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iac47d85e9621531a4a95b99c4becc8cc95242721
|
|
Current implementation of NCMP handle de-registration relies on
DataNodeNotFoundException being thrown to report errors.
- Make deleteDataNodes throw DataNodeFoundExceptionBatch
- Update performance test timings
Issue-ID: CPS-1481
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib833bdb4a23d24e1784bdaf4e5e5e8a9acb41c54
|
|
List elements cannot be lists. Filtering list elements before trying
to delete lists by xpaths doubles performance when deleting list
elements (which doubles performance of CM handle de-registration).
Issue-ID: CPS-1511
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ieb6002212d006396d468f27f853708b5aa1e31f2
|
|
- Extract method getFragmentEntities from getDataNodes
- Remove unused code from getFragmentEntity
- Use bulk getFragmentEntities in updateDataNodesAndDescendants
- Update performance test timings
Issue-ID: CPS-1504
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Icd2c6a0e6009e152de43090cbc23a21349703da2
|
|
- Use SQL IN operator instead of temp table when deleting nodes
- Use Postgresql LIKE ANY array operator when deleting lists
- Update delete perf test timings
- Refactor adding cascade delete constraint
Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic90b867e7c71ec1981f05a9122322ece84dd8bde
|
|
|
|
|
|
- Modified the GET Data Node API so it returns all the data nodes when
xpath set to root "/"
- Fragment Repository now returns a collection of Fragment Entities
- Instead of returning only the first Fragment Entity now all fragment
entities are returned when xpath is set to root
- The Fragemnt Entities are further processed to a Collection of Data
Nodes. As opposed to singular Data Node in current implementation.
- Finally the DataRestController also returns a Collection of Data
Nodes when xpath is set to root and valid data is present
- Response body changed from JSON object to JSON Array.
- Exception handling for invalid xpath and non-existing xpath is now
done separately at persistence layer.
- Refactored code against CPS-1422
- Deprecated getDataNode method from Service and Persistence layer
- Modified V1 of Get Data Node API to use the getDataNodes method and
get the first data node from the collection returned.
- Modified NCMP to use getDataNodes method
- NCMP still does not support multiple data nodes. It retrieves the
first data node from the collection returned by getDataNodes
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e
|
|
- exclude childFragments from toString method
Issue-ID: CPS-1493
Change-Id: I3d0c7ef601f06ed36265d88cadb6fb0ab0cb6ea2
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
|
|
- Bulk delete anchors and datanodes associated with schemasets.
Improves de-registration performance by approx 10%
Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714
|