aboutsummaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/functions/python-executor/src
AgeCommit message (Collapse)AuthorFilesLines
2023-06-14Convert scripts to python3Dan Timoney2-4/+4
Convert python2 scripts to python3 Issue-ID: CCSDK-3906 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I68b82108c5cd51a99735f65f964b8ebc16181c1b
2022-07-18Aligned attributes of CDS componentsLukasz Rajewski2-1/+31
Issue-ID: CCSDK-3698 Signed-off-by: Lukasz Rajewski <lukasz.rajewski@t-mobile.pl> Change-Id: Ie5c78a9a347373b5faf588627e42138806d69c0c
2022-06-07CCSDK-3684 ComponentRemotePYthonExecutor eliminate recursive callOleg Mitsura1-2/+2
Issue-ID: CCSDK-3684 prepareEnv (mistakenly) had a recursive call, but no decement. Simplified it in favour of the inner/outter flag instead of retry counter as in Frankfurt release. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I5d7f623d6e206a844d33d95a15f85758da3a6681
2021-11-30CCSDK-3531 improve cmd-exec returned err msgOleg Mitsura1-1/+5
Issue-ID: CCSDK-3531 Improve end-user CBA's Python scripts ability to return error messages on failues. See JIRA story for examples: https://jira.onap.org/browse/CCSDK-3531 adds functions send_response_err_msg(err_message) and send_response_err_msg_and_exit(ret_err_msg, code=1) Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ideda64dd108f987c3c4515c70daf9f7b6d2b1f2f
2021-10-20Add missing code for Metrics and PV/PVC eliminationJozsef Csongvai1-3/+0
Earlier patch introducing MeterRegistry to AbstractComponentFunction was not initializing the lateinit property and causing NPE. Also add additional code to handle compatibility issues after the introduction of PV/PVC elminiation for CommandExecutor. This allows blueprintsprocessor to communicate with earlier versions of command- executor which still use the shared pvc. Issue-ID: CCSDK-3471 Change-Id: I84a04601c4fe09c5f3a06664ce877800a30531f1 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-09-21Metrics on command-executor blueprints processor sideMichael Pruss1-25/+69
Issue-ID: CCSDK-3471 Signed-off-by: Michael Pruss <michprus@amdocs.com> Change-Id: I871a032ff5c9ebfd408aaa28a7d24185dee7c953
2021-07-28Revert "Renaming Files having BluePrint to have Blueprint"Jozsef Csongvai13-87/+87
The renaming in CCSDK-3098 caused breaking changes to the grpc api and compile issues for kotlin scripts. Issue-ID: CCSDK-3385 Change-Id: I0d745cb858371678eabcb2284671c1fd76a1ab6d Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-07-27Removed redundant timeout handling for executeCommandJulien Fontaine1-19/+3
executeCommand was called from a GlobalScope.async coroutine to handle executeCommand timeout. Execution timeout is already handled on the gRPC side, there's no need to handle this timeout by wrapping the gRPC call with a coroutine timeout. When a lot of long running processes in executeCommand were running, it was causing process to queue on BP side to get their executeCommand executed because all the capacity of couroutines were being used by some long running process. This was causing a delay on the execution of prepareEnv and executeCommand and would even some time make those process timeout because of component timeout. Issue-ID: CCSDK-3386 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: If3aef4b6b65006e874525436bf3f6d1b1a3d8c9a
2021-03-19Merge "Refactoring to enable on_failure for imperative workflow"KAPIL SINGAL2-5/+4
2021-03-18ComponentRemotePythonExecutor prepareEnv bugOleg Mitsura1-37/+11
Issue-ID: CCSDK-3086 commit 1: orig commit commit 2: update retry counter commit 3: bug fix for packages being null should still trigger creation of python environment Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Id9e7024df8c604c719828211cce31d6f0452330b
2021-03-17Refactoring to enable on_failure for imperative workflowJozsef Csongvai2-5/+4
BlueprintError needs to associate errors with the steps in which they occurred in order for imperative workflow to handle on_failure properly. Made stepName more accessible and corrected places where stepName was assigned to nodeTemplateName. Issue-ID: CCSDK-3219 Change-Id: I7e5805745c63558cff6be533e1b99c32ad06c3db Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-01-26PV/PVC eliminationOleg Mitsura2-18/+114
Issue-ID: CCSDK-3086 1. initial commit 2. fix accidental paste / rebased cleaned up unneeded validation call in cmd-exec upload Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ife5460c5be59aa8d8592d82099b27c507b08c6c6
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL13-89/+89
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)10-140/+164
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-09-02Adding parent pom referenceSingal, Kapil (ks220y)2-2/+2
Adding parent pom to cba-assembly-descriptor as it needs to inherit repository from parent so maven-deploy-plugin works Issue-ID: CCSDK-2720 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I44ce3721f280555b81475580ca47539ae83d07ca
2020-08-24Add request IDs in command-executor logJulien Fontaine1-0/+5
* Update gRPC interface to handle SubRequestID and Originator ID * Update command-executor log format to handle request IDs in the log statement * Add SubRequest ID and Originator ID in BP log statement Issue-ID: CCSDK-2687 Change-Id: If261bb0912df5800ed2fc65371c5ee784d6a45fb Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-08-04cds-exec shouldn't return "N/A" as outputOleg Mitsura1-8/+8
Issue-ID: CCSDK-2603 * execute-command-logs" should always be a list (in some cases, we had a string causing possible parsing problems) * response-data should be JSON and "N/A" not acceptible. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ie6fceb6e1f4c86bfcdeb7671bacd74b909628096
2020-07-23Disable jython testsJozsef Csongvai1-1/+3
BlueprintJythonServiceTest started failing in Jenkins for unknown reasons. Seeing as jython is deprecated and should be removed, its not worth tracing the cause. Issue-ID: CCSDK-2602 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I495409a18fc324e00c373c99bcc6da803666240a
2020-07-14cmd-exec payload should be payload, not err msg.Oleg Mitsura1-67/+50
Issue-ID: CCSDK-2549 fixes: 1. CMD-exec returns payload as JSON object. There was some regression where we overwrite returned JSON with errMsg. 2. adds 100ms delta to executionTimeout, as was in our downstream version (want to guarantee that we get grpc timeout exception on BP side rather than coroutine w/o waiting extra) 3. setNodeOutputErrors should not have ability to set execution status to success. rev1. initial import rev2. reworked the calls to setNodeOutputErrors: addError was brought out of it as we can set the node execution error based on prepare.env or execution. rev3. did not mean to add BlueprintRuntimeService change Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I6d509df5ae51598f33ab9f0ea53806d653cf79c0
2020-07-08cmd-exec server-side timeout.Oleg Mitsura1-1/+1
Issue-ID: CCSDK-2535 Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I897678a5a8a23503a878f2d3eb836ba4597a6e6e
2020-07-02Command Executor : Invalid response_data when executed script failsJulien Fontaine1-18/+18
* Modified command exec returned value in case of failure during execution. It now prints the response_data defined by the user * Modified truncation method of the gRPC returned object to use ByteSize() to get the exact sizxe consumed within the buffer Issue-ID: CCSDK-2501 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ie1db8db265623b5137ab3946ff4e3abda1c54a78
2020-06-26cmdexec should return status=FAILURE on errorsOleg Mitsura1-40/+47
Issue-ID: CCSDK-2445 In certain cases, status message is returned on error instead of status=FAILURE. rev 2: reuse same msg rev 3: past elalto, compoment level timeout is implementation.timeout Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I2651e5706b7e62894c5e071dc1563fb1b9dd67e1
2020-06-18Missing note of timeout differences.Oleg Mitsura1-3/+8
rev1. initial commit rev2. spacing (silly klint!) Issue-ID: CCSDK-2445 Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ie8cc62c89e212efbf5240304856b79c173c88abb
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine2-25/+83
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-09Merge "Improving CMD-exec err msgs/handling."Dan Timoney1-16/+51
2020-04-08Improving CMD-exec err msgs/handling.Oleg Mitsura1-16/+51
Issue-ID: CCSDK-2049 rev1. initial commit rev2. added better logging/passing errors. rev3. bugfixes rev4. bugfix for CommandExecutorServer.prepareEnv return status was still returning bool. (was caught only during a defective mariadb pip package install!) Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ifbddaeb0e15857f42b121af10e30e29dec5c20b2
2020-04-08Merge "Refactoring BP Code with ErrorCatalog"Dan Timoney1-0/+6
2020-04-06Refactoring BP Code with ErrorCatalogBrinda Santh1-0/+6
Issue-ID: CCSDK-2124 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ief468a56f9e7b3ef86c357965aa7b15f0a4cfa22
2020-03-30execution timeout not respected...Oleg Mitsura2-9/+27
Issue-ID: CCSDK-2012 rev1: initial commit rev2: reworked + added grpc deadline rev3: wrong err msg.. rev4: timeouts in millis rev5: timeout defaulting was after logging... Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I02eb414904cc5dfcf51ff7b7552dafe54857ed1e (cherry picked from commit 3f56fcfec7d14e7351280b9e2a1d220638626693)
2020-03-27Change relative paths in Python Executor testMichal Jagiello1-2/+2
Add "babel/compat-data" package to Designer Client devDependencies Remove package-lock.json from .gitignore Issue-ID: CCSDK-2192 Signed-off-by: Michal Jagiello <michal.jagiello@t-mobile.pl> Change-Id: I2676412dc4994fa5314cd411ce64d19df2790e5a
2019-12-30Runtime services for Relationship Type and TemplatesBrinda Santh2-2/+2
Added and improved blueprint runtime service property resolution api. Relationship template CBA sample Performance improvement on Artifact resolutions. Added connection config DSL for DB, GRPC connections. Added starter relationship connection config models. Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I73e1deed708dc924352867ddd1d1b2271c8ea106
2019-12-11Metadata for name, version, tags and typeBrinda Santh1-8/+7
Mandate Tosca.meta template name, version, type and tags. Auto copy metadata from Tosca.meta to ServiceTemplate definitions. Optimize Blueprint context and runtime creation from file path. Removed attached CBA zip file in test repository dirs Issue-ID: CCSDK-1992 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I5d9d7a4599234a38d431328dbd9b74bd831e0115
2019-12-11Separate jython implementation and dependencies.Brinda Santh10-10/+484
Refactored cds python capabilities to python-executor function, so that it will easy to disable and deprecation This change is not jython depreciation, just separating out Issue-ID: CCSDK-1865 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I3727d72d7195c49d670c30a6fc4245f5a8599e3a
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)8-121/+193
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-19Changes return value from boolean to typical process return code;Serge Simard1-23/+29
Also ensure that if environment preparation does not succeed the processing fails. Issue-ID: CCSDK-1855 Signed-off-by: Serge Simard <serge@agilitae.com> Change-Id: Iab2ec8b44a0e9e7edef68d0d5c13860eedc5be67 Signed-off-by: Serge Simard <serge@agilitae.com>
2019-10-24Improve Remote Python Executor error handling and allow for structured responseSerge Simard2-9/+43
Issue-ID: CCSDK-1855 Signed-off-by: Serge Simard <serge@agilitae.com> Change-Id: I924918fb06a5a0f118772dedaa41c536c7eca9e7 Signed-off-by: Serge Simard <serge@agilitae.com>
2019-09-18Optimizing Imports and Formatting codeSingal, Kapil (ks220y)1-4/+4
No Business logic change, no code change.. just optimizing imports and reformatting code Issue-ID: CCSDK-1744 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I1fb2ed10525a6f392f3c0d963c2ed03cfe8dec7e
2019-09-18Refactoring Netconf NetconfMessageUtilsSingal, Kapil (ks220y)1-5/+4
Appending New Line before and after request body within doWrapRpc and removing new line is child API's which forms the request body Issue-ID: CCSDK-1743 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic58566e24a21c71d545c45fb95a667c17de2fe66
2019-09-16Changing line separator type to LFSINGAL, KAPIL (ks220y)2-66/+66
Changed Line Separator for all files using find . -type f -not -name .* -exec dos2unix {} \; Issue-ID: CCSDK-1733 Signed-off-by: SINGAL, KAPIL (ks220y) <ks220y@att.com> Change-Id: I2efa3276c35c0a0159c6d4debe21c0bd0b4a7b85
2019-08-09Add resource source template DSL.Brinda Santh1-38/+40
Change-Id: If68ce2e944700cb19380d34ff3b466a9c8633472 Issue-ID: CCSDK-1577 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-08-09Add property definition type DSL.Brinda Santh2-6/+6
Change-Id: I83a3d1391c60cd426236b55d0d399e273e7c15dc Issue-ID: CCSDK-1577 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-08-01Add remote python executor DSL propertiesBrinda Santh3-0/+220
Change-Id: I7888ac302da2a51ca34c531b7553d04af13786ac Issue-ID: CCSDK-1380 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-23Fix for python executor unit test case failure.Brinda Santh2-48/+49
Change-Id: I193e2a092ff8cefe48419031096e1336e5c84fc1 Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-17Enable dynamic remote python executorAlexis de Talhouët2-13/+20
Change-Id: I77a60de87acec862ce47256557a1223fefe82a12 Issue-ID: CCSDK-1497 Signed-off-by: Alexis de Talhouët <adetalhouet89@gmail.com>
2019-07-10Python executor parameters sorted again.Oleg Mitsura1-10/+12
Issue-ID: CCSDK-1468 Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: If8eb50fc2e8dfa084e486b02ace163eee1bfac77
2019-07-08Merge "Enforce resolutionKey or resourceId/resourceType"Dan Timoney1-37/+41
2019-07-08Merge "Remote Python executor unescapes script parameter values"Dan Timoney1-2/+14
2019-07-04Enforce resolutionKey or resourceId/resourceTypeAlexis de Talhouët1-37/+41
There are three existing ways to perform the resolution: either we don't store the results at all, whether for resource or template either we store using the resolution key. The combination of blueprintName, blueprintVersion, artifactName and resolutionKey has to be unique. If it is re-used, it is considered as a new attempt for that specific resolution request, and process will only try to resolve resources not marked at SUCCESS in the database. either we store using the resourceId and resourceType. As previous point, the combination of blueprintName, blueprintVersion, artifactName and resolutionKey has to be unique. If it is re-used, it is considered as a new attempt for that specific resolution request, and process will only try to resolve resources not marked at SUCCESS in the database. TBD: add uni tests Issue-ID: CCSDK-1423 Change-Id: I6b7198453cf0002edfa7a0c9ea3179555211b5dc Signed-off-by: Alexis de Talhouët <adetalhouet89@gmail.com>
2019-06-25python-executor will provide output line-by-lineOleg Mitsura2-6/+7
* python-executor STDOUT contains cleaner logs * return status has been reworked to return a list of lines Issue-ID: CCSDK-1404 Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ib17495e92a136ff4f950bb6e049246a9e100eb7d
2019-06-21Remote Python executor unescapes script parameter valuesSerge Simard1-2/+14
Issue-ID: CCSDK-1426 Signed-off-by: Serge Simard <serge@agilitae.com> Change-Id: I919e30b76bb472b4e0aeeb2294f0b3087a655efd