aboutsummaryrefslogtreecommitdiffstats
path: root/ice_validator/tests/conftest.py
AgeCommit message (Collapse)AuthorFilesLines
2019-12-04[VVP] Support pluggable data sources for preload dataLovett, Trevor1-11/+21
Change-Id: Ia7fcfa25203a93eac93381f472e3ba1e6c11235f Issue-ID: VVP-339 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-10-29[VVP] Make Preload Generation more reslient to failuresLovett, Trevor1-1/+5
Ensure that errors during preload do not prevent validation report creation Issue-ID: VVP-335 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Change-Id: I0cfd411b6a034422e349b96c462e68a4f0ceb876
2019-10-16[VVP] Allow any_of and all_of in categories decoratorLovett, Trevor1-6/+2
Change-Id: I76ff9ebd7108d2ef19a3f167c5a253a3ce5f07f7 Issue-ID: VVP-331 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-10-16[VVP] Support any_of in categories decoratorLovett, Trevor1-22/+22
If specified, then the test will be selected if any of the passed categories from the command line are in the tests categories. This is different than the default behavior where all categories must match Change-Id: Iee08556d6c07eac2663ff2ff2e89bcd7a18cd392 Issue-ID: VVP-330 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-09-06Enforce black code format via pre-commit hookLovett, Trevor1-2/+2
Issue-ID: VVP-203 Change-Id: If0de5b4bc1be0c9514decea1f4ff6f5ec79dc41b Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-08-27[VVP] Generated completed preload from env filesLovett, Trevor1-1/+19
User can supply an optional directory containing .env files and/or CSAR VSP which can be used to generate populated preloads in the requested format. The nested directories can be used to create sub-environments that inherit their settings from the parent directories. Optionally, values can be specified in a defaults.yaml and they will be used if that value is not defined in the .env file. This is useful if the parameter name and value will be the same in all modules. Issue-ID: VVP-278 Change-Id: Icd9846c63463537793db908be8ce5dba13c4bda3 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-08-20Merge "[VVP] Adding preload generation functionality"Steven Wright1-14/+13
2019-08-16[VVP] Adding preload generation functionalityLovett, Trevor1-14/+13
preload.py discovers and loads implementations of AbstractPreloadGenerator from any module on sys.path prefixed with preload_* Initial support is provided for VNF-API and GR-API. The templates will provide a guide for users to provide their values. Known limitations: - No support for Contrail. Preload will be created, but contrail parameters will be skipped. This will be addressed in the future. Issue-ID: VVP-227 Signed-off-by: stark, steven <steven.stark@att.com> Change-Id: I081d50ac379062fbf1bffebd687e920220d32571 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-08-13VNFRQTS - Update the rst table contentHagop Bozawglanian1-2/+1
Issue-ID: VNFRQTS-651 Signed-off-by: Hagop Bozawglanian <hagop.bozawglanian@att.com> Change-Id: If4f535004e78cf3abaf0113eda3cbfa4e562fba1
2019-07-24[VVP] Adding bandit security scans and fixesLovett, Trevor1-1/+1
Issue-ID: VVP-244 Change-Id: Ia782f4cc7bf5a379ff8cdcce96cd2e7235998345 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-07-19[VVP] Properly escape error messages in HTML outputLovett, Trevor1-2/+7
Issue-ID: VVP-159 Change-Id: I741e5abbc80fbcbbb5d7c0332e13217305532bfd Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-07-01[VVP] Bug fix in conftest.pyLokur, Manish1-1/+1
Relatived Path function failed if the value is an empty string. Issue-ID: VVP-229 Signed-off-by: Lokur, Manish <manish.shivakumar.lokur@att.com> Change-Id: I811234cfc7c3c438bd8934568a3ccbb02bac013a
2019-06-14[VVP] Allow multi-line error messages on error reportsLovett, Trevor1-5/+5
Issue-ID: VVP-225 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com> Change-Id: Iebee13d29f8025b3dd5c147f2024a484789464f7
2019-06-06[VVP] Improved readability of reportsLovett, Trevor1-61/+33
- Removed resolution steps from all reports - Added error number to all reports - Only show base file names - Add auto-filter to excel reports - Shift test ID to right-most column of Excel - Removed raw output from Excel and CSV Change-Id: Ic022b0a14d53fbeab591def631b8743228e1d5c0 Issue-ID: VVP-184 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-06-05[VVP] Removed yamllint libraryLovett, Trevor1-5/+12
Change-Id: If2c26b8650e8aba0a183d0cf47b1b3744dbfaf95 Issue-ID: VVP-201 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-06-05[VVP] Performance Enhancements (report generation and test collection)Lovett, Trevor1-92/+38
Issue-ID: VVP-222 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com> Change-Id: I9dd3506097cb0d6e69bd1434b2d42f6d1965023b
2019-05-30[VVP] Exempt output section from nested get_param checkLovett, Trevor1-3/+3
Change-Id: Ib87b0fb1c70c5689fc7687a21a27ea60934096b1 Issue-ID: VVP-220 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-05-23[VVP] adding heat template-validate teststark, steven1-0/+3
Issue-ID: VVP-218 Signed-off-by: stark, steven <steven.stark@att.com> Change-Id: If88f9b4b620aaffe61ead3b6f7d5c74dcfd14cba
2019-04-16[VVP] Reports show test file and case nowLovett, Trevor1-3/+10
Change-Id: I017c1830b61b186f52ba2fe29db03dd41cb68b26 Issue-ID: VVP-192 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-04-08[VVP] adding base test failure message to reportstark, steven1-1/+4
Change-Id: Ibdb75750624d1482d87466b17aa2486f794a5fd6 Issue-ID: VVP-189 Signed-off-by: stark, steven <steven.stark@att.com>
2019-03-20VVP - Fixing script to include static validationsBozawglanian, Hagop (hb755d)1-57/+104
Change-Id: I0b210d788439e856c8229430abf7264768843867 Issue-ID: VVP-187 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2019-03-07[VVP] Ensure report.json always producedLovett, Trevor1-4/+5
Also fixed error where rst.csv did not use the requested output directory if the default is overridden Change-Id: Ife7bead97e5172ee115a7d07bd4ac917046f3b31 Issue-ID: VVP-185 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-03-04[VVP] Handle missing requirement in needs.jsonLovett, Trevor1-2/+2
Change-Id: Id22840cf58fdc0063f925a55672de58a9753b59c Issue-ID: VVP-182 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-21[VVP] JSON report shows relative paths for "files"Lovett, Trevor1-2/+8
Template path is also normalized to Unix style path separators. Change-Id: I2961e09300ed5ab8487bb21b8a302c7b55594474 Issue-ID: VVP-169 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-08[VVP] Fixing internal network checkLovett, Trevor1-4/+4
The test_neutron_port_internal_network test is intended to check that internal networks referenced in **incremental** modules are defined and exported in the base template. The current version of the test applies this check to nested modules as well causing false violations to be flagged. Issue-ID: VVP-163 Change-Id: I42cc81acf0cc2b6827ae8cf4e9a7faa4af91f9b4 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-06[VVP] Fix errors on tests with no fixturesLovett, Trevor1-5/+7
Change-Id: I4ad24979128902b4e0e502c76d6503e2876b960a Issue-ID: VVP-161 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-01-31[VVP] Creating table for RST ingestionBozawglanian, Hagop (hb755d)1-0/+50
Issue-ID: VVP-155 Change-Id: I16b9c54121302b78d05cd6ee40a499a599a35f17 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2019-01-30[VVP] Add new columns to traceability reportLovett, Trevor1-46/+51
Added the following columns: * Keyword - MUST, MUST NOT, etc. * Validation Mode - contents of :validation_mode: from vnfrqts * Is Testable - TRUE if requirement is testable based on keyword and validation mode, FALSE otherwise Also fixed issue following issues: * Traceability and mapping error reports were not going to the configured output directory if a custom value was provided * Removed duplicate rows from traceability report * Traceability report shows all Heat requirements Change-Id: I339b42e9e662f1681b78d1a49bd7e71e87cae21a Issue-ID: VVP-148 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-29[VVP] Removing dynamic download of needs.jsonLovett, Trevor1-23/+1
Also resolving some flake8 failures Change-Id: I2de20975106de0308bc20c2a8944e3552f6f7cca Issue-ID: VVP-140 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-29[VVP] stand alone tool, script updatesstark, steven1-67/+93
1) Validation script updates: Various bug fixes and script enhancements New tests for: R-18683 R-94669 R-304011 R-01455 R-86476 R-708564 R-85734 R-18683 R-94669 R-304011 R-01455 R-86476 R-86476 R-589037 Removed tests for: R-75202 R-62954 R-22441 R-49177 R-16576 R-86237 R-44491 R-70757 R-01896 R-26124 2) Stand Alone tool introduced 3) Added optional "categories" capability Change-Id: I193cd5c267750791d97b350e91fc36faa72d8d5f Issue-ID: VVP-143 Signed-off-by: stark, steven <steven.stark@att.com>
2019-01-22[VVP-132] Add new JSON output reportLovett, Trevor1-171/+583
Added a new JSON output format with complete results of the test run. This can be produced using the existing --report-format option and passing the json value to it. Additionally users can now customize where output files are written using the --output-directory option. Issue-ID: VVP-132 Change-Id: I7b8a01bd3d5179ecb75b1db588da3bb057b9b573 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2018-12-17[VVP] updating validation scripts in dublinstark, steven1-1/+1
- adding backlog of new validation scripts for dublin - updating existing tests - removing outdated tests Issue-ID: VVP-123 Change-Id: Ib8260889ac957c1dd28d8ede450fc8edc6fb0ec0 Signed-off-by: stark, steven <steven.stark@att.com>
2018-11-06[VVP] validation scripts warning functionstark, steven1-1/+1
Change-Id: Ib639b5eb271e48ded9d5c8bb8857b5e12260deed Issue-ID: VVP-116 Signed-off-by: stark, steven <steven.stark@att.com>
2018-09-17VVP - Fixing script for traceability csvBozawglanian, Hagop (hb755d)1-4/+13
Fixing try catch to account for service outage. Issue-ID: VNFRQTS-236 Change-Id: Ic7ef03215fa4ed6319d7b5e986828bc83c17465a Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2018-09-13[VVP] udpating scripts for casablancastark, steven1-142/+201
adding new "infrastructure" scripts addresses bugs VVP-100, VVP-101, VVP-102 adding base tests updating tests where arguments have changed Adds traceability for task VVP-92 Change-Id: I067d8e80934403039e66fbc9fc93766587f67b4e Issue-ID: VVP-80 Signed-off-by: stark, steven <ss820f@att.com>
2018-09-13VVP - Fixing script for traceability csvBozawglanian, Hagop (hb755d)1-1/+8
Fixed the conftest.py script so traceability wouldn't have repeats and it pulls the newest JSON it runs. Issue-ID: VNFRQTS-236 Change-Id: Id33965b64f939bed0a6bbb7ab3a9e692829b4603 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2018-08-30[VVP] pytest --self-test xfailstark, steven1-3/+7
validation scripts pytest --self-test "xfail" being skipped Issue-ID: VVP-108 Change-Id: I1bdb146db3de78b393315499491b3ad65597213c Signed-off-by: stark, steven <ss820f@att.com>
2018-08-22[VVP] Added new three new reportsLovett, Trevor (tl2972)1-45/+494
Resolves VVP-81 by introducing three new reports to the validation script output. Each report is written to the output directory under ice_validator. * mapping_errors.csv - each row represents an issue where a test is mapped to a requirement that does not exist in the requirement.json file * traceability.csv - Shows a mapping between all test cases and requirements. Rows will also be shown where a requirement does not have a tests, and where tests are not mapped to a requirement * report.{html|csv|xlsx} - Formatted error report that shows each failure that was detected and maps it to a the file name, requirement(s) violated if known, possible resolution steps, error message, and raw output from pytest. The report can be written in html, csv, or Excel format Other minor changes: * Replaced smart quotes with regular quotes to resolve issues with py3 * Updated copyright of modified files * Fixed Flake-8 errors Issue-ID: VVP-81 Change-Id: I659836cb20262ae44652c03639281a817dd051ae Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2018-07-09[VVP] Track Requirements to Test Casesstark, steven1-0/+11
Change-Id: I35c7630d0aabb970e67323e1e895a57cbcb23fb2 Issue-ID: VVP-79 Signed-off-by: stark, steven <ss820f@att.com>
2017-09-05Commit seed code for validation-scriptsMichael F. Lamb1-0/+149
This imports the initial seed code for validation scripts. These files were imported from a tarball with the SHA1SUM ce2ae49c82546b987c8ad2f68ac43d94b4934818. From the contents of the tarball, some errant .pyc files were removed, and a .gitignore file was added. The result matches exactly the contents of the origin private repository at hash 88c656e. Change-Id: I97d19b11495e116890ca6577e83037b0934519cc Issue-Id: VVP-11 Signed-off-by: Michael F. Lamb <mike@datagrok.org>