aboutsummaryrefslogtreecommitdiffstats
path: root/ice_validator/tests
AgeCommit message (Collapse)AuthorFilesLines
2019-04-15[VVP] Updated network param validations per reqtsLovett, Trevor25-578/+878
Change-Id: Idb0e051d6063cd94b733ed68093989d527592c9f Issue-ID: VVP-193 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-04-12Merge "[VVP] updating v6 regex for fip/ip and fip/subnet"Steven Wright2-6/+6
2019-04-11Merge "[VVP] adding base test failure message to report"Steven Wright1-1/+4
2019-04-11[VVP] updating v6 regex for fip/ip and fip/subnetstark, steven2-6/+6
Change-Id: Ia7acc252d6a1429ec0c73f720bb1308dd70178b5 Issue-ID: VVP-188 Signed-off-by: stark, steven <steven.stark@att.com>
2019-04-09[VVP] restricting ip index check to resourcesstark, steven7-204/+73
Restricting check for ip parameter indices to only those parameters that are used in OS::Neutron::Port Issue-ID: VVP-190 Change-Id: Ifaf1a40ee47bcaf83657fe58a33e229c4126b4e8 Signed-off-by: stark, steven <steven.stark@att.com>
2019-04-08[VVP] adding base test failure message to reportstark, steven1-1/+4
Change-Id: Ibdb75750624d1482d87466b17aa2486f794a5fd6 Issue-ID: VVP-189 Signed-off-by: stark, steven <steven.stark@att.com>
2019-03-21Merge "VVP - Fixing script to include static validations"Steven Wright1-57/+104
2019-03-21[VVP] updating OS::Neutron::Port parameter testsstark, steven21-2972/+316
AAP, FIP/IP, FIP/Subnet parameters use same validation (ports.py) REGEX moved to specific tests Added tests for R-159016, R-717227, R-805572 Removed mapping for R-98748 Updated env file test to exclude internal AAP params Change-Id: Iddde8ee42400b8df9be49b8c15718b451101b37b Issue-ID: VVP-181 Signed-off-by: stark, steven <steven.stark@att.com>
2019-03-20VVP - Fixing script to include static validationsBozawglanian, Hagop (hb755d)1-57/+104
Change-Id: I0b210d788439e856c8229430abf7264768843867 Issue-ID: VVP-187 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2019-03-07[VVP] Ensure report.json always producedLovett, Trevor1-4/+5
Also fixed error where rst.csv did not use the requested output directory if the default is overridden Change-Id: Ife7bead97e5172ee115a7d07bd4ac917046f3b31 Issue-ID: VVP-185 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-03-05[VVP] Allow AZ CDL params in nested templatesLovett, Trevor1-0/+3
Issue-ID: VVP-183 Change-Id: Iad69ca26eb54a2200186ddf1b8d410e1248d1bcd Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-03-04[VVP] Handle missing requirement in needs.jsonLovett, Trevor1-2/+2
Change-Id: Id22840cf58fdc0063f925a55672de58a9753b59c Issue-ID: VVP-182 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-28Merge "[VVP-171] R-84123 and R-76160 test update"Steven Wright7-152/+138
2019-02-27[VVP-171] R-84123 and R-76160 test updatestark, steven7-152/+138
R-84123 and R-76160 test_neutron_port_internal_fixed_ips should only be checking incremental modules Change-Id: I7ef3cd4aa2b01273c2592f0b17751c5cb79c002b Issue-ID: VVP-171 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-26[VVP] R-95430 modified in vnfrqtsstark, steven3-12/+0
Removing test for R-95430 because requirement was changed from MUST to MAY Change-Id: I3430c54d319561b26a98380146e365a3b3b90f3c Issue-ID: VVP-178 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-21[VVP] JSON report shows relative paths for "files"Lovett, Trevor1-2/+8
Template path is also normalized to Unix style path separators. Change-Id: I2961e09300ed5ab8487bb21b8a302c7b55594474 Issue-ID: VVP-169 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-20[VVP] Fix error msg on test_network_formatLovett, Trevor1-1/+1
Change-Id: I945f7f56c9457ec49f10cebe0fc45bbc1a2e7129 Issue-ID: VVP-174 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-18Merge "[VVP] remove tests for 88540, 20947, 54458"Steven Wright2-33/+0
2019-02-15[VVP] remove tests for 88540, 20947, 54458stark, steven2-33/+0
the reqs 88540, 20947, 54458 were removed from read the docs, the corresponding tests for these should also be removed. Change-Id: I1912321e6c68df34b41c32b62d0ae2a25873b959 Issue-ID: VVP-168 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-15[VVP] ports.py check port typestark, steven1-4/+7
get_invalid_ip_addresses needs to be enhanced to check the neutron port type (internal or external) before validating the IP parameter format. The IP parameter format should not be validated against both internal and external, it should only be validated against the formats for the ports type. Change-Id: Ic8b66b83729f6ea5d268d55816e03dcbbfec12fd Issue-ID: VVP-167 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-12VVP - Fix nested templateHagop Bozawglanian1-2/+2
Change-Id: I814804ccffaeb20cc758428c62a1d789868d7ff8 Issue-ID: VVP-166 Signed-off-by: Hagop Bozawglanian <hagop.bozawglanian@att.com>
2019-02-12[VVP] Allow lists in test_availability_zoneLovett, Trevor2-53/+4
Change-Id: I590830db83b0c7dd5df2ac59992a7771483eab03 Issue-ID: VVP-165 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-08[VVP] Improved message on test_base_template_nameLovett, Trevor3-15/+134
Change-Id: I5974ed3b8ff6e20ab747b3e96af37f84a133ecbb Issue-ID: VVP-141 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-02-08[VVP] Fixing internal network checkLovett, Trevor5-211/+113
The test_neutron_port_internal_network test is intended to check that internal networks referenced in **incremental** modules are defined and exported in the base template. The current version of the test applies this check to nested modules as well causing false violations to be flagged. Issue-ID: VVP-163 Change-Id: I42cc81acf0cc2b6827ae8cf4e9a7faa4af91f9b4 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-06[VVP] Fix errors on tests with no fixturesLovett, Trevor1-5/+7
Change-Id: I4ad24979128902b4e0e502c76d6503e2876b960a Issue-ID: VVP-161 Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2019-02-06Merge "[VVP] revert nested resource section"Steven Wright3-6/+6
2019-02-05[VVP] Removing invalid testsLovett, Trevor16-1768/+0
Change-Id: I9336fa834b08748cabb5ac49495f1591f15939f8 Issue-ID: VVP-150 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-02-05[VVP] revert nested resource sectionstark, steven3-6/+6
- R-40551 specifies nested templates MAY or MAY NOT contain a resource section - remove check for get_param in allowed_address_pairs, no requirement states get_param MUST be used Change-Id: If133b4ec4d39c25a95c77482b1743562e2ae74a8 Issue-ID: VVP-158 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-04[VVP] enhance tests for nested templatesstark, steven42-129/+129
Change-Id: I17047fa3df555fafcdae566fad50b8c19ddca16f Issue-ID: VVP-158 Signed-off-by: stark, steven <steven.stark@att.com>
2019-02-01[VVP] pseudo-parameters nested parametersstark, steven2-3/+10
Change-Id: Iff7630a6f62fa8b83c80048d6efc12d187037412 Issue-ID: VVP-157 Signed-off-by: stark, steven <steven.stark@att.com>
2019-01-31Merge "[VVP] Creating table for RST ingestion"Steven Wright1-0/+50
2019-01-31[VVP] Creating table for RST ingestionBozawglanian, Hagop (hb755d)1-0/+50
Issue-ID: VVP-155 Change-Id: I16b9c54121302b78d05cd6ee40a499a599a35f17 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2019-01-30[VVP] Removing unnecessary trademark linesLovett, Trevor436-578/+164
Issue-ID: VVP-152 Change-Id: I6b978768324f69adba50c75a187509c4a7605341 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-30[VVP] Add new columns to traceability reportLovett, Trevor1-46/+51
Added the following columns: * Keyword - MUST, MUST NOT, etc. * Validation Mode - contents of :validation_mode: from vnfrqts * Is Testable - TRUE if requirement is testable based on keyword and validation mode, FALSE otherwise Also fixed issue following issues: * Traceability and mapping error reports were not going to the configured output directory if a custom value was provided * Removed duplicate rows from traceability report * Traceability report shows all Heat requirements Change-Id: I339b42e9e662f1681b78d1a49bd7e71e87cae21a Issue-ID: VVP-148 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-29[VVP] Fixing mapping issuesLovett, Trevor1-35/+3
Updated to the latest version of the needs.json R-52530 was removed and replaced with R-511776 which is already covered by a different test. test_07_nested_template_in_same_directory: the only remaining purpose of this method was to test for the existence of the nested file, but this is already covered by test_valid_nesting I mapped the unmapped tests in test_inital_configuration to to the requirement that states the templates must be valid Heat Change-Id: Ia71d774df8873ebb4182f44eb66b108f9ff24d20 Issue-ID: VVP-142 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-29[VVP] Removing dynamic download of needs.jsonLovett, Trevor1-23/+1
Also resolving some flake8 failures Change-Id: I2de20975106de0308bc20c2a8944e3552f6f7cca Issue-ID: VVP-140 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com>
2019-01-29[VVP] stand alone tool, script updatesstark, steven74-1566/+2817
1) Validation script updates: Various bug fixes and script enhancements New tests for: R-18683 R-94669 R-304011 R-01455 R-86476 R-708564 R-85734 R-18683 R-94669 R-304011 R-01455 R-86476 R-86476 R-589037 Removed tests for: R-75202 R-62954 R-22441 R-49177 R-16576 R-86237 R-44491 R-70757 R-01896 R-26124 2) Stand Alone tool introduced 3) Added optional "categories" capability Change-Id: I193cd5c267750791d97b350e91fc36faa72d8d5f Issue-ID: VVP-143 Signed-off-by: stark, steven <steven.stark@att.com>
2019-01-22[VVP-132] Add new JSON output reportLovett, Trevor1-171/+583
Added a new JSON output format with complete results of the test run. This can be produced using the existing --report-format option and passing the json value to it. Additionally users can now customize where output files are written using the --output-directory option. Issue-ID: VVP-132 Change-Id: I7b8a01bd3d5179ecb75b1db588da3bb057b9b573 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2018-12-17[VVP] updating validation scripts in dublinstark, steven267-3838/+14928
- adding backlog of new validation scripts for dublin - updating existing tests - removing outdated tests Issue-ID: VVP-123 Change-Id: Ib8260889ac957c1dd28d8ede450fc8edc6fb0ec0 Signed-off-by: stark, steven <steven.stark@att.com>
2018-11-06[VVP] validation scripts warning functionstark, steven1-1/+1
Change-Id: Ib639b5eb271e48ded9d5c8bb8857b5e12260deed Issue-ID: VVP-116 Signed-off-by: stark, steven <steven.stark@att.com>
2018-09-17VVP - Fixing script for traceability csvBozawglanian, Hagop (hb755d)1-4/+13
Fixing try catch to account for service outage. Issue-ID: VNFRQTS-236 Change-Id: Ic7ef03215fa4ed6319d7b5e986828bc83c17465a Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2018-09-14[VVP] udpating scripts for casablanca 2stark, steven86-1212/+3197
adding scripts updates adding docrators for sections 5.2.6 - end closes out VVP 80 Change-Id: Iecac476949aa3ae3982f8c77671869583eb313d6 Issue-ID: VVP-80 Signed-off-by: stark, steven <ss820f@att.com>
2018-09-13[VVP] udpating scripts for casablancastark, steven35-2041/+983
adding new "infrastructure" scripts addresses bugs VVP-100, VVP-101, VVP-102 adding base tests updating tests where arguments have changed Adds traceability for task VVP-92 Change-Id: I067d8e80934403039e66fbc9fc93766587f67b4e Issue-ID: VVP-80 Signed-off-by: stark, steven <ss820f@att.com>
2018-09-13VVP - Fixing script for traceability csvBozawglanian, Hagop (hb755d)1-1/+8
Fixed the conftest.py script so traceability wouldn't have repeats and it pulls the newest JSON it runs. Issue-ID: VNFRQTS-236 Change-Id: Id33965b64f939bed0a6bbb7ab3a9e692829b4603 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2018-08-30[VVP] pytest --self-test xfailstark, steven1-3/+7
validation scripts pytest --self-test "xfail" being skipped Issue-ID: VVP-108 Change-Id: I1bdb146db3de78b393315499491b3ad65597213c Signed-off-by: stark, steven <ss820f@att.com>
2018-08-22[VVP] Added new three new reportsLovett, Trevor (tl2972)285-1613/+2083
Resolves VVP-81 by introducing three new reports to the validation script output. Each report is written to the output directory under ice_validator. * mapping_errors.csv - each row represents an issue where a test is mapped to a requirement that does not exist in the requirement.json file * traceability.csv - Shows a mapping between all test cases and requirements. Rows will also be shown where a requirement does not have a tests, and where tests are not mapped to a requirement * report.{html|csv|xlsx} - Formatted error report that shows each failure that was detected and maps it to a the file name, requirement(s) violated if known, possible resolution steps, error message, and raw output from pytest. The report can be written in html, csv, or Excel format Other minor changes: * Replaced smart quotes with regular quotes to resolve issues with py3 * Updated copyright of modified files * Fixed Flake-8 errors Issue-ID: VVP-81 Change-Id: I659836cb20262ae44652c03639281a817dd051ae Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
2018-08-09[VVP] create new validation scriptsstark, steven39-148/+1910
- create validation scripts for R-04747, R-30804, R-11690, R-46839, R-98407 - other small script enhancements Change-Id: I12c42a43c83e5cb4217895c19fd0beef928dc955 Issue-ID: VVP-103 Signed-off-by: stark, steven <ss820f@att.com>
2018-07-31[VVP] Updating error reporting for helpersBozawglanian, Hagop (hb755d)3-56/+85
Change-Id: Ib93b6ff452613b2ee1f2804d958f4c6f66d6dee4 Issue-ID: VVP-80 Signed-off-by: Bozawglanian, Hagop (hb755d) <hb755d@att.com>
2018-07-27[VVP] decorators for 5.2.7 and 5.2.8stark, steven2-0/+4
update decorator for validation script corresponding to 5.2.8 of VNFRTQS update decorator for validation script corresponding to 5.2.7 of VNFRTQS Change-Id: Ibb5bb639908538c8ce07e2f2913a0fb447c27746 Issue-ID: VVP-94 Signed-off-by: stark, steven <ss820f@att.com>
2018-07-26[VVP] add decorators for heat 5.2.5.6stark, steven3-0/+14
Change-Id: I422fad86438380092254e46e3940f32fe6dc732f Issue-ID: VVP-92 Signed-off-by: stark, steven <ss820f@att.com>