FieldsTests
Test generation mechanism to verify the field extractions of an Add-on
TestTemplates
Includes the test scenarios to check the field extractions of an Add-on.
- class standard_lib.fields_tests.test_templates.FieldTestTemplates[source]
Bases:
object
Test templates to test the knowledge objects of an App
- test_datamodels(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_datamodels, record_property, caplog)[source]
Test case to check if correct datamodels are assigned to the event.
This test case checks if tags assigned to the event match assigned datamodel and also checks if there is no additional wrongly assigned datamodel.
- Parameters
splunk_search_util (splunksplwrapper.SearchUtil.SearchUtil) – object that helps to search on Splunk.
splunk_ingest_data (fixture) – Unused but required to ensure data was ingested before running test
splunk_setup (fixture) – Unused but required to ensure that test environment was set up before running test
splunk_searchtime_fields_datamodels (fixture) – pytest parameters to test.
record_property (fixture) – pytest fixture to document facts of test cases.
caplog (fixture) – fixture to capture logs.
- test_eventtype(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_eventtypes, record_property, caplog)[source]
Tests if all eventtypes in eventtypes.conf are generated in Splunk.
- Parameters
splunk_search_util (fixture) – Fixture to create a simple connection to Splunk via SplunkSDK
splunk_searchtime_fields_eventtypes (fixture) – Fixture containing list of eventtypes
record_property (fixture) – Used to add user properties to test report
caplog (fixture) – Access and control log capturing
- Returns
Asserts whether test case passes or fails.
- test_props_fields(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_positive, record_property)[source]
This test case checks that a field value has the expected values.
- Parameters
splunk_search_util (SearchUtil) – Object that helps to search on Splunk.
splunk_searchtime_fields_positive (fixture) – Test for stanza field.
record_property (fixture) – Document facts of test cases.
caplog (fixture) – fixture to capture logs.
- test_props_fields_no_dash_not_empty(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_negative, record_property)[source]
This test case checks negative scenario for the field value.
- Parameters
splunk_search_util (SearchUtil) – Object that helps to search on Splunk.
splunk_searchtime_fields_negative (fixture) – Test for stanza field.
record_property (fixture) – Document facts of test cases.
caplog (fixture) – fixture to capture logs.
- test_requirements_fields(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_requirements, record_property)[source]
This test case checks that a field value has the expected values.
- Parameters
splunk_search_util (SearchUtil) – Object that helps to search on Splunk.
splunk_searchtime_fields_positive (fixture) – Test for stanza field.
record_property (fixture) – Document facts of test cases.
caplog (fixture) – fixture to capture logs.
- test_savedsearches(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_savedsearches, record_property, caplog)[source]
Tests if all savedsearches in savedsearches.conf are being executed properly to generate proper results.
- Parameters
splunk_search_util (fixture) – Fixture to create a simple connection to Splunk via SplunkSDK
splunk_searchtime_fields_savedsearches (fixture) – Fixture containing list of savedsearches
record_property (fixture) – Used to add user properties to test report
caplog (fixture) – Access and control log capturing
- Returns
Asserts whether test case passes or fails.
- test_splunk_internal_errors(splunk_search_util, ignore_internal_errors, record_property, caplog)[source]
This test case checks that there are not any unexpected internal errors
- Parameters
splunk_search_util (SearchUtil) – Object that helps to search on Splunk.
ignore_internal_errors (fixture) – common list of errors to be ignored
record_property (fixture) – Document facts of test cases.
caplog (fixture) – fixture to capture logs.
- test_tags(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_tags, record_property, caplog)[source]
Test case to check tags mentioned in tags.conf
This test case checks if a tag is assigned to the event if enabled, and also checks that a tag is not assigned to the event if disabled.
- Parameters
splunk_search_util (splunksplwrapper.SearchUtil.SearchUtil) – object that helps to search on Splunk.
splunk_searchtime_fields_tags (fixture) – pytest parameters to test.
record_property (fixture) – pytest fixture to document facts of test cases.
caplog (fixture) – fixture to capture logs.
TestGenerator
Module include class to generate the test cases to test the knowledge objects of an Add-on.
- class standard_lib.fields_tests.test_generator.FieldTestGenerator(app_path, tokenized_events, field_bank=None)[source]
Bases:
object
Generates test cases to test the knowledge objects of an Add-on.
Provides the pytest parameters to the test templates.
Supports field_bank: List of fields with patterns and expected values which should be tested for the Add-on.
- Parameters
app_path (str) – Path of the app package
field_bank (str) – Path of the fields Json file
- generate_eventtype_tests()[source]
Generate test case for eventtypes
- Yields
pytest.params for the test templates
- generate_field_tests(is_positive)[source]
Generate test case for fields
- Parameters
is_positive (bool) – Test type to generate
- Yields
pytest.params for the test templates
- generate_requirements_datamodels_tests()[source]
Generate test case for tags
- Yields
pytest.params for the test templates
- generate_requirements_tests()[source]
Generate test cases for fields defined for datamodel These function generates tests previously covered by requirement tests
- Yields
pytest.params for the test templates
- generate_savedsearches_tests()[source]
Generate test case for savedsearches
- Yields
pytest.params for the test templates
- generate_tag_tests()[source]
Generate test case for tags
- Yields
pytest.params for the test templates
- generate_tests(fixture)[source]
Generate the test cases based on the fixture provided supported fixtures:
splunk_searchtime_fields_positive
splunk_searchtime_fields_negative
splunk_searchtime_fields_tags
splunk_searchtime_fields_eventtypes
splunk_searchtime_fields_savedsearches
splunk_searchtime_fields_requirements
- Parameters
fixture (str) – fixture name
sample_generator (SampleGenerator) – sample objects generator
store_events (bool) – variable to define if events should be stored
FieldBank
To enhance the test cases while verifying the field extractions.
- class standard_lib.fields_tests.field_bank.FieldBank[source]
Bases:
object
- Supports field_bank: List of fields with patterns and expected
values which should be tested for the Add-on.
Steps to use:
Create a json file with the list of fields.
Example:
{ "stanza_name": [ # Key should be stanza_name { "name": "action", "condition": "| regex _raw=\"success\"" "validity": "action=if(action=\"unknown\", null(), action)" "expected_values": ["success", "failure"] "negative_values": ["", "-", "unknown"] } ] }
Parameter
Description
condition
A filtering SPL command.
validity
An EVAL statement. Filter out invalid value of a field
expected_fields
List of expected_fields
negative_fields
The list of values the field should not have
supported stanza_type:
source
sourcetype
Provide path of the json file with –field-bank=path parameter