FieldsTests

Test generation mechanism to verify the field extractions of an Add-on

TestTemplates

Includes the test scenarios to check the field extractions of an Add-on.

class standard_lib.fields_tests.test_templates.FieldTestTemplates[source]

Bases: object

Test templates to test the knowledge objects of an App

test_datamodels(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_datamodels, record_property, caplog)[source]

Test case to check if correct datamodels are assigned to the event.

This test case checks if tags assigned to the event match assigned datamodel and also checks if there is no additional wrongly assigned datamodel.

Parameters
  • splunk_search_util (splunksplwrapper.SearchUtil.SearchUtil) – object that helps to search on Splunk.

  • splunk_ingest_data (fixture) – Unused but required to ensure data was ingested before running test

  • splunk_setup (fixture) – Unused but required to ensure that test environment was set up before running test

  • splunk_searchtime_fields_datamodels (fixture) – pytest parameters to test.

  • record_property (fixture) – pytest fixture to document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

test_eventtype(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_eventtypes, record_property, caplog)[source]

Tests if all eventtypes in eventtypes.conf are generated in Splunk.

Parameters
  • splunk_search_util (fixture) – Fixture to create a simple connection to Splunk via SplunkSDK

  • splunk_searchtime_fields_eventtypes (fixture) – Fixture containing list of eventtypes

  • record_property (fixture) – Used to add user properties to test report

  • caplog (fixture) – Access and control log capturing

Returns

Asserts whether test case passes or fails.

test_props_fields(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_positive, record_property)[source]

This test case checks that a field value has the expected values.

Parameters
  • splunk_search_util (SearchUtil) – Object that helps to search on Splunk.

  • splunk_searchtime_fields_positive (fixture) – Test for stanza field.

  • record_property (fixture) – Document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

test_props_fields_no_dash_not_empty(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_negative, record_property)[source]

This test case checks negative scenario for the field value.

Parameters
  • splunk_search_util (SearchUtil) – Object that helps to search on Splunk.

  • splunk_searchtime_fields_negative (fixture) – Test for stanza field.

  • record_property (fixture) – Document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

test_requirements_fields(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_requirements, record_property)[source]

This test case checks that a field value has the expected values.

Parameters
  • splunk_search_util (SearchUtil) – Object that helps to search on Splunk.

  • splunk_searchtime_fields_positive (fixture) – Test for stanza field.

  • record_property (fixture) – Document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

test_savedsearches(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_savedsearches, record_property, caplog)[source]

Tests if all savedsearches in savedsearches.conf are being executed properly to generate proper results.

Parameters
  • splunk_search_util (fixture) – Fixture to create a simple connection to Splunk via SplunkSDK

  • splunk_searchtime_fields_savedsearches (fixture) – Fixture containing list of savedsearches

  • record_property (fixture) – Used to add user properties to test report

  • caplog (fixture) – Access and control log capturing

Returns

Asserts whether test case passes or fails.

test_splunk_internal_errors(splunk_search_util, ignore_internal_errors, record_property, caplog)[source]

This test case checks that there are not any unexpected internal errors

Parameters
  • splunk_search_util (SearchUtil) – Object that helps to search on Splunk.

  • ignore_internal_errors (fixture) – common list of errors to be ignored

  • record_property (fixture) – Document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

test_tags(splunk_search_util, splunk_ingest_data, splunk_setup, splunk_searchtime_fields_tags, record_property, caplog)[source]

Test case to check tags mentioned in tags.conf

This test case checks if a tag is assigned to the event if enabled, and also checks that a tag is not assigned to the event if disabled.

Parameters
  • splunk_search_util (splunksplwrapper.SearchUtil.SearchUtil) – object that helps to search on Splunk.

  • splunk_searchtime_fields_tags (fixture) – pytest parameters to test.

  • record_property (fixture) – pytest fixture to document facts of test cases.

  • caplog (fixture) – fixture to capture logs.

TestGenerator

Module include class to generate the test cases to test the knowledge objects of an Add-on.

class standard_lib.fields_tests.test_generator.FieldTestGenerator(app_path, tokenized_events, field_bank=None)[source]

Bases: object

Generates test cases to test the knowledge objects of an Add-on.

  • Provides the pytest parameters to the test templates.

  • Supports field_bank: List of fields with patterns and expected values which should be tested for the Add-on.

Parameters
  • app_path (str) – Path of the app package

  • field_bank (str) – Path of the fields Json file

generate_eventtype_tests()[source]

Generate test case for eventtypes

Yields

pytest.params for the test templates

generate_field_tests(is_positive)[source]

Generate test case for fields

Parameters

is_positive (bool) – Test type to generate

Yields

pytest.params for the test templates

generate_requirements_datamodels_tests()[source]

Generate test case for tags

Yields

pytest.params for the test templates

generate_requirements_tests()[source]

Generate test cases for fields defined for datamodel These function generates tests previously covered by requirement tests

Yields

pytest.params for the test templates

generate_savedsearches_tests()[source]

Generate test case for savedsearches

Yields

pytest.params for the test templates

generate_tag_tests()[source]

Generate test case for tags

Yields

pytest.params for the test templates

generate_tests(fixture)[source]

Generate the test cases based on the fixture provided supported fixtures:

  • splunk_searchtime_fields_positive

  • splunk_searchtime_fields_negative

  • splunk_searchtime_fields_tags

  • splunk_searchtime_fields_eventtypes

  • splunk_searchtime_fields_savedsearches

  • splunk_searchtime_fields_requirements

Parameters
  • fixture (str) – fixture name

  • sample_generator (SampleGenerator) – sample objects generator

  • store_events (bool) – variable to define if events should be stored

FieldBank

To enhance the test cases while verifying the field extractions.

class standard_lib.fields_tests.field_bank.FieldBank[source]

Bases: object

Supports field_bank: List of fields with patterns and expected

values which should be tested for the Add-on.

Steps to use:

  1. Create a json file with the list of fields.

    Example:

    {
        "stanza_name": [    # Key should be stanza_name
            {
                "name": "action",
                "condition": "| regex _raw=\"success\""
                "validity": "action=if(action=\"unknown\", null(), action)"
                "expected_values": ["success", "failure"]
                "negative_values": ["", "-", "unknown"]
            }
        ]
    }
    

    Parameter

    Description

    condition

    A filtering SPL command.

    validity

    An EVAL statement. Filter out invalid value of a field

    expected_fields

    List of expected_fields

    negative_fields

    The list of values the field should not have

    supported stanza_type:

    1. source

    2. sourcetype

  2. Provide path of the json file with –field-bank=path parameter

classmethod init_field_bank_tests(field_bank_path)[source]

Parse the field JSON file and return the list of fields

Parameters

field_bank_path (str) – Path of the field JSON file

Yields

dict – details of the fields including stanza and stanza_type