DataGenerator

PytestSplunkAddonDataParser

class standard_lib.sample_generation.pytest_splunk_addon_data_parser.PytestSplunkAddonDataParser(addon_path: str, config_path: str)[source]

Bases: object

This class parses pytest-splunk-addon-data.conf file.

Parameters: addon_path – Path to the Splunk App

get_sample_stanzas()[source]

Converts a stanza in pytest-splunk-addon-data.conf to an object of SampleStanza.

Returns: List of SampleStanza objects.

SampleStanza

class standard_lib.sample_generation.sample_stanza.SampleStanza(sample_path, psa_data_params)[source]

Bases: object

This class represents a stanza of the pytest-splunk-addon-data.conf. It contains all the parameters for the stanza such as:

Sample Name

Tokens

Sample file’s raw data

Tokenized events

Sample ingestion type

Parameters

sample_path (str) – Path to the sample file
psa_data_params (dict) – Dictionary representing pytest-splunk-addon-data.conf

break_events(sample_raw)[source]

Break sample file into list of raw events using breaker

Parameters: sample_raw (str) – Raw sample
Returns: List of raw events
Return type: event_list (list)

get_eventmetadata()[source]: Return the unique host metadata for event.

get_raw_events()[source]: Gets the raw events from the sample file.

get_tokenized_events()[source]: Yields the tokenized events

static populate_requirement_test_data(event)[source]

Analyze event’s datamodels, cim_fields, missing_recommended_fields, exception

Parameters: event (dict) – event data from xml file
Returns: datamodels, cim_fields, missing_recommended_fields, exception
Return type: requirement_test_data (dict)

tokenize(conf_name)[source]

Tokenizes the raw events by replacing all the tokens in it.

Parameters: conf_name (str) – Name of the conf file, “psa-data-gen”

class standard_lib.sample_generation.sample_stanza.token_value(key, value)

Bases: tuple

property key: Alias for field number 0

property value: Alias for field number 1

SampleEvent

class standard_lib.sample_generation.sample_event.SampleEvent(event_string, metadata, sample_name, requirement_test_data=None)[source]

Bases: object

This class represents an event which will be ingested in Splunk.

Parameters

event_string (str) – Event content
metadata (dict) – Contains metadata for the event
sample_name (str) – Name of the file containing this event

classmethod copy(event)[source]

Copies the SampleEvent object into a new one. :param event: Event object which has to be copied :type event: SampleEvent

Returns: Copy of the SampleEvent object

get_field_fqdn(rule)[source]

Returns unique fqdn value for the key fields src, dest, host, dvc

Parameters: rule (str) – Type of rule either src, host, dest, dvc

get_field_host(rule)[source]

Returns unique host value for the key fields src, dest, host, dvc

Parameters: rule (str) – Type of rule either src, host, dest, dvc

get_host()[source]: Returns a unique host value

get_ipv4(rule)[source]

Returns Ipv4 Address as per the rule.

Parameters

rule (str) – Type of rule either src, host, dest, dvc.
address. (If the value is not one of the key field it will return a randomly generated Ipv4) –

get_ipv6(rule)[source]

Returns Ipv6 Address as per the rule.

Parameters

rule (str) – Type of rule either src, host, dest, dvc.
address. (If the value is not one of the key field it will return a randomly generated Ipv6) –

get_key_fields()[source]: Returns the key field value from event

get_token_count(token)[source]

Returns the token count in event

Parameters: token (str) – Token name

get_token_extractions_count(token)[source]

Returns minimum number of occurrence count if token not found in event but is in extracted fields

Parameters: token (str) – Token name

register_field_value(field, token_values)[source]

Registers the value for the key fields in its SampleEvent object

Parameters

field (str) – Token field name
token_values (list/str) – Token value(s) which are replaced in the key fields

replace_token(token, token_values)[source]

Replaces the token value in event

Parameters

token (str) – Token name
token_values (list/str) – Value(s) to be replaced in the token

update(new_event)[source]

This method updates the event content

Parameters: new_event (str) – Event content

update_metadata(event, metadata, key_fields)[source]

Processes the syslog formated samples Format:

'***SPLUNK*** source=<source> sourcetype=<sourcetype>             field_1       field2        field3             ##value1##    ##value2##   ##value3##'

Parameters

event (str) – event string containing raw syslog data
metadata (dict) – Contains metadata for the event

Returns

Syslog event and the updated metadata

Rule

Provides Rules for all possible replacements for tokens.

class standard_lib.sample_generation.rule.DestPortRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random port value from [80, 443, 25, 22, 21]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.DestRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random dest replacement value from the list of values mentioned in token. Possible values: [“host”, “ipv4”, “ipv6”, “fqdn”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.DvcRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random dvc replacement value from the list of values mentioned in token. Possible values: [“host”, “ipv4”, “ipv6”, “fqdn”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.EmailRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random email from lookupsuser_email.csv file.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.FileRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

get_file_path()[source]: Returns the relative sample file path and index value

indexed_sample_file(sample, file_path, index, token_count)[source]

Yields the column value of token by reading files.

Parameters

sample (SampleEvent) – Instance containing event info
file_path (str) – path of the file mentioned in token.
index (int) – index value mentioned in file_path i.e. <file_path>:<index>
token_count (int) – No. of token in sample event where rule is applicable

lookupfile(sample, file_path, index, token_count)[source]

Yields the column value of token by reading files.

Parameters

sample (SampleEvent) – Instance containing event info
file_path (str) – path of the file mentioned in token.
index (int) – index value mentioned in file_path i.e. <file_path>:<index>
token_count (int) – No. of token in sample event where rule is applicable

replace(sample, token_count)[source]

Yields the values of token by reading files.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.FloatRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random float no. between the range mentioned in token.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.GuidRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random guid.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.HexRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random hex value.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.HostRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random host replacement value from the list of values mentioned in token. Possible values: [“host”, “ipv4”, “ipv6”, “fqdn”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.IntRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random int between the range mentioned in token.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.Ipv4Rule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random ipv4 address.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.Ipv6Rule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random ipv6 address

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.ListRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random value from the list mentioned in token.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.MacRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random mac address

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.Rule(token, psa_data_params=None, sample_path=None)[source]

Bases: object

Base class for all the rules.

Parameters

token (dict) – Dictionary containing token and its data
psa_data_params (dict) – PSA data stanzas dictionary
sample_path (str) – Path to the samples directory

apply(events)[source]

Replaces the token with appropriate values as per rules mapped with the tokens in the event. For replacement_type = all it will generate an event for each replacement value. i.e. integer[1:50] => will generate 50 events

Parameters: events (list) – List of events(SampleEvent)

get_lookup_value(sample, key, headers, value_list)[source]

Common method to read csv and get a random row.

Parameters

sample (SampleEvent) – Instance containing event info
key (str) – fieldname i.e. host, src, user, dvc etc
headers (list) – Headers of csv file in list format
value_list (list) – list of replacement values mentioned in configuration file.

Returns

list of mapped columns(int) as per value_list csv_row (list): list of replacement values for the rule.

Return type

index_list (list)

get_rule_replacement_values(sample, value_list, rule)[source]

Common method for replacement values of SrcRule, Destrule, DvcRule, HostRule.

Parameters

sample (SampleEvent) – Instance containing event info
value_list (list) – list of replacement values mentioned in configuration file.
rule (str) – fieldname i.e. host, src, user, dvc etc

Returns

list of mapped columns(int) as per value_list csv_row (list): list of replacement values for the rule.

Return type

index_list (list)

classmethod parse_rule(token, psa_data_params, sample_path)[source]

Returns appropriate Rule object as per replacement type of token.

Parameters

token (dict) – Dictionary containing token and its data
psa_data_params (dict) – PSA data stanzas dictionary
sample_path (str) – Path to the samples directory

class token_value(key, value)

Bases: tuple

property key: Alias for field number 0

property value: Alias for field number 1

class standard_lib.sample_generation.rule.SrcPortRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random port value from the range 4000-5000

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.SrcRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random src replacement value from the list of values mentioned in token. Possible values: [“host”, “ipv4”, “ipv6”, “fqdn”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.StaticRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields the static value mentioned in token.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.TimeRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Returns time according to the parameters specified in the input.

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.UrlRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

generate_url_query_params()[source]

Generates random query params for url

Returns: Return the query param string

replace(sample, token_count)[source]

Yields a random url replacement value from the list of values mentioned in token.

Possible values: [“ip_host”, “fqdn_host”, “path”, “query”, “protocol”, “full”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

class standard_lib.sample_generation.rule.UserRule(token, psa_data_params=None, sample_path=None)[source]

Bases: standard_lib.sample_generation.rule.Rule

replace(sample, token_count)[source]

Yields a random user replacement value from the list of values mentioned in token. Possible values: [“name”, “email”, “domain_user”, “distinquised_name”]

Parameters

sample (SampleEvent) – Instance containing event info
token_count (int) – No. of token in sample event where rule is applicable

standard_lib.sample_generation.rule.raise_warning(warning_string)[source]

To raise a pytest user warning along with a log.

Parameters: warning_string (str) – warning string