EventIngestor

HEC Event Ingestor

class standard_lib.event_ingestors.hec_event_ingestor.HECEventIngestor(required_configs)

Bases: standard_lib.event_ingestors.base_event_ingestor.EventIngestor

Class to ingest event via HEC Event

The format for required_configs is:

{
    hec_uri: {splunk_hec_scheme}://{splunk_host}:{hec_port}/services/collector,
    session_headers(dict):
    {
        "Authorization": f"Splunk <hec-token>",
    }
}

Parameters

required_configs (dict) – Dictionary containing hec_uri and session headers

ingest(events, thread_count)

Ingests event and metric data into splunk using HEC token via event endpoint.

For batch ingestion of events in a single request at event endpoint provide a list of event dict to be ingested.

The format of dictionary for ingesting a single event:

{
    "sourcetype": "sample_HEC",
    "source": "sample_source",
    "host": "sample_host",
    "event": "event_str"
}

The format of dictionary for ingesting a batch of events:

[
    {
        "sourcetype": "sample_HEC",
        "source": "sample_source",
        "host": "sample_host",
        "event": "event_str1"
    },
    {
        "sourcetype": "sample_HEC",
        "source": "sample_source",
        "host": "sample_host",
        "event": "event_str2"
    },
]

Parameters

events (list) – List of events (SampleEvent) to be ingested

HEC Raw Ingestor

class standard_lib.event_ingestors.hec_raw_ingestor.HECRawEventIngestor(required_configs)

Bases: standard_lib.event_ingestors.base_event_ingestor.EventIngestor

Class to ingest event via HEC Raw

The format for required_configs is:

{
    hec_uri: {splunk_hec_scheme}://{splunk_host}:{hec_port}/services/collector,
    session_headers(dict):
    {
        "Authorization": f"Splunk <hec-token>",
    }
}

Parameters

required_configs (dict) – Dictionary containing hec_uri and session headers

ingest(events, thread_count)

Ingests data into splunk via raw endpoint.

For batch ingestion of events in a single request at raw endpoint provide a string of events in data to be ingested.

The format of event and params for ingesting a single event:

'127.0.0.1 - admin [28/Sep/2016:09:05:26.875 -0700] "GET /servicesNS/admin/launcher/data/ui/views?count=-1 HTTP/1.0" 200 126721 - - - 6ms'

{
    "sourcetype": "sample_HEC",
    "source": "sample_source",
    "host": "sample_host",
}

The format of event and params for ingesting a batch of events:

    '''127.0.0.1 - admin [28/Sep/2016:09:05:26.875 -0700] "GET /servicesNS/admin/launcher/data/ui/views?count=-1 HTTP/1.0" 200 126721 - - - 6ms
    127.0.0.1 - admin [28/Sep/2016:09:05:26.917 -0700] "GET /servicesNS/admin/launcher/data/ui/nav/default HTTP/1.0" 200 4367 - - - 6ms
    127.0.0.1 - admin [28/Sep/2016:09:05:26.941 -0700] "GET /services/apps/local?search=disabled%3Dfalse&count=-1 HTTP/1.0" 200 31930 - - - 4ms'''

{
    "sourcetype": "sample_HEC",
    "source": "sample_source",
    "host": "sample_host",
}

Parameters

• events (list) – List of events (SampleEvent) to be ingested

• params (dict) – dict with the info of the data to be ingested.

SC4S Event Ingestor

class standard_lib.event_ingestors.sc4s_event_ingestor.SC4SEventIngestor(required_configs)

Bases: standard_lib.event_ingestors.base_event_ingestor.EventIngestor

Class to Ingest Events via SC4S

The format for required_configs is:

{
    sc4s_host (str): Address of the Splunk Server. Do not provide http scheme in the host.
    sc4s_port (int): Port number of the above host address
}

Parameters

required_configs (dict) – Dictionary containing splunk host and sc4s port

ingest(events, thread_count)

Ingests events in the splunk via sc4s (Single/Batch of Events)

Parameters

events (list) – Events with newline character or LineBreaker as separator

File Monitor Ingestor

class standard_lib.event_ingestors.file_monitor_ingestor.FileMonitorEventIngestor(required_configs)

Bases: standard_lib.event_ingestors.base_event_ingestor.EventIngestor

Class to ingest event via File monitor This ingestor will only work if splunk_type is docker and container of universal forwarder is linked with container of splunk instance as ‘splunk’ service.

The format for required_configs is:

{
    uf_host: Host of universal forwarder
    uf_port: Management port of universal forwarder
    uf_username: Name of user for universal forwarder
    uf_password: Password of universal forwarder
}

Parameters

required_configs (dict) – Dictionary containing information about universal forwarder

create_event_file(event)

Write each tokenized event in files with host name as name of file. The host of all events will be unique.

Parameters

event (SampleEvent) – Instance containing event info

create_inputs_stanza(event)

Create stanza in inputs.conf on universal forwarder for each tokenized event.

Parameters

event (SampleEvent) – Instance containing event info

create_output_conf()

Create stanza in outputs.conf file of universal forwarder to send on splunk(indexer).

get_file_path(event)

Returns absolute path for tokenized events.

Parameters

event (SampleEvent) – Instance containing event info

ingest(events, thread_count)

Ingests data into splunk via file monitor. :param events: List of events (SampleEvent) to be ingested :type events: list