Threat Intelligence Framework functions in Splunk ES

By Kamal Dorairaj, Senior Splunk Consultant

Overall Framework

In the Splunk Enterprise Security App, navigate to Configure -> Data Enrichment -> Threat Intelligence Management. Here, you’ll find a variety of out-of-the-box threat intelligence sources. Enabling any of these sources triggers a process where a Modular Input downloads and integrates them into the KV Store. Subsequently, various searches extract this data for use in threat lookups, which are then utilized in threat intelligence correlation searches and threat intelligence dashboards.

Enabling Threat Source and Threat Matching

To activate threat intelligence, go to ES -> Configure -> Data Enrichment -> Threat Intelligence Management. Enable relevant threat sources for your organization. Next, navigate to the “Threat Matching” tab.

Within Threat Matching, you can view CIM fields from data models that are checked for matches against IOCs from the threat intel lookups. When a match is found, an event is written in the threat_activity index which also populates the Threat_Intelligence data model. The `Threat Activity Detected` Correlation search will then trigger a notable.

Threat Intel Collections

All data onboarded from Threat Intel feeds in Step 1 flows into the nine supported threat intel collections in ES. To view these collections, navigate to Settings -> Advanced Search -> Search Macros and search for “all_threat_intel”:

These macros represent the IOC collections. You can inspect the contents of each collection using commands like:

| inputlookup ip_intel

| inputlookup file_intel | stats count by threat_key

For details on the fields supported by each collection, visit Settings -> Lookups -> Lookup Definitions and search for the respective collection (or search for *_intel). Only the supported fields defined for each collection will be utilized in the framework.

Enabling Threat Correlation Search

To enable the out-of-the-box threat intelligence correlation search, navigate to Configure -> Content -> Content Management. Select “Correlation Search” from the “type” dropdown menu and filter for “Threat Activity Detected”.

Once this search is enabled, any identified IOCs will automatically trigger a notable event. To review the resulting incidents, proceed to Incident Review and filter for “Threat Activity Detected” to display a list of alerts generated by the correlation search. Expand any specific incident alert and click on “Contributing Events” located on the right side to access detailed information.

Threat Dashboards

Threat Activity and Threat Artifacts are two essential threat intelligence dashboards in ES. To access these dashboards navigate to Security Intelligence -> Threat Intelligence and chose either `Threat Activity` or `Threat Artifacts`.

Note that these dashboards utilize the Threat_Intelligence data model, which is populated by the threat_activity index.

Custom Threat Source

You can also upload custom threat sources into the framework. Ensure that the fields in your CSV file correspond with those in the nine collections; any mismatches will be ignored.

When debugging threat intelligence, two log sources found with the following query are particularly useful:

index=_internal source=”*threat*”

The logs of interest are threatmatch.log, which monitors the state of threat matching work, and threatlist.log, which tracks the onboarding of threat lists. When reviewing these logs, the status field will display their respective phases.

There are threat match lookup generating searches that separate out the data in the nine collections into match lookups. We can see those lookups here.

Settings -> Lookups -> lookup definitions

Clear the filters and search for threatintell_by

These lookups are used with the threat match searches that we looked at the Step3.

This completes the overview of the data flow of threat framework in Splunk ES.

Reference the Following Links for More Information on Threat Intelligence

Threat Intelligence Framework:

https://dev.splunk.com/enterprise/docs/devtools/enterprisesecurity/threatintelligenceframework

MISP Integration:

https://conf.splunk.com/files/2020/slides/SEC1579C.pdf

https://www.splunk.com/en_us/blog/security/integrating-covid-or-any-threat-indicators-with-misp-and-enterprise-security.html

Reference Upload (#2):

https://lantern.splunk.com/Security/Product_Tips/Enterprise_Security/Using_threat_intelligence_in_Splunk_Enterprise_Security

Use Cases:

https://research.splunk.com/detections

Threat Intel API Reference:

https://docs.splunk.com/Special:SpecialLatestDoc?t=Documentation/ES/latest/API/ThreatIntelligenceAPIreference

Adding threat intel:

https://docs.splunk.com/Documentation/ES/7.0.1/Admin/Addthreatintel

Add threat intel from events:

https://docs.splunk.com/Documentation/ES/7.0.1/Admin/Addthreatintelfromevents

Change Threat intel:

https://docs.splunk.com/Documentation/ES/7.0.1/Admin/Changethreatintel

Customize/Create Threat Matching:

https://docs.splunk.com/Documentation/ES/7.1.0/Admin/Createthreatmatchspecs

Add-ons and conf links:

https://www.splunk.com/en_us/blog/security/integrating-covid-or-any-threat-indicators-with-misp-and-enterprise-security.html

https://www.splunk.com/en_us/blog/security/how-do-i-add-covid-threat-intelligence-from-the-internet-to-enterprise-security.html

https://splunkbase.splunk.com/app/4335

https://conf.splunk.com/files/2020/slides/SEC1579C.pdf

https://www.splunk.com/en_us/blog/security/threat-intel-and-splunk-enterprise-security-part-1-what-s-the-point-of-threat-intel-in-es.html

Mission Control:

https://www.youtube.com/watch?v=K9D2_Ar9biA

https://docs.splunk.com/Documentation/MC/Current/Detect/Intelligence

Learn more about our Splunk services here.

About the Author

Kamal Dorairaj has over 22 years of diverse IT experience with full cycle development of various Applications and Systems. In his first 10 years; he worked as a CRM Consultant at the client locations includes Middle East, India, US; in various domains includes Retail, Telecom, eCommerce. Later he worked as a Lead SRE at Stubhub (was eBay company) for 7 years as a SME in Splunk, GCP. He is a PMP certified, ITIL certified, CompTia Security+ certified and Splunk Architect, Splunk ES Admin, Splunk Developer certified. He also completed Cybersecurity for Managers, Business Analytics from MIT Sloan and MBA in Technology Management. His core interest area has been, ‘Get more business value out of big data’.

I welcome opportunities for new challenges and creative thinking. I am a lifetime learner, continue to learn new technology and skills quickly, and apply them to solve real problems and drive to innovate.