Using an External Application to Pull Splunk Search Results

By: Aaron Dobrzeniecki | Splunk Consultant

Have you ever wanted to pull logs from Splunk without actually being physically signed into the Splunk Search Head? With an external application, such as Postman, you can query the Splunk REST API endpoint to actually provide you with results from a search being run.

When Splunk runs a search, it creates a search ID which we can use to grab the results from the REST endpoint. We will be testing out two ways to get the results of a search. The first way is to grab the name of the Splunk search and query it against the /services/saved/searches/{search_name}/dispatch endpoint, which will provide us with the sid. We then use the sid to grab the results of the search, which will fire off the search and will poll for results as they come in. The second way to get the search results is by doing an export on the search name which will run the search and get the results without polling.

First things first, you need to make sure that the user you are authenticating to Splunk with has the “Search” capability, as well as access to search the necessary indexes. It’s that simple! If you are setting up a user for a particular person make sure they only have access to what they need. Giving further access is not necessary and can cause security issues.
In this example we are using the Postman application to query the Splunk REST API to grab search results from a couple of different reports/saved searches. Things we are going to need include:

  • Splunk user account with the Search capability. We need that user to be able to search the index we are going to be grabbing our data from.
  • We also need to know the Splunk URL we are going to be pulling from. In this case, I am using my localhost as an example. We will also be querying the Splunk management port of 8089 to get our results set.

The image above shows the type of request I am doing (POST), the REST API being used to query my search ID (/services/saved/searches/{name_of_search}/dispatch), and the authentication type of username and password. What the URL above is doing is it is reaching out to Splunk and grabbing the SID (search id) of the search named Index Retention Getting Close. With this search id we will be able to run a GET on the Splunk REST API and grab the results of the search.

Below I will be showing you two Splunk REST API endpoints that you can query (using POST) to get the Search ID for a specified search. The first endpoint is for searches that do not have Global permissions. As long as the user you are authenticating with has a role that has access to read the search, you can query the endpoint of /servicesNS/nobody/{app}/saved/searches/{name}/dispatch to retrive the Search ID. The second endpoint you can query if the search has Global permissions and you have read access is simply /services/saved/searches/{name}/dispatch to retrieve the Search ID. The two scenarios are below.

The image above shows the rest endpoint that can be used to grab a specific search ID that is in an app and has specific permissions. As long as my account has access to the app and search inside the app, I will be able to query it. For this example, we have changed the permissions of the search to be App only.

The image above is the results of the search in json, using the search ID we queried from the REST API.

The image above gives us the same results except they are in xml format.

The image above shows the search ID of the search with REST API I am querying. Since that search now has Global permissions, we do not need to use the ServicesNS endpoint. When you do a POST with a dispatch on the name of a search/report you will get the Search ID. As you can see the search ID is circled. We will be using this search ID to query the results of the search and show the actual search results in the Postman application. The Splunk REST API you will want to query next is the /services/search/jobs/{sid}/results?output_mode= (atom | csv | json | json_cols | json_rows | raw | xml). Any of those values will get you the results of the search in the format selected. In this example, I will be showing you json and xml.

As you can see above, the data results are shown in xml format for the search we were wanting to get results from.

This image shows the same results but in json format. With the options above for data output, you can query the Splunk REST API to get the search results and have them show in your preferred format.

Way 2: Query the REST API to show the results by using an export on the search name which will run the search and get the results without polling. Take a look at the screenshot below which queries the /services/search/jobs endpoint to stream in the results of the search as they come in.

Remember, you need to have the Search capability in Splunk, as well as you have to be able to read the results of the search. Whether that is setting Global permissions or having a role that has read access to the app and search. Below are some links referencing the Splunk REST API. If you have any questions at all regarding querying the Splunk REST API from an external application, please let me know!

https://docs.splunk.com/Documentation/Splunk/8.0.6/RESTTUT/RESTsearches

https://docs.splunk.com/Documentation/Splunk/8.0.6/RESTREF/RESTsearch#search.2Fjobs.2Fexport)

Want to learn more about using an external application to pull Splunk search results? Contact us today!