Create Splunk Indexes and HEC Inputs with Ansible

By: Brandon Mesa | Splunk Consultant

Managing Splunk .conf files is a day to day routine for most, if not all, Splunk admins. As your Splunk environment matures, you’ll find yourself making constant .conf changes to improve operational efficiency. For example, as new data sources are onboarded, new indexes and parsing settings are implemented to maintain efficiency and the appropriate data segregation controls in place. To access this new data or index you might also have to create a new role or manage an existing one in order to set the appropriate data permissions to a specific set of users. You may also explore alternate data inputs such as making use of the HTTP Event Collector.

Manually completing these tasks can become time-consuming and error-prone. While you can’t automate every change on the back end, you may be able to standardize some of the common configuration changes. For example, common tasks include creating a new index, role, HEC token, and many more. You can use a variety of automation tools to manage your .conf files and reduce time spent making manual .conf changes. This blog will show you how to use Ansible playbooks to automate common Splunk tasks including index and HEC input creation.

To keep this blog simple, examples will be applied to a local standalone instance in the $SPLUNK_HOME/etc/system/local path. The location of .conf changes will vary depending on your specific environment.

The following Ansible playbooks are used in this blog:

create_index.yaml

 

create_hec_token.yaml

 

Create an Index

To create a new index with Ansible playbooks, run the following command:

% ansible-playbook create_index.yaml -e ‘{“index_name”:”ansible_index”}’

Shown below, you can see the new index “ansible_index” has now been created on the indexes.conf.

 

If you run the playbook again to create a new index with an existing index name, an error will be returned and escape the playbook execution. For example, if we try to create the “ansible_index” index a second time, the playbook escapes execution and returns the following message:

“ansible_index – Index string already found in indexes.conf”

 

Take a look at the returned message for the “Confirm if index already exists” task. The playbook reads the indexes.conf file and looks for the index_name variable passed at the time the CLI command is run. If the string is found in the file, the playbook skips the stanza creation.

 

Create a HEC Token

We’ve created a new index for all the Ansible related data. Now let’s create a new HEC input that will constraint incoming data to the new index. To create a new HEC token, run the following Ansible playbook:

% ansible-playbook create_hec_token.yaml -e ‘{“username”:”admin”,”password”:”Pa$$w0rd”,”token_name”:”ansible_token”,”index”:”ansible_index”,”indexes”:”ansible_index”}’

Playbook execution will look something like this:

 

Now let’s validate our token has been created:

 

Automation tools can facilitate day-to-day operations related to your Splunk infrastructure. It’s not likely that all .conf changes will be automated in your environment as you’ll come across unique use cases that will require specific configurations. However, you can automate some of the common manual tasks, like the ones shown above, to reduce time spent and avoid any silly mistakes.

Want to learn more about creating Splunk indexes and HEC inputs with Ansible? Contact us today!