Format JSON Data at Search Time

By Forrest Lybarger | Splunk Consultant

JSON data is a very common format in Splunk and users like to have control of the data. Splunk’s collection of json_* commands help users format JSON data at search time so that it can be presented and used without any permanent changes to the indexed data. This guide will help users learn how to use these commands themselves, so they can have full control of their JSON data. It is important to note that the commands are often looking for the name of a field containing a whole JSON event, so if the whole event is one JSON event then _raw can be used to reference the whole event as one field.

First is the json_valid command. Using this command, strings can be validated as using proper JSON. It has simple syntax where you input the field containing the JSON data and it returns true or false. If you want to store the value in a field, you could use an if statement.

    • Example:
    • | eval valid = if(json_valid(_raw), 1, 0)
    • Result:
    • valid=1

Second is json_array. This command lets you create an array with JSON formatting. You can use an array field or multi-value field in this command instead of hard coded values. This can be useful in cases like if you need a sub-search to return an array.

    • Example:
    • | eval array=json_array(field_name)
    • Result:
    • array=[“string1”, “string2″, “string3”]

Third is json_object. This command creates JSON objects from the inputs given to the command. Other JSON commands can be used as inputs including another json_object command to create nested objects. The first input is just the name of the object, then you can add one more input that can be a nested JSON object, array, or hard-coded value.

    • Example:
    • | eval obj=json_object(“object1”, json_object(“object2”, json_array(“item1”, “item2”))
    • Result:
    • Obj={“object1”:{“object2”:[“item1″,”item2”]}}

Lastly is json_extract. This command essentially lets you do a field extraction on the fly with JSON data. You just give the command the field containing the JSON data (if the event is one big JSON event, then do _raw) then tell it the path you want extracted.

    • Example:
    • | eval ext=json_extract(_raw, “array{0}”)
    • Event:
    • {
    •   “array”: [
    •     {
    •       “name”: “item1”,
    •       “subarray1”: [
    •         { “name”: “subitem1” },
    •         { “name”: “subitem2” }
    •       ]
    •     },
    •     {
    •       “name”: “item2”,
    •       “subarray2”: [
    •         { “name”: “subitem1” },
    •         { “name”: “subitem2” },
    •         { “name”: “subitem3” }
    •       ]
    •     },
    •     {
    •       “name”: “item3”,
    •       “subarray3”: [
    •         { “name”: “subitem1” },
    •         { “name”: “subitem2” }
    •       ]
    •     }
    •   ]
    • }
    •             Result:
    •             ext={“name”: “item1″,”subarray1”: [{ “name”: “subitem1” },{ “name”: “subitem2” }]}

These commands are very useful for users that deal with a lot of web data or API related events. Some notable data that these commands could be useful for are AWS Cloudtrail events. Users might want to extract whole arrays as fields or do other JSON manipulation and these commands make all that possible within Splunk’s search bar.