Splunk - Diff examples and One-Way diff

Splunk’s ‘set’ command will allow you to ‘diff’ two result sets. What this means is that say you have two sets:

Set A: “event1 event2 event3”
Set B: “event2 event3 event4”

Splunk will tell you all the differences that occur between these two sets. When you run the following command, the output will be “event1 event4” because they are not common in both sets.

| set diff [set A] [set B]

A full splunk search search to generate the following example might look like:

| set diff
[| makeresults | eval events=“event1 event2 event3” | makemv events | mvexpand events | table events]
[| makeresults | eval events=“event2 event3 event4” | makemv events | mvexpand events | table events]

This would be considered a two-way diff. It can be very useful for determining that there are events from both sets (search results) that can’t be found in the opposite set.

To break down the commands used:

makeresults

This will make a new event. Without setting the ‘count’ parameter, this will make a single event

eval events=“event1 event2 event3”

Make a space-delimited field called events

makemv events

Turn events into a multivalued field, delimited by the space

mvexpand events

Because events is a multivalued field, we can split it into multiple events (in this case, 3 events)

table events

List out the 3 events in table format

 Real world examples

Goal: Take a list of unchanging users (maybe they have left the company), and verify that they don’t appear in any logs for compliance reasons.

 Two-Way

| set diff
[| makeresults | eval users=“user1 user2 user6” | makemv users | mvexpand users | table users]
[ search sourcetype=linux_secure logged_in_user!=“” | stats values(logged_in_user) as users | mvexpand users | table users]

The goal of this command would be to see the list of users we made part of the users variable show up if they don’t appear in the following list. Say the second list results in a set of “user3, user4, user5”, the output of this set command will be the entire list of users pulled from both sets. This isn’t very useful. If both sets are entirely unique, it would provide no benefit.

Enter, the one-way diff.

 One-Way

Splunk doesn’t have a one-way diff option, and there’s probably a couple different ways to do something like this.

In our first example we did:

| set diff [Set A] {Set B]

This will involve:

| set diff [Set A + Set B] [Set B]

The reasoning for this is that if both sets contain all data from Set B, then only the data from Set A that isn’t contained in Set B will end up in the results. Yes, it’s inefficient to search Set B twice.

| set diff
[| makeresults count=1 | eval users=“user1 user2 user6” | makemv users | mvexpand users | table users | append [search sourcetype=linux_secure logged_in_user!=“” | table users] | stats values(users) as users | mvexpand users | table users]
[search sourcetype=linux_secure logged_in_user!=“” | stats values(logged_in_user) as users | mvexpand users | table users]

Now, if you’re looking to determine that user1 user2 and user6 don’t appear in the 2nd result set, you can confirm this as those 3 users should be listed if this ran successfully. If they do appear in Set B, then they won’t be displayed.

 
4
Kudos
 
4
Kudos

Now read this

RetroPie: Raspberry Pie Retro Gaming Setup

What I’m going to show you how to configure Raspberry Pi running the Raspbian OS Boot into emulationstation, a GUI frontend to RetroArch allowing you to select any emulators’ games Use of xboxdrv to use an xbox controller to play with... Continue →