on November 21, 2018 Automation Splunk

Is your Splunk thriving?

Subscribe to Email Updates

It’s budget time, and as always you’re being asked to shave 10% off last year’s spending without compromising IT services. On your desk is a request for a 10GB/day increase in your Splunk licence – and that’s the third one this year!  You’ve seen Splunk being used around the place by lots of different teams, but is it really worth the ever-increasing licence cost?

It is extremely easy to get machine data into Splunk to make it accessible for searching – after all that’s one of its key selling points. And Splunk costs money, in the form of its GB/day of indexed data licensing model. The combination of these two factors can unfortunately lead to tense conversations with budget-holders, both when licence increases are requested and when annual maintenance invoices arrive, which boil down to “Are we getting sufficient value from our Splunk investment?”.

In this post I’d like to share a few ideas to help Splunk customers develop thriving Splunk environments where the focus is on realising ever-increasing business value from Splunk, rather than on constraining licence costs.

Take small bites of the tomato

“Rome wasn’t built in a day” and neither is a Splunk environment (although the initial deployment may well be done in a day!). Clearly define the pain point you are wanting to address, how you will address it, what new data sources, if any, Splunk will need, and how success will be measured. Develop the new app or dashboard, assess success and document. Rinse and repeat.

Understand the cost of doing nothing

Whether you are hoping to use Splunk to reduce your MTTR, or identify trends in customer behaviour for marketing campaigns, or automate compliance audits, you need to assign a cost to how things are done now. For example, how many high-severity incidents do we deal with each week, and how long on average do they take to resolve, and what is the cost to the business? How many person-hours does it take to complete annual security audits? Don’t get bogged down on the numbers (you can always fine tune those later) – focus instead on identifying all the different activities and resources that contribute to the overall cost.

Celebrate (and track) your successes

After each planned Splunk development task is complete, measure the effect it has had on the target systems or processes. Confirm that the agreed success criteria have been met, and calculate the resulting benefit or cost reduction. Publish this info proudly on the company intranet, so that everyone can see the ongoing benefits being derived from Splunk (via a new Splunk dashboard perhaps?). And keep an eye on what your Splunk users are up to – if someone has created a private dashboard which saves them 4 hours per week then you want to know about it, and maybe share it with other users, delivering further value. The end result will be an up-to-date estimate of the value Splunk has delivered to the business since deployment, and hopefully an end to those awkward budget conversations.  

Feed, water, fertilise and prune

Don’t let your Splunk environment grow stale. Have regular catch-ups with managers to talk about the challenges that are keeping them awake at night and how Splunk might help. Arrange Splunk user group sessions to discuss new features and best practices, and to share what people are up to with Splunk. Periodically review dashboard and index activity to ensure licence is being used effectively, and make use of the Splunk Chargeback Analysis app (available from Splunkbase) to track index usage and costs by team. 

In a future post I will discuss the tools Splunk provide to assist with the creation of a business case for Splunk and with the subsequent tracking and maximising of value realised from a Splunk deployment, namely the Interactive Value Assessment and Data Source Assessment.

If you would like to sit down with one of our OSS-Group Splunk consultants to discuss how best to get your Splunk environment thriving please get in touch.


Justin Farmiloe

My current role at OSS is leader of our Data Management Capability team, which includes storage, backup, data analytics and data migration. I'm also responsible for development of our Splunk-related consulting and managed services offerings.