Ever-growing data piles lead to an ever-growing Splunk

Ever-growing data piles lead to an ever-growing Splunk

Splunk has been a data collector at heart from its inception. At one point, it wasn’t entirely clear what could be done with said collected data, but over the years the company has kept expanding the applications it offers. At .conf23 in Las Vegas Splunk touted the credo “Ready for Anything”, with cyber resilience and the convergence of security and observability as key elements. AI was a common thread throughout the keynotes and conversations we had, as one might expect these days. What does Splunk do in the here and now, or rather: what doesn’t it do?

Much has been said about the data explosion that has gripped organizations globally in recent years. The rise of hybrid work and the migration to the cloud have only increased the complexity of IT environments. Another truism for IT companies is that expertise is thin on the ground. These are challenges that have forced the likes of Splunk to look at their own operations differently.

Anyone looking at old .conf coverage will be met with omnipresent discussions of “data innovation” as the top billing. Splunk is still trying to answer the question: “What can you do with all the data your systems generate?” However, given the continuous cybercrime threat, they’ve approached the issue they’re tackling from a somewhat different angle. The question has now been expanded to include: “How do you leverage data to arm your business against malicious actors?”

Converging security and observability

Security’s importance won’t be articulated more clearly by anyone within Splunk than Mike Horn, SVP & GM of Security. Previously, Horn worked at TwinWave, which was acquired by Splunk last year. “I always talk about being one small step behind the attackers, because you never know what they’ll do tomorrow,” he says. Indeed, hackers will have been looking closely at all the innovations the company presented last month along with Splunk’s userbase.

The internet is teeming with threat actors, and if anything the problem is getting worse. Security reports worldwide show cybercriminals running rampant. At Splunk, the increased threat cybercriminals pose seems to be leading to a convergence of security and observability. These two fields of work have a considerable inherent overlap. For example, SecOps could detect suspicious activity via signals emerging from system performance. They have common aims, according to Horn: “An observability trace can be useful for data logs,” he cites as an example.

This convergence narrative is best illustrated by the new Mission Control application, which packages as much information as possible in one place. Horn mentions that this should prevent the endless browser tab switching that Splunk users have been accustomed to. Mission Control merges several data points to provide unified case management: a lot can be managed from within the UI, from quarantining a suspicious user to checking system performance. For some organizations, this convergence will be most welcome. Consider the aims of a large media company that doesn’t want a separation between its network operations and its SOC, as they may look to fix the same issue independently if there’s no direct line of communications.

Incidentally, Mission Control already was an application within Splunk, available only to cloud users. As it happens, the new iteration is not yet ready for on-prem, but it should be later this year. In principle, smaller companies could also benefit from this summarization and organization of data. However, Splunk is focused solely on Fortune 2000 companies as CEO Gary Steele lets us know during a press conference.

Splunk AI: an adaptable security aid, a helpful teacher for beginners

If there’s one thing that sounds like a good use case for AI, it’s the summarization and transformation of data. The newly announced Splunk AI is not a specific application, but a catch-all term for every AI-infused implementation within the Splunk platform. For example, Splunk Attack Analyzer is designed to eliminate as many manual tasks as possible when analyzing cyber threats. In combination with Splunk SOAR, users can deploy Attack Analyzer to automatically play a response workflow based on the analysis it has put forward. AI tools also offer insights in Splunk Enterprise Security and User Behavior Analytics.

This form of data aggregation can simplify the work of security specialists, freeing up their time for other things. That’s quite handy given the glaring worker shortage in the industry. In that regard, AI should be a step in the right direction. Splunk AI Assistent is available in preview and can help SPL novices gain insights into Splunk data. SPL is the language you normally need to speak to talk to the platform properly. The new generative AI tool provides the ability to perform natural language searches and then converts this query to SPL. It’s the only time we hear about gen-AI specifically in the Splunk narrative at .conf.

Tip: Splunk integrates security and observability with Splunk AI

AI also features heavily in our joint conversation with GVP and Chief Strategy Advisor EMEA James Hodge and Chief Technical Advisor EMEA Mark Woods. Even before the ChatGPT hype took the industry by storm, they were heavily involved as Splunk representatives at the World Economic Forum to help companies implement AI, as Hodge explains.

Much of what the company is talking about during .conf23 is not “new new”, as Woods puts it. For example, the Machine Learning Toolkit (MLTK) has been around for a while, which includes the option to run third-party AI models (provided they support ONNX, or Open Neural Network Exchange). Still, Woods warns of the problems that large models can bring to the table – there’s a lot of extra noise in the outputs that a human will have to sift through. In other words, you need to have a good understanding of what AI is actually doing with your data under the hood. “I always talk about a glass box, not a black box,” Woods reveals. “If you apply a glass box to your data, you’ve got a good starting point for any kind of AI.”

A wider view: Edge Hub, observability

Splunk’s ambition to collect as much (useful) data as possible now even extends to hardware. The company is not a hardware manufacturer, but it has partnered with one to produce what it’s calling the Edge Hub. It provides OT infrastructure with data collections options as well. The device is bursting with sensor to measure such things as humidity, movements and light levels. On the show floor, we were met with a treasure trove of applications, from protecting a chicken coop from outside threats (which includes the exhibitor’s son, a threat actor to eggs) all the way to making sure a sewer system runs as planned. Edge Hub is currently US-only, but Europeans can expect availability to arrive later in 2023. Apparently, the Edge Hub concept emerged from an explicit desire from the manufacturing industry to acquire more OT visibility.

Also read: Splunk aims to simplify OT security with Edge Hub hardware

GM of Observability Spiros Xanthos sees it as an extension to the existing observability package, making OT infrastructure measurable for IT teams. This is an issue we’ve seen crop up time and again, especially as the industrial sector has a habit of running woefully vulnerable and dated OT environments. Protecting industrial equipment starts with data collection, which is why Splunk arrived on the scene. “The problems we solve in terms of security, IT and observability are ultimately all data problems,” Xanthos states. “Most of this data is unstructured, so you have to be able to make sense of it afterwards.”

Tip: Zero Trust must be a pivotal part of OT security

To make all that extra data manageable requires processing up front. Not just because Splunk users aren’t helped by an endless pile of data, but also because storing it isn’t free. Earlier this year it announced Edge Processor, aimed at filtering the amount of data coming from the edge. In the process, it’s possible to see exactly what’s being done with the data before it is sent on its way to on-prem or the cloud. Data tiering is a chief aim for the company, as Xanthos explains.

In terms of data collection, Splunk has adopted OpenTelemetry (or OTel). Its own OTel Collector hooks into all sorts of sources, from Kubernetes environments to managed databases from the client company. OpenTelemetry’s popularity has ballooned since its beginning in 2018, with Xanthos as one of its founders. It’s the second most supported CNFC project, only behind Kubernetes. This level of popularity surprised Xanthos. He’s noticed that customers have quickly adopted it as the de facto standard: any provider that hasn’t built support for it, is said to have little chance of sealing a contract.

But… there are limits

Let’s say you’re a Fortune 2000 company and a Splunk power user. Perhaps you’re a manufacturer and the abilities of Edge Hub suit your needs. You make use of all security and observability solutions available on the platform. What’s still missing? We return to security chief Mike Horn: he lists firewalls, web proxies and EDR as three areas where Splunk does not offer anything.

Above all, Horn says the focus is still on data insights and visibility, even if it’s broadly defined. Ultimately, everything Splunk does is still related to data problems, and the tech industry has simply shifted to make this field of play bigger and bigger. As for AI: it’s refreshing to see that the company refrains from making near-magical promises. Splunk AI is not a panacea, it assists where it can and is a straightforward evolutionary step from what the company has already developed. In addition, the story that describes why security and observability should go together was already discussed at length at .conf22.

In short: Splunk has become increasingly ambitious, but ultimately still relies on the events, logs, metrics and traces that underpin IT operations. With the Edge Hub, it offers more ways to generate this data where it had been missing. Mission Control and Attack Analyzer let security specialists focus on what’s relevant. Although the various Splunk representatives have had a lot to say about AI, it is hardly the same hype-inducing rhetoric we’ve seen in other parts of the industry in 2023. The expanded range of offerings may make it hard to keep track of what Splunk does, and by extension: what it doesn’t do. This is not because the company is overstepping its bounds, but because there’s simply more data to collect.

Also read: Partnership with Microsoft brings Splunk to Azure Marketplace