Last Updated: 13th August, 2022

Effective Threat Hunting Process: No one ever responded to a threat that they were unable to identify. Simply ask Capital One who is currently facing public criticism on one of the biggest data breaches ever: one that lasted for months in the company’s cloud-based network but was not reported until the corporation was tipped off by external analysts. Thus, effective threat hunting process for machine learning (ML) and automation in security in inevitable in today’s scenario.

Like with Capital One, the vast majority of today’s breaches root back to a failure of detection. The challenge of protecting low hanging fruit has been solved – alerting, signature-based rules, blocking at the packet/signature level – it works, but it’s easy. Most failures in detection happen in the gray area between black and white.

Phishing is a great example of this, as those attacks continue to become stealthier by the day. Over the last year, ninety-eight percent of phishing attacks that made it past enterprise email security controls and into user inboxes contained no malware, which makes them much harder for enterprises to detect.

What’s scarier than that? As cyber-criminal activity grows more sophisticated by the day, the gray area is expanding, and the damage is particularly debilitating because symptoms are hard to detect and present a morbid outcome.

The good news is that there are a number of best practices companies can follow to limit the amount of detection failure across the network.

Securing The “Gray” Area With Effective Threat Hunting Process For Machine Learning (ML) And Automation

One of my favorite quotes is “the absence of evidence is not evidence of absence.” The fact that you don’t know you have been breached, doesn’t mean you haven’t. While everything may appear to be fine, it could simply mean that you aren’t looking at things in your network you should be looking at.

Increasing feedback and knowledge transfer across the teams that test the effectiveness of an organization’s defense (red team) and the teams that keep the organization safe from real-world attackers by understanding their TTPs (blue team), also known as purple-teaming, is essential.

However, the reality for many companies is that the red and blue teams can be completely separate and disconnected entities. By collaborating and continuously enhancing security controls across teams, analysts can improve the effectiveness of their threat hunting skills, vulnerability detection efforts and network monitoring techniques through imitating common threat scenarios and aiding the creation of procedures designed to prevent and detect these new threats.

While purple teaming is a helpful strategy, organizations see the most value by investing heavily in the blue team and proactive threat hunting. Actively “hunting” for threats targeting your organization helps your security team to learn to seek out sophisticated threat behaviors and identify them before they turn into full on security breaches.

While it can be costly and time consuming to invest in, proactive threat hunting can help your security team uncover things that you thought you had defended against, but in reality, are still quite vulnerable to.

Lastly, automating the threat hunting process is the final level of defense to ensure your network is as secure as possible. Attackers are not scanning everything by hand, and neither should you. People’s intuitions are simply no match for today’s machine learning (ML) capabilities.

While many analysts have good awareness and the ability to detect threats, there are often things happening inside the network that go unnoticed. Machine learning (ML) and automation help support an analyst during the threat hunting process, by helping them understand and access all the data across a network in seconds versus hours. This is crucial in a market where finding analysts that can accurately detect the non-trivial attacks across a network is difficult.

Achieving Reasonable Machine Learning (ML) And Automation

Even though machine learning (ML) and automation are the supposed silver bullets of modern cybersecurity, application of the transformative technology is misconstrued and often overhyped by vendors. During the threat hunting process, not only are organizations limited by the manpower available, but analysts are simultaneously equipped with more data and tools than they know what to do with.

Machine learning (ML) and automation have the ability to provide analysts with deep, reasonable analysis to support them during the threat hunting process by uncovering the value in the data. With machine learning (ML) and automation, analysts can easily see what they can learn from the data and improve the organization’s threat hunting process.

Machine learning (ML) and automation help analysts better understand the protocols they have in place to detect threats, the gaps that may exist across the network, the number of false positives, etc.

With automation, security analysts’ work becomes streamlined, so they can focus on doing what they do best: analyzing incidents and protecting their organization’s IT infrastructure. However, it is a constant journey, where you need to strive to continuously improve.

If organizations start investing in reasonable automation now, they’ll be set up to succeed for the next horizon of automation – where automation systems proactively suggest what organizations can and should automate – and empower their SOC teams to become tomorrow’s super humans, today.