Last Updated: 8th November, 2019
Graph Search Tools: As with the rising threat of undetected malware patterns of behavior, companies and individuals are greatly encouraged to properly address cyber threats as quickly and efficiently as possible, especially true those in the financial sector, by using the right threat detection tools.
By 2021, the estimated amount of damages due to malware will be $6 trillion – as cyber-criminals place malicious software inside computer networks to gain access to bank accounts, exfiltrate account information, transfer funds, or extort money through ransomware.
There have already been around 3,500 successful cyber-attacks against financial institutions this year, according to reports filed with the Treasury Department’s Financial Crimes Enforcement Network. A single hack into Capital One yielded the personal data of over 100 million people.
All of this illustrates just how mission-critical it has become for security specialists to identify and neutralize threats instantly. Yet today’s threat detection tools to face formidable challenges, particularly when analyzing massive data sets for undetected malware patterns.
In response, banks and other large organizations are exploring new search and compute technologies to find malware patterns faster. New in-memory computing and graph search tools can identify cyber risks in near real-time, condensing what typically takes weeks down to just minutes.
For security experts who work with particularly large data sets, and who are vexed by the time it takes to find and neutralize undetected malware on their networks, there are steps which can help.
Graph Search Tools: Use Tools With Greater Scale
There is too much data for conventional tools to scan in a reasonable amount of time. Organizations must regularly scan their network log data to identify lateral movement. Yet banks can generate multiple terabytes of network log data per day, which means threats cannot be found in a meaningful time-frame.
Conventional tools will simply never catch up to the amount of data being generated and the number of incoming threats hitting the network. That’s one reason why the mean dwell-time for malware patterns is 71 days, and that interval exposes organizations to a whole lot of potential damage.
You need to look beyond conventional graph databases to find malware patterns. Graph databases are scalable both vertically and horizontally, without introducing data integrity or consistency issues, and work very well for smaller data sets.
The challenge is that graph databases don’t scale well once you get into terabytes of data. They lose steam in the small terabytes of data sizes with dramatic declines in performance on larger data sets. This happens for two reasons:
- Since scaling leads in almost all memory fetch (edge traversal) enabling a message being sent to certain other nodes over a network.
- Keeping data on the disk and working with only a small part of that data in memory results in thrashing of data between RAM and disk in order to traverse edges.
Graph search tools, on the other hand, are built for very large data sets. The Department of Defense helped develop the Trovares graph search tool which is now commercially available. The technology adopts super-computing techniques such as extreme multithreading and fine-grain locks to achieve orders of magnitude increases in speed and scale.
A team of data scientists applied analytics and super-computing expertise to deliver a significantly different graph search tool that returns queries hundreds of times faster than conventional search tools. It supports very large in-memory graphs for fast queries and enables the direct ingest of data into the system to avoid database performance issues.
Graph Search Tools: Look Beyond Clusters
Secondly, you need to consider computing platforms designed for extreme performance. Server clusters are not ideal for graph search; the typical computation over a graph data structure is among the worst for clusters. Symmetric multiprocessor systems (SMP) however, are excellent for graph search. Implementations from the team commercializing the DoD technology were built on SMP systems such as HPE’s Superdome Flex.
Currently, a single SMP device can vary in size from 3 to 48 memory terabytes and over a 1000 execution threads, ensuring the balance between memory performance and computing power to achieve scaling graph search performance standards.
These platforms are built on industry standard x86 processor technology and PCIe-based IO to enable high performance ingest of data and support for the full range of software needed to complete a workflow around the graph search tool.
The graph search efficiency is outstanding over an SMP system. Benchmark data show near linear scalability when querying three terabytes of cyber data with 20 billion graph’s edges and 212 billion edge properties.
The combination of the graph search tool and an SMP system demonstrates orders of magnitude improvements in speed, reducing the query time from 179 hours to 12 minutes. You should expect these search tools on SMP systems to outperform conventional tools on datasets of all sizes, but excel when data exceed a billion records.
Graph Search Tools: Search All The Data Sets Within Your Database
Rather than take a small slice of data and looking at it, graph search tools are able to ingest more of the data and answer complex data searches. The performance boost of graph search combined with SMP systems lets it leapfrog conventional search tools.
In this way it can quickly find intrusions that have continued to reside in the data. More importantly, speed and scale allow organizations to approach zero malware risks, with no unsigned data, and no multi-hour or multi-day scans.
The malware threat patterns challenge has grown and so the advancement of the threat detection tools simultaneously. The costs are higher, and the technical challenges are greater. New graph search tools combined with SMP systems are showing how companies can win the battle against malware and malware patterns.
Large enterprise organizations, including banks, telecom companies, biosciences and other industries are adding graph search and SMP systems to their cybersecurity roadmaps. They have found the performance needed to overcome the speed and scale challenges of finding malware patterns.