Big Data analytics tools will be crucial to enterprise security as criminals deploy faster and more sophisticated methods to steal valuable data, according to security firm RSA.
"We are really at the beginning of intelligence-driven security: it is just the tip of the iceberg. Looking forward we are going to have to be smarter [to deal with threats], and we are going to be looking at better data science," said RSA's head of knowledge delivery and business development, Daniel Cohen.
"It's not 'if' we are going to be breached, but 'when' we are going to be breached, so there is a need to focus more on detection. We saw with the Target breach it was the human factor that slipped there, so we have to be able to bring in more automation."
The number of successful attacks against high-profile businesses have clearly increased in recent years, with the compromise of Target's point of sale systems just one example of the variety of methods that cyber criminals are using to steal data on a large scale.
Businesses are threat from a number of sources - from criminal gangs, to hacktivist and insider threats, as evidenced recently by the theft of payroll data for thousands of employees at Morrisons in the UK last week, and more famously, Edward Snowden at the NSA.
Businesses slow to adopt data analytics
However, the adoption of big data analytics within businesses for security - and the maturity of offerings from vendors - remains at an early stage.
While banks are already deploying analytics for fraud prevention purposes, and have begun engaging with big data start-ups for security services, there are few wider businesses that have adopted new techniques and tools to monitor threats.
A recent Gartner study highlighted that adoption of big data analytics currently stands at only eight percent of large enterprises, though this is set to grow to 25 percent by 2016 as businesses get to grips with the information being generated across their business.
"We are still at the stage where we are collecting huge amounts of data, and we need to improve the mining of that data," said Cohen, speaking at an event at the company's cyber security HQ in Herzliya, Israel, this week.
'Cat and mouse game'
According to the RSA's security analytics director Dr Alon Kaufman, the current siloed detection processes employed by businesses and the large volumes of data generated across an organisation make swift threat detection difficult.
"Investigation today is something very time-consuming, and adding or removing rules is a very manual process," he said. "To have a good analysis you need people with very good knowledge and experience.
"Big data can improve the analyst's abilty to deal with the more human intellignce tasks, and not have to do a lot of the optimisation and statistical work that machines can do."
Large firms are likely to generate terabytes of data each day which can be monitored for the anomalous behaviour that may indicate malicious activity. This can be external and internal information, such as monitoring user profiles to identify changes in location, device used to access the network, or visits to high risk domains which are flagged up to security analysts, who can then make decision whether to take further action.
Sifting through these large volumes of information at speed is not possible for for humans, but by using big data analytics tools to process risk in real-time business can react more quickly, which is vital if there is any chance of stopping an attack in progress.
"If you look at search engine data science - for example, how does Google find a needle in the haystack in 0.1 seconds - the difference is in our world is that the search results are actually acting against us: they don't want to be found," said Cohen.
"The cat and mouse game we are playing is going to call for better data science, and so we have to be able to detect these anomalies much faster, and that means better use of big data."
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.