Since 1965, the tech industry has considered Moore’s Law the defining trajectory of big data.
Microchips have since become more affordable, efficient, and smaller.
Today, the big data industry is moving beyond Moore’s Law as enormous amounts of data are generated every millisecond worldwide – and with no end in sight.
As the amount of data organizations are responsible for handling continues to rapidly grow, how can IT professionals keep up? In recent years, overwhelming amounts of data often are stored in a single, static depository referred to as a data lake.
Data stored in the “lake” can be later accessed for analysis.
However, because data stored in the data lake is static, security analytics are only available for past events.
This presents a myriad of issues for security professionals whose job it is to stop data breaches in real-time.
Outdated Data is a Problem googletag.
cmd.
push(function() { googletag.
display(div-gpt-ad-1439400881943-0); }); Alternatively, organizations should aspire to analyze data as it flows and better pinpoint anomalies in user activity patterns.
If security professionals are armed with more instantaneous data, they are empowered to make more informed decisions with regard to network events and user behavior anomalies.
This can be illustrated in the below example.
Imagine a pedestrian preparing to cross the street.
In order to cross the street safely, the pedestrian must have a full and clear vision of street traffic.
If the pedestrian does not know exactly where nearby cars are on the road at that moment, the risks associated with crossing the street increase dramatically.
This is comparable to traditional network security analytics, which is like using a photo of the intersection from minutes earlier to inform when to cross the street at present.
Focus on Data Flow Analysis Traditional security analytics only provide snapshots of past network events.
However, the next generation of network security analytics aims to bridge the delay so that security professionals can make more informed decisions based on a clear and immediate snapshot of data events.
In this aspect, the data lake becomes more of a data river – constantly flowing.
As organizations look to scale security analytics with data growth, there are new capabilities to consider when exploring network security solutions.
Identify a solutions provider that offers results versus the infrastructure needed to get results.
Find a solutions provider that can deliver ground truth, or a complete snapshot, of who is accessing what data, when, and where from.
For example, a network user who is inactive is shown to have accessed the network at 4 AM on a Tuesday.
Add to it that the network was accessed from overseas and the incident is escalated greatly.
This detail may be obvious in retrospect but is most valuable when known as-it-happens.
Another example of security events to monitor in real-time is failed login attempts.
Scaling Security Through Automation Automation also is an added benefit of next generation network flow analysis.
Most traditional network analysis providers require human input and multiple queries to piece together all necessary details surrounding a security event.
Machine learning is the next frontier of network security analytics and allows for queries to be automated as network event anomalies arise.
While it is easy to assume that details such as who, what when, and where are typically included when security events are flagged, the unfortunate truth is that this simply is not the case.
Next generation data flow analysis solves this issue through automation, initiating the relevant data queries upon identifying network event anomalies.
Ultimately, today’s network flow analysis providers are using automation to take people out of the process and move it along as far as possible without human input.
The fact is that processes scale – people do not.
About the Author Fluency CEO and Founder Chris Jordan founded Endeavor Security, a cutting-edge, threat detection and analysis company focused on helping enterprises and governments protect their most sensitive networks.
Acquired by McAfee in 2009, he then continued with a role as Vice President of Threat Intelligence.
Well known for establishing some of the largest Government security operations centers, Chris changed his career, starting a security service company in 2003 and a research & development company in 2004.
Both companies have since been acquired, and with retiring from McAfee in 2012 founded Fluency® with longtime friend and coworker Kun Luo.
Sign up for the free insideBIGDATA newsletter.
.