You have a tremendous amount of information all around you - structured data, unstructured data, sensitive data, compliant data, and more. From enterprise databases to social media posts, imagine a scenario where you could take your data in all its various forms, transform it to understandable data, apply the right analytics to it, and afterwards use it to better secure your business, become more security intelligent, and even drive more revenue as a result of that knowledge?
This is Big Data analytics is all about. Big data technologies are seen as increasingly valuable by organizations seeking to maximize their security intelligence. According to a recent study conducted by ESG based on interviewing 257 IT/Information security professionals at enterprise-class organizations based in North America, currently, 44% of companies are collecting and analyzing “Big Data” for security purposes.
The more you know about Big Data, the more you and your organization can use it for the betterment of security intelligence.
But First How Analysis is Different from Analytics?
When I was doing my post-graduation, one of my faculty asked us a question, what’s the difference between “analysis” and “analytics”. Interestingly, nobody in the class knew the correct answer. We come across these two terms almost every day and still, there is a gray area between the meanings.
Before starting, it is first necessary to understand the difference between terms “analysis” and “analytics”. One way to distinguish the difference between analysis and analytics is to think in terms of past and future. Analysis looks backward over time, providing users with a historical view of what has happened. Typically, analytics look forward to model the future or predict a result. The topic of analysis is very broad, as it can include practically any means of gaining insight from data. Even simply looking at data to gain a high-level understanding of it is a form of analysis. When we refer to analytics, however, we are generally implying the use of methods, tools, or algorithms beyond merely looking at the data.
How to Describe Analytics in Security?
Analytics integrates concepts, procedures, and techniques from a wide range of fields, such as statistics, computer engineering, visualization, and research operations. Any idea or method enabling you to recognize patterns and insights from data could be considered analytics
Incident response is one of the core areas of an effective security program. Good incident response capabilities enable organizations to contain incidents and eradicate and recoup from the effects of an incident to their information resources.
In any case, to effectively eradicate and recover from a security incident, an incident responder should have the capacity to identify the root cause of an incident. For instance, suppose your organization’s corporate website got hacked. The organization can simply restore the site using backups however without knowing the root cause, you would neither know the vulnerability causing the hack nor would you know what to fix so that the website does not get hacked again. You also might not know the full degree of the harm and damage done, or what information may have been stolen.
How does an incident responder know what to fix? To start with, the responder must have the capacity to trace the activities attributed to the intruder. These can be found in different data sources such as logs, alerts, traffic captures, and attacker artifacts. In most cases, a responder will begin off with logs, as they can help with discovering activities that can be traced back to the intruder. By tracing the activities of the intruder, an incident responder is able to create a history of the attack, thereby detect and identify possible “points of entry” of the intrusion.
What are these logs and how do we get them? This really depends on the type of intrusion to which you are responding. For example, in web compromises, an incident responder will ordinarily take a look at web server logs, but remember that this is not always the case. Some attack vectors show up in completely different data sources, which is why evaluating diverse and different data sources is important.
Problems in Collecting Data for Security Analytics
Much of the challenge in performing security analytics comes from the irregular and unpredictable data that the analyst must deal with. There is no single standard data arrangement or set of data definitions relating to data produced by computer systems and networks. For instance, each server software package produces its own log file format. Additionally, these formats can generally be customized by users, which adds to the difficulty of building standard software tools for analyzing the data.
Another factor additionally complicating the analysis is that log files and other source data are generally created in plain text format, instead of being organized into tables or columns. This can make it troublesome or even impossible to import the data directly into recognizable and familiar analytical tools, such as Microsoft Excel.
Additionally, security-related data is increasingly becoming too large to analyze with standard tools and methods. Large organizations may have multiple large data centers with an ever-growing collection of servers that are together by sprawling networks. All of this generates a huge volume of log files, which takes us into the realm of Big Data.
How can Big Data Come to Rescue?
Throughout the years, organizations have expanded the amount of data they collect. They are presently at the point where maintaining large data repositories is part of their business model—which is where the buzzword phrase “big data” emerges.
The main driving force behind the “hype” for big data is the need for businesses to have the intelligence to make business decisions. Innovative technology is not the primary reason for the growth of the big data industry—in fact, many of the technologies used in data analysis, such as parallel and distributed processing, and analytics software and tools, were already available. Changes in business practices (e.g., a shift to the cloud) and the application of techniques from other fields (engineering, uncertainty analysis, behavioral science, etc.) are what is driving the development and growth of data analytics. This emerging area created a new industry with experts (data scientists), who are able to examine and configure the distinctive sorts of data into usable business intelligence.
Many of the same analytical methods can be applied to security. These methods can be utilized to uncover relationships within data produced by servers and networks to uncover and reveal intrusions, denial of service attacks, attempts to install malware or even fraudulent activity.
One of the main challenges in incident response is the sheer amount of data to review. Even reviewing the logs from a busy web server for one day can be a challenge. Imagine a scenario in which a responder has to review several years of logs? Aside from this, what if a responder had to review multiple server logs during the same time period? The data an incident responder has to sift through would be immense—potentially millions of lines of log information!
This is where analytics and big data techniques become possibly the most important factor. Using big data techniques, an incident responder will have the capacity to combine many data sources with different structures together. Once that is finished, analytics techniques such as fuzzy searches, outlier detection, and time aggregations can be utilized to “crunch” the data into more manageable data sets so responders can focus their investigations on a smaller, more relevant subset of the data.
Security analysis can range from simple observation by querying or visualizing the data, to applying sophisticated artificial intelligence applications. It can involve the use of simple spreadsheets on small samples of data, to applying big data, parallel computing technologies to store, process and analyze terabytes, or even petabytes of data.
Four Ways in Which Big Data will Transform Security Analytics
The goal of Big Data analytics for security is to obtain actionable intelligence in real time.
New Big Data technologies are empowering the storage and analysis of large heterogeneous data sets at an extraordinary scale and speed. These technologies will transform security analytics by (a) gathering data at a gigantic scale from numerous internal enterprise sources and external sources such as vulnerability databases, (b) performing further deeper analytics on the data, (c) giving a consolidated view of security-related information and (d) accomplishing real-time analysis of streaming data. It is imperative to note that Big Data tools still require system architects and analysts to have a profound learning of their system in order to properly configure the Big Data analysis tools.
Authored By - Aditya Vats
TCS Cyber Security Practice
TCS Cyber Security Practice
Rate this article: