The key focus for most of Cyber Threat Defense and Response program is to achieve effective and efficient threat hunting capability. Organizations are investing signification effort and money in building the required competency and infrastructure.
Threat hunting approach thus uses the intuitive and analytical ability of the human supported by the insights provided by tools using statistical, data modeling methods, machine learning, and artificial intelligence.
Rule-based threat detection approach has many limitations (like static nature i.e. detection based on a pre-defined threshold value, inability to process data in high throughput scenario, the requirement to exponentially increase computing resources as the number of rules increases etc.) and threat hunting effectively compliments the rule-based approach by addressing the above-mentioned limitations.
The difference between Rule-Based Approach and Threat Hunting can be understood in simple form as
Rules: Watchman (Gate) or Traps (Multiple Locations)
Threat Hunting: Patrolling or Raids
The threat hunting approach works primarily on two principals first cut the available set of information in the granular form and second aggregation of a smaller set of information to make the meaningful.
The following approach or techniques may be used in performing the threat hunting.
- Visualization ( Top View )
Visualization techniques are used to obtain the top level view and empower the analysts in the identification of the areas which needs immediate attention.
- Historical Search ( Search back in time)
Historical search technique is used as starting with initial attack indicators (confirmative/non-confirmative) and move back in time to find the origin of attack or identifying any other systems or applications which are impacted (IOC sweeping across the landscape).
- Sub Searches (Narrow Down the parameters)
Sub Searches techniques are used to narrow down the detected sample by further filtering based on observed parameters to locate the attacker as well as understand the modus operandi.
- Pivot (One-dimensional view Asset or Identity or business function )
Pivot techniques are used to slice and dice the detected attack indicators pattern in multiple dimensions (asset/identity/business function/location etc.)
- Decision tree (choose the Best Path)
Decision tree technique is used in a scenario when the detected attack indicators don’t offer any conclusive pieces of evidence and there exist more than one options to explore further. Decision tree approach is thus iteratively moving further on each available path and detect the attacker.
- Anomaly Detection (Detecting Outliers against Baselining)
Anomaly detection technique is used to effectively to detect abnormal activities by finding deviations from the baseline (i.e. understanding of activity pattern over past data).
- Recursive or Repeated Pattern over time ( Automated attacks)
The recursive or repeated pattern over time technique is used to detect potentially automated attack activities.
- Cluster Analysis ( Consolidating and Aggregating )
Cluster analysis techniques are used to obtain the insight into the activities within the organizational landscape by combining the few parameters of the informational dataset.
- Association Rules ( If then Relation)
Association rules technique is used to detect the attack by establishing a relation between the available nature of data and potential attack scenario.
There exists many modeling and associated algorithms and effectiveness of the detection depend on which algorithm, test data used for enabling the system to learn etc. The security analytics practices are still in evolving. Judicial use of the one or more of above-mentioned technique and any other techniques can result in significant improvement in the organizational threat management capabilities.
Authored By - Himanshu Porwal
TCS Cyber Security Practice