Detection Engineering is a critical element of any security program for both endpoints and networks, helping detect attacker patterns that bypass other defenses. Through using detection rules to identify these patterns of attack, detection Engineering allows security programs to better defend their infrastructure against threats.
Effective security teams often rely on frameworks such as YARA and Sigma to write detections in an organized, automated fashion before deploying them through a continuous integration/continuous deployment pipeline, helping shift security right and reduce response times.
Detection-as-Code
Detection engineering is an emerging approach to automating threat detection processes. Traditional Security Information and Event Management (SIEM) and Security Operations Centers (SOC) rely heavily on static rules and manual processes; with detection engineering’s innovative techniques creating more efficient ways of detecting, prioritizing, and responding to security alerts.
Deploying detection engineering combines the advantages of software development – expressibility, testing and version control – with its ability to identify behaviors which could lead to breaches. This new model for creating and deploying threat detection content could bring standardization, sustainability and reliability in its detection outputs.
Early attempts at detection engineering focused on malware hashes and communication patterns between threats to bypass firewalls; but today it takes on more of a targeted approach, scrutinizing specific system events and filename changes. With automation tools like Python scripts and YARA rules becoming more popular among cybersecurity teams, this strategy has become more widely practiced.
At its core, detection engineering aims to strike an equilibrium between coverage, precision and alert volume. One effective strategy to do so is implementing a flexible detection-as-code strategy that supports an agile CI/CD process; this may include automated tests, linting as well as using a repository for revision review or reuse of detections.
Effective detection-as-code processes must include an easy-to-use GUI that allows users to rapidly create and modify rules without altering source code. This ensures that detections can be tested by engineers before being deployed to production automatically and thus helping reduce mean time-to-response times.
Implementing a detection-as-code strategy takes both time and effort, but its rewards far outweigh its initial costs. By supporting agile CI/CD pipelines with reliable detection-as-code workflows that deliver consistent quality detections while saving both detection engineers and security analysts time by eliminating manual work required for threat identification, detection-as-code can accelerate SOC success significantly.
Continuous Integration/Continuous Deployment
Continuous deployment takes Continuous Integration a step further by automating the release process so every change that passes automated testing is automatically deployed into production. This approach ensures application updates can be deployed as rapidly as possible, increasing delivery speed and improving business value.
Organizations with functioning continuous integration workflows will also typically possess functional continuous deployment (CD) pipelines, and the two processes should complement one another. However, CD should be seen as part of continuous integration rather than its final stage – and must complete all stages before moving onto production deployment, which may take considerable time and effort.
Continuous integration entails developers committing small changes at least daily and, often multiple times a day, and having them tested by an automated testing server before being integrated into the main codebase. If any problems with a developer’s work are discovered by this server, it will stop merging altogether and alert their team immediately – in an ideal world, this should occur concurrent with development so any issues can be rectified quickly.
Once a developer feels that their code is ready for release, they should merge it into the Master/Trunk/Mainline branch and push it out through our Continuous Integration server to staging or QA environments for further testing before voting amongst developers about whether or not it should go live in production.
Once a change passes QA, it will then be sent into production and made available to customers. Depending on your company’s needs, this process can be modified in order to gradually rollout new features or minimize risks through canary releases or blue/green releases.
Continuous deployment processes offer numerous advantages to developers, chief among them the speed at which you can develop without waiting for manual releases to complete development and more stable and reliable software release processes. But continuous deployment should not be seen as a replacement for traditional development methodologies – which remain highly effective in some situations.
Repository
At the center of detection engineering is creating a repository for your threat detection rules, commonly referred to as “detection signatures.” This repository includes both physical and virtual rules as well as metadata on when and how each one was deployed to production, which your security tools use to search against threat data for matches and trigger alerts upon finding matches; additionally, using such a repository provides insight into its success as part of DE program monitoring efforts.
Utilizing a central repository allows for easier collaboration, test-driven development and version control management. Furthermore, eliminating duplicate work helps reduce false positives while keeping security teams focused on the most pressing threats and incidents first. To maximize effectiveness of such an initiative, its methods should include threat models, pen testing/purple teaming/sandboxing/adaptive honeypot deployment among others.
Detection engineers should use intelligence sources such as MITRE’s ATT&CK framework and other threat intel sources like cyber news feeds and industry information sharing and analysis centers in order to rapidly detect threats that have bypassed other security defenses and reduce dwell time quickly by quickly recognizing them when they occur. A good intelligence strategy entails working closely with threat hunters, content developers and red team members in order to establish an efficient detection system capable of recognizing attacks as they happen.
As threat actors constantly adapt their tactics, attackers, and techniques, so should detection content. But this shouldn’t overwhelm a security team with alerts; to avoid this scenario, leverage a detection engine like Falcon LogScale Community Edition (formerly Humio).
At this final stage of Detection Engineering, it’s crucial that detections you’ve created are tailored precisely against the threats you face. This requires striking an ideal balance among factors like relevance, importance and impact – with relevance ensuring your content doesn’t become too broad for alert fatigue; while narrow detection might mean missing attacks against your organization.
Maintenance
As detection capabilities are designed, tested and deployed into production, security teams must maintain them. This involves minimizing false positives while increasing detection content quality and closing gaps in detection capabilities. Furthermore, threat actor behavior needs to be prioritized so the team has enough resources to address new threats as they emerge.
Detection Engineering seeks to design a system capable of quickly recognizing malicious activity without delay, using intelligence gleaned from various sources like threat hunters, content developers and red team members in an organized fashion.
Detection engineers are responsible for designing, testing, and maintaining detection capability – whether in the form of rules, saved searches, reports that focus on specific artifacts or meta-characteristics of threats; as well as ensuring their system has sufficient coverage against attackers exploiting weaknesses elsewhere; this practice is known as Endpoint Detection and Response (EDR) on endpoints and Network Detection and Response (NDR) for networks.
Not only is new detections necessary, but identifying detections that no longer apply – for instance if their threat model has changed – should also be an essential aspect. A central repository storing versioned and shared detections would greatly facilitate collaboration while simultaneously decreasing time to resolution in production environments.
Falcon LogScale Community Edition – now powered by the ATT&CK threat hunting framework – is an example of modern log management platform, providing a centralized repository to facilitate collaboration and faster detection development. It utilizes an industry-standard coding framework and test-driven development (TDD) process, to expose blind spots quickly while expediting design processes faster while improving detection consistency.
Detection Engineering is an essential and rapidly expanding part of cybersecurity, and professionals with skills in this field are expected to become increasingly sought after in coming years. To pursue a career as a Detection Engineer, individuals interested may pursue degrees in computer science or information technology, take internships or entry-level positions in this area, obtain industry certifications and network with others within the field.
