I’ve decided to tie this post and one other on Building an Effective Security Operations Program together instead of posting them weeks apart. I wanted to focus on the high level aspect of building out a proper security operations focused on Detection and Response, but I also felt like we needed to really dig into the weeds a bit.

That is what I’m hoping to accomplish with this post. Now, some pieces may be redundant if you’ve read the first part as every section contains some summaries, but the core of it digs a bit deeper into the matter at hand. I do plan on following up this post with an in-depth look at some of the SIEMs on the market and how to approach some of these key topics.

1. The Foundation of SIEM Logging in SecOps

A SIEM is the beating heart of any detection and response effort, and it relies heavily on comprehensive logging to provide visibility into the environment. Effective SIEM logging allows SecOps teams to collect, correlate, and analyze massive volumes of data from diverse sources, which is crucial for identifying and responding to incidents in real time.

Key Principles of SIEM Logging:

• Collect Everything (but with Focus): While it’s tempting to log “all the things,” a targeted approach can help avoid data overload. Prioritize high-value log sources that provide critical insights, such as:
  • Network Security Logs from firewalls, VPNs, and Intrusion Detection/Prevention Systems (IDS/IPS) that show inbound and outbound traffic patterns.
  • Endpoint Logs that capture details on user activity and system behavior.
  • Authentication and Identity Logs that track login events across cloud services, Active Directory, and local machines.
  • Application and Database Logs to monitor activity on sensitive data repositories.
  • Cloud Logs from providers like AWS, Azure, and GCP, which reveal access and activity in cloud resources.

Side Note: If you’re just starting out and you’re not sure where to start, a good rule of thumb is around the biggest attack surface, Identity/Authentication & Endpoint logs, you’ll catch a great deal of attacks with just those logs and you can continually build from there.

• Normalization and Parsing of Logs: Normalizing logs ensures they are structured in a way that makes correlation across sources easier. Parsing these logs into a standard format allows SIEM systems to interpret them consistently, enabling more accurate detections and faster analysis.
• Log Storage and Retention: Storing logs is essential for both compliance and incident analysis. Establish retention policies that balance regulatory requirements and storage costs, ensuring data remains accessible for future analysis or forensic investigations.

2. Crafting Threat Detection within the SIEM

Once the data is ingested into the SIEM, the next step is setting up robust detection rules. Effective threat detection requires continuous refinement of alerts to accurately identify threats while minimizing false positives.

Key Components of Threat Detection:

• Baseline Normal Behavior: One of the most effective ways to detect threats is by knowing what “normal” looks like in your environment. By establishing a baseline, SOC teams can quickly spot anomalies that may indicate malicious activity.
• Event Correlation Rules: Event correlation ties different log events together to form a pattern of suspicious behavior. For example, a failed login followed by a successful login from an unusual location might indicate a compromised account. These correlation rules can help surface threats that would be missed by single-event detections.
• Utilize MITRE ATT&CK Framework for Detection Development: Aligning detections with the MITRE ATT&CK framework can be highly beneficial. Each detection can map to specific tactics, techniques, and procedures (TTPs), allowing the team to focus on identifying behaviors known to be indicative of malicious actors.
• Threshold-Based Alerts vs. Behavioral Analytics: Threshold-based alerts (e.g., alert if more than 10 failed logins occur within a minute) are effective but can be noisy. Behavioral analytics, on the other hand, identifies abnormal patterns over time, such as unusual network traffic from a single host. While threshold alerts can be noisy, behavioral alerts are prone to have high false positives in the beginning. Behavioral alerts tend to lean on ML which always requires a baselines before becoming effective.

3. Detection Engineering: Crafting Precise and Actionable Alerts

Detection engineering is a critical discipline within SecOps, focusing on developing, testing, and refining detection rules that accurately identify threats with minimal false positives. This process requires deep knowledge of the threat landscape, technical expertise, and continuous tuning to stay effective against evolving attacks.

Best Practices in Detection Engineering:

• Create Custom Detections Tailored to Your Environment: Generic detection rules often result in a high number of false positives. Customizing rules to fit your specific environment and user behaviors reduces noise, enabling analysts to focus on high-fidelity alerts.
• Test and Validate Detections Regularly: Every detection should go through a rigorous testing process to ensure it works as expected without generating excessive noise. Regular testing allows detection engineers to adjust for changes in the environment or in threat actor techniques.
• Enhanced Coverage and Granularity: Mapping detections to MITRE techniques provides insight into specific actions taken by attackers, allowing SOCs to cover the full scope of potential threat behaviors.
• Standardization Across Detections: By mapping detections to MITRE, SOC teams have a structured language to communicate threat behaviors, which is especially helpful for reporting and improvement.
• Emphasize Threat Context in Alerts: Alerting without context can overwhelm SOC teams. Including context—such as which tactic the alert relates to (e.g., credential access, privilege escalation) or links to recent threat intelligence—helps analysts triage alerts more efficiently.
• False Positive Management and Alert Tuning: A common issue in SIEM is “alert fatigue,” where too many alerts overwhelm analysts. By fine-tuning alerts and adjusting thresholds, detection engineers can maintain a balance between sensitivity and accuracy.

4. Leveraging Threat Intelligence in Detection Engineering

Threat intelligence plays a significant role in making detections more relevant and targeted by incorporating information on real-world adversaries and emerging threats. Integrating threat intelligence into the SIEM allows detection engineers to build alerts based on Indicators of Compromise (IoCs) and known tactics used by threat actors.

Ways to Incorporate Threat Intelligence into SIEM:

• Enrich Alerts with IoCs: Adding known malicious IPs, domains, and file hashes into SIEM detections can enhance alert quality. However, IoCs alone are insufficient and should be part of a broader detection strategy.
• Identify Relevant Threat Actors and Techniques: Not every organization faces the same threats. By identifying adversaries and techniques relevant to your industry, SOC teams can prioritize detections and threat-hunting activities that are most relevant.
• Drive Proactive Threat Hunting: Threat intelligence helps guide threat hunters on what to look for, such as specific TTPs that are likely to be used by threat actors targeting similar organizations. This proactive hunting approach helps uncover hidden threats.

5. Detection Engineering with Sigma: A Flexible and Adaptable Approach

One of my all time favorite things to be open sourced in the last decade is Sigma. Florian Roth is the founder and pioneer of Sigma which will help elevate any detection engineer/SOC when it comes to building out proper detection engineering practices. Sigma is an open-source, rule-based format for writing detections that can be converted into queries for various SIEM platforms. By adopting Sigma, SOC teams can write detection rules in a platform-agnostic way, simplifying rule creation, sharing, and maintenance.

https://github.com/SigmaHQ/sigma

Example Sigma-Based Detection Rules:

Here are some examples of detections written in the Sigma format. These are aligned with common techniques in the MITRE framework to provide actionable threat detection:

Dump Credentials from Windows Credential Manager With PowerShell (MITRE ATT&CK: T1555 – Credential-Access):

• Adversaries may search for common password storage locations to obtain user credentials. Passwords are stored in several places on a system, depending on the operating system or application holding the credentials.

Compress Data and Lock With Password for Exfiltration With 7-ZIP (MITRE ATT&CK: T1560.001 – Collection):

• An adversary may compress or encrypt data that is collected prior to exfiltration using 3rd party utilities.

Potential Invoke-Mimikatz PowerShell Script (MITRE ATT&CK: T1003 – Credential-Access):

• Detects Invoke-Mimikatz PowerShell script and alike. Mimikatz is a credential dumper capable of obtaining plaintext Windows account logins and passwords.

By leveraging Sigma, SOC teams can easily implement and share detection rules across different SIEM platforms, helping to standardize detection engineering practices while providing flexibility and scalability. Sigma is an all-time must for security teams looking to mature their detection engineering.

5. Continuous Improvement: Refining SIEM Detections Over Time

New attacks are released every single day and they continue to evolve, and so should your detection capabilities. Continuous improvement processes ensure that your SIEM detections remain effective against new and emerging threats.

Steps for Continuous Improvement:

• Regular Review and Optimization of Detections: Detection rules should be revisited periodically. This review can involve checking for any redundant or outdated rules and optimizing detection logic to reduce noise.
• Incident Post-Mortem Analysis: After a significant incident, conduct a post-mortem to analyze which detections triggered (or didn’t). This feedback loop is crucial for refining existing detections and identifying gaps.
• Stay Updated on Threat Trends: Regularly update detection logic based on new threat reports, TTPs, and trends observed within your industry. Leveraging sources like the MITRE ATT&CK updates and threat intelligence feeds keeps your defenses current.
• Red Team / Blue Team Exercises: Engage in red/blue/purple team exercises to simulate attacks and test the effectiveness of your detections. These exercises provide direct insight into how well your detection engineering holds up under real-world attack scenarios.

6. Future State: Automating Detection Engineering with a CI/CD Pipeline

We’ve implemented all of the above including Sigma which moves us towards Detection as Code. Now, we’re ready to really mature our detection engineering process by introducing CI/CD into the mess. I’m going to do a future write up on this entire process and really walk you through how to do it, but for now we’ll keep it high level.

A CI/CD pipeline for detection engineering allows security teams to iterate and deploy detection logic quickly while maintaining quality and accuracy. This automation can accelerate response times, reduce human error, and ensure detections are kept current with the evolving threat landscape.

Key Components of a CI/CD Pipeline for Detection Engineering:

1. Version Control for Detection Rules: Storing detection rules in a version control system (like Git) is foundational to CI/CD. Version control allows teams to track changes, manage rule versions, and collaborate on updates. It also provides a history of detection modifications, useful for auditing and rollbacks.
   • I honestly, cannot tell you how many SIEM vendors would benefit so much by keeping up with version history for their detection rules.
2. Automated Testing and Validation: Automated testing ensures that new detection rules are validated for quality and effectiveness before deployment. This might involve:
   • Unit Testing: Checking that the rule logic functions as expected in different scenarios.
   • False Positive Testing: Running detection rules against normal activity logs to identify potential noise and ensure rules don’t generate excessive false positives.
   • Simulation Testing: Using red team data or attack simulations to test the detection rule’s ability to identify actual threat activity.
3. Automated Deployment: After passing all tests, detection rules can be automatically deployed to the SIEM or EDR environment. This can be achieved using CI/CD tools like Jenkins, GitLab CI/CD, or other automation platforms that support rule deployment workflows.
4. Feedback Loop for Continuous Improvement: Once deployed, detection rules should feed back into a continuous improvement process.
5. Environment-Specific Rule Adjustments: In larger organizations, different environments (e.g., production, staging, testing) may require tailored rules. A CI/CD pipeline can manage environment-specific configurations, allowing deployment to the correct environment with the appropriate modifications.

Conclusion:

I just blabbered on a whole lot there, but I’m finding more and more everyday from my own experiences over the last few years and talking with colleagues, this is becoming an industry standard approach. Detection Engineering with your SIEM or EDR or NDR or any other tools that allow it, needs to be a constant and iterative approach, it’s not something you can set and forget because it’s going to change quick. While we focused on SIEM, a lot of these core principals can be applied to pretty much any tool that allows you to develop detections.

Again, thank you so much for reading this post! Please if you have questions, feedback, etc reach out to me on Linkedin or Twitter (X), I’d love to chat, SecOps/Detection Engineering is truly one of my great passions and I’d be happy to nerd out.

Stay tuned for the next posts, not sure which will come out next but we have more security goodness coming soon!