security-tools/sigma-rules
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Add documentation and update license notice

Browse Source
This commit is contained in:
Ross Wolf
2020-06-29 23:21:06 -06:00
parent d51474f0a7
commit fb0d36941c
5 changed files with 463 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+258
View File
@@ -0,0 +1,258 @@
# Contributing to Detection Rules
Thank you for your interest in contributing to Detection Rules. We've crafted this document to make it simple and easy for you to contribute. We recommend that you read these contribution guidelines carefully so that you spend less time working on GitHub issues and PRs and can be more productive contributing to this repository.
These guidelines will also help you post meaningful issues that will be more easily understood, considered and resolved. These guidelines are here to help you whether you are creating a new rule, opening an issue to report a false positive or requesting a feature.
## Table of Contents
- [Effective issue creation in Detection Rules](#effective-issue-creation-in-detection-rules)
- [Why we create issues before contributing code or new rules](#why-we-create-issues-before-contributing-code-or-new-rules)
- [What a good issue looks like](#what-a-good-issue-looks-like)
- ["My issue isn’t getting enough attention"](#my-issue-isnt-getting-enough-attention)
- ["I want to help!"](#i-want-to-help)
- [How We Use Git and GitHub](#how-we-use-git-and-github)
- [Forking](#forking)
- [Branching](#branching)
- [Commit Messages](#commit-messages)
- [What Goes Into a Pull Request](#what-goes-into-a-pull-request)
- [Our approach to detection engineering](#our-approach-to-detection-engineering)
- [Rule metadata](#rule-metadata)
- [Using Elastic Common Schema (ECS)](#using-elastic-common-schema-ecs)
- [Creating a rule with the CLI](#creating-a-rule-with-the-cli)
- [Testing a rule with the CLI](#testing-a-rule-with-the-cli)
- [Writing Style](#writing-style)
- [Signing the contributor license agreement](#signing-the-contributor-license-agreement)
- [Submitting a Pull Request](#submitting-a-pull-request)
- [What to expect from a code review](#what-to-expect-from-a-code-review)
- [How we handle merges](#how-we-handle-merges)
## Effective issue creation in Detection Rules
### Why we create issues before contributing code or new rules
We generally create issues in GitHub before contributing code or new rules. This helps front-load the conversation before the rules. There are many rules that will make sense in one or two environments, but don't work as well in general. Some rules are over-fitted to a particular indicator or tool. By creating an issue first, it creates an opportunity to bounce our ideas off each other to see what's feasible and what ways to approach detection.
By contrast, starting with a pull request makes it more difficult to revisit the approach. Many PRs are treated as mostly done and shouldn't need much work to get merged. Nobody wants to receive PR feedback that says "start over" or "closing: won't merge." That's discouraging to everyone, and we can avoid those situations if we have the discussion together earlier in the development process. It might be a mental switch for you to start the discussion earlier, but it makes us all more productive and and our rules more effective.
### What a good issue looks like
We have a few types of issue templates to [choose from](https://github.com/elastic/detection-rules/issues/new/choose). If you don't find a template that matches or simply want to ask a question, create a blank issue and add the appropriate labels.
* **Bug report**: Create a report to help us improve (not pertaining to rules)
* **Feature request**: Suggest an idea for this project (not pertaining to rules)
* **New rule**: Suggestions and ideas for new rules for the Detection Engine
* **Release package**: Used by the Elastic team to track a rules release
* **Rule deprecation**: Recommend deprecating a rule that doesn't work or isn't useful anymore
* **Tune existing rule**: Suggest changes to make to an existing rule to address false positives or negatives
When requesting a **New rule**, please create an issue of the **New rule** type. The issue contains a handful of questions about the targeted behavior and the approach to detection:
* What are the matching MITRE ATT&CK® technique and tactics?
* What data sources are needed?
* Does a detection need fields that aren't in ECS yet?
* Is the technique behavior-based, or is it based on indicators of compromise?
### "My issue isn't getting enough attention"
First of all, **sorry about that!** We want you to have a great time with Detection Rules.
We'll tag issues and pull requests with the target release. If a rule is blocked by a feature, we'll add a label to reflect that. With all of the issues, we need to prioritize according to impact and difficulty, so some issues can be neglected while we work on more pressing issues.
Of course, feel free to bump your issues if you think they've been neglected for a prolonged period.
### "I want to help!"
**Now we're talking**. If you have a bug fix or new rule that you would like to contribute to Detection Rules, please **find or open an issue about it before you start working on it.** Talk about what you would like to do. It may be that somebody is already working on it, or that there are particular issues that you should know about before implementing the change.
We enjoy working with contributors to get their code accepted. There are many approaches to fixing a problem and it is important to find the best approach before writing too much code.
## How We Use Git and GitHub
### Forking
We follow the [GitHub forking model](https://help.github.com/articles/fork-a-repo/) for collaborating on Detection Rules rules. This model assumes that you have a remote called `upstream` which points to the official Detection Rules repo, which we'll refer to in later code snippets.
### Branching
This repository follows a similar approach to other repositories within the [Elastic](https://github.com/elastic) organization, with a few exceptions that make our life easier. One way this repository is simpler is the lack of major version breaking changes. This means we have less backport commits to worry about and makes us a little more productive.
The basic branching workflow we follow for Detection Rules:
* All changes for the next release of rules are made to the `main` branch
* During feature freeze for a release, we will create a branch from `main` for the release version `{majorVersion.minorVersion}`. This means that we can continue contributing to `main`, even during feature freeze, and it will just target `{majorVersion.minorVersion+1}`
* For bug fixes and other changes targeting the pending release during feature freeze, we will make those contributions to `{majorVersion.minorVersion}`. Periodically, we will then backport those changes from `{majorVersion.minorVersion}` to `main`
### Commit Messages
* Feel free to make as many commits as you want, while working on a branch.
* Please use your commit messages to include helpful information on your changes. Commit messages that look like `update` are unhelpful to reviewers. Try to be clear and concise with the changes in a commit. For example: `Add Sysmon support to MsBuild network rule`. Here's a [good blog](https://chris.beams.io/posts/git-commit/) on general best practices for commit messages.
### What Goes Into a Pull Request
* Please include an explanation of your changes in your PR description.
* Links to relevant issues, external resources, or related PRs are very important and useful.
* Please try to explain *how* and *why* your rule works. Can you explain what makes the logic sound? Does it actually detect what it's supposed to? If you include the screenshot, please make sure to crop out any sensitive information!
* Please try to capture the expectations for noise levels: is the rule prone to false positives or false negatives?
* See [Submitting a Pull Request](#submitting-a-pull-request) for more info.
## Our approach to detection engineering
Contributions to Detection Rules are ultimately integrated with the Detection Engine within the Security Application of Kibana. The rules in this repository[*](#maturity-note) will be bundled in the next release and available to all users with access to the Detection Engine. For that reason, we want to keep the bar high and avoid rules that lead to high volumes of false-positives (FPs) or have significant performance impact on a cluster. You can use *Exceptions* in the Detection Engine to add allowlist exceptions when a rule generates an FP. That gives some tolerance of FPs, but we still want to keep numbers as low as we can.
For more information on our approach to writing threat-based detection logic, please read our [philosophy](PHILOSOPHY.md) page.
<a id="maturity-note">\* Note:</a> Specifically, rules that contain `maturity = "production"` will be included in the next stack release.
### Rule metadata
Detection logic in itself is not enough to be useful to practitioners. Rules need to contain more information, like a name, description and severity levels within the metadata. Try to be thorough with the metadata you provide for a rule. Some of the information is required for a rule to run, other information is provided to the user enabling the rule, and some information is also invaluable context to users that triage the alerts generated by a rule.
Some of the required metadata captured in a rule file:
| field | required | description |
| -------------------- | -------- | ------------------------------------------------------------------------------- |
| **description** | ✓ | Brief one-two sentence description for what the rule detects |
| **enabled** | | Default status of the rule, automatically enabled if `true` |
| **false_positives** | | Array of markdown strings for guidance on triaging false positives |
| **filters** | | Array of query DSL filters to `and` with the query |
| **from** | | Relative start time for a rule (e.g. `now-6m`) |
| **index** | | List of index patterns that stores the needed events |
| **interval** | | Interval between executions of the rule |
| **language** | ✓ | Query language for language-based rules (e.g. `kuery`, `lucene`, `eql`) |
| **max_signals** | ✓ | Cutoff for the maximum number of signals in an execution before dropped results |
| **name** | ✓ | A short title for the rule |
| **note** | | Additional triage notes or details on the rule beyond `description` |
| **query** | ✓ | The query language code for rules of type `query` |
| **risk_score** | | Integer to rank the risk relative to other rules. Leave blank if unknown |
| **rule_id** | ✓ | Automatically generated UUID for the rule |
| **severity** | ✓ | Severity of the matching results (e.g. `low`, `medium`, `high`, `critical`) |
| **tags** | | Array of tags for grouping the rule (e.g. `APM`, `Linux`, `Packetbeat`, ...) |
| **threat** | ✓ | Mapping to a threat framework, such as MITRE ATT&CK® |
| **to** | | Relative end time of a rule (e.g. `now`) |
| **type** | ✓ | Execution type of the rule (`query` or `machine_learning`) |
### Using Elastic Common Schema (ECS)
Our rules should be written generically when possible. We use [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html) to standardize data before ingesting into Elasticsearch. ECS gives a set of field sets, field names and categories to standardize events across various tools. By writing rules using ECS fields and values, you can reuse the same logic regardless of data source. ECS is an ongoing effort, and we may encounter fields that aren't present yet. If you need to make any requests to ECS, see the [elastic/ecs](https://github.com/elastic/ecs) GitHub repository.
Sometimes, it might not make sense for ECS to standardize a field, value or category. Occasionally, we may encounter fields that specific to a single use-case or vendor. When that happens, we add an exception in [etc/non-ecs-schema.json](etc/non-ecs-schema.json). We automatically detect beats by looking at the index patterns used in a rule. If we see `winlogbeat-*`, for example, then we can validate the rule against ECS + Winlogbeat. When using a particular beat, please use `event.module` and `event.dataset` to make the rule more precise and to better nudge the validation logic.
When a Pull Request is missing a necessary ECS change, please add an issue to [elastic/ecs](https://github.com/elastic/ecs) and link it from the pull request. We don't want to leave PRs blocked for too long, so if the ECS issue isn't progressing, then we can add a note and use the vendor- or beat-specific fields. We'll create another issue, reminding us to update the rule logic to switch to the ECS field when it becomes available. To maximize compatibility, we may add an `or` clause for a release or two to handle the different permutatations. After a few releases, we'll remove this and strictly require the ECS fields.
### Creating a rule with the CLI
We manage our repository with a command line tool that automatically creates TOML files, validates rules and bundles all rules for the Detection Engine. There's a lot of metadata for each rule, and manually copying and pasting rule files is error prone and tedious. To create a new rule, run the command below, which iterates through the required metadata, and prompts for each field.
For example, to create a new rule file for `rules/windows/defense_evasion_msbuild_child.toml`, run the command
```console
$ python -m detection_rules create-rule rules/windows/defense_evasion_msbuild_child.toml
```
The command will prompt you for each required field in the metadata
```
Rule type (machine_learning, query, saved_id): query
actions (multi, comma separated):
description (required): Look for child processes of MsBuild
enabled [false] ("n/a" to leave blank):
from [now-6m] ("n/a" to leave blank):
false_positives (multi, comma separated):
filters (multi, comma separated):
interval [5m] ("n/a" to leave blank):
exceptions_list (multi, comma separated):
max_signals [100] ("n/a" to leave blank):
meta:
name (required): Suspicious Child of MsBuild
note:
references (multi, comma separated):
risk_score [21] ("n/a" to leave blank) (required):
rule_id [90d0c543-e197-46d8-934d-0320b2c83486] ("n/a" to leave blank) (required):
severity [low] ("n/a" to leave blank) (required): medium
tags (multi, comma separated): Windows
throttle:
timeline_id:
timeline_title:
to [now] ("n/a" to leave blank):
threat (multi, comma separated):
index (multi, comma separated): winlogbeat-*
language [kuery] ("n/a" to leave blank) (required): kuery
query (required): event.category:process and process.parent.name:msbuild.exe
ecs_version (multi, comma separated): 1.4.0
```
Pending no errors, you'll see this output upon success
```
Rule Suspicious Child of MsBuild saved to rules/windows/defense_evasion_msbuild_child.toml
Did not set the following values because they are un-required when set to the default value
- from
- interval
- max_signals
- to
```
### Testing a rule with the CLI
When a rule is ready, it can be tested with unit tests. Detection Rules has several tests that run locally to validate rules in the repository. These tests make sure that rules are syntactically correct, use ECS or Beats schemas correctly, and ensure that metadata is also validated. There are also internal tests to make sure that the tools and functions to manager the repository are working as expected.
To run tests, simply run the command `test` with the CLI
```console
$ python -m detection_rules test
============================================================= test session starts ==============================================================
collected 73 items
tests/test_all_rules.py::TestValidRules::test_all_rule_files PASSED [ 1%]
tests/test_all_rules.py::TestValidRules::test_all_rule_queries_optimized PASSED [ 2%]
tests/test_all_rules.py::TestValidRules::test_all_rules_as_rule_schema PASSED [ 4%]
tests/test_all_rules.py::TestValidRules::test_all_rules_tuned PASSED [ 5%]
...
tests/kuery/test_parser.py::ParserTests::test_number_exists PASSED [ 98%]
tests/kuery/test_parser.py::ParserTests::test_number_wildcard_fail PASSED [100%]
========================================================================== 73 passed in 45.47s ==========================================================================
```
## Writing Style
Our rules are much more than queries. We capture a lot of metadata within the rules, such as severity, indexes and noise level. We also have several fields that are user-readable text, such as `name`, `description`, `false_positives`, `investigation_notes` and `name`. Those fields, which are populated with English text[*](#i18n-note), should follow the [Elastic UI writing guidelines](https://elastic.github.io/eui/#/guidelines/writing). We want our text to be *clear* and *concise*, *consistent* and *conversational*.
<a id="i18n-note">\* Note</a>: We currently don't have i18n support for Detection Rules.
## Signing the contributor license agreement
Please make sure you've signed the [Contributor License Agreement](http://www.elastic.co/contributor-agreement/). We're not asking you to assign copyright to us, but to give us the right to distribute your code without restriction. We ask this of all contributors in order to assure our users of the origin and continuing existence of the code. You only need to sign the CLA once.
## Submitting a Pull Request
Push your local changes to your forked copy of the repository and submit a Pull Request. In the Pull Request, describe what your changes do and mention the number of the issue where discussion has taken place, e.g., "Closes #123".
Always submit your pull against `main` unless you are making changes for the pending release during feature freeze (see [Branching](#branching) for our branching strategy).
Then sit back and wait. We will probably have a discussion in the pull request and may request changes before merging. We're not trying to get in the way, but want to work with you to get your contributions in Detection Rules.
### What to expect from a code review
After a pull is submitted, it needs to get to review. If you have commit permissions on the Detection Rules repo you will probably perform these steps while submitting your Pull Request. If not, a member of the Elastic organization will do them for you, though you can help by suggesting a reviewer for your changes if you've interacted with someone while working on the issue.
Most likely, we will want to have a conversation in the pull request. We want to encourage contributions, but we also want to keep in mind how changes may affect other Elastic users. Please understand that even if a rule is working in your environment, it still may not be a good fit for all users of Elastic Security.
### How we handle merges
We recognize that Git commit messages are a history of all changes to the repository. We want to make this history easy to read and as concise and clear as possible. When we merge a pull request, we squash commits using GitHub's "Squash and Merge" method of merging. This keeps a clear history to the repository, since we rarely need to know about the commits that happen *within* a working branch for a pull request.
The exception to this rule is backport PRs. We want to maintain that commit history, because the commits within a release branch have already been squashed. If we were to squash again to a single commit, we would just see a commit "Backport changes from `{majorVersion.minorVersion}`" show up in main. This would obscure the changes. For backport pull requests, we will either "Create a Merge Commit" or "Rebase and Merge." For more information, see [Branching](#branching) for our branching strategy.
NOTICE.txt
+32
View File
@@ -1,2 +1,34 @@
Detection Rules Detection Rules
Copyright 2020 Elasticsearch B.V. Copyright 2020 Elasticsearch B.V.
---
This product bundles rules based on https://github.com/BlueTeamLabs/sentinel-attack
which is available under a "MIT" license. The files based on this license are:
- defense_evasion_via_filter_manager.toml
- discovery_process_discovery_via_tasklist_command.toml
- persistence_priv_escalation_via_accessibility_features.toml
- persistence_via_application_shimming.toml
- defense_evasion_execution_via_trusted_developer_utilities.toml
MIT License
Copyright (c) 2019 Edoardo Gerosa, Olaf Hartong
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
PHILOSOPHY.md
+65
View File
@@ -0,0 +1,65 @@
# Philosophy
Rule development can be hotly debated and there are many ideas for what makes a detection rule *good*. We hear about arguments between *Indicators of Compromise* vs. *Indicators of Attack* and *signatures* vs. *rules*. Instead of boring ourselves with those re-hashed discussions, we want to share our approach for rule writing and our expectations of this repository.
## Approach
Our goal is to improve detection within Elastic Security, while combating alert fatigue. When we create a rule, we often approach it from this perspective. To make sure a rule is a complete and a good candidate for Detection Rules, consider asking these questions:
* Does this rule improve our detection or visibility?
* Does it strike a good balance between true positives, false positives and false negatives?
* How difficult is it for an attacker to evade the rule?
* Is the rule written with [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html) in mind? Is the logic data-source agnostic or does it depend on specific beats or agents?
### Behavioral Rules
Based on our approach, we tend to prefer rules that are more *behavioral* in nature. Behavioral rules are rules that focus on the attacker technique, and less on a specific tool or indicator. This might mean more research and effort is needed to figure out how a technique works. By taking this approach, we do a better job detecting and stopping the attacks of today and tomorrow, instead of the attacks of yesterday.
### Signatures and Indicators
Even though we gravitate towards behavioral or technique-based rules, we don't want to automatically disqualify a rule just because it uses indicators of a specific actor or tool. Though those are typically more brittle, they tend to have less noise, because they are specifically written to detect exactly one thing. Sometimes tools are used across multiple actors or red teams, and a signature could go a long way.
One example would be a detection for the common open-source tool [mimikatz](http://github.com/gentilkiwi/mimikatz), which is used by many red teams and in real world incidents. It dumps credentials by requesting read access to the `lsass.exe` process and decrypt passwords from memory. This technique is often too low-level for some tools. One way to detect it would be to look for special flags in the command line or inside the file itself, such as `sekurlsa::logonpasswords` or `sekurlsa::wdigest`. Those indicator-based detections are less effective these days, because `mimikatz` mostly runs in memory, so there's no command line or even a file to observe.
A better approach is to focus on the technique: remotely reading memory for `lsass.exe`. Defenders now have tools and solutions that can detect a process requesting memory access to `lsass.exe` and block or defend the behavior natively. One tool, Microsoft Sysmon, has Event ID 10: [ProcessAccess](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon#event-id-10-processaccess) which can detect the access request to `lsass.exe`. From there, the logic will need to tune out legitimate software that requests access or focus on specific flags when requesting access. Then, you get a detection that doesn't just find `mimikatz`, but can also detect other tools like ProcDump.exe requesting memory access to `lsass.exe`.
## Review questions
There are a few ways that we strive to improve our detection rates and performance when writing rules. We ask a handful of questions while developing or reviewing rules. When contributing, consider this questionnaire:
### Does the rule detect what it's supposed to?
This probably seems like an obvious question, but is a crucial and regular part of any review for a new rule. Sometimes we work backwards from a specific indicator to a general rule. But when we get there, how do we know that we're detecting other instances of the technique?
Another good reason for asking this question: others may have experience with this type of data. Someone else may be aware of false positives which original author didn't anticipate.
Maybe you're looking for suspicious privilege escalation on Windows by looking for services that spawn with `cmd /c ...` in the command line. This is behavior metasploit does when calling `getsystem`. You might write a rule like this: `process.parent.name: services.exe and process.child.name: cmd.exe`, and it would detect what you expected. But what *else* did it detect? There are [failure actions](https://docs.microsoft.com/en-us/windows/win32/api/winsvc/ns-winsvc-service_failure_actionsw) for services that can run arbitrary commands when a service fails.
Knowing this, there are a few good options to take:
* Try switching to registry events to look for the `binPath` key
* Look for Windows Event Logs for [Event ID 4697](https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventid=4697) to detect service creations
* Leave the logic in there. Maybe you're more worried about failure actions also being used maliciously and don't want to risk the false negatives by making the rule more precise
Regardless of the approach you took, document any caveats in the rule's description, false positive notes or investigation notes. This information helps users to both understand what the rule is trying to detect and will also give good information when triaging an alert.
### Does a rule have trivial evasions?
We don't want our rules to be trivial to evade. When looking for evasions in a rule, try putting on the hat of the adversary and ask yourself *how could I perform this action while going undetected by this rule?*
One way that we've seen evasions before is when matching the command line for process events. Those rules can be trivial to evade. For instance, consider the command `wmic process call create whoami.exe`.
If you search for the substring `process call create`, then all an attacker has to do is add a few more spaces: `process  call   create`. Voilà! Undetected.
Maybe the next iteration of the rule tried to avoid whitespace evasions and then opted for `* process *` and `* call *` and `* create *`. But the rule is still easy to evade by quoting the individual args with the command `wmic "process" "call" "create" "whoami.exe"`.
The **ideal** way to write the rule would be to use a parsed command line and not rely on wildcards or regular expressions. Thankfully, [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html) has the [`process.args`](https://www.elastic.co/guide/en/ecs/current/ecs-process.html#_process_field_details) field which contains exactly this. Then the KQL for the rule is simple: `process.args:(process and call and create)`.
## Resources
- [MITRE ATT&CK](https://attack.mitre.org)
- [MITRE ATT&CK philosophy](https://attack.mitre.org/docs/ATTACK_Design_and_Philosophy_March_2020.pdf)
- [Finding Cyber Threats with ATT&CK-Based Analytics](https://www.mitre.org/publications/technical-papers/finding-cyber-threats-with-attck-based-analytics)
README.md
+91
View File
@@ -0,0 +1,91 @@
# Detection Rules
Detection Rules is the home for rules used by Elastic Security. This repository is used for the development, maintenance, testing, validation, and release of rules for Elastic Security’s Detection Engine.
## Table of Contents
- [Overview of this repository](#overview-of-this-repository)
- [Getting started](#getting-started)
- [How to contribute](#how-to-contribute)
- [Licensing](#licensing)
## Overview of this repository
Detection Rules contains more than just static rule files. This repository also contains code for unit testing in Python and integrating with the Detection Engine in Kibana.
| folder | description |
|-------------------------------------- |------------------------------------------------------------------------------------ |
| [`etc/`](etc) | Miscellaneous files, such as ECS and Beats schemas |
| [`kibana/`](kibana) | Python library for handling the API calls to Kibana and the Detection Engine |
| [`kql/`](kql) | Python library for parsing and validating Kibana Query Language |
| [`rta/`](rta) | Red Team Automation code used to emulate attacker techniques, used for rule testing |
| [`rules/`](rules) | Root directory where rules are stored |
| [`detection_rules/`](detection_rules) | Python module for rule parsing, validating and packaging |
| [`tests/`](tests) | Python code for unit testing rules |
## Getting started
Although rules can be added by manually creating `.toml` files, we don't recommend it. This repository also consists of a python module that aids rule creation and unit testing. Assuming you have Python 3.7+, run the below command to install the dependencies:
```console
$ pip install -r requirements.txt
Collecting jsl==0.2.4
Downloading jsl-0.2.4.tar.gz (21 kB)
Collecting jsonschema==3.2.0
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
|████████████████████████████████| 56 kB 318 kB/s
Collecting requests==2.22.0
Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 1.2 MB/s
Collecting Click==7.0
Downloading Click-7.0-py2.py3-none-any.whl (81 kB)
|████████████████████████████████| 81 kB 2.6 MB/s
...
```
To confirm that everything was properly installed, run with the `--help` flag
```console
$ python -m detection_rules --help
Usage: detection_rules [OPTIONS] COMMAND [ARGS]...
Commands for detection-rules repository.
Options:
-h, --help Show this message and exit.
Commands:
build-release Assemble all the rules into Kibana-ready release files.
create-rule Create a new rule TOML file.
es Helper commands for integrating with Elasticsearch.
kibana-diff Diff rules against their version represented in...
load-from-file Load rules from file(s).
mass-update Update multiple rules based on eql results.
rule-search Use EQL to search the rules.
test Run unit tests over all of the rules.
toml-lint Cleanup files with some simple toml formatting.
update-lock-versions Update rule hashes in version.lock.json file...
validate-all Check if all rules validates against a schema.
validate-rule Check if a rule staged in rules dir validates...
view-rule View an internal rule or specified rule file.
```
The [contribution guide](CONTRIBUTING.md) describes how to use the `create-rule` and `test` commands to create and test a new rule when contributing to Detection Rules.
## How to contribute
We welcome your contributions to Detection Rules! Before contributing, please familiarize yourself with this repository, its [directory structure](#overview-of-this-repository), and our [philosophy](PHILOSOPHY.md) about rule creation. When you're ready to contribute, read the [contribution guide](CONTRIBUTING.md) to learn how we turn detection ideas into production rules and validate with testing.
## Licensing
Everything in this repository — rules, code, RTA, etc. — is licensed under the [Elastic License](LICENSE.txt). These rules are designed to be used in the context of the Detection Engine within the Elastic Security application. If you are using the Elastic Stack with at least the [Basic](https://www.elastic.co/subscriptions) subscription, then you can use these rules for free.
Occasionally, we may want to import rules from another repository that already have a license, such as MIT or Apache 2.0. This is welcome, as long as the license permits sublicensing under the Elastic License. We keep those license notices in `NOTICES.txt` and sublicense as the Elastic License with all other rules. We also require contributors to sign a [Contributor License Agreement](https://www.elastic.co/contributor-agreement) before contributing code to any Elastic repositories.
## Questions? Problems? Suggestions?
- Want to know more about the Detection Engine? Check out the [overview](https://www.elastic.co/guide/en/siem/guide/current/detection-engine-overview.html) in Kibana.
- If you’d like to report a false positive or other type of bug, please create a GitHub issue and check if there's an existing one first.
- Need help with Detection Rules? Post an issue or ask away in our [Security Discuss Forum](https://discuss.elastic.co/c/security/) or the **#detection-rules** channel within [Slack workspace](https://www.elastic.co/blog/join-our-elastic-stack-workspace-on-slack).
rules/README.md
+17
View File
@@ -0,0 +1,17 @@
# rules/
Rules within this folder are organized by solution or platform. The structure is flattened out, because nested file hierarchies are hard to navigate and find what you're looking for. Each directory contains several [.toml](https://github.com/toml-lang/toml) files, and the primary ATT&CK tactic is included in the file name when it's relevant (i.e. [`windows/execution_via_compiled_html_file.toml`](windows/execution_via_compiled_html_file.toml))
| folder | description |
|-------------------------------------|----------------------------------------------------------------------|
| `.` | Root directory where rules are stored |
| [`apm/`](apm) | Rules that use Application Performance Monitoring (APM) data sources |
| [`aws/`](aws) | Rules written for the Amazon Web Services (AWS) module of filebeat |
| [`cross-platform/`](cross-platform) | Rules that apply to multiple platforms, such as Windows and Linux |
| [`endpoint/`](endpoint) | Rules specifically for Elastic Endpoint Security solution |
| [`linux/`](linux) | Rules for Linux or other Unix based operating systems |
| [`macos/`](macos) | Rules for macOS |
| [`ml/`](ml) | Rules that use machine learning jobs (ML) |
| [`network/`](network) | Rules that use network data sources |
| [`okta/`](okta) | Rules written for the Okta module of filebeat |
| [`windows/`](windows) | Rules for the Microsoft Windows Operating System |