Update Advanced Analytics config guides (#3302)

* Updating config guides for Advanced Analytics rules

* More updates

* Update setup instructions for LMD

* Adding more guides

* update TestRuleTiming unit test to ignore advanced analytic rules

* fixed flake error

* Moving config guides under setup instead of note

* Removing leading and trailing whitespace

* Updates as requested by PM

* Updating related integrations, minor updates to setup guides

* fixing unit tests to ignore analytic packages with multiple integration tags

* Update tests/test_all_rules.py

* fixing linting errors

---------

Co-authored-by: Kirti Kirti <kirti.kirti@elastic.co>
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
This commit is contained in:
Apoorva Joshi
2023-12-13 07:53:41 -08:00
committed by GitHub
parent a39a52360a
commit 9a9f5437f2
33 changed files with 1159 additions and 162 deletions
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["beaconing"]
integration = ["beaconing", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "Beaconing package updates and support"
min_stack_version = "8.10.1"
updated_date = "2023/10/26"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -18,20 +18,33 @@ index = ["ml_beaconing.all"]
language = "kuery"
license = "Elastic License v2"
name = "Statistical Model Detected C2 Beaconing Activity"
note = """## Setup
setup = """
The rule requires the Network Beaconing Identification integration assets to be installed, as well as network logs collected by the Elastic Defend or Network Packet Capture integrations.
The Beaconing integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Network Beaconing Identification Setup
The Network Beaconing Identification integration consists of a statistical framework to identify C2 beaconing activity in network logs.
#### Prerequisite Requirements:
- Fleet is required for Network Beaconing Identification.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Network Beaconing Identification integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Network Beaconing Identification and select the integration to see more details about it.
- Under Settings, click "Install Network Beaconing Identification assets" and follow the prompts to install the assets.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/beaconing",
"https://www.elastic.co/security-labs/identifying-beaconing-malware-using-elastic"
"https://www.elastic.co/security-labs/identifying-beaconing-malware-using-elastic",
]
risk_score = 21
rule_id = "5397080f-34e5-449b-8e9c-4c8083d7ccc6"
severity = "low"
tags = ["Domain: Network", "Use Case: C2 Beaconing Detection", "Tactic: Command and Control"]
timestamp_override = "event.ingested"
type = "query"
query = '''
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["beaconing"]
integration = ["beaconing", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "Beaconing package updates and support"
min_stack_version = "8.10.1"
updated_date = "2023/10/26"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -18,20 +18,33 @@ index = ["ml_beaconing.all"]
language = "kuery"
license = "Elastic License v2"
name = "Statistical Model Detected C2 Beaconing Activity with High Confidence"
note = """## Setup
setup = """
The rule requires the Network Beaconing Identification integration assets to be installed, as well as network logs collected by the Elastic Defend or Network Packet Capture integrations.
The Beaconing integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Network Beaconing Identification Setup
The Network Beaconing Identification integration consists of a statistical framework to identify C2 beaconing activity in network logs.
#### Prerequisite Requirements:
- Fleet is required for Network Beaconing Identification.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Network Beaconing Identification integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Network Beaconing Identification and select the integration to see more details about it.
- Under Settings, click "Install Network Beaconing Identification assets" and follow the prompts to install the assets.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/beaconing",
"https://www.elastic.co/security-labs/identifying-beaconing-malware-using-elastic"
"https://www.elastic.co/security-labs/identifying-beaconing-malware-using-elastic",
]
risk_score = 21
rule_id = "0ab319ef-92b8-4c7f-989b-5de93c852e93"
severity = "low"
tags = ["Domain: Network", "Use Case: C2 Beaconing Detection", "Tactic: Command and Control"]
timestamp_override = "event.ingested"
type = "query"
query = '''
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -19,13 +19,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_sent_bytes_destination_geo_country_iso_code"
name = "Potential Data Exfiltration Activity to an Unusual ISO Code"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your network data. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `logs-network_traffic.*` if you used Network Packet Capture.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "e1db8899-97c1-4851-8993-3a3265353601"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -19,13 +19,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_sent_bytes_destination_ip"
name = "Potential Data Exfiltration Activity to an Unusual IP Address"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your network data. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `logs-network_traffic.*` if you used Network Packet Capture.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "cc653d77-ddd2-45b1-9197-c75ad19df66c"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -18,13 +18,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_sent_bytes_destination_port"
name = "Potential Data Exfiltration Activity to an Unusual Destination Port"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your network data. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `logs-network_traffic.*` if you used Network Packet Capture.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "ef8cc01c-fc49-4954-a175-98569c646740"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -19,13 +19,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_sent_bytes_destination_region_name"
name = "Potential Data Exfiltration Activity to an Unusual Region"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Network events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) or [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your network data. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `logs-network_traffic.*` if you used Network Packet Capture.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "bfba5158-1fd6-4937-a205-77d96213b341"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -19,13 +19,34 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_bytes_written_to_external_device"
name = "Spike in Bytes Sent to an External Device"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the Elastic Defend integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "35a3b253-eea8-46f0-abd3-68bdd47e6e3d"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -20,13 +20,34 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_high_bytes_written_to_external_device_airdrop"
name = "Spike in Bytes Sent to an External Device via Airdrop"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the Elastic Defend integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "e92c99b6-c547-4bb6-b244-2f27394bc849"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/22"
integration = ["ded"]
integration = ["ded", "endpoint"]
maturity = "production"
min_stack_comments = "New rule"
min_stack_version = "8.9.0"
updated_date = "2023/10/14"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -19,13 +19,34 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "ded_rare_process_writing_to_external_device"
name = "Unusual Process Writing Data to an External Device"
note = """## Setup
setup = """
The rule requires the Data Exfiltration Detection integration assets to be installed, as well as network and file events collected by integrations such as Elastic Defend and Network Packet Capture (for network events only).
The Data Exfiltration Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Data Exfiltration Detection Setup
The Data Exfiltration Detection integration detects data exfiltration activity by identifying abnormalities in network and file events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Data Exfiltration Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the Elastic Defend integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Data Exfiltration Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Data Exfiltration Detection and select the integration to see more details about it.
- Under Settings, click Install Data Exfiltration Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Data Exfiltration Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/ded/kibana/ml_module/ded-ml.json) configuration file, you will see a card for Data Exfiltration Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/ded"
"https://docs.elastic.co/en/integrations/ded",
"https://www.elastic.co/blog/detect-data-exfiltration-activity-with-kibanas-new-integration",
]
risk_score = 21
rule_id = "4b95ecea-7225-4690-9938-2a2c0bad9c99"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/14"
integration = ["dga","endpoint","network_traffic"]
integration = ["dga", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "DGA package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -17,13 +17,55 @@ index = ["logs-endpoint.events.*", "logs-network_traffic.*"]
language = "kuery"
license = "Elastic License v2"
name = "Machine Learning Detected DGA activity using a known SUNBURST DNS domain"
note = """## Setup
setup = """
The rule requires the Domain Generation Algorithm (DGA) Detection integration assets to be installed, as well as DNS events collected by integrations such as Elastic Defend, Network Packet Capture, or Packetbeat.
The Domain Generation Algorithm (DGA) integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### DGA Detection Setup
The DGA Detection integration consists of an ML-based framework to detect DGA activity in DNS events.
#### Prerequisite Requirements:
- Fleet is required for DGA Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- DNS events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint), [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration, or [Packetbeat](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
- To set up and run Packetbeat, follow [this](https://www.elastic.co/guide/en/beats/packetbeat/current/setting-up-and-running.html) guide.
#### The following steps should be executed to install assets associated with the DGA Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Domain Generation Algorithm Detection and select the integration to see more details about it.
- Under Settings, click Install Domain Generation Algorithm Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich DNS events with predictions from the Supervised DGA Detection model. This is done via the ingest pipeline named `<package_version>-ml_dga_ingest_pipeline` installed with the DGA Detection package.
- If using an Elastic Beat such as Packetbeat, add the DGA ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `packetbeat.yml`.
- If adding the DGA ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the DGA ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"ml_is_dga": {
"properties": {
"malicious_prediction": {
"type": "long"
},
"malicious_probability": {
"type": "float"
}
}
}
}
}
```
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/dga"
"https://docs.elastic.co/en/integrations/dga",
"https://www.elastic.co/security-labs/detect-domain-generation-algorithm-activity-with-new-kibana-integration",
]
risk_score = 99
rule_id = "bcaa15ce-2d41-44d7-a322-918f9db77766"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/14"
integration = ["dga","endpoint","network_traffic"]
integration = ["dga", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "DGA package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,62 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "dga_high_sum_probability"
name = "Potential DGA Activity"
note = """## Setup
setup = """
The rule requires the Domain Generation Algorithm (DGA) Detection integration assets to be installed, as well as DNS events collected by integrations such as Elastic Defend, Network Packet Capture, or Packetbeat.
The Domain Generation Algorithm (DGA) integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### DGA Detection Setup
The DGA Detection integration consists of an ML-based framework to detect DGA activity in DNS events.
#### Prerequisite Requirements:
- Fleet is required for DGA Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- DNS events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint), [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration, or [Packetbeat](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
- To set up and run Packetbeat, follow [this](https://www.elastic.co/guide/en/beats/packetbeat/current/setting-up-and-running.html) guide.
#### The following steps should be executed to install assets associated with the DGA Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Domain Generation Algorithm Detection and select the integration to see more details about it.
- Under Settings, click Install Domain Generation Algorithm Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich DNS events with predictions from the Supervised DGA Detection model. This is done via the ingest pipeline named `<package_version>-ml_dga_ingest_pipeline` installed with the DGA Detection package.
- If using an Elastic Beat such as Packetbeat, add the DGA ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `packetbeat.yml`.
- If adding the DGA ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the DGA ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"ml_is_dga": {
"properties": {
"malicious_prediction": {
"type": "long"
},
"malicious_probability": {
"type": "float"
}
}
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched DNS events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `logs-network_traffic.*` if you used Network Packet Capture.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/dga/kibana/ml_module/dga-ml.json) configuration file, you will see a card for DGA under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/dga"
"https://docs.elastic.co/en/integrations/dga",
"https://www.elastic.co/security-labs/detect-domain-generation-algorithm-activity-with-new-kibana-integration",
]
risk_score = 21
rule_id = "ff0d807d-869b-4a0d-a493-52bc46d2f1b1"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/14"
integration = ["dga","endpoint","network_traffic"]
integration = ["dga", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "DGA package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -17,13 +17,55 @@ index = ["logs-endpoint.events.*", "logs-network_traffic.*"]
language = "kuery"
license = "Elastic License v2"
name = "Machine Learning Detected a DNS Request With a High DGA Probability Score"
note = """## Setup
setup = """
The rule requires the Domain Generation Algorithm (DGA) Detection integration assets to be installed, as well as DNS events collected by integrations such as Elastic Defend, Network Packet Capture, or Packetbeat.
The Domain Generation Algorithm (DGA) integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### DGA Detection Setup
The DGA Detection integration consists of an ML-based framework to detect DGA activity in DNS events.
#### Prerequisite Requirements:
- Fleet is required for DGA Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- DNS events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint), [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration, or [Packetbeat](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
- To set up and run Packetbeat, follow [this](https://www.elastic.co/guide/en/beats/packetbeat/current/setting-up-and-running.html) guide.
#### The following steps should be executed to install assets associated with the DGA Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Domain Generation Algorithm Detection and select the integration to see more details about it.
- Under Settings, click Install Domain Generation Algorithm Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich DNS events with predictions from the Supervised DGA Detection model. This is done via the ingest pipeline named `<package_version>-ml_dga_ingest_pipeline` installed with the DGA Detection package.
- If using an Elastic Beat such as Packetbeat, add the DGA ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `packetbeat.yml`.
- If adding the DGA ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the DGA ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"ml_is_dga": {
"properties": {
"malicious_prediction": {
"type": "long"
},
"malicious_probability": {
"type": "float"
}
}
}
}
}
```
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/dga"
"https://docs.elastic.co/en/integrations/dga",
"https://www.elastic.co/security-labs/detect-domain-generation-algorithm-activity-with-new-kibana-integration",
]
risk_score = 21
rule_id = "da7f5803-1cd4-42fd-a890-0173ae80ac69"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/14"
integration = ["dga","endpoint","network_traffic"]
integration = ["dga", "endpoint", "network_traffic"]
maturity = "production"
min_stack_comments = "DGA package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -17,13 +17,55 @@ index = ["logs-endpoint.events.*", "logs-network_traffic.*"]
language = "kuery"
license = "Elastic License v2"
name = "Machine Learning Detected a DNS Request Predicted to be a DGA Domain"
note = """## Setup
setup = """
The rule requires the Domain Generation Algorithm (DGA) Detection integration assets to be installed, as well as DNS events collected by integrations such as Elastic Defend, Network Packet Capture, or Packetbeat.
The Domain Generation Algorithm (DGA) integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### DGA Detection Setup
The DGA Detection integration consists of an ML-based framework to detect DGA activity in DNS events.
#### Prerequisite Requirements:
- Fleet is required for DGA Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- DNS events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint), [Network Packet Capture](https://docs.elastic.co/integrations/network_traffic) integration, or [Packetbeat](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To add the Network Packet Capture integration to an Elastic Agent policy, refer to [this](https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html) guide.
- To set up and run Packetbeat, follow [this](https://www.elastic.co/guide/en/beats/packetbeat/current/setting-up-and-running.html) guide.
#### The following steps should be executed to install assets associated with the DGA Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Domain Generation Algorithm Detection and select the integration to see more details about it.
- Under Settings, click Install Domain Generation Algorithm Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich DNS events with predictions from the Supervised DGA Detection model. This is done via the ingest pipeline named `<package_version>-ml_dga_ingest_pipeline` installed with the DGA Detection package.
- If using an Elastic Beat such as Packetbeat, add the DGA ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `packetbeat.yml`.
- If adding the DGA ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the DGA ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"ml_is_dga": {
"properties": {
"malicious_prediction": {
"type": "long"
},
"malicious_probability": {
"type": "float"
}
}
}
}
}
```
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/dga"
"https://docs.elastic.co/en/integrations/dga",
"https://www.elastic.co/security-labs/detect-domain-generation-algorithm-activity-with-new-kibana-integration",
]
risk_score = 21
rule_id = "f3403393-1fd9-4686-8f6e-596c58bc00b4"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_mean_rdp_process_args"
name = "High Mean of Process Arguments in an RDP Session"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "36c48a0c-c63a-4cbc-aee1-8cac87db31a9"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_mean_rdp_session_duration"
name = "High Mean of RDP Session Duration"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "a74c60cb-70ee-4629-a127-608ead14ebf1"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -20,13 +20,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_file_size_remote_file_transfer"
name = "Unusual Remote File Size"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "0678bc9c-b71a-433b-87e6-2f664b6b3131"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_var_rdp_session_duration"
name = "High Variance in RDP Session Duration"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "a8d35ca0-ad8d-48a9-9f6c-553622dca61a"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_rare_file_path_remote_transfer"
name = "Unusual Remote File Directory"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "be4c5aed-90f5-4221-8bd5-7ab3a4334751"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -18,13 +18,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_rare_file_extension_remote_transfer"
name = "Unusual Remote File Extension"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "814d96c7-2068-42aa-ba8e-fe0ddd565e2e"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_rdp_distinct_count_destination_ip_for_source"
name = "Spike in Number of Connections Made from a Source IP"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "3e0561b5-3fac-4461-84cc-19163b9aaa61"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_rdp_distinct_count_source_ip_for_destination"
name = "Spike in Number of Connections Made to a Destination IP"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "18a5dd9a-e3fa-4996-99b1-ae533b8f27fc"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -18,13 +18,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_sum_rdp_number_of_processes"
name = "Spike in Number of Processes in an RDP Session"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "19e9daf3-f5c5-4bc2-a9af-6b1e97098f03"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -20,13 +20,35 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_high_count_remote_file_transfer"
name = "Spike in Remote File Transfers"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- File events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your file events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "e9b0902b-c515-413b-b80b-a8dcebc81a66"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/12"
integration = ["lmd"]
integration = ["lmd", "endpoint"]
maturity = "production"
min_stack_comments = "LMD package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/12"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 70
@@ -19,13 +19,36 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "lmd_unusual_time_weekday_rdp_session_start"
name = "Unusual Time or Day for an RDP Session"
note = """## Setup
setup = """
The rule requires the Lateral Movement Detection integration assets to be installed, as well as file and Windows RDP process events collected by the Elastic Defend integration.
The Lateral Movement Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### Lateral Movement Detection Setup
The Lateral Movement Detection integration detects lateral movement activity by identifying abnormalities in file and Windows RDP events. Anomalies are detected using Elastic's Anomaly Detection feature.
#### Prerequisite Requirements:
- Fleet is required for Lateral Movement Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows RDP process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration.
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
#### The following steps should be executed to install assets associated with the Lateral Movement Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Lateral Movement Detection and select the integration to see more details about it.
- Under Settings, click Install Lateral Movement Detection assets and follow the prompts to install the assets.
#### Anomaly Detection Setup
Before you can enable rules for Lateral Movement Detection, you'll need to enable the corresponding Anomaly Detection jobs.
- Before enabling the Anomaly Detection jobs, confirm that the Pivot Transform asset is installed and actively gathering data in the destination index `ml-rdp-lmd`.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your transformed RDP process data i.e.`ml-rdp-lmd`.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/lmd/kibana/ml_module/lmd-ml.json) configuration file, you will see a card for Lateral Movement Detection under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection jobs and datafeeds.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/lmd"
"https://docs.elastic.co/en/integrations/lmd",
"https://www.elastic.co/blog/detecting-lateral-movement-activity-a-new-kibana-integration",
"https://www.elastic.co/blog/remote-desktop-protocol-connections-elastic-security",
]
risk_score = 21
rule_id = "3f4e2dba-828a-452a-af35-fe29c5e78969"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/09/19"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/23"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -20,14 +20,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_rare_process_by_host"
name = "Unusual Process Spawned by a Host"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "56004189-4e69-4a39-b4a9-195329d226e9"
@@ -48,7 +98,9 @@ id = "T1218"
name = "System Binary Proxy Execution"
reference = "https://attack.mitre.org/techniques/T1218/"
[rule.threat.tactic]
id = "TA0005"
name = "Defense Evasion"
reference = "https://attack.mitre.org/tactics/TA0005/"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/23"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -20,14 +20,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_rare_process_by_parent"
name = "Unusual Process Spawned by a Parent Process"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "ea09ff26-3902-4c53-bb8e-24b7a5d029dd"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/23"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -21,14 +21,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_rare_process_by_user"
name = "Unusual Process Spawned by a User"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "40155ee4-1e6a-4e4d-a63b-e8ba16980cfb"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild","endpoint"]
integration = ["problemchild", "endpoint"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -18,14 +18,57 @@ index = ["endgame-*", "logs-endpoint.events.process-*", "winlogbeat-*"]
language = "eql"
license = "Elastic License v2"
name = "Machine Learning Detected a Suspicious Windows Event Predicted to be Malicious Activity"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "13e908b9-7bf0-4235-abc9-b5deb500d0ad"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild","endpoint"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/23"
updated_date = "2023/12/12"
[rule]
author = ["Elastic"]
@@ -18,14 +18,57 @@ index = ["endgame-*", "logs-endpoint.events.process-*", "winlogbeat-*"]
language = "eql"
license = "Elastic License v2"
name = "Machine Learning Detected a Suspicious Windows Event with a High Malicious Probability Score"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "994e40aa-8c85-43de-825e-15f665375ee8"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -22,14 +22,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_high_sum_by_host"
name = "Suspicious Windows Process Cluster Spawned by a Host"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "bdfebe11-e169-42e3-b344-c5d2015533d3"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -22,14 +22,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_high_sum_by_parent"
name = "Suspicious Windows Process Cluster Spawned by a Parent Process"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "f5d9d36d-7c30-4cdb-a856-9f653c13d4e0"
@@ -1,10 +1,10 @@
[metadata]
creation_date = "2023/10/16"
integration = ["problemchild"]
integration = ["problemchild", "endpoint", "windows"]
maturity = "production"
min_stack_comments = "LotL package job ID and rule removal updates"
min_stack_version = "8.9.0"
updated_date = "2023/10/16"
updated_date = "2023/12/12"
[rule]
anomaly_threshold = 75
@@ -22,14 +22,64 @@ interval = "15m"
license = "Elastic License v2"
machine_learning_job_id = "problem_child_high_sum_by_user"
name = "Suspicious Windows Process Cluster Spawned by a User"
note = """## Setup
setup = """
The rule requires the Living off the Land (LotL) Attack Detection integration assets to be installed, as well as Windows process events collected by integrations such as Elastic Defend or Winlogbeat.
The Living-off-the-Land (LotL) Detection integration must be enabled and related ML jobs configured for this rule to be effective. Please refer to this rule's references for more information.
### LotL Attack Detection Setup
The LotL Attack Detection integration detects living-off-the-land activity in Windows process events.
#### Prerequisite Requirements:
- Fleet is required for LotL Attack Detection.
- To configure Fleet Server refer to the [documentation](https://www.elastic.co/guide/en/fleet/current/fleet-server.html).
- Windows process events collected by the [Elastic Defend](https://docs.elastic.co/en/integrations/endpoint) integration or Winlogbeat(https://www.elastic.co/guide/en/beats/winlogbeat/current/_winlogbeat_overview.html).
- To install Elastic Defend, refer to the [documentation](https://www.elastic.co/guide/en/security/current/install-endpoint.html).
- To set up and run Winlogbeat, follow [this](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html) guide.
#### The following steps should be executed to install assets associated with the LotL Attack Detection integration:
- Go to the Kibana homepage. Under Management, click Integrations.
- In the query bar, search for Living off the Land Attack Detection and select the integration to see more details about it.
- Under Settings, click Install Living off the Land Attack Detection assets and follow the prompts to install the assets.
#### Ingest Pipeline Setup
Before you can enable this rule, you'll need to enrich Windows process events with predictions from the Supervised LotL Attack Detection model. This is done via the ingest pipeline named `<package_version>-problem_child_ingest_pipeline` installed with the LotL Attack Detection package.
- If using an Elastic Beat such as Winlogbeat, add the LotL ingest pipeline to it by adding a simple configuration [setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats) to `winlogbeat.yml`.
- If adding the LotL ingest pipeline to an existing pipeline, use a [pipeline processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html).
#### Adding Custom Mappings
- Go to the Kibana homepage. Under Management, click Stack Management.
- Under Data click Index Management and navigate to the Component Templates tab.
- Templates that can be edited to add custom components will be marked with a @custom suffix. Edit the @custom component template corresponding to the beat/integration you added the LotL ingest pipeline to, by pasting the following JSON blob in the "Load JSON" flyout:
```
{
"properties": {
"problemchild": {
"properties": {
"prediction": {
"type": "long"
},
"prediction_probability": {
"type": "float"
}
}
},
"blocklist_label": {
"type": "long"
}
}
}
```
### Anomaly Detection Setup
Before you can enable this rule, you'll need to enable the corresponding Anomaly Detection job.
- Go to the Kibana homepage. Under Analytics, click Machine Learning.
- Under Anomaly Detection, click Jobs, and then click "Create job". Select the Data View containing your enriched Windows process events. For example, this would be `logs-endpoint.events.*` if you used Elastic Defend to collect events, or `winlogbeat-*` if you used Winlogbeat.
- If the selected Data View contains events that match the query in [this](https://github.com/elastic/integrations/blob/main/packages/problemchild/kibana/ml_module/problemchild-ml.json) configuration file, you will see a card for "Living off the Land Attack Detection" under "Use preconfigured jobs".
- Keep the default settings and click "Create jobs" to start the anomaly detection job and datafeed.
"""
references = [
"https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html",
"https://docs.elastic.co/en/integrations/problemchild",
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration"
"https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration",
]
risk_score = 21
rule_id = "1224da6c-0326-4b4f-8454-68cdc5ae542b"
+9 -1
View File
@@ -641,7 +641,8 @@ class TestRuleMetadata(BaseRuleTest):
integration_string = "|".join(indices)
if not re.search(rule_integration, integration_string):
if rule_integration == "windows" and re.search("winlog", integration_string) or \
rule_integration in [*map(str.lower, definitions.MACHINE_LEARNING_PACKAGES)]:
any(ri in [*map(str.lower, definitions.MACHINE_LEARNING_PACKAGES)]
for ri in rule_integrations):
continue
err_msg = f'{self.rule_str(rule)} {rule_integration} tag, index pattern missing.'
failures.append(err_msg)
@@ -990,6 +991,13 @@ class TestRuleTiming(BaseRuleTest):
for rule in self.all_rules:
if rule.contents.data.type not in ('eql', 'query'):
continue
if rule.contents.metadata.get('integration'):
integrations = rule.contents.metadata.get('integration')
if not isinstance(integrations, list):
integrations = [integrations]
machine_learning_packages_lower = [pkg.lower() for pkg in definitions.MACHINE_LEARNING_PACKAGES]
if any(tag in machine_learning_packages_lower for tag in integrations):
continue
if isinstance(rule.contents.data, QueryRuleData) and 'endgame-*' in rule.contents.data.index:
continue