Commit Graph

201 Commits

Author SHA1 Message Date
Terrance DeJesus 3c97d34615 adding adjustment to route c of manage_versions (#2307) 2022-09-19 14:50:46 -04:00
Samirbous acdfe5ddab [New Rule] Process Creation via Secondary Logon (#2282)
* [New Rule] Process Creation via Secondary Logon

https://github.com/elastic/detection-rules/issues/2164

Create process using alternate creds (i.g. runas) :

* Update privilege_escalation_create_process_as_different_user.toml

* Update privilege_escalation_create_process_as_different_user.toml

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
2022-09-19 13:04:08 -05:00
Isai 963d01ba89 [New Rule] Kubernetes Suspicious Assignment of Controller Service Account (#2298)
* [New Rule] Kubernetes Suspicious Assignment of Controller Service Account

Issues
--
#2034

Summary
--
This rule detects a request to attach a controller service account to an existing or new pod running in the kube-system namespace. By default, controllers running as part of the API Server utilize admin-equivalent service accounts hosted in the kube-system namespace. Controller service accounts aren't normally assigned to running pods and could indicate adversary behavior within the cluster. An attacker that can create or modify pods or pod controllers in the kube-system namespace, can assign one of these admin-equivalent service accounts to a pod and abuse their powerful token to escalate privileges and gain complete cluster control.

* Update privilege_escalation_suspicious_assignment_of_controller_service_account.toml

updated query after testing

* Update non-ecs-schema.json

added new field used in query update

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
2022-09-19 13:35:37 -04:00
Isai a9364beef9 [New Rule] Kubernetes Denied Service Account Request (#2299)
* [New Rule] Kubernetes Denied Service Account Request

## Issue
#2040

## Summary
This rule detects when a service account makes an unauthorized request for resources from the API server. Service accounts follow a very predictable pattern of behavior. A service account should never send an unauthorized request to the API server. This behavior is likely an indicator of compromise or of a problem within the cluster. An adversary may have gained access to credentials/tokens and this could be an attempt to access or create resources to facilitate further movement or execution within the cluster.

* Update discovery_denied_service_account_request.toml

updated the query after testing to reduce false positives

* Update rules/integrations/kubernetes/discovery_denied_service_account_request.toml

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2022-09-19 13:22:20 -04:00
Samirbous 99dcfe2055 [New Rule] Multiple Vault Web credentials were read (#2281)
* [New Rule] Multiple Vault Web credentials were read

https://github.com/elastic/detection-rules/issues/2164

* Update credential_access_saved_creds_vault_winlog.toml

* Update non-ecs-schema.json

* Update rules/windows/credential_access_saved_creds_vault_winlog.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-09-19 19:07:05 +02:00
Samirbous 4609a5e8fe [New Rule] Scheduled Task Creation using winlog (#2277)
* [New Rule] Scheduled Task Creation using winlog

https://github.com/elastic/detection-rules/issues/2164 (T1053.005 - Scheduled Task)

- A scheduled task was created
- A scheduled task was updated
- Temp scheduled task (creation followed by deletion, rare and can be sign of proxy execution via schedule service)

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* toml-lint

* remote task

* Update non-ecs-schema.json

* waaaaaaaaaaaaaa

* Update persistence_scheduled_task_updated.toml

* Update persistence_scheduled_task_creation_winlog.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update lateral_movement_remote_task_creation_winlog.toml

* event.ingested

* Update lateral_movement_remote_task_creation_winlog.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update defense_evasion_persistence_temp_scheduled_task.toml

* Update rules/windows/lateral_movement_remote_task_creation_winlog.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-09-19 18:50:45 +02:00
Samirbous fc8ec668b1 [New Rule] Brute Force Detection - Windows (#2275)
* [New Rule] Brute Force Detection - Windows

https://github.com/elastic/detection-rules/issues/2164 (T1110 - Brute Force)

- multiple logon failure from same source address in 10s maxspan
- 5 logon failure followed by success from same source address in 5s maxspan

* non ecs

* Update credential_access_bruteforce_multiple_logon_failure_followed_by_success.toml

* fix error

* added bruteforce admin account and linted tomls

* Update credential_access_bruteforce_admin_account.toml

* Update rules/windows/credential_access_bruteforce_admin_account.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* related_rules

* 4625_errorcode_notes

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-09-19 18:43:28 +02:00
Justin Ibarra 2ee5a185c7 Add test command to verify version collisions do not occur (#2272)
* Add test command to verify version collisions do not occur
* add max_allowable_version to schema and lock flow
* add max_allowable_version to all entries in version.lock
* add test-version-lock command
* use min supported stack if > locked min stack
* share lock conversion code with rule and lock to fix M.m bug
2022-09-19 09:53:30 -06:00
Mika Ayenson c2e7011ec6 break out the logic to a script and manual workflow (#1908)
* Break out the logic to a script and manual workflow with an option to skip staging files
2022-09-16 13:34:04 -04:00
Mika Ayenson e3040d8019 [Bug] Keyerror on rule-survey hits (#2293) 2022-09-13 11:38:29 -04:00
Mika Ayenson 0358ec9d9a Release ER Production RTAs to DR (#2270) 2022-09-08 12:50:39 -04:00
Justin Ibarra 332ea40100 Cleanup rule survey code (#1923)
* Cleanup rule survey code

* default to only unique-ing on process name for lucene rules

* fix bug in kibana url parsing by removing redundant port from domain

* update search-alerts columns and nest fields

* fix rule.contents.data.index

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-09-06 15:53:47 -06:00
Justin Ibarra d37eac8d9d Add test that newly introduced build-time fields for a min_stack for … (#2262)
* add test that newly introduced build-time fields for a min_stack for applicable rules.

* account for rules without min_stack_version

* limit test to >= stack ver
2022-08-25 21:56:16 -06:00
Terrance DeJesus 5a04aaf671 [Bug] Integrations-Pr Command (Elastic-Package Linting and Version Adjustments) (#2054)
* started solution for integrations-pr bug

* Update devtools.py

* Update detection_rules/devtools.py

* Update detection_rules/devtools.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-08-24 14:01:30 -04:00
github-actions[bot] 6ff7d2284d Lock versions for releases: 7.16,8.0,8.1,8.2,8.3,8.4 (#2261)
* Locked versions for releases: 7.16,8.0,8.1,8.2,8.3,8.4

* adjusting version lock file to increase current version by 100

Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <terrance.dejesus@elastic.co>
2022-08-24 13:26:35 -04:00
github-actions[bot] cb2ca45d56 Locked versions for releases: 7.16,8.0,8.1,8.2,8.3,8.4 (#2236)
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2022-08-10 09:18:59 -04:00
Terrance DeJesus 2a3b584433 Prep for 8.5 branch (#2220)
* adding first commit

* renamed branch

* adjusted packages, stack schema and updated schemas

* updated integrations manifest

* adjusted comments to be a little more organized

* adjusted stack-schema-map

* refreshed ecs and beats schema, adjusted stack schema map accordingly
2022-08-09 17:14:42 -04:00
Mika Ayenson 89cdae87c5 only add related_integration if on the correct stack (#2234) 2022-08-08 18:41:56 -04:00
Mika Ayenson 7d973a3b07 add new field related_integrations to the post build (#2060)
* add new field `related_integrations` to the post build

* add exception for endpoint `integration`

* Skip rules without related integrations

* lint

* refactor related_integrations to TOMLRuleContents class

* update to reflect required_fields updates

* add todo

* add new line for linting

* related_integrations updates, get_packaged_integrations returns list of dictionaries, started work on integrations py

* build_integrations_manifest command completed

* initial test completed for post-building related_integrations

* removed get_integration_manifest method from rule, removed global integrations path

* moved integration related methods to integrations.py and fixed flake issues

* adjustments for PipedQuery from eql sequence rules and packages with no integration

* adjusted github client import for integrations.py

* Update detection_rules/devtools.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/devtools.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* added integration manifest schema, made adjustments

* Update detection_rules/integrations.py

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* removed get_integrations_package to consolidate code

* removed type list return

* adjusted import flake errors

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* adjusted indentation error

* adjusted rule.get_packaged_integrations to account for kql.ast.OrExpr if event.dataset is not set

* Update detection_rules/devtools.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/devtools.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* adjusted find_least_compatible_version in integrations.py

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* fixed flake issues

* adjusted get_packaged_integrations

* iterate the ast for literal event.dataset values

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* made small adjustments to address errors during build manifests command

* addressing integrations.find_least_compatible method to return None instead of raise error only

* Update detection_rules/integrations.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <terrance.dejesus@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2022-08-08 13:44:36 -04:00
Samirbous b15f0de9a4 [Rules Tuning] Diverse Windows Rules - FPs reduction (#2213)
* [Rules Tuning] 7 diverse Windows rules

Excluding FP patterns while avoiding breaking compat with winlogbeat and 4688 events lack of codesign metadata.

* Update initial_access_suspicious_ms_exchange_process.toml

* Update privilege_escalation_persistence_phantom_dll.toml

* Update execution_psexec_lateral_movement_command.toml

* Update persistence_remote_password_reset.toml

* Update non-ecs-schema.json

* Update persistence_remote_password_reset.toml

* Update non-ecs-schema.json

* Update discovery_privileged_localgroup_membership.toml
2022-08-02 18:37:07 +02:00
Isai c1486407aa [New Rule] Kubernetes Pod Created with Sensitive hostPath Volume (#2094)
* [New Rule] Kubernetes Pod Created with Sensitive hostPath Volume

created new rule toml and updated non-ecs-schema with k8s fields

* Update rules/integrations/kubernetes/privilege_escalation_pod_created_with_sensitive_hospath_volume.toml

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2022-07-28 13:09:26 -04:00
Terrance DeJesus e8c39d19a7 [Rule Tuning] Missing MITRE ATT&CK Mappings (#2073)
* initial commit with eggshell mitre mapping added

* adding updated rules

* [Rule Tuning] MITRE for GCP rules

I've added Mitre references for the 4 GCP rules missing. Changed 3 of the rules from "Impact" to "Defense Evasion" based on the technique used and it's matched tactic.

* [Rule Tuning] Endgame Rule name updates for Mitre

Updated Endgame rule names for those with Mitre tactics to match the tactics.

* Update rules/integrations/aws/persistence_redshift_instance_creation.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* Update rules/integrations/aws/exfiltration_rds_snapshot_restored.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* adding 10 updated rules for google_workspace, ml and o365

* adding 22 rule updates for mitre att&ck mappings

* adding 24 rule updates related mainly to ML rules

* adding 3 rules related to detection via ML

* adding adjustments

* adding adjustments with solutions to recent pytest errors

* removed tabs from tags

* adjusted mappings and added techniques

* adjusted endgame rule mappings per review

* adjusted names to match different tactics

* added execution and defense evasion tag

* adjustments to address errors from merging with main

* added newlines to rules missing them at the end of the file

Co-authored-by: imays11 <59296946+imays11@users.noreply.github.com>
Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
2022-07-22 14:30:34 -04:00
Samirbous d312f49117 [New Rule] Suspicious HTML File Creation (#2068)
* [New Rule] Suspicious HTML File Creation

* Update initial_access_evasion_suspicious_htm_file_creation.toml

* Update non-ecs-schema.json

* Update initial_access_evasion_suspicious_htm_file_creation.toml

* Update rules/windows/initial_access_evasion_suspicious_htm_file_creation.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
2022-07-22 16:21:53 +02:00
Mika Ayenson a52751494e 2058 add setup field to metadata (#2061)
* Convert config header to setup in note field
* Parse note field into separate setup and note field with marko gfm
* only validate and parse note on elastic authored rules and add CLI description for new DR_BYPASS_NOTE_VALIDATION_AND_PARSE environment variable

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2022-07-18 15:41:32 -04:00
Mika Ayenson c76a397969 Add new required_fields as a build-time restricted field (#2059)
* Add new `require_field` restricted field
* validate new fields against BaseRuleData schema and global constant

Co-authored-by: Terrance DeJesus <terrance.dejesus@elastic.co>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2022-07-06 11:49:44 -04:00
Justin Ibarra cc01d3fb1a Add support for restricted fields (#2053)
* Add support for restricted fields (fields valid only in min/max stack versions)
* add test to ensure rule backports wont exceed min compat
2022-06-27 10:02:15 -05:00
Mika Ayenson 4ef1a1a627 Update cli documentation for search-alerts (#2051)
* Add cli documentation for search-alerts and table fields
2022-06-24 09:58:58 -04:00
Mika Ayenson 4fdd978183 test automatically prevent future merges when a backport fails (#1909)
automatically prevent future merges when a backport fails
2022-06-23 14:59:25 -04:00
github-actions[bot] fd9c9f8abf Locked versions for releases: 7.16,8.0,8.1,8.2,8.3 (#2041)
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com>
2022-06-17 11:44:07 -04:00
Isai 63fda01fdd [New Rule] Kubernetes execution_user_exec_to_pod (#1979)
* Create execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* Update rules/integrations/kubernetes/execution_user_exec_to_pod.toml

* Update non-ecs-schema.json

* Update execution_user_exec_to_pod.toml

* Update rules/integrations/kubernetes/execution_user_exec_to_pod.toml

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>

* Update execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* toml-linted file and add to false positive

toml-linted the file and added to the false positive description

* Create notepad.sct

Added this back into the repo, deleted by mistake.

* added min_stack_version based on integration

min stack version determined by integration support of necessary fields

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
2022-06-09 17:52:45 -04:00
Justin Ibarra 744f56d98e [Bug] resolves bug in Rule version methods (#2021)
* [Bug] resolves bug in Rule version methods

* comment out unused code with notes
2022-06-07 15:40:46 -08:00
Justin Ibarra e850f39526 [Bug] Fix test_matrix_to_lock_version_defaults test (#2014) 2022-06-02 16:34:54 -08:00
Justin Ibarra f57950a3c9 Collapse unsupported previous version entries (#2013)
* Collapse unsupported previous version entries
* drop the last entry in the matrix test
2022-06-02 15:18:12 -08:00
Terrance DeJesus 35b1a69ff5 Prep for Creation of 8.4 Branch (#2001)
* prepping for 8.4 branch

* adjusted schemas init file

* adjusted target matrix to only backport to 7.16, updated api schemas

* adjusted the lock-versions workflow to account for 7.16 and up support only

* Add test for version lock to schema map correlation

* decouple from static 7.13 references

* keep patch version for lock

* Update detection_rules/etc/packages.yml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2022-06-02 14:59:18 -04:00
Justin Ibarra 0428e161a8 Refresh ECS/beats schemas up to 8.2 (#1995) 2022-05-25 11:51:43 -08:00
Mika Ayenson e1266a6fd3 Skip previous validation on pre/post load/dump (#1942)
* Build out the dataclasses for a base entry and version lock explicitly
* Ensure previous field does not have a nested previous
* Test validation on version lock for previous fields.
2022-05-25 13:34:03 -04:00
Samirbous 19ff825a91 [New rule] Remote Computer Account DnsHostName Update (#1962)
* [New rule] Remote Computer Account DnsHostName Update

Identifies remote update to a computer account DnsHostName attribute, if the new value is set a valid domain controller DNS hostname and the subject computer name is not a domain controller then it's high likely a preparation step to exploit CVE-2022-26923 in an attempt to elevate privileges from a standard domain user to domain admin privileges :

* added MS ref url

* Update rules/windows/privilege_escalation_suspicious_dnshostname_update.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* Update rules/windows/privilege_escalation_suspicious_dnshostname_update.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
2022-05-11 19:40:34 +02:00
Justin Ibarra 8168551c59 Manually reconciled versions from forked rule package generation bug (#1950) 2022-05-04 10:04:10 -08:00
Justin Ibarra 22679e16d2 Add delta command to determine changes to endpoint rules between tags (#1943)
* update git tag loader to be compatible with lock validation
* add diff command
* default to query for missing rules
2022-05-03 12:30:11 -08:00
Mika Ayenson 6219fc06b9 Move etc under detection_rules (#1885)
* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-05-02 10:11:21 -04:00
Justin Ibarra c803160e4f Validate version lock and deprecation files on load and save (#1884)
* Validate version lock and deprecation files on load and save
* add missing types for previous lock entries
* bump marshmallow_dataclass
2022-04-26 22:17:20 -08:00
Justin Ibarra b3e789a202 Remove deprecated elasticsearch parameter (#1913) 2022-04-12 12:06:11 -08:00
Justin Ibarra ad99c6b489 Update elasticsearch dependency to 8.1 (#1911) 2022-04-06 11:52:22 -08:00
Justin Ibarra 6bdfddac8e Expand timestamp override tests (#1907)
* Expand timestamp_override tests
* removed timestamp_override from eql sequence rules
* add config entry for eql rules with beats index and t_o
* add timestamp_override to missing fields
2022-04-01 15:27:08 -08:00
Terrance DeJesus 648daf1237 Prep for Creation of 8.3 Branch (#1906)
* updating with changes for 8.3 prep
* adding updates
* adjusted version in packages.yml
2022-04-01 13:33:18 -08:00
Terrance DeJesus e72031a71a added comprehensive timeline template definitions (#1905) 2022-04-01 08:51:54 -08:00
Justin Ibarra 5214209f8d reset evasion rules (#1902) 2022-03-29 15:47:48 -08:00
Justin Ibarra 17ef6c558c [Bug] Fix bug in version_lock.py (#1880) 2022-03-24 15:41:16 -08:00
Justin Ibarra 11ec9c230e Prevent changes to rule type for locked rules (#1855)
* add rule type to the rule lock_info
* add check in VersionLock; add type to version.lock
* print changes only on save
2022-03-24 11:56:27 -08:00
Justin Ibarra f4c94af994 [Bug] Version bump with previous (#1870)
* save changes to top level for route C; verbose prints
* update top level on forked rule without overriding min_stack_version
* add check to ensure previous version !> current
2022-03-24 11:12:06 -08:00