Commit Graph

27 Commits

Author SHA1 Message Date
Terrance DeJesus 141b00ec41 [Rule Tuning] Missing MITRE ATT&CK Mappings (#2073)
* initial commit with eggshell mitre mapping added

* adding updated rules

* [Rule Tuning] MITRE for GCP rules

I've added Mitre references for the 4 GCP rules missing. Changed 3 of the rules from "Impact" to "Defense Evasion" based on the technique used and it's matched tactic.

* [Rule Tuning] Endgame Rule name updates for Mitre

Updated Endgame rule names for those with Mitre tactics to match the tactics.

* Update rules/integrations/aws/persistence_redshift_instance_creation.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* Update rules/integrations/aws/exfiltration_rds_snapshot_restored.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* adding 10 updated rules for google_workspace, ml and o365

* adding 22 rule updates for mitre att&ck mappings

* adding 24 rule updates related mainly to ML rules

* adding 3 rules related to detection via ML

* adding adjustments

* adding adjustments with solutions to recent pytest errors

* removed tabs from tags

* adjusted mappings and added techniques

* adjusted endgame rule mappings per review

* adjusted names to match different tactics

* added execution and defense evasion tag

* adjustments to address errors from merging with main

* added newlines to rules missing them at the end of the file

Co-authored-by: imays11 <59296946+imays11@users.noreply.github.com>
Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

Removed changes from:
- rules/integrations/google_workspace/application_added_to_google_workspace_domain.toml
- rules/integrations/google_workspace/domain_added_to_google_workspace_trusted_domains.toml
- rules/integrations/google_workspace/google_workspace_admin_role_deletion.toml
- rules/integrations/google_workspace/google_workspace_mfa_enforcement_disabled.toml
- rules/integrations/google_workspace/google_workspace_policy_modified.toml
- rules/integrations/google_workspace/mfa_disabled_for_google_workspace_organization.toml
- rules/ml/ml_linux_anomalous_compiler_activity.toml
- rules/ml/ml_linux_anomalous_metadata_process.toml
- rules/ml/ml_linux_anomalous_metadata_user.toml
- rules/ml/ml_linux_anomalous_process_all_hosts.toml
- rules/ml/ml_linux_anomalous_sudo_activity.toml
- rules/ml/ml_linux_anomalous_user_name.toml
- rules/ml/ml_linux_system_information_discovery.toml
- rules/ml/ml_linux_system_network_configuration_discovery.toml
- rules/ml/ml_linux_system_network_connection_discovery.toml
- rules/ml/ml_linux_system_process_discovery.toml
- rules/ml/ml_linux_system_user_discovery.toml
- rules/ml/ml_rare_process_by_host_linux.toml
- rules/ml/ml_rare_process_by_host_windows.toml
- rules/ml/ml_suspicious_login_activity.toml
- rules/ml/ml_windows_anomalous_metadata_process.toml
- rules/ml/ml_windows_anomalous_metadata_user.toml
- rules/ml/ml_windows_anomalous_path_activity.toml
- rules/ml/ml_windows_anomalous_process_all_hosts.toml
- rules/ml/ml_windows_anomalous_process_creation.toml
- rules/ml/ml_windows_anomalous_script.toml
- rules/ml/ml_windows_anomalous_service.toml
- rules/ml/ml_windows_anomalous_user_name.toml
- rules/ml/ml_windows_rare_user_runas_event.toml
- rules/ml/ml_windows_rare_user_type10_remote_login.toml
- rules/windows/defense_evasion_execution_lolbas_wuauclt.toml

(selectively cherry picked from commit e8c39d19a7)
2022-07-22 18:31:42 +00:00
Samirbous 25493a90c9 [New Rule] Suspicious HTML File Creation (#2068)
* [New Rule] Suspicious HTML File Creation

* Update initial_access_evasion_suspicious_htm_file_creation.toml

* Update non-ecs-schema.json

* Update initial_access_evasion_suspicious_htm_file_creation.toml

* Update rules/windows/initial_access_evasion_suspicious_htm_file_creation.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

(cherry picked from commit d312f49117)
2022-07-22 14:22:56 +00:00
Mika Ayenson 62298d92f4 2058 add setup field to metadata (#2061)
* Convert config header to setup in note field
* Parse note field into separate setup and note field with marko gfm
* only validate and parse note on elastic authored rules and add CLI description for new DR_BYPASS_NOTE_VALIDATION_AND_PARSE environment variable

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>

Removed changes from:
- rules/cross-platform/impact_hosts_file_modified.toml
- rules/integrations/google_workspace/application_added_to_google_workspace_domain.toml
- rules/integrations/google_workspace/domain_added_to_google_workspace_trusted_domains.toml
- rules/integrations/google_workspace/google_workspace_admin_role_deletion.toml
- rules/integrations/google_workspace/google_workspace_mfa_enforcement_disabled.toml
- rules/integrations/google_workspace/google_workspace_policy_modified.toml
- rules/integrations/google_workspace/mfa_disabled_for_google_workspace_organization.toml
- rules/integrations/google_workspace/persistence_google_workspace_admin_role_assigned_to_user.toml
- rules/integrations/google_workspace/persistence_google_workspace_api_access_granted_via_domain_wide_delegation_of_authority.toml
- rules/integrations/google_workspace/persistence_google_workspace_custom_admin_role_created.toml
- rules/integrations/google_workspace/persistence_google_workspace_role_modified.toml
- rules/integrations/kubernetes/execution_user_exec_to_pod.toml
- rules/windows/credential_access_lsass_memdump_file_created.toml
- rules/windows/defense_evasion_adding_the_hidden_file_attribute_with_via_attribexe.toml
- rules/windows/defense_evasion_execution_lolbas_wuauclt.toml
- rules/windows/defense_evasion_suspicious_certutil_commands.toml
- rules/windows/execution_command_shell_started_by_svchost.toml

(selectively cherry picked from commit a52751494e)
2022-07-18 21:25:32 +00:00
Mika Ayenson 06ce0015df Add new required_fields as a build-time restricted field (#2059)
* Add new `require_field` restricted field
* validate new fields against BaseRuleData schema and global constant

Co-authored-by: Terrance DeJesus <terrance.dejesus@elastic.co>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>

(cherry picked from commit c76a397969)
2022-07-06 15:51:18 +00:00
Justin Ibarra 179a3bd284 Add support for restricted fields (#2053)
* Add support for restricted fields (fields valid only in min/max stack versions)
* add test to ensure rule backports wont exceed min compat

(cherry picked from commit cc01d3fb1a)
2022-06-27 15:03:32 +00:00
Mika Ayenson eb6deea9ac Update cli documentation for search-alerts (#2051)
* Add cli documentation for search-alerts and table fields

(cherry picked from commit 4ef1a1a627)
2022-06-24 14:00:01 +00:00
Mika Ayenson 6c5e101e6f test automatically prevent future merges when a backport fails (#1909)
automatically prevent future merges when a backport fails

(cherry picked from commit 4fdd978183)
2022-06-23 19:00:24 +00:00
github-actions[bot] fafe1e0ab6 Locked versions for releases: 7.16,8.0,8.1,8.2,8.3 (#2041)
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com>

(cherry picked from commit fd9c9f8abf)
2022-06-17 15:44:35 +00:00
Isai fa5fc6094e [New Rule] Kubernetes execution_user_exec_to_pod (#1979)
* Create execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* Update rules/integrations/kubernetes/execution_user_exec_to_pod.toml

* Update non-ecs-schema.json

* Update execution_user_exec_to_pod.toml

* Update rules/integrations/kubernetes/execution_user_exec_to_pod.toml

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>

* Update execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* Update execution_user_exec_to_pod.toml

* toml-linted file and add to false positive

toml-linted the file and added to the false positive description

* Create notepad.sct

Added this back into the repo, deleted by mistake.

* added min_stack_version based on integration

min stack version determined by integration support of necessary fields

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>

(cherry picked from commit 63fda01fdd)
2022-06-09 21:53:15 +00:00
Justin Ibarra 8564185a7d [Bug] resolves bug in Rule version methods (#2021)
* [Bug] resolves bug in Rule version methods

* comment out unused code with notes

(cherry picked from commit 744f56d98e)
2022-06-07 23:41:40 +00:00
Justin Ibarra c16442517e [Bug] Fix test_matrix_to_lock_version_defaults test (#2014)
(cherry picked from commit e850f39526)
2022-06-03 00:35:19 +00:00
Justin Ibarra 3a1a5fe12b Collapse unsupported previous version entries (#2013)
* Collapse unsupported previous version entries
* drop the last entry in the matrix test

(cherry picked from commit f57950a3c9)
2022-06-02 23:18:45 +00:00
Terrance DeJesus 220996b1b8 Prep for Creation of 8.4 Branch (#2001)
* prepping for 8.4 branch

* adjusted schemas init file

* adjusted target matrix to only backport to 7.16, updated api schemas

* adjusted the lock-versions workflow to account for 7.16 and up support only

* Add test for version lock to schema map correlation

* decouple from static 7.13 references

* keep patch version for lock

* Update detection_rules/etc/packages.yml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>

Removed changes from:
- detection_rules/etc/packages.yml

(selectively cherry picked from commit 35b1a69ff5)
2022-06-02 18:59:56 +00:00
Justin Ibarra 6199bd4524 Refresh ECS/beats schemas up to 8.2 (#1995)
(cherry picked from commit 0428e161a8)
2022-05-25 19:53:52 +00:00
Mika Ayenson 3988b2ed5e Skip previous validation on pre/post load/dump (#1942)
* Build out the dataclasses for a base entry and version lock explicitly
* Ensure previous field does not have a nested previous
* Test validation on version lock for previous fields.

(cherry picked from commit e1266a6fd3)
2022-05-25 17:36:12 +00:00
Samirbous ca7a148f5a [New rule] Remote Computer Account DnsHostName Update (#1962)
* [New rule] Remote Computer Account DnsHostName Update

Identifies remote update to a computer account DnsHostName attribute, if the new value is set a valid domain controller DNS hostname and the subject computer name is not a domain controller then it's high likely a preparation step to exploit CVE-2022-26923 in an attempt to elevate privileges from a standard domain user to domain admin privileges :

* added MS ref url

* Update rules/windows/privilege_escalation_suspicious_dnshostname_update.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

* Update rules/windows/privilege_escalation_suspicious_dnshostname_update.toml

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

Co-authored-by: Jonhnathan <jonhnathancesar@gmail.com>

(cherry picked from commit 19ff825a91)
2022-05-11 17:42:44 +00:00
Justin Ibarra 4b92b42b45 Manually reconciled versions from forked rule package generation bug (#1950)
(cherry picked from commit 8168551c59)
2022-05-04 18:06:16 +00:00
Justin Ibarra d7713cea73 Add delta command to determine changes to endpoint rules between tags (#1943)
* update git tag loader to be compatible with lock validation
* add diff command
* default to query for missing rules

(cherry picked from commit 22679e16d2)
2022-05-03 20:32:29 +00:00
Mika Ayenson 2ccbdcb773 Move etc under detection_rules (#1885)
* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-05-02 14:06:33 -04:00
Mika Ayenson cc8af968e3 Move etc under detection_rules (#1885)
* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Removed changes from:
- etc/packages.yml

(selectively cherry picked from commit 6219fc06b9)
2022-05-02 14:13:36 +00:00
Justin Ibarra fbd217ae53 Validate version lock and deprecation files on load and save (#1884)
* Validate version lock and deprecation files on load and save
* add missing types for previous lock entries
* bump marshmallow_dataclass

(cherry picked from commit c803160e4f)
2022-04-27 06:19:24 +00:00
Justin Ibarra a911907422 Remove deprecated elasticsearch parameter (#1913)
(cherry picked from commit b3e789a202)
2022-04-12 20:08:06 +00:00
Justin Ibarra bd5ada51e3 Update elasticsearch dependency to 8.1 (#1911)
(cherry picked from commit ad99c6b489)
2022-04-06 19:54:26 +00:00
Justin Ibarra eeb8ab7744 Expand timestamp override tests (#1907)
* Expand timestamp_override tests
* removed timestamp_override from eql sequence rules
* add config entry for eql rules with beats index and t_o
* add timestamp_override to missing fields

Removed changes from:
- rules/cross-platform/impact_hosts_file_modified.toml
- rules/windows/credential_access_lsass_memdump_file_created.toml
- rules/windows/defense_evasion_adding_the_hidden_file_attribute_with_via_attribexe.toml
- rules/windows/defense_evasion_execution_lolbas_wuauclt.toml
- rules/windows/defense_evasion_suspicious_certutil_commands.toml
- rules/windows/execution_command_shell_started_by_svchost.toml

(selectively cherry picked from commit 6bdfddac8e)
2022-04-01 23:28:54 +00:00
Terrance DeJesus 2edb1e0ee7 Prep for Creation of 8.3 Branch (#1906)
* updating with changes for 8.3 prep
* adding updates
* adjusted version in packages.yml

Removed changes from:
- etc/packages.yml

(selectively cherry picked from commit 648daf1237)
2022-04-01 21:35:14 +00:00
Terrance DeJesus 1ca68f9d85 added comprehensive timeline template definitions (#1905)
(cherry picked from commit e72031a71a)
2022-04-01 16:53:55 +00:00
Colson Wilhoit 150ff0502e Linux Shell Evasion Rule Tuning (#1878)
* Linux Shell Evasion Rule Tuning

* Update execution_python_tty_shell.toml

* Update rules/linux/execution_apt_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_apt_binary.toml

* Update rules/linux/execution_awk_binary_shell.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_awk_binary_shell.toml

* Update rules/linux/execution_c89_c99_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_c89_c99_binary.toml

* Update rules/linux/execution_cpulimit_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_cpulimit_binary.toml

* Update rules/linux/execution_expect_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_expect_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_expect_binary.toml

* Update rules/linux/execution_find_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_find_binary.toml

* Update rules/linux/execution_gcc_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_gcc_binary.toml

* Update rules/linux/execution_mysql_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_mysql_binary.toml

* Update rules/linux/execution_nice_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_nice_binary.toml

* Update rules/linux/execution_ssh_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_ssh_binary.toml

* Update execution_perl_tty_shell.toml

* Update execution_python_tty_shell.toml

* Update rules/linux/execution_apt_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_awk_binary_shell.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_c89_c99_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_cpulimit_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_expect_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_find_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_gcc_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_mysql_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_nice_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/linux/execution_ssh_binary.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-03-29 21:03:35 -04:00