Commit Graph

470 Commits

Author SHA1 Message Date
Isai 09ea35f33a [New Rule] AWS STS AssumeRole with New MFA Device [Rule Tuning] AWS IAM Deactivation of MFA Device (#4210)
* [New Rule] [Rule Tuning] AWS STS AssumeRole with New MFA Device, AWS IAM Deactivation of MFA Device

New terms rule for new MFA device with AssumeRole action. Rule tuning to add MITRE technique to "AWS IAM Deactivation of MFA Device"

* add serialNumber to non-ecs schema file

* fixed misspelled toml file name

* Update rules/integrations/aws/persistence_sts_assume_role_with_new_mfa.toml

Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>

---------

Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>
2024-11-05 02:09:05 -05:00
Jonhnathan 81292aee8a [Rule Tuning] 3rd Party EDR - Add Crowdstrike FDR support - 1 (#4220)
* [Rule Tuning] 3rd Party EDR - Add Crowdstrike FDR support - 1

* Update Integrations unit tests

* Update test_all_rules.py
2024-11-04 11:32:22 -03:00
github-actions[bot] 5d2940fa7c Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4217) 2024-10-28 21:07:46 +05:30
shashank-elastic 275c7288a3 Add testcase to check for related_integrations based on index (#4096) 2024-10-22 00:17:30 +05:30
github-actions[bot] c1ce0d43d1 Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4159) 2024-10-16 10:23:33 +05:30
shashank-elastic acb01cf9ee Refresh to fetch latest ECS & Beats schemas, Integration manifests & schemas. (#4140) 2024-10-10 11:30:00 +05:30
github-actions[bot] afbca3ee75 Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4147) 2024-10-09 20:56:57 -05:00
Terrance DeJesus 06319b7a13 [Rule Tuning] Add KEEP Command to all ES|QL Rules (#4146)
* updating ES|QL rules to include KEEP command

* fixed some ES|QL rules with typos; added validation for KEEP command

* fixed ES|QL errors from missing fields

* fixed flake errors

* updated date

* added best practices to hunt docs
2024-10-09 21:08:38 -04:00
Eric Forte 4edef2ea80 [FR][DAC] Import Rules Verbose Message (#4093)
* Draft Verbose Message

* Fix Linting

* Made more descriptive

* Updated for readability
2024-10-09 17:19:59 -04:00
Terrance DeJesus 281926052c [Rule Tuning] Add METADATA checks for non-aggregate ES|QL queries and fix existing (#4126)
* fixed existing rules;added query checks

* fixed flake errors

* added re.DOTALL to regex pattern, adjusted pattern slightly; reverted some rules

* removed valueError and replaced ValidationError

* adjusted validation error output based on feedback

* Update rules/integrations/aws/persistence_iam_user_created_access_keys_for_another_user.toml

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>

* Update rules/integrations/aws/persistence_iam_user_created_access_keys_for_another_user.toml

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>

* added space for failure

* updated to use re.compile

---------

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>
2024-10-09 15:25:36 -04:00
Terrance DeJesus 50e23ba242 [Hunting] Re-factor Hunting Library Code (#4085)
* updating python code for hunting library

* fixed okta queries; added MITRE search capability

* fixed hunting unit test imports

* fixed duplicate UUID; fixed duplicate index entry bug

* fixed technique finding sub-technique in search

* added more unit tests

* linted

* flake errors addressed; fixed unit test import; fixed markdown generate bug

* added description for generate-markdown command

* updated README

* adjusted YAML index, adjusted code for index changes

* adjusted relative imports; updated CODEOWNERS

* adding updates; moving to different branch for main dependencies

* finished run-query command; made some code adjustments

* removed some comments

* revised makefile; fixed unit tests; adjusted detection rules pyproject

* updated README

* updated README

* adjusted unit tests; adjusted hunt guidelines; updated makefile; adjusted several commands

* adjusted package to be more object-oriented

* removed unused variable

* Add simple breakdown stats

* addressed feedback; added keyword option for search

* Update hunting/README.md

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Update detection_rules/etc/test_hunting_cli.bash

Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>

* addressing feedback

* addressed feedback

* added message for unknown index; fixed function call

* fixed search command

* fixed flake error

---------

Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
2024-10-03 12:47:40 -04:00
github-actions[bot] 80143b23b2 Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4116) 2024-10-01 18:14:03 +05:30
shashank-elastic e2f1fcefa8 Add flag to update the docs/ATT&CK-coverage.md with markdown URL(s) (#4077) 2024-09-19 23:12:01 +05:30
Samirbous 5e0fb4a63e [Tuning] Add logs-panw.panos index to Network rules (#4089)
* [Tuning] Add logs-panw.panos index to Network rules

https://github.com/elastic/detection-rules/issues/3998

This PR adds to the PANOS traffic index `.ds-logs-panw.panos-default-*` to the network rules using fields that are compatible.

* add tag and integration

* Update command_and_control_fin7_c2_behavior.toml

* Build Manifest and Schema for panw integration

* Update definitions.py

* Update definitions.py

* Fix definitions declaration

---------

Co-authored-by: Shashank K S <Shashank.Suryanarayana@elastic.co>
2024-09-19 08:01:44 +01:00
Mika Ayenson df31c002ca [Bug] Handle formatting empty list (#4086) 2024-09-17 13:25:17 -05:00
github-actions[bot] 574064272d Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4082) 2024-09-16 21:43:16 +05:30
Terrance DeJesus bb9a772870 [New Rule] Okta Public Client App OAuth Token Request with Client Credentials (#4074)
* adding new rule for Okta public client app OAuth token request with client credentials

* Update detection_rules/etc/non-ecs-schema.json

* changing new terms to okta.actor.display_name

* linted; added references
2024-09-13 14:57:49 -04:00
shashank-elastic eda179bbe1 Skip Development Rules from Security Docs (#4073) 2024-09-13 19:57:00 +05:30
Thijs Xhaflaire df1f0bc98e [New Rule] Add Jamf Protect detection rules (#4047)
* Create privilege_escalation_user_added_to_admin_group.toml

* Update privilege_escalation_user_added_to_admin_group.toml

* Update privilege_escalation_user_added_to_admin_group.toml

* Adding pbpaste detection rule and minor adjustments to user added to group

* Update credential_access_high_volume_of_pbpaste.toml

* Update credential_access_high_volume_of_pbpaste.toml

* Adding two rules to validate our approach.

* Updated index to "logs-jamf_protect*"

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Moved to rules/macos folder

* Removed rules from integration/jamf folder

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* minstack rules and support jamf_protect non-dataset

---------

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
2024-09-12 15:03:56 -05:00
shashank-elastic 8618b1ad73 Support toml lint for investigate transforms (#4066) 2024-09-11 20:45:36 +05:30
github-actions[bot] 6a1ba19f7c Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#4050) 2024-09-03 17:40:44 +05:30
Eric Forte 0c38662cf3 [FR] [DAC] Add Support for Known Types to Auto-generated Schemas (#3985)
* Add support for autogen known type

* Add support for ML packages

* rename known_type to field_type
2024-08-28 10:48:00 -04:00
Eric Forte f7b7a04d53 [FR] Add Better Error Handling for CUSTOM_RULES_DIR (#3990)
* Add better error handling for CUSTOM_RULES_DIR

* Update detection_rules/config.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-08-28 10:30:45 -04:00
Eric Forte ba76c20b3d Update import rules to repo help text. (#4013) 2024-08-26 10:20:32 -04:00
Eric Forte 589aa33508 [Bug] Add historical Rules as Default when Build Package (#4003)
* Add historical Rules as Default

* Update num latest rule versions

* Update split for parsing

* Update saved version

* Remove if else

* write historical rules with versions

---------

Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
2024-08-21 18:00:02 -04:00
shashank-elastic c77356c0f2 Refresh Integration Manifest and Schema (#4001) 2024-08-21 22:24:05 +05:30
github-actions[bot] fbe47298cf Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#3997) 2024-08-20 23:46:25 +05:30
shashank-elastic 0c25cfb82e Remove unused @click.pass_context (#3996) 2024-08-20 23:11:22 +05:30
github-actions[bot] 760d9f6398 Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#3995) 2024-08-20 21:32:43 +05:30
Terrance DeJesus 2559b7bb41 [Rule Tuning] Tuning AWS Rules for SAML Provider Updates and Assumed Roles via STS (#3898)
* tuning AWS rules for SAML provider updates and assumed roles via STS

* fixed mitre mapping

* adjusted new terms and added user ID to query

* reverting new terms value change

* adding non-ecs to new term checks

* fixing mitre mapping

* Update rules/integrations/aws/privilege_escalation_sts_temp_creds_via_assume_role.toml

* reverting file removal to add diff changes

* changeing rule contents

* reverting rule changes

* added rule contents

* changed file name

* linted

* reverting lint
2024-08-20 11:53:46 -04:00
shashank-elastic d3dc231315 Refresh ECS, Beats manifest and schemas (#3993) 2024-08-20 20:45:20 +05:30
Mika Ayenson 10ba6ad5a6 [FR] Add Alert Suppression for Addtional Rule Types (#3986) 2024-08-15 15:03:45 -05:00
Eric Forte 400b4dbd23 [Bug] [DAC] Fix Kibana action connector export to export details with action connectors (#3984)
* Create Nested Directories

* Fix Kibana export  not exporting connector info
2024-08-13 14:28:17 -04:00
Eric Forte d0597e4260 Create Nested Directories (#3980) 2024-08-13 09:40:49 -04:00
Eric Forte 47d7a3acaa [DaC] Beta Release (#3889)
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
2024-08-06 18:07:12 -04:00
github-actions[bot] f9717e71bb Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#3961) 2024-08-06 19:37:36 +05:30
github-actions[bot] 823e8fd140 Lock versions for releases: 8.10,8.11,8.12,8.13,8.14,8.15 (#3926) 2024-07-25 18:38:08 +05:30
shashank-elastic f3b0dc1954 Prep for next release 8.16 (#3919) 2024-07-24 11:19:56 -04:00
eric-forte-elastic baee89de9b Revert "Prep for next release 8.16 (#3914)"
This reverts commit 4245a815d2.
2024-07-23 14:06:04 -04:00
shashank-elastic 4245a815d2 Prep for next release 8.16 (#3914)
* Prep for Release 8.16

* Add subscription

* Remove double subscription

* Formatting

* Formatting

* Revert Beaconing rules minstack and lock version
2024-07-23 13:04:03 -04:00
Mika Ayenson 03c99d22d3 Revert "Prep for Release 8.16 (#3913)"
This reverts commit 01135085f6.
2024-07-23 09:50:04 -05:00
shashank-elastic 01135085f6 Prep for Release 8.16 (#3913) 2024-07-23 09:42:26 -05:00
Mika Ayenson 2110ad53f0 [FR] Support new_terms schema import/export w/custom format (#3890)
* [FR] Support new_terms schema import/export w/custom format

* fix formatter for filters

* handle both rule formats when parsing data view
2024-07-12 17:17:09 -05:00
Justin Ibarra 361e97a256 [FR] Add API auth to Kibana module (#3815)
* [FR] Add API auth to Kibana module

* update make file to properly install all deps

* Bump Kibana Version

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
2024-07-11 17:19:41 -04:00
George Papakyriakopoulos 80ac2794f2 [Rule BugFix] Google Workspace Oauth2 new app (#3436)
* [Rule BugFix] Google Workspace Oauth2 new app

In our extended testing the changed rule with latest Google Workspace
integration generates the following errors which make the rule fail everytime:

```
unsupported_operation_exception: [wildcard] queries are not currently supported on keyed [flattened] fields.
```

After careful investigation this happens since the field google_workspace.token.scope.data is a flattened
JSON filed that contains one or more key/value pairs and ES does not support wildcard matches withing flattened
fields as the error suggests.

We instead query the whole field (that contains the flattened fields) with the wildcard characters and achieve
the same outcome without the error.

* [Rule BugFix] Google Workspace Oauth2 new app update (#3436)

In our extended testing the changed rule with latest Google Workspace
integration generates the following errors which make the rule fail everytime:

```
unsupported_operation_exception: [wildcard] queries are not currently supported on keyed [flattened] fields.
```

After careful investigation this happens since the field google_workspace.token.scope.data is a flattened
JSON filed that contains one or more key/value pairs and ES does not support wildcard matches withing flattened
fields as the error suggests.

We instead query the whole field (that contains the flattened fields) with the wildcard characters and achieve
the same outcome without the error.

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-07-11 10:45:17 -04:00
Eric Forte ec6038b9d9 Added Schema Check for Data View ID and Index (#3830) 2024-07-09 15:05:12 -04:00
github-actions[bot] 6a28881b5f Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3880) 2024-07-09 19:13:24 +05:30
ar3diu 5048bc26bd [Rule Tuning] Suspicious Inter-Process Communication via Outlook #3803 (#3806)
* Add "by host.id" argument to the sequence command in the rule query.

* Update collection_email_outlook_mailbox_via_com.toml

* Update non-ecs-schema.json

---------

Co-authored-by: Andrei Rediu <andrei.rediu@bit-sentinel.com>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-07-03 10:39:15 -04:00
Terrance DeJesus 99a4d629c9 [New Rule] Entra ID Device Code Auth with Broker Client (#3819)
* new rule 'Entra ID Device Code Auth with Broker Client'

* updated azure integration, non-ecs updated, rule date updated

* updates tags

* updated query to add Azure activity logs

* merging in main

* updated azure manifest and schemas

* updated azure manifest and schemas

* updated index map for summary and changelog

* removed string imports

* reverting packaging.py updates

* adjusted query

* adjusted query to be more optimized

---------

Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
2024-07-01 10:31:26 -04:00
shashank-elastic 949ceccc0f Generate Better Index Keys (#3826)
* Generate Better Index Keys

* More Robust index mapping

* Remove unused import

* Remove unused import

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-06-28 13:48:09 -04:00