Commit Graph

130 Commits

Author SHA1 Message Date
shashank-elastic 1ce072a4e5 Prep for Release 9.3 (#5548) 2026-01-12 21:07:07 +05:30
Mika Ayenson, PhD f40a383b7e [New Rules] Add MITRE ATLAS framework support and GenAI threat detection rules (#5352)
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>
2025-12-05 12:26:56 -06:00
Eric Forte 634de61d6d [FR] ES|QL remote validation support newline split indices (#5356)
* Updated regex pattern for multiline

* Add line split unit test
2025-12-03 11:50:51 -05:00
Eric Forte 29d4aeb37a [Bug] [DAC] Auto Gen Schema Fails on Certain Subqueries (#5256)
* Add alignment checking for sub-queries

* Allow field to be over written with original field

* Update rule prompt to allow for int 0 values

* Support custom schema index overwrite

---------

Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
2025-11-12 11:21:53 -05:00
shashank-elastic 818978975d Prep 9.2 (#5231) 2025-10-17 21:01:13 +05:30
Sergey Polzunov c7246313f7 feat: ESQL query validation against Elastic cluster (#4955)
* Add remote ESQL validation
---------

Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
2025-10-15 15:17:07 -04:00
Eric Forte 80c01cf665 [Bug] Annotated Fields Ignored (#5125)
* Add Note for stop gap
2025-09-17 17:34:42 -04:00
Mika Ayenson, PhD 3c1de72f6b [FR] Add support for 5 group_by fields in threshold rules (>=9.2) (#5040) 2025-09-04 09:24:36 -05:00
Mika Ayenson, PhD 1dc3926203 [New Rules] External Promotion Alerts (#4903) 2025-07-31 11:00:50 -05:00
shashank-elastic 9b292b97ea Prep 8.19/9.1 (#4869)
* Prep 8.19/9.1 Release

* Download Beats Schema

* Download API Schema

* Download 8.18.3 Beats Schema

* Download Latest Integrations manifest and schema

* Comment old schemas

* Update Patch version
2025-07-07 11:27:48 -04:00
Sergey Polzunov 1fb60d6475 fix: type hinting fixes and additional code checks (#4790)
* first pass

* Adding a dedicated code checking workflow

* Type fixes

* linting config and python version bump

* Type hints

* Drop incorrect config option

* More fixes

* Style fixes

* CI adjustments

* Pyproject fixes

* CI & pyproject fixes

* Proper version bump

* Tests formatting

* Resolve cirtular dependency

* Test fixes

* Make sure the tests are formatted correctly

* Check tweaks

* Bumping python version in CI images

* Pin marshmallow do 3.x because 4.x is not supported

* License fix

* Convert path to str

* Making myself a codeowner

* Missing kwargs param

* Adding a missing kwargs to `set_score`

* Update .github/CODEOWNERS

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>

* Dropping unnecessary raise

* Dropping skipped test

* Drop unnecessary var

* Drop unused commented-out func

* Disable typehinting for the whole func

* Update linting command

* Invalid type hist on the input param

* Incorrect field type

* Incorrect value used fix

* Stricter values check

* Simpler function call

* Type condition fix

* TOML formatter fix

* Simpligy output conditions

* Formatting

* Use proper types instead of aliases

* MITRE attack fixes

* Using pathlib.Path for an argument

* Use proper method to update a set from a dict

* First round of `ruff` fixes

* More fixes

* More fixes

* Hack against cyclic dependency

* Ignore `PLC0415`

* Remove unused markers

* Cleanup

* Fixing the incorrect condition

* Update .github/CODEOWNERS

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>

* Set explicit default values for optional fields

* Update the guidelines

* Adding None Defaults

---------

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
2025-07-01 08:20:55 -05:00
Jonhnathan 364d9dd3bc [New Rule] Threat Intel Email Indicator Match (#4598)
* [New Rule] Threat Intel Email Indicator Match

* Update threat_intel_indicator_match_email.toml

* Update pyproject.toml

* Adds IG

* Update rules/threat_intel/threat_intel_indicator_match_email.toml

* Update rules/threat_intel/threat_intel_indicator_match_email.toml

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>

* Update rules/threat_intel/threat_intel_indicator_match_email.toml

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>

---------

Co-authored-by: Isai <59296946+imays11@users.noreply.github.com>
2025-04-22 12:15:36 -03:00
shashank-elastic e8c54169a4 Prep main for 9.1 (#4555)
* Prep for Release 9.1

* Update Patch Version

* Update Patch version

* Update Patch version
2025-03-26 11:04:14 -04:00
shashank-elastic 059d7efa25 Prep for Release 9.0 (#4550) 2025-03-20 20:32:07 +05:30
Eric Forte 5ccb7ed4af Min stack rules from 4516 (#4549) 2025-03-19 20:27:30 -04:00
Eric Forte 5b3dc4a4a7 Revert "Add new ML detection rules for Privileged Access Detection (#4516)" (#4548)
This reverts commit 2ff8d1bb56.
2025-03-19 20:08:08 -04:00
Kirti Sodhi 2ff8d1bb56 Add new ML detection rules for Privileged Access Detection (#4516)
Add detection-rules for privileged access detection integration
2025-03-19 11:02:28 -04:00
Sergey Polzunov 5f54eb8006 chore: Removing RTAs (#4437)
* Delete RTAs

* Delete RTA-related orchestration code

* Drop RTAs from tests

* Remove RTAs from README

* Further cleanup

* Readme update

* Version bump and no more RTAs

* Styling fixes

* Drop RTAs from config files

* Drop `rule-mapping.yaml`

* Bring back event collector / normalizer

* Drop rta mention

* Cleanup rta leftovers

* Style fix

---------

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
2025-03-05 12:35:57 +01:00
shashank-elastic 2c848c5111 Prep for Release 8.18 (#4288) 2024-12-09 18:25:13 +05:30
Jonhnathan 81292aee8a [Rule Tuning] 3rd Party EDR - Add Crowdstrike FDR support - 1 (#4220)
* [Rule Tuning] 3rd Party EDR - Add Crowdstrike FDR support - 1

* Update Integrations unit tests

* Update test_all_rules.py
2024-11-04 11:32:22 -03:00
shashank-elastic 275c7288a3 Add testcase to check for related_integrations based on index (#4096) 2024-10-22 00:17:30 +05:30
Samirbous 5e0fb4a63e [Tuning] Add logs-panw.panos index to Network rules (#4089)
* [Tuning] Add logs-panw.panos index to Network rules

https://github.com/elastic/detection-rules/issues/3998

This PR adds to the PANOS traffic index `.ds-logs-panw.panos-default-*` to the network rules using fields that are compatible.

* add tag and integration

* Update command_and_control_fin7_c2_behavior.toml

* Build Manifest and Schema for panw integration

* Update definitions.py

* Update definitions.py

* Fix definitions declaration

---------

Co-authored-by: Shashank K S <Shashank.Suryanarayana@elastic.co>
2024-09-19 08:01:44 +01:00
Thijs Xhaflaire df1f0bc98e [New Rule] Add Jamf Protect detection rules (#4047)
* Create privilege_escalation_user_added_to_admin_group.toml

* Update privilege_escalation_user_added_to_admin_group.toml

* Update privilege_escalation_user_added_to_admin_group.toml

* Adding pbpaste detection rule and minor adjustments to user added to group

* Update credential_access_high_volume_of_pbpaste.toml

* Update credential_access_high_volume_of_pbpaste.toml

* Adding two rules to validate our approach.

* Updated index to "logs-jamf_protect*"

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

* Update rules/integrations/jamf/credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/integrations/jamf/privilege_escalation_user_added_to_admin_group.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Moved to rules/macos folder

* Removed rules from integration/jamf folder

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update credential_access_high_volume_of_pbpaste.toml

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* minstack rules and support jamf_protect non-dataset

---------

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
2024-09-12 15:03:56 -05:00
Eric Forte 47d7a3acaa [DaC] Beta Release (#3889)
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
2024-08-06 18:07:12 -04:00
Jonhnathan 54d5b442cf [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs (#3825)
* [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs

* .

* Update integration-schemas.json.gz

* Fix integration manifests
2024-06-26 11:06:27 -03:00
Terrance DeJesus 020ca4be24 [New Rule] Rapid7 Threat Command CVEs Correlation (#3718)
* new rule 'Rapid7 Threat Command CVEs Correlation'

* Update rules/threat_intel/threat_intel_rapid7_threat_command.toml

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* updated threat index and tags

* changed 'indicator match' to 'threat match' for tags

* removed timeline

* updating integrations to match main

* re-adding rapid7 threat command integration manifest and schema

* reverting changes; removing timeline

* changed max signals to 10000

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2024-06-12 18:01:44 -04:00
Eric Forte f43fbfba0d [FR] Update utility path computation to use pathlib (#3699)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object

* Fix typo
2024-05-23 17:36:51 -04:00
shashank-elastic f73022b900 Package Manifest changes to add capabilities (#3706) 2024-05-23 15:46:35 -05:00
Mika Ayenson 371e24b2ed Revert "[FR] Update Utility Path Computation to use Pathlib (#3659)"
This reverts commit 23567c1d0c.
2024-05-21 16:14:45 -05:00
Eric Forte 23567c1d0c [FR] Update Utility Path Computation to use Pathlib (#3659)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object
2024-05-21 14:19:20 -04:00
Jonhnathan d023ad66b1 [Rule Tuning] Add Initial SentinelOne Compatibility to Windows DRs (#3627)
* [Rule Tuning] Add Initial SentinelOne Compatibility

* updated definitions.py; updated tags; fixed unit tests

* added prerelease versions for s1 integration; updated build CLI commands to allow prerelease; bumped min-stacks

* updating manifests and integrations

* fixing flake errors

* min_stack

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
2024-05-20 09:50:57 -03:00
shashank-elastic 50a8b52cd5 Prepare For Next Elastic Stack 8.15 (#3670) 2024-05-15 00:31:02 +05:30
Mika Ayenson 2ffb0e7fe2 [New Rule] Potential Abuse of Resources by High Token Count and Large Response Sizes (#3644) 2024-05-03 18:01:53 -05:00
Eric Forte fa75876322 [Bug] New Terms Rule Import Failing (#3569)
* initial patch

* Update definitions to allow for brackets in name

* Update to prompt for required fields.

* Update detection_rules/cli_utils.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-04 17:37:13 -04:00
Jonhnathan 67ca13c1ce [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions (#3505)
* [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions

* update min_stack

* build out schema in more detail for Filters

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Remove enum for definition

* remove unused import

* remove $state store

* transform state

* add call to super

* add return type hint

* use dataclass metadata

* use Literal type

---------

Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-01 17:44:50 -03:00
shashank-elastic a4094df732 Prepare For Next Elastic Stack Minor Release (#3490) 2024-03-06 21:26:54 +05:30
Ruben Groenewoud a637bcec38 [FR] NON_DATASET_PACKAGE list & Data Source tag for Auditd_manager (#3430)
* [FR] Add Auditd_Manager to NON_DATASET_PACKAGE

* Changed alphabetical order

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-02-19 09:37:02 +01:00
Mika Ayenson c3ca01ebcc [FR] Add support for Threshold Alert Suppression (#3433) 2024-02-12 09:55:46 -06:00
Eric Forte 90a2043bc4 [FR] 8.12 Release Preparation update Main Branch to 8.13 (#3313)
* 8.12 Release Prep update Main Branch to 8.13

* Fix typo in integrations

* Updated Schemas
2023-12-11 14:58:06 -05:00
Mika Ayenson 7514c0a206 [FR] Add Support for ES|QL Rule Type and Remote Validation (#3281)
* add suuport for esql type
* add unit tests
* set clients in RemoteConnector from auth methods
* thread remote rules; add engine test
* Add versions to remote validation results

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2023-12-08 12:46:28 -07:00
Jonhnathan aeb1f91320 [Security Content] Introduce Investigate Plugin in Investigation Guides (#3080)
* [Security Content] Introduce Investigate Plugin in Investigation Guides
* Add compatibility note
* Update Transform format
* update transform unit tests for investigate
* updated docs with transform

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2023-12-08 11:54:40 -07:00
Mika Ayenson bc39c20eaf FR] Add Core Support for ES|QL Rule Type (#3292) 2023-11-28 13:03:09 -06:00
Mika Ayenson 93ad4b0959 Add UEBA Tag (#3277) 2023-11-20 13:51:13 -06:00
Terrance DeJesus cdeb398ab3 [FR] Adjust Prebuilt Rules Packaging to Use Elastic Package v3 (#3252)
* Adding support for elastic package version 3

* replaced OS with Pathlib where applicable

* added sub-dataclasses for V3

* fixed flake errors

* adjusted registry dataclasses to inherit base
2023-11-01 12:47:40 -04:00
Mika Ayenson d0b0216362 [FR] Support missing events (#3153) 2023-10-31 16:20:52 -05:00
Mika Ayenson a808130390 Cleanup saved_query references (#3205) 2023-10-26 18:07:33 -05:00
Terrance DeJesus 3ab57fb8a7 [FR] Adding Support for missing_field_strategy Field in Alert Suppression (#3201)
* adding missing field strategy option to alert suppression

* fixed linting errors

* added validate methods for alertsuppression dataclass

* fixed linting errors

* replaced old variable with new variable

* removing test rule

* adding post_load to queryruledata

* changed post_load to validates_schema

* updated unit testing for alert suppression

* fixed linting errors

* changed validates method name to validates_exceptions

* removed min compat for fields
2023-10-19 18:16:54 -04:00
Terrance DeJesus b4f8fc3290 [FR] 8.11 Release Preparation and Update Main Branch to 8.12 (#3182)
* prepping for 8.12 branch

* added ananlytic manifests and schemas

* fix linting issues

* updated analytic package manifests and schemas
2023-10-13 13:37:21 -04:00
Justin Ibarra 7f8a9849c4 [New Rule] File Compressed or Archived into Common Format (#3173)
* [New Rule] File Compressed or Archived into Common Format
* new build-threat-map-entry-command

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
2023-10-11 11:34:34 -07:00
Apoorva Joshi 747ee7d593 [New Rule] Adding Lateral Movement Rules from Advanced Analytic LMD Package (#3119)
* Adding Lateral Movement Detection rules

* added tags; adjusted tests; updated manifests and schemas

* added default value to build_integrations_schema

* combined analytic and non-dataset packages for related integrations

* adjusted machine learning definitions

* adjusted machine learning definitions

* removed splat for machine learning list due to 3.8 constraints

---------

Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2023-09-27 14:53:38 -04:00