Commit Graph

106 Commits

Author SHA1 Message Date
Jonhnathan 54d5b442cf [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs (#3825)
* [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs

* .

* Update integration-schemas.json.gz

* Fix integration manifests
2024-06-26 11:06:27 -03:00
Terrance DeJesus 020ca4be24 [New Rule] Rapid7 Threat Command CVEs Correlation (#3718)
* new rule 'Rapid7 Threat Command CVEs Correlation'

* Update rules/threat_intel/threat_intel_rapid7_threat_command.toml

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* updated threat index and tags

* changed 'indicator match' to 'threat match' for tags

* removed timeline

* updating integrations to match main

* re-adding rapid7 threat command integration manifest and schema

* reverting changes; removing timeline

* changed max signals to 10000

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2024-06-12 18:01:44 -04:00
Eric Forte f43fbfba0d [FR] Update utility path computation to use pathlib (#3699)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object

* Fix typo
2024-05-23 17:36:51 -04:00
shashank-elastic f73022b900 Package Manifest changes to add capabilities (#3706) 2024-05-23 15:46:35 -05:00
Mika Ayenson 371e24b2ed Revert "[FR] Update Utility Path Computation to use Pathlib (#3659)"
This reverts commit 23567c1d0c.
2024-05-21 16:14:45 -05:00
Eric Forte 23567c1d0c [FR] Update Utility Path Computation to use Pathlib (#3659)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object
2024-05-21 14:19:20 -04:00
Jonhnathan d023ad66b1 [Rule Tuning] Add Initial SentinelOne Compatibility to Windows DRs (#3627)
* [Rule Tuning] Add Initial SentinelOne Compatibility

* updated definitions.py; updated tags; fixed unit tests

* added prerelease versions for s1 integration; updated build CLI commands to allow prerelease; bumped min-stacks

* updating manifests and integrations

* fixing flake errors

* min_stack

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
2024-05-20 09:50:57 -03:00
shashank-elastic 50a8b52cd5 Prepare For Next Elastic Stack 8.15 (#3670) 2024-05-15 00:31:02 +05:30
Mika Ayenson 2ffb0e7fe2 [New Rule] Potential Abuse of Resources by High Token Count and Large Response Sizes (#3644) 2024-05-03 18:01:53 -05:00
Eric Forte fa75876322 [Bug] New Terms Rule Import Failing (#3569)
* initial patch

* Update definitions to allow for brackets in name

* Update to prompt for required fields.

* Update detection_rules/cli_utils.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-04 17:37:13 -04:00
Jonhnathan 67ca13c1ce [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions (#3505)
* [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions

* update min_stack

* build out schema in more detail for Filters

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Remove enum for definition

* remove unused import

* remove $state store

* transform state

* add call to super

* add return type hint

* use dataclass metadata

* use Literal type

---------

Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-01 17:44:50 -03:00
shashank-elastic a4094df732 Prepare For Next Elastic Stack Minor Release (#3490) 2024-03-06 21:26:54 +05:30
Ruben Groenewoud a637bcec38 [FR] NON_DATASET_PACKAGE list & Data Source tag for Auditd_manager (#3430)
* [FR] Add Auditd_Manager to NON_DATASET_PACKAGE

* Changed alphabetical order

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-02-19 09:37:02 +01:00
Mika Ayenson c3ca01ebcc [FR] Add support for Threshold Alert Suppression (#3433) 2024-02-12 09:55:46 -06:00
Eric Forte 90a2043bc4 [FR] 8.12 Release Preparation update Main Branch to 8.13 (#3313)
* 8.12 Release Prep update Main Branch to 8.13

* Fix typo in integrations

* Updated Schemas
2023-12-11 14:58:06 -05:00
Mika Ayenson 7514c0a206 [FR] Add Support for ES|QL Rule Type and Remote Validation (#3281)
* add suuport for esql type
* add unit tests
* set clients in RemoteConnector from auth methods
* thread remote rules; add engine test
* Add versions to remote validation results

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2023-12-08 12:46:28 -07:00
Jonhnathan aeb1f91320 [Security Content] Introduce Investigate Plugin in Investigation Guides (#3080)
* [Security Content] Introduce Investigate Plugin in Investigation Guides
* Add compatibility note
* Update Transform format
* update transform unit tests for investigate
* updated docs with transform

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2023-12-08 11:54:40 -07:00
Mika Ayenson bc39c20eaf FR] Add Core Support for ES|QL Rule Type (#3292) 2023-11-28 13:03:09 -06:00
Mika Ayenson 93ad4b0959 Add UEBA Tag (#3277) 2023-11-20 13:51:13 -06:00
Terrance DeJesus cdeb398ab3 [FR] Adjust Prebuilt Rules Packaging to Use Elastic Package v3 (#3252)
* Adding support for elastic package version 3

* replaced OS with Pathlib where applicable

* added sub-dataclasses for V3

* fixed flake errors

* adjusted registry dataclasses to inherit base
2023-11-01 12:47:40 -04:00
Mika Ayenson d0b0216362 [FR] Support missing events (#3153) 2023-10-31 16:20:52 -05:00
Mika Ayenson a808130390 Cleanup saved_query references (#3205) 2023-10-26 18:07:33 -05:00
Terrance DeJesus 3ab57fb8a7 [FR] Adding Support for missing_field_strategy Field in Alert Suppression (#3201)
* adding missing field strategy option to alert suppression

* fixed linting errors

* added validate methods for alertsuppression dataclass

* fixed linting errors

* replaced old variable with new variable

* removing test rule

* adding post_load to queryruledata

* changed post_load to validates_schema

* updated unit testing for alert suppression

* fixed linting errors

* changed validates method name to validates_exceptions

* removed min compat for fields
2023-10-19 18:16:54 -04:00
Terrance DeJesus b4f8fc3290 [FR] 8.11 Release Preparation and Update Main Branch to 8.12 (#3182)
* prepping for 8.12 branch

* added ananlytic manifests and schemas

* fix linting issues

* updated analytic package manifests and schemas
2023-10-13 13:37:21 -04:00
Justin Ibarra 7f8a9849c4 [New Rule] File Compressed or Archived into Common Format (#3173)
* [New Rule] File Compressed or Archived into Common Format
* new build-threat-map-entry-command

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
2023-10-11 11:34:34 -07:00
Apoorva Joshi 747ee7d593 [New Rule] Adding Lateral Movement Rules from Advanced Analytic LMD Package (#3119)
* Adding Lateral Movement Detection rules

* added tags; adjusted tests; updated manifests and schemas

* added default value to build_integrations_schema

* combined analytic and non-dataset packages for related integrations

* adjusted machine learning definitions

* adjusted machine learning definitions

* removed splat for machine learning list due to 3.8 constraints

---------

Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2023-09-27 14:53:38 -04:00
Mika Ayenson 20de1d8d1d [FR] Add support for samples in eql 0.9.18 (#3000) 2023-09-07 09:01:28 -05:00
Terrance DeJesus 08b646aa94 [FR] 8.10 Release Preparation and Update Main Branch to 8.11 (#3012)
* prepping for 8.11 branch

* fixed lint errors

* added 8.11 to stack schema map

* trimmed version lock file; adjusted new terms validation

* reverting changes to version lock, stack schema and workflow
2023-08-16 14:23:44 -04:00
Mika Ayenson 3f9e7aced1 [Bug] Strip Non-Public Fields Prior to Uploading Rules (#2986) 2023-08-02 12:38:48 -05:00
Mika Ayenson 2ff4584456 load unsupported rule type from schema (#2893) 2023-06-29 15:32:32 -04:00
Terrance DeJesus 35d373b2bd [FR] 8.9 Release Preparation and Update Main Branch to 8.10 (#2891)
* adding new branch and refreshed schema

* fixed flake errors
2023-06-29 11:39:11 -04:00
Mika Ayenson cec41b4072 [FR Build a limited compatible rule ndjson for older stacks (#2885) 2023-06-29 10:18:24 -04:00
Jonhnathan a7e605a0e5 [Rule Tuning] [BUG] Revert PowerShell Query modifications from #2823 (#2889)
* Revert query mods done in https://github.com/elastic/detection-rules/pull/2823

* Add exception to unit test

* fixed linting

* proper linting fix

* updated to add to definitions.py

* fix linting

---------

Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
2023-06-28 15:55:43 -03:00
Terrance DeJesus 48cf95c8eb [Rule Tuning] Change Network Rules to Use Network Packet Capture Integration (#2665)
* updated indexes and updated dates

* added network_traffic integration tag to rules

* reverting changes to resolve conflicts

* metadata changes; indexes changed; schemas and manifest updated

* updated default telnet port connection rule

* updating integration manifests

* adjusted rules; updated integrations; deduplicate packages
2023-06-26 17:35:49 -04:00
Terrance DeJesus d829b145ef [Bug] Fix Tag Navigator Generation (#2875)
* bug fix for tag navigator generation

* addressing flake errors

* added unit test to ensure prefix exists

* updated unit test case sensitivity

* moved expected tags to definitions.py

* removed expected prefixes

* revert downloadable updates JSON file
2023-06-23 10:44:55 -04:00
eric-forte-elastic 6449cecd08 [FR] Add support for building block rules (BBR) (#2822)
* added test bbr

* initial implementation

* Added Unit test and exempted bbr from integrations

* fixed linting

* Add schema validation to building block rules

* add separate error messages

* fixed linting

* Add testing bbr validation

* fixed linting

* Add default values

* fixed linting

* added defaults

* fixed linting

* cleaned up test rule

* removed .gitkeep

* read .gitkeep

* Switch to using validates_schema

* addressing some linting

* fixed linting

* Update detection_rules/schemas/definitions.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* add env variable check

* fix skip function

* updated name

* Update detection_rules/schemas/definitions.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Add bbr validation unit test

* Clean up comments

* fix linting

* Move convert time to utils

* Moved to rules_building_block

* Add check for only bbr in bbr dir

* fix linting

* additional linting fix

* Changed to bbr rule loader

* fixed bbr default

* Updated error messages and README

* fixed more linting

* Updating root level README

* Fixed convert_time_span calls

* fixed typo in unit test logic and updated txt

* fixed error message

* updated comment for clarity

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Updated validation methods for clarity

* fix doctring location

* Fixed typo

* updated error messages.

* removed excess whitespace

* Add per rule bypass

* Add single rule bypass

* Split unit tests

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-06-20 09:00:30 -04:00
Terrance DeJesus 792da36fb9 [Bug] Add Cloud Defend to definitions.NON_DATASET_PACKAGES (#2764)
* updating code to include cloud defend package

* updated integration manifests and schemas
2023-04-28 11:23:48 -04:00
Terrance DeJesus fadb5c2343 [FR] 8.8 Release Preparation and Update Main Branch to 8.9 (#2734)
* [FR] 8.8 Release Preparation and Update Main Branch to 8.9

* fixed flake errors
2023-04-24 10:13:07 -04:00
Mika Ayenson e9ebb1f2d8 [Bug] Rename 8.7 schemas from *.master and strip build time fields (#2707) 2023-04-11 10:56:20 -04:00
Terrance DeJesus 7e28b8fc50 [FR] Support Rule Alert Suppression in Rule Schema (#2660)
* adding initial solution for alert suppression support in rule schema

* reverting rule changes

* fixing flake errors

* reverting rule changes

* adding unit tests

* addressing flake errors

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Update detection_rules/schemas/definitions.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* adjusting rule.py after commits

* adjusted test_group_field_in_schemas to check integrations

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* nested AlertSuppressDuration class under mapping class

* adjusted dataclass naming

* added unit test to ensure rule is KQL

* fixing flake errors

* added docstrings

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2023-03-27 15:37:35 -04:00
Terrance DeJesus 8a7ad13611 [FR] 8.7 Release Preparation and Update Main Branch to 8.8 (#2533)
* adding preparations for 8.8 release

* addressed flake single new line error

* froze and updated API schemas

* updated get_intregration_manifests

* adjusted boolean in find_latest_integration_version
2023-02-08 17:27:21 -05:00
Terrance DeJesus fb2b4529c5 [FR] Adapt PyPi semver Library and Remove Custom (#2503)
* removed custom semver and replaced with pypi

* updated beats.py version references

* updated bump-versions CLI command to use semver and change logic

* updated schemas __init__, test_version_lock and unstage incompatible rules CLI

* updated test_stack_schema_map in TestVersions unittest

* updated test_all_rules unit testing Version() references

* updated stack_compat.py for get_restricted_field references)

* updated version_lock.py Version() references

* updated docs.py Version() reference for parse_registry

* updated devtools.py Version() reference for trim-version-lock

* updated mixins.py Version() reference in validate_field_compatibility

* adjusted schemas.__init__ Version() reference in get_stack_schemas

* adjusted ecs.py Version() references

* adjusted integrations.py Version() references

* adjusted rule.py Version() references

* sorted imports

* replaced custom semver with pypi semver in unit test files

* addressed unit test and flake errors

* changed semver strings casted to version_lock.py

* fixed sorting in integrations.py

* updated bump-pkgs-versions CLI command

* adjusted semantic version in unstage-incompatible-rules command

* adjusted semver import to VersionInfo

* added semver 3 and adjusted import names

* added option_minor_and_patch parameter where version is major.minor

* updated bump-pkg-versions to always save to packages.yml

* removed leftover split call & updated find latest compatible version command

* updated integrations.py, version_lock.py and schemas.__init__.py

* changed fstring reference in downgrade function

* reverted formatting changes for detection_rules __init__.py

* added newline to detection_rules __init__.py

* adjusted finding latest_release for attack package logic

* adjusted unstage-incompatible-rules command logic comparing versions

* removing changes from misc.py related to auto-formatting

* adding newline to misc.py

* fixed bug in downgrade function calling decorators

* added semantic version validation on migrate decorator function

* added expected type returned from find_latest_integration_version in integrations.py

* add comment about stripped versions for version lock file

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-02-07 14:26:29 -05:00
Mika Ayenson 1784429aa7 [FR] Add Integration Schema Query Validation (#2470) 2023-02-02 16:22:44 -05:00
Jonhnathan 0acbe1d832 [New Rule] Multiple Alerts Involving a User (#2401)
* [New Rule] Multiple Alerts Involving a User

* Update definitions.py

* update query

* Update multiple_alerts_involving_user.toml

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
2023-01-03 12:25:40 -03:00
Terrance DeJesus 7e459dd585 [FR] Add support for New Terms Fields and Window Start History (#2360)
* adding support new_terms_fields and window_start_history

* adjusted rule.py to address flake errors

* added assertion error if history_window_start does not exist

* removed sample rule

* removed self.rule_id from DataValidator

* added new_terms to RuleType

* changed new terms to its own class in rule.py

* removed nonexisting function call in DataValidator class

* adjusted new_terms field value in dataclass

* changed literal type for history_window_start; view-rule working

* removing test TOML rule

* addressed flake errors for missing newlines

* added validation option and adjusted object referencing

* adjusted validation method call in post_validation

* addressed flake errors for multiple spaces

* added transform method to NewTermsRuleData class

* added validation for min stack version and new terms array length restraints

* added validation for unique new terms array

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* removed historywindowstart definition and adjusted subclass

* removed test rule from commit

* adjusted if/else for data transform method check

* adjusted stack-schema-map; validation method name

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* added assertion for history_window_start field value

* added variables for feature min stack and extended field min stack

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* addressed flake errors for continuation line with same indent

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-12-05 14:07:33 -05:00
Terrance DeJesus 57b8f630de initial commit with changes for 8.7 branch creation (#2406) 2022-11-21 12:55:01 -05:00
Jonhnathan a7caa4baf3 [New Rule] Multiple Alerts in Different ATT&CK Tactics on a Single Host (#2399)
* [New Rule] Multiple Alerts in Different ATT&CK Tactics on a Single Host

* Update definitions.py

* Update rules/cross-platform/multiple_alerts_different_tactics_host.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-11-18 17:38:27 -03:00
Terrance DeJesus facaef1389 Prep for 8.6 Branch Creation (#2308)
* adding prep for 8.6

* updated ecs and beats packages

* addressed flake errors
2022-09-21 17:01:02 -04:00
Terrance DeJesus 2a3b584433 Prep for 8.5 branch (#2220)
* adding first commit

* renamed branch

* adjusted packages, stack schema and updated schemas

* updated integrations manifest

* adjusted comments to be a little more organized

* adjusted stack-schema-map

* refreshed ecs and beats schema, adjusted stack schema map accordingly
2022-08-09 17:14:42 -04:00
Justin Ibarra cc01d3fb1a Add support for restricted fields (#2053)
* Add support for restricted fields (fields valid only in min/max stack versions)
* add test to ensure rule backports wont exceed min compat
2022-06-27 10:02:15 -05:00