fe9217892f
Deprecate Releasing to a patch kibana version workflow ( #3552 )
...
(cherry picked from commit 3fbffa24ed
2024-04-03 03:12:07 +00:00
112ae41cd3
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 ( #3567 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13
* Update detection_rules/etc/deprecated_rules.json
---------
Co-authored-by: shashank-elastic <shashank-elastic@users.noreply.github.com >
Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com >
(cherry picked from commit 8d5bd3b0f6
2024-04-02 18:37:42 +00:00
7838042839
[Rule Tuning] Replace KQL exceptions for Query DSL Exceptions ( #3505 )
...
* [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions
* update min_stack
* build out schema in more detail for Filters
* Update detection_rules/rule.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
* Remove enum for definition
* remove unused import
* remove $state store
* transform state
* add call to super
* add return type hint
* use dataclass metadata
* use Literal type
---------
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com >
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
Removed changes from:
- rules/windows/collection_mailbox_export_winlog.toml
- rules/windows/collection_posh_clipboard_capture.toml
- rules/windows/defense_evasion_posh_assembly_load.toml
- rules/windows/defense_evasion_posh_compressed.toml
- rules/windows/discovery_posh_suspicious_api_functions.toml
- rules/windows/discovery_privileged_localgroup_membership.toml
- rules/windows/execution_posh_hacktool_functions.toml
- rules/windows/execution_posh_psreflect.toml
- rules_building_block/collection_posh_compression.toml
- rules_building_block/defense_evasion_powershell_clear_logs_script.toml
- rules_building_block/discovery_posh_generic.toml
- rules_building_block/lateral_movement_posh_winrm_activity.toml
(selectively cherry picked from commit 67ca13c1ce
2024-04-01 20:53:09 +00:00
e74f7a4d6b
[FR] Add support for investigation_fields ( #3550 )
...
(cherry picked from commit bb907a4d76
2024-04-01 16:59:59 +00:00
69d2f4b607
Fix create PR in release workflow ( #3528 )
...
(cherry picked from commit 8b215eac41
2024-04-01 15:54:59 +00:00
e7416a6a68
[FR] Add required-fields option to import-rules ( #3546 )
...
(cherry picked from commit b6a7e7ebda
2024-03-28 23:37:15 +00:00
6bf3a82f51
Update sort parameter ( #3531 )
...
(cherry picked from commit 3503786154
2024-03-25 15:54:13 +00:00
dda6a33f70
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 ( #3526 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13
* Update detection_rules/etc/deprecated_rules.json
---------
Co-authored-by: shashank-elastic <shashank-elastic@users.noreply.github.com >
Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com >
(cherry picked from commit eaf4658620
2024-03-21 15:09:40 +00:00
edf52a578c
[FR] Update Python Dependency Versions ( #3515 )
...
(cherry picked from commit 5c3523954e
2024-03-19 19:15:12 +00:00
434b3ffcc0
[FR] Independently package kql / kibana and bump to py3.12 ( #3514 )
...
(cherry picked from commit d26981f712
2024-03-15 01:26:12 +00:00
2af0c64945
[FR] Add support for dataviews in the rule schema ( #3510 )
...
(cherry picked from commit 8724077a0e
2024-03-14 22:48:44 +00:00
59812dac4e
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 ( #3491 )
...
(cherry picked from commit bf3932f384
2024-03-06 17:45:52 +00:00
7043173371
Prepare For Next Elastic Stack Minor Release ( #3490 )
...
Removed changes from:
- detection_rules/etc/packages.yml
(selectively cherry picked from commit a4094df732
2024-03-06 16:03:19 +00:00
f8a7fe9cec
[Bug] Fix URL links in autogenerated security docs ( #3474 )
...
* added content() class method for guide and setup
* removed non-existent variable
* removed unnecessary newlines
* adjusted levels for titles
* reverting changes
* added method to convert markdown links to asciidoc
* adjusted regex to include trailing periods
* fixing linting errors
* adjusted regex pattern
* added content() class method for guide and setup
* stripped # out of investigation guide, setup or note
* adjusted formatting outcome
* changed function call
* fixed linting errors
* fixing auto-formatting for rule asciidoc
* fixing URL link removal
* fixing URL link removal
* removed strip() from string for setup
* fixed linting errors
* fixed linting errors
* adjusting code formatting for convert_markdown_to_asciidoc
(cherry picked from commit 8e0ca421ca
2024-02-23 21:55:30 +00:00
2312455d7a
[FR] Skip eql optimizations on parsing query for unique fields ( #3443 )
...
(cherry picked from commit 542053719b
2024-02-21 02:31:01 +00:00
c772b2a842
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12 ( #3459 )
...
(cherry picked from commit 7815d23110
2024-02-20 17:32:25 +00:00
984f2a6fbf
[FR] NON_DATASET_PACKAGE list & Data Source tag for Auditd_manager ( #3430 )
...
* [FR] Add Auditd_Manager to NON_DATASET_PACKAGE
* Changed alphabetical order
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit a637bcec38
2024-02-19 08:42:19 +00:00
bde05d63c6
[FR] Add support for Threshold Alert Suppression ( #3433 )
...
(cherry picked from commit c3ca01ebcc
2024-02-12 16:01:10 +00:00
00fe4c8283
[Bug] Adjust build-release CLI and fix links when generating security docs ( #3434 )
...
* removed historical argument; added setup string; fixed links
* fixing flake errors
* added types for command arguments
* adjusted get_release_diff to append strings for release tags
* set fetch-depth to 0 for integrations checkout in workflow
* changed the name of the workflow
* removed TODOs
* adjusted release docs workflow to remove prefix for release tags
* adjusted URL replacement only if pointed to docs site
* added elastic website to regex pattern
* add docstrings; adjusted regex; add note for stopgap
* added a note about the regex pattern for elastic URLs
(cherry picked from commit 06b97ec79b
2024-02-12 15:13:42 +00:00
10d36f6872
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12 ( #3431 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12
* Update detection_rules/etc/version.lock.json
* updated downloadable updates file to reconcile changes
* Removed spacing from downloadable updates file
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit 827dfa7327
2024-02-06 19:54:15 +00:00
7201490af1
[Bug] Update Prebuilt Detection Rules Release Process ( #3403 )
...
* release fleet workflow updates; build package integration reference changes
* updated commit hash extraction to output to env
* adjusted bump-pkg-versions to only include release if necessary
* fixed flake errors
* add historical argument for build-release set to yes by default
* Update detection_rules/devtools.py
* fixed fleet workflow; updated registry data references
* updated job names
* removed extract commit hash job and consolidated into fleet pr job
* added echo statement for current branch before checkout
* removed id from extract commit hash
(cherry picked from commit 7df7ab5101
2024-02-06 14:04:40 +00:00
df82c11b4a
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12 ( #3402 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12
* Update detection_rules/etc/version.lock.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit d093336125
2024-01-23 21:42:17 +00:00
cfb4f1a013
[New Rules] UEBA GItHub BBRs and Rules ( #3174 )
...
* [New Rules] UEBA GItHub BBRs and Rules
A new set of BBRs and rules that will be used to trigger new UEBA GitHub threshold Rules.
* Update rules/integrations/github/impact_github_member_removed_from_organization.toml
* Apply suggestions from code review
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com >
* edited BBR rules
-removed newly added member rule
* updated integration manifests and schemas
* Updated min_stack for some rules based on newest GitHub integration schema manifest
* testing min_stack bump to 8.8 for new fields
* removing offending rule to troubleshoot seperately
* added UEBA tags and created UEBA threshold rule
* updated non-ecs-schema to add signal.rule.tags
* updated non-ecs-schema with kibana.alert.workflow_status
* updated rule.threat.tactic
* added user.name to non-ecs-schema
* added quotes to kibana.alert.workflow_status value
* removed trailing space from rule name
* update tags and optimize query for UEBA threshold rule
* removed integration field from Higher-Order rule
* Apply suggestions from code review
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
* adjusted new_terms order and rule types based on review feedback
* Apply suggestions from code review
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com >
* remove user.name from detection_rules/etc/non-ecs-schema.json
* fix json formatting
---------
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com >
Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com >
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
(cherry picked from commit 442435830f
2024-01-22 17:53:42 +00:00
8a80d74136
[FR] Update Validate Integrations to Check Fields Across All Schema Variations ( #3372 )
...
(cherry picked from commit a873abbb5b
2024-01-18 21:47:51 +00:00
968814ddbb
[FR] Update _event_sort to use datetime instead of time ( #3375 )
...
* Update _event_sort to use datetime
* remove unused time
* added type hints
(cherry picked from commit 6170db6231
2024-01-09 16:04:18 +00:00
667df1b714
[FR] Add --include-metadata argument to export-rules command ( #3365 )
...
* added --include-metadata argument to export-rules command
* added type hinting in method definitions
* changed add_metadata to include_metadata
* adjusted argument name to include_metadata in command
* Update detection_rules/main.py
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com >
* fixed flake error
* Update detection_rules/rule.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
---------
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com >
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
(cherry picked from commit d7b62395e7
2024-01-04 21:07:56 +00:00
b319d0e68b
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12 ( #3358 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12
* Update detection_rules/etc/deprecated_rules.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit f37d13f29b
2024-01-02 17:30:46 +00:00
d9652ad592
[Bug] Fix BBR Folder Location Requirements for Specific Integrations ( #3348 )
...
* fixing bug in BBR rule folder location
* fixed export rules missing BBR rules
* adjusted directory loading
* Update tests/test_all_rules.py
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com >
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com >
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com >
(cherry picked from commit eafec1d857
2023-12-19 20:42:09 +00:00
389ac555e2
[Tuning] Remote Scheduled Task Creation ( #3337 )
...
* Update non-ecs-schema.json
* add timestamp override
---------
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com >
(cherry picked from commit 07b952b7bc
2023-12-14 23:45:08 +00:00
69f9bb416d
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12 ( #3319 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12
* Update detection_rules/etc/version.lock.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit a39a52360a
2023-12-12 18:28:19 +00:00
7b7ca3fdc9
[New Rule] Adding Detection for Stolen Credentials Used to Login to Okta Account After MFA Reset ( #3265 )
...
* adding new rule 'Stolen Credentials Used to Login to Okta Account After MFA Reset'
* updated non-ecs; linted rule; updated description
* adjusted interval and maxspan
* Update rules/integrations/okta/persistence_stolen_credentials_used_to_login_to_okta_account_after_mfa_reset.toml
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com >
* Update rules/integrations/okta/persistence_stolen_credentials_used_to_login_to_okta_account_after_mfa_reset.toml
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com >
* Update rules/integrations/okta/persistence_stolen_credentials_used_to_login_to_okta_account_after_mfa_reset.toml
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com >
* Update rules/integrations/okta/persistence_stolen_credentials_used_to_login_to_okta_account_after_mfa_reset.toml
---------
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com >
(cherry picked from commit 93d71acb91
2023-12-12 15:37:32 +00:00
908168725a
[FR] 8.12 Release Preparation update Main Branch to 8.13 ( #3313 )
...
* 8.12 Release Prep update Main Branch to 8.13
* Fix typo in integrations
* Updated Schemas
Removed changes from:
- detection_rules/etc/packages.yml
(selectively cherry picked from commit 90a2043bc4
2023-12-11 20:03:26 +00:00
2303db8486
[Bug] Use integration schemas for required_field types ( #3303 )
...
(cherry picked from commit face95058f
2023-12-11 17:37:43 +00:00
53f924d52e
[FR] Add Support for ES|QL Rule Type and Remote Validation ( #3281 )
...
* add suuport for esql type
* add unit tests
* set clients in RemoteConnector from auth methods
* thread remote rules; add engine test
* Add versions to remote validation results
---------
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com >
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com >
(cherry picked from commit 7514c0a206
2023-12-08 19:52:16 +00:00
094f3ead92
[Security Content] Introduce Investigate Plugin in Investigation Guides ( #3080 )
...
* [Security Content] Introduce Investigate Plugin in Investigation Guides
* Add compatibility note
* Update Transform format
* update transform unit tests for investigate
* updated docs with transform
---------
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com >
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com >
(cherry picked from commit aeb1f91320
2023-12-08 18:59:53 +00:00
7df6661596
Adjust ESQLRuleData to Inherit QueryRuleData Dataclass ( #3297 )
...
* adjusting inheritance of ESQL rule data
* update tests to handle missing index from QueryRuleData
* removed test es|ql rule
---------
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com >
(cherry picked from commit 5358361754
2023-11-30 14:12:26 +00:00
5bceaa3e01
Update QueryRuleData ( #3294 )
...
(cherry picked from commit f7b9a1f8df
2023-11-29 15:48:08 +00:00
53c4ff1fdc
FR] Add Core Support for ES|QL Rule Type ( #3292 )
...
(cherry picked from commit bc39c20eaf
2023-11-28 19:08:40 +00:00
1d05f49436
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11 ( #3291 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11
* Update detection_rules/etc/version.lock.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit ba7b2722c2
2023-11-28 17:35:57 +00:00
61bbcfec52
Add UEBA Tag ( #3277 )
...
(cherry picked from commit 93ad4b0959
2023-11-20 19:56:28 +00:00
dd49bcc535
[Bug] Fix typo in downgrade_contents_from_rule ( #3272 )
...
* Fix missing to_dict()
* Update pyproject.toml
(cherry picked from commit 66c1d7f3b4
2023-11-15 04:11:25 +00:00
b342660c3a
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11 ( #3270 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11
* Update detection_rules/etc/version.lock.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit 9195eedb9c
2023-11-13 19:51:15 +00:00
96f7803431
[Bug] Add Integration Schema Validation to NewTermsRuleData.validate Method ( #3227 )
...
* adjusted validation method to include integration schema checks
* fixed linting errors
* re-factored NewTermsRuleData and added unit testing
(cherry picked from commit 829f5ea885
2023-11-02 20:58:10 +00:00
4c6fc3496c
[FR] Adjust Prebuilt Rules Packaging to Use Elastic Package v3 ( #3252 )
...
* Adding support for elastic package version 3
* replaced OS with Pathlib where applicable
* added sub-dataclasses for V3
* fixed flake errors
* adjusted registry dataclasses to inherit base
Removed changes from:
- detection_rules/etc/packages.yml
(selectively cherry picked from commit cdeb398ab3
2023-11-01 16:54:01 +00:00
8415bedf0f
[FR] Support missing events ( #3153 )
...
(cherry picked from commit d0b0216362
2023-10-31 21:27:08 +00:00
9191b3e9f1
[New Rule] Adding Beaconing Rules from Advanced Analytic Beaconing Package ( #3128 )
...
* Adding beaconing rules
* Update rules/integrations/beaconing/command_and_control_beaconing_high_confidence.toml
Co-authored-by: Kirti Sodhi <109447885+sodhikirti07@users.noreply.github.com >
* Update rules/integrations/beaconing/command_and_control_beaconing.toml
Co-authored-by: Kirti Sodhi <109447885+sodhikirti07@users.noreply.github.com >
* Updating min stack version
* added beaconing to manifests and schemas; updated rules
---------
Co-authored-by: Kirti Sodhi <109447885+sodhikirti07@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co >
Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com >
(cherry picked from commit a4f9cf4616
2023-10-30 14:12:37 +00:00
55661b1239
Cleanup saved_query references ( #3205 )
...
(cherry picked from commit a808130390
2023-10-26 23:13:57 +00:00
38bc110dc5
Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11 ( #3223 )
...
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11
* Update detection_rules/etc/deprecated_rules.json
---------
Co-authored-by: terrancedejesus <terrancedejesus@users.noreply.github.com >
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com >
(cherry picked from commit ab6f28a380
2023-10-24 18:07:33 +00:00
8e5464be56
Move Setup information into setup filed ( #3206 )
...
(cherry picked from commit 7254c582c5
2023-10-23 14:05:04 +00:00
9c38931287
[FR] Adding Support for missing_field_strategy Field in Alert Suppression ( #3201 )
...
* adding missing field strategy option to alert suppression
* fixed linting errors
* added validate methods for alertsuppression dataclass
* fixed linting errors
* replaced old variable with new variable
* removing test rule
* adding post_load to queryruledata
* changed post_load to validates_schema
* updated unit testing for alert suppression
* fixed linting errors
* changed validates method name to validates_exceptions
* removed min compat for fields
(cherry picked from commit 3ab57fb8a7
2023-10-19 22:23:31 +00:00
shashank-elastic