Commit Graph

427 Commits

Author SHA1 Message Date
Justin Ibarra 361e97a256 [FR] Add API auth to Kibana module (#3815)
* [FR] Add API auth to Kibana module

* update make file to properly install all deps

* Bump Kibana Version

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
2024-07-11 17:19:41 -04:00
George Papakyriakopoulos 80ac2794f2 [Rule BugFix] Google Workspace Oauth2 new app (#3436)
* [Rule BugFix] Google Workspace Oauth2 new app

In our extended testing the changed rule with latest Google Workspace
integration generates the following errors which make the rule fail everytime:

```
unsupported_operation_exception: [wildcard] queries are not currently supported on keyed [flattened] fields.
```

After careful investigation this happens since the field google_workspace.token.scope.data is a flattened
JSON filed that contains one or more key/value pairs and ES does not support wildcard matches withing flattened
fields as the error suggests.

We instead query the whole field (that contains the flattened fields) with the wildcard characters and achieve
the same outcome without the error.

* [Rule BugFix] Google Workspace Oauth2 new app update (#3436)

In our extended testing the changed rule with latest Google Workspace
integration generates the following errors which make the rule fail everytime:

```
unsupported_operation_exception: [wildcard] queries are not currently supported on keyed [flattened] fields.
```

After careful investigation this happens since the field google_workspace.token.scope.data is a flattened
JSON filed that contains one or more key/value pairs and ES does not support wildcard matches withing flattened
fields as the error suggests.

We instead query the whole field (that contains the flattened fields) with the wildcard characters and achieve
the same outcome without the error.

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-07-11 10:45:17 -04:00
Eric Forte ec6038b9d9 Added Schema Check for Data View ID and Index (#3830) 2024-07-09 15:05:12 -04:00
github-actions[bot] 6a28881b5f Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3880) 2024-07-09 19:13:24 +05:30
ar3diu 5048bc26bd [Rule Tuning] Suspicious Inter-Process Communication via Outlook #3803 (#3806)
* Add "by host.id" argument to the sequence command in the rule query.

* Update collection_email_outlook_mailbox_via_com.toml

* Update non-ecs-schema.json

---------

Co-authored-by: Andrei Rediu <andrei.rediu@bit-sentinel.com>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2024-07-03 10:39:15 -04:00
Terrance DeJesus 99a4d629c9 [New Rule] Entra ID Device Code Auth with Broker Client (#3819)
* new rule 'Entra ID Device Code Auth with Broker Client'

* updated azure integration, non-ecs updated, rule date updated

* updates tags

* updated query to add Azure activity logs

* merging in main

* updated azure manifest and schemas

* updated azure manifest and schemas

* updated index map for summary and changelog

* removed string imports

* reverting packaging.py updates

* adjusted query

* adjusted query to be more optimized

---------

Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
2024-07-01 10:31:26 -04:00
shashank-elastic 949ceccc0f Generate Better Index Keys (#3826)
* Generate Better Index Keys

* More Robust index mapping

* Remove unused import

* Remove unused import

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-06-28 13:48:09 -04:00
github-actions[bot] aef9fe8ec4 Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3845) 2024-06-28 17:49:18 +05:30
Mika Ayenson 357204e1c5 [FR] Limit historical rules to the latest 2 (#3842) 2024-06-28 06:42:10 -05:00
Jonhnathan 54d5b442cf [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs (#3825)
* [Rule Tuning] Add Initial Microsoft Defender for Endpoint Compatibility to Windows DRs

* .

* Update integration-schemas.json.gz

* Fix integration manifests
2024-06-26 11:06:27 -03:00
github-actions[bot] 6f43d1f535 Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3821) 2024-06-25 17:58:37 +05:30
Mika Ayenson 259efaf716 [FR] Loosen Filters Schema Validation (#3753) 2024-06-18 15:57:14 -05:00
Terrance DeJesus 020ca4be24 [New Rule] Rapid7 Threat Command CVEs Correlation (#3718)
* new rule 'Rapid7 Threat Command CVEs Correlation'

* Update rules/threat_intel/threat_intel_rapid7_threat_command.toml

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* updated threat index and tags

* changed 'indicator match' to 'threat match' for tags

* removed timeline

* updating integrations to match main

* re-adding rapid7 threat command integration manifest and schema

* reverting changes; removing timeline

* changed max signals to 10000

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2024-06-12 18:01:44 -04:00
github-actions[bot] e3a72c6c47 Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3778) 2024-06-11 20:57:01 +05:30
Ruben Groenewoud ec223a4a05 [New Rule] Suspicious File Modification (#3746)
* [New Rule] Suspicious File Modification

* Update persistence_suspicious_file_modifications.toml

* Update rules/linux/persistence_suspicious_file_modifications.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Update rules/linux/persistence_suspicious_file_modifications.toml

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>

* Updates

* Update rules/integrations/fim/persistence_suspicious_file_modifications.toml

---------

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2024-06-11 13:03:20 +02:00
shashank-elastic e357a2c050 Refresh MITRE Attack v15.1.0 (#3725) 2024-06-04 20:14:58 +05:30
github-actions[bot] 259bab7a5a Lock versions for releases: 8.9,8.10,8.11,8.12,8.13,8.14 (#3716) 2024-05-29 19:48:22 +05:30
Terrance DeJesus 527f785a60 [New Rule] AWS EC2 VPC Security Group Rule Added for Any Address or Remote Access Ports (#3599)
* new rule 'AWS EC2 VPC Security Group Rule Added for Any Address or Remote Access Ports'

* updated rule name

* changed file name; added false-positive note

* changed rule UUID

* adjusted file name

* updated tags

* added investigation guide; updated query logic

* Update rules/integrations/aws/defense_evasion_vpc_security_group_ingress_rule_added_for_remote_connections.toml

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* updated query and name

* updated query optimization

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
2024-05-28 10:49:20 -04:00
Eric Forte f43fbfba0d [FR] Update utility path computation to use pathlib (#3699)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object

* Fix typo
2024-05-23 17:36:51 -04:00
shashank-elastic f73022b900 Package Manifest changes to add capabilities (#3706) 2024-05-23 15:46:35 -05:00
shashank-elastic 63e91c2f12 Back-porting Version Trimming (#3704) 2024-05-23 00:45:10 +05:30
Mika Ayenson 2c3dbfc039 Revert "Back-porting Version Trimming (#3681)"
This reverts commit 71d2c59b5c.
2024-05-22 13:51:46 -05:00
shashank-elastic 71d2c59b5c Back-porting Version Trimming (#3681) 2024-05-23 00:11:50 +05:30
Mika Ayenson 371e24b2ed Revert "[FR] Update Utility Path Computation to use Pathlib (#3659)"
This reverts commit 23567c1d0c.
2024-05-21 16:14:45 -05:00
Eric Forte 23567c1d0c [FR] Update Utility Path Computation to use Pathlib (#3659)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object
2024-05-21 14:19:20 -04:00
Jonhnathan d023ad66b1 [Rule Tuning] Add Initial SentinelOne Compatibility to Windows DRs (#3627)
* [Rule Tuning] Add Initial SentinelOne Compatibility

* updated definitions.py; updated tags; fixed unit tests

* added prerelease versions for s1 integration; updated build CLI commands to allow prerelease; bumped min-stacks

* updating manifests and integrations

* fixing flake errors

* min_stack

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
2024-05-20 09:50:57 -03:00
Eric Forte 707ca32ab1 [FR] Add --force flag to update-lock-versions (#3693)
* Add --force flag to update-lock-versions

* Add type hinting
2024-05-17 20:25:08 -04:00
Mika Ayenson 79f575b33c [FR] Normalize yml ext to yaml (#3675) 2024-05-15 15:18:39 -05:00
github-actions[bot] f3585da503 Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13,8.14 (#3676) 2024-05-15 17:04:22 +05:30
shashank-elastic 50a8b52cd5 Prepare For Next Elastic Stack 8.15 (#3670) 2024-05-15 00:31:02 +05:30
Mika Ayenson 78837549e8 [FR] Bundle KQL & Kibana libs into base dependencies (#3662) 2024-05-13 14:29:03 -05:00
Eric Forte 094ef22604 [Bug] Update Rule Formatter (#3668)
* Update Rule Formatter

* Only apply fix to Note
2024-05-13 15:00:01 -04:00
github-actions[bot] 84437bac03 Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 (#3650)
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13

* Bumping status checks

* undo bump

---------

Co-authored-by: eric-forte-elastic <eric-forte-elastic@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
2024-05-06 12:44:32 -04:00
Eric Forte a4a0bc6a7e [Bug] Query validation failing to capture InSet edge case with ip field types (#3572)
* Move test case to separate file

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
2024-05-06 07:58:42 -04:00
Mika Ayenson 2ffb0e7fe2 [New Rule] Potential Abuse of Resources by High Token Count and Large Response Sizes (#3644) 2024-05-03 18:01:53 -05:00
Justin Ibarra 2668f5f762 [Bug] Fix missing indexes on navigator build (#3636)
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2024-05-01 15:50:54 -06:00
Justin Ibarra 54ff270c62 [New Rule] AWS S3 Bucket Enumeration or Brute Force (#3635)
* [New Rule] AWS S3 Bucket Enumeration or Brute Force
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>
---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>
2024-05-01 15:00:33 -06:00
github-actions[bot] ca78f550fd Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 (#3630) 2024-04-30 18:06:01 +05:30
Justin Ibarra c567d3731a Refresh Kibana module with API updates (#3466)
* Refresh Kibana module with API updates
* add import/export commands
* rename repo commands
* add RawRuleCollection and DictRule objects
* save exported rules to files; rule.from_rule_resource
* strip unknown fields in schema
* add remote cli test
* update docs
* bump kibana lib version

---------

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2024-04-26 11:12:50 -06:00
github-actions[bot] 374f21fbc4 Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 (#3615) 2024-04-23 17:59:01 +05:30
Jonhnathan d0dfa479bb [Rule Tuning] Windows BBR Rule Tuning - 1 (#3579)
* [Rule Tuning] Windows BBR Rule Tuning - 1

* Update non-ecs-schema.json

* Update rules_building_block/command_and_control_certutil_network_connection.toml

Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>

* Update rules_building_block/collection_common_compressed_archived_file.toml

Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>

* Update defense_evasion_dll_hijack.toml

---------

Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Ruben Groenewoud <78494512+Aegrah@users.noreply.github.com>
2024-04-08 10:38:41 -03:00
Terrance DeJesus 0cb42983c1 updated to v14.0 mitre ATT&CK (#3289)
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
2024-04-05 14:30:23 -04:00
Eric Forte fbb6df506e Update default (#3574) 2024-04-04 20:27:14 -04:00
Eric Forte 1566c29bae [Bug] KQL fails validation on uppercase keywords (#3568)
* add todo

* Add a normalize_kql_keywords function to utils

* update rule loader to normalize and warn

* optimized loading

* fix linting

* Moved conversion to kql module.

* Updated unit test

* Refactor KQL parser to normalize keywords via flag

* Fix logic typo

* Update detection_rules/utils.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Update lib/kql/kql/__init__.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

* Updated to fix unit tests and remove warnings

* linting typo

* Added comments

* remove unused imports

* Update kql.parse default

---------

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-04 18:03:30 -04:00
Eric Forte fa75876322 [Bug] New Terms Rule Import Failing (#3569)
* initial patch

* Update definitions to allow for brackets in name

* Update to prompt for required fields.

* Update detection_rules/cli_utils.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-04 17:37:13 -04:00
Mika Ayenson c35652c8c8 [Bug] Add explicit format preserver (#3566) 2024-04-04 15:50:48 -05:00
Eric Forte a9cc323d09 [Bug] Threshold Rule Importing Failures (#3560)
* remove threshold specific req

* fix test event override

---------

Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
2024-04-03 14:15:09 -04:00
shashank-elastic 3fbffa24ed Deprecate Releasing to a patch kibana version workflow (#3552) 2024-04-03 08:34:45 +05:30
github-actions[bot] 8d5bd3b0f6 Lock versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13 (#3567)
* Locked versions for releases: 8.3,8.4,8.5,8.6,8.7,8.8,8.9,8.10,8.11,8.12,8.13

* Update detection_rules/etc/deprecated_rules.json

---------

Co-authored-by: shashank-elastic <shashank-elastic@users.noreply.github.com>
Co-authored-by: shashank-elastic <91139415+shashank-elastic@users.noreply.github.com>
2024-04-02 23:59:42 +05:30
Jonhnathan 67ca13c1ce [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions (#3505)
* [Rule Tuning] Replace KQL exceptions for Query DSL Exceptions

* update min_stack

* build out schema in more detail for Filters

* Update detection_rules/rule.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Remove enum for definition

* remove unused import

* remove $state store

* transform state

* add call to super

* add return type hint

* use dataclass metadata

* use Literal type

---------

Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2024-04-01 17:44:50 -03:00