Commit Graph

762 Commits

Author SHA1 Message Date
Khristinin Nikita 2bb9fdb724 Add default timestamp condition for threat_query (#1486)
(cherry picked from commit 10a977914b)
2021-09-20 19:20:58 +00:00
Justin Ibarra 143afc4f38 [KQL] Add support for date fields in parser (#1487)
* [KQL] Add support for date fields in parser

* add test for parsing date value

(cherry picked from commit 582a842e32)
2021-09-16 17:26:26 +00:00
Justin Ibarra 0a3bd9130d Allow CLi config to be multiple formats (#1485)
(cherry picked from commit 7179942be3)
2021-09-16 04:13:51 +00:00
dstepanic17 c864538606 [rule-tuning] Adding more context with triage/investigation (#1481)
* [rule-tuning] Adding more context with triage/investigation

* Adding mimikatz rule

* Fixed updated date on mimikatz rule

* Adding Defender update

* Adding scheduled task

* Adding AdFind

* Adding rare process

* Adding cloudtrail country

* Adding cloudtrail spike

* Adding threat intel

* Fixed minor spelling/syntax

* Fixed minor spelling/syntax p2

* Update rules/cross-platform/threat_intel_module_match.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/integrations/aws/ml_cloudtrail_error_message_spike.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/ml/ml_rare_process_by_host_windows.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/credential_access_mimikatz_powershell_module.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/credential_access_mimikatz_powershell_module.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/defense_evasion_defender_exclusion_via_powershell.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update rules/windows/discovery_adfind_command_activity.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Removed MITRE link, added Microsoft

* Update ml_cloudtrail_error_message_spike.toml

* Update ml_cloudtrail_rare_method_by_country.toml

* Update ml_rare_process_by_host_windows.toml

* Update credential_access_mimikatz_powershell_module.toml

* Update defense_evasion_defender_exclusion_via_powershell.toml

* Update discovery_adfind_command_activity.toml

* Update lateral_movement_dns_server_overflow.toml

* Update lateral_movement_scheduled_task_target.toml

* Update persistence_evasion_registry_startup_shell_folder_modified.toml

* Update defense_evasion_defender_exclusion_via_powershell.toml

* Update lateral_movement_scheduled_task_target.toml

* Update persistence_evasion_registry_startup_shell_folder_modified.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

(cherry picked from commit 9ff3873ee7)
2021-09-16 01:08:23 +00:00
Justin Ibarra 31202bf4f6 [Rule tuning] Fix typo in ML rule descriptions (#1484)
(cherry picked from commit 51a2bc815b)
2021-09-14 16:37:55 +00:00
Justin Ibarra 938cc5b8b5 [Bug] CLI Fixes (#1073)
* add support for self-signed certs in es and kibana
* allow Kibana to auth against any providerType
* fix export-rules command
* fix kibana upload-rule command
* fix view-rule command
* fix validate-rule command
* fix search-rules command
* fix dev kibana-diff command
* fix dev package-stats command
* fix dev search-rule-prs command
* fix dev deprecate-rule command
* replace toml with pytoml to fix import-rules command
* use no_verify in get_kibana_client
* use Path for rule-file type in view-rule
* update schemas to resolve additionalProperties type bug
* fix missing unique_fields in package rule filter
* fix github pr loader
* Load gh rules as TOMLRule instead of dict
* remove unnecessary version insertion

(cherry picked from commit 5b24eca0bc)
2021-09-10 18:07:10 +00:00
Samirbous 105a1fd023 [New Rule] Behavior Rule for CVE-2021-40444 Exploitation (#1479)
* [New Rule] Behavior Rule for CVE-2021-40444 Exploitation

* added a ref

* replaced \ with /

* removed unecessary wildcard

(cherry picked from commit 0875c1e4c4)
2021-09-08 19:27:16 +00:00
dstepanic17 88bfc67638 Adding control.exe (#1477)
(cherry picked from commit cb27c686e0)
2021-09-08 18:31:51 +00:00
github-actions[bot] 2ed00c3f95 Lock versions for releases: 7.13,7.14,7.15 (#1474)
* Locked versions for releases: 7.13,7.14,7.15
* remove extra previous sections
* add backport label to workflow

(cherry picked from commit 58a4483222)
2021-09-07 20:33:39 +00:00
David French f77e18977a Generate detection rule to alert on traffic to typosquatting/homonym domains (#1199)
* create new cli commands

* add kibana object to create_dnstwist_rule

* Adding code for index-dnstwist-results

* Changed es to es_client

* Tested. it works!

* flake8-ed

* Adding timestamps

* use eql.utils.load_dump to load json file

* rename data to dnstwist_data

* start working on create-dnstwist-rule command

* add print statements for user

* tweak formatting for line length

* add template threat match rule file

* continue working on threat match rule creation

* create rule using TomlRuleContents

* save rule to toml file

* Moving rule creation to eswrap.py

* Moving create dnstwist rule stuff to eswrap

* Fixed imports

* flake8 fixes

* More flake8 fixes

* fix usage of @add_client('kibana')

* use ctx.invoke to upload rule

* cleanup record assembly and use bulk api

* swap order of notes in `note` for sample rule

* small modifications

* move command to root click group

* remove unused click group

* Update detection_rules/main.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* remove rule upload and convert template to ndjson

* Adding docs for typosquatting rule

* renaming the file

* Adding a note

* separate index and rule prep commands

* Final changes

Co-authored-by: Apoorva <appujo@gmail.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Apoorva Joshi <30438249+ajosh0504@users.noreply.github.com>

(cherry picked from commit 90aa65aed3)
2021-09-03 20:36:52 +00:00
Ross Wolf 2ef59e918f Revert #1440 new endpoint promotion rule (#1470)
* Revert #1440 new endpoint promotion rule
* Set the updated_at date

Removed changes from:
- rules/integrations/endpoint/elastic_endpoint_security_behavior_protection.toml

(selectively cherry picked from commit c9d6527280)
2021-09-03 14:08:22 +00:00
Justin Ibarra eb37f07417 Add DeprecatedCollection to RuleCollection to bypass validation (#1454)
* Add DeprecatedCollection to RuleCollection to bypass validation
* use DeprecatedRule properties in RuleCollection
* use RuleCollection filter for max/min filtering in Package

(cherry picked from commit 7710e2b798)
2021-09-01 23:31:06 +00:00
Ross Wolf e9d67898d9 [CI] Notify slack on backport failure (#1468)
(cherry picked from commit c395d799b4)
2021-09-01 12:48:45 +00:00
Justin Ibarra 21628611a9 [Bug] Community label: use getMembershipForUser (#1469)
Use getMembershipForUser to determine the proper org membership status

(cherry picked from commit 2a7d036443)
2021-09-01 05:33:32 +00:00
Justin Ibarra 7371608d39 [Bug] RuleTOMLContents.to_dict serialize with proper schema (#1460)
(cherry picked from commit 9d10458be4)
2021-09-01 05:07:14 +00:00
Justin Ibarra 2a2bcbd870 [Rule tuning] Fix spacing in reference URLs (#1455)
(cherry picked from commit 655f7d91d0)
2021-09-01 00:00:06 +00:00
Nic 20a814c47f [Rule tuning] Azure Active Directory High Risk Sign-in (#1463)
* Add Aggregated Risk Level
* There can be a risk_level_during_signin:low but have a risk_level_aggregated:high which is also just as concerning and must be alerted on.
* An example is a password spray attack and have a successful login. Which makes me consider a new rule for interesting risk event types

(cherry picked from commit 8b2c8c2e03)
2021-08-30 22:34:47 +00:00
Ross Wolf 3204a5c366 Update main to point to 7.16 (#1457)
* Update main to point to 7.16
* Add 7.16 -> 7.15 migration
* Update stack-schema-map
* Update conditions.kibana.version

Removed changes from:
- etc/packages.yml

(selectively cherry picked from commit 7b8b18cb20)
2021-08-26 20:24:53 +00:00
Ross Wolf 79d3b60c9a [CI] Add GitHub actions workflow to lock versions across branches (#1456)
* Start job to lock versions
* Update lock-versions workflow
* Call lock-multiple script
* Fix script
* Add the lock file to staging
* pass branches to the job
* Fetch all branches and tags
* Push the branch first
* Push with upstream
* Change PR params
* Remove protections machine token
* Add 7.14.0 to the lock for min_stack_version=7.14.0
* Fix branch prefix
* Add trailing newline
* Trailing newline
* Restrict to main branch

(cherry picked from commit 4adad703fc)
2021-08-26 20:18:34 +00:00
Ross Wolf 1f7c404548 Remove the 7.15+ behavior protection promotion rule 2021-08-26 08:51:38 -06:00
Apoorva Joshi b883415914 Small update to docs (#1442)
(cherry picked from commit 227b67e636)
2021-08-26 06:41:40 +00:00
Ross Wolf 34ab6c81d3 [New Rule] Endpoint Security Behavior Protection (#1440)
* [New Rule] Endpoint Security Behavioral Protection
* Update readme and labeler for endpoint integration
* Fix new rule to use event.code
* Fix old rule to use event.code
* Changed from behavioral to behavior
* Rename elastic_endpoint_security_behavioral.toml to elastic_endpoint_security_behavior_protection.toml
* Back from the future (updated_date)

Co-authored-by: David French <56409778+threat-punter@users.noreply.github.com>

(cherry picked from commit 3b338baab0)
2021-08-25 15:58:03 +00:00
Ross Wolf 8a3220ef6a Track multiple stacks in lock (#1434)
* Save the stack versions in the lock file
* Support tracking of multiple stacks in the lock
* Update the version locking logic
* Fix bugs and test lock file
* Restore version lock
* Fix lint errors
* Call both click.echo and verbose echo separately
* Change when the change_rules message is output

(cherry picked from commit 0d47cb324a)
2021-08-24 22:57:14 +00:00
dstepanic17 689e690f8c [New rule] Webshell Detection (#1448)
* [new-rule] Webshell Detection

* Update rules/windows/persistence_webshell_detection.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Added FP note section

* Update rules/windows/persistence_webshell_detection.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

(cherry picked from commit 8ddffc298b)
2021-08-24 20:19:32 +00:00
Justin Ibarra cc75f645b6 [Rule Tuning] Add technique T1005 to 2 rules (#1405)
(cherry picked from commit 8099e1c733)
2021-08-20 08:20:32 +00:00
Ross Wolf 632a322431 Fix encoding of 'Any' type in jsonschema (#1438)
(cherry picked from commit 11c443ba26)
2021-08-19 16:16:40 +00:00
Justin Ibarra 60caedc026 Bump package versions (#1418)
* Bump package versions
* Add 7.14 migration; use master schema map if one does not exist
* add test to ensure an entry exists in the stack-schema-map for the current package version

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>

Removed changes from:
- etc/packages.yml

(selectively cherry picked from commit 2d517432e3)
2021-08-19 05:26:47 +00:00
Ross Wolf c1b774cdb6 Skip etc/packages.yml from backport: auto (#1437)
(cherry picked from commit d647c7b809)
2021-08-18 22:57:34 +00:00
Austin Songer 94190321c1 [Rule Tuning] AWS Security Group Configuration Change Detection (#1426)
* move rule "AWS Security Group Configuration Change Detection" to integrations directory and add "aws" integration

(cherry picked from commit 3b29498907)
2021-08-15 04:35:07 +00:00
Christian Clauss 604fd2a18f Fix typos discovered by codespell (#1430)
(cherry picked from commit ddec37b731)
2021-08-15 04:30:11 +00:00
Justin Ibarra 16bc2a24f1 Remove labeling from community workflow (#1432)
(cherry picked from commit 4a3bacae48)
2021-08-14 10:44:37 +00:00
Justin Ibarra 52dee0d0c6 Add revised workflow for community label (#1431)
(cherry picked from commit f63a72f1ac)
2021-08-14 10:19:55 +00:00
Justin Ibarra 986a515a62 Add label workflow for community issues and pulls (#1406)
* Add label workflow for community issues and pulls
* run on label changes

(cherry picked from commit 006cb0e702)
2021-08-14 06:37:59 +00:00
Justin Ibarra 4bd62ef5c9 Add botelastic workflow for stale issues and PRs (#1414)
(cherry picked from commit 5c8029ad55)
2021-08-14 06:25:51 +00:00
Justin Ibarra 764cb5d0b4 Add paths-labeller workflow (#1407)
* add botelastic workflow

(cherry picked from commit 75d6d76926)
2021-08-14 06:14:32 +00:00
Justin Ibarra c2b7b22496 Pull latest ECS+beats schemas and update schema-map (#1417)
(cherry picked from commit b27a20fc3a)
2021-08-12 21:10:22 +00:00
Austin Songer e170935f1f [New Rule] AWS EC2 Security Group Configuration Change Detection (#1144)
(cherry picked from commit 67ba66c8e7)
2021-08-12 19:38:05 +00:00
David French 9e6c107de5 [New Rule] Whitespace Padding in Process Command Line (#1392)
* Create defense_evasion_whitespace_padding_in_command_line.toml

* add newline

* update description

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

(cherry picked from commit 14493689b9)
2021-08-11 16:16:05 +00:00
Justin Ibarra dca8f2b712 [Bug] Flatten method improperly added subtechniques (#1404)
(cherry picked from commit 95486ecfdf)
2021-08-05 19:17:17 +00:00
Ross Wolf 5a33f634a7 Add RuleCollection.load_git_branch (#1403)
(cherry picked from commit 17bf3c1e16)
2021-08-05 07:16:38 +00:00
dishadasgupta 91e1d1abfc Adding docs for URL Spoofing (#1400)
* Adding docs for urlspoof

* Fixing typo in readme

* Editing documentation to reflect rule upload process

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

(cherry picked from commit 7be58b7b09)
2021-08-05 00:14:12 +00:00
Justin Ibarra 121431b40b Refresh ATT&CK mappings to v9.0 (#1401)
* Refresh ATT&CK mappings to v9.0
* Update rules to reflect ATT&CK changes

(cherry picked from commit d31ea6253e)
2021-08-04 22:17:11 +00:00
Justin Ibarra 742253c61d [Rule tuning] Revise rule description and other text (#1398)
(cherry picked from commit f8f643041a)
2021-08-03 21:08:48 +00:00
Austin Songer fcd2071ca9 [Rule Tuning] NTDS or SAM Database File Copied (#1378)
* Update credential_access_copy_ntds_sam_volshadowcp_cmdline.toml to include esentutl.exe

Co-authored-by: David French <56409778+threat-punter@users.noreply.github.com>

(cherry picked from commit d2365783fa)
2021-08-03 20:29:19 +00:00
Apoorva Joshi 99c9995967 Update Host Risk Score docs (#1397)
(cherry picked from commit 06a9ba6463)
2021-08-03 04:53:06 +00:00
Apoorva Joshi 197bb86459 Adding host risk score docs (#1390)
* Adding host risk score docs
* Highlighting caveats around hostname
* Update host-risk-score.md
* Adding host risk score to the experimental detections readme

(cherry picked from commit c283d2a2f3)
2021-08-02 21:44:26 +00:00
Justin Ibarra 05d01bbfe0 [Rule Tuning] Rule description tweaks (#1388)
(cherry picked from commit b736d6e748)
2021-07-29 18:57:11 +00:00
Ross Wolf 06849a82d8 [CI] Add missing clone for Fleet on-demand job (#1387)
(cherry picked from commit 2e8f7cd13f)
2021-07-27 22:56:37 +00:00
Ross Wolf f6d9295ead [CI] Fix kibana PR command again (#1386)
(cherry picked from commit 92937a1ad1)
2021-07-27 22:30:54 +00:00
Ross Wolf 51f8ea7526 Fix kibana_pr for click.Context (#1385)
(cherry picked from commit 64977b01bd)
2021-07-27 22:04:31 +00:00