Commit Graph

636 Commits

Author SHA1 Message Date
Justin Ibarra 0a992d716a [Rule Tuning] Update EQL rules for 7.10 (#399)
* update syntax to reflect eql changes
* use more case-insensitivity
* comment out missing fields for winlogbeat compatibility
2020-10-21 12:35:18 -08:00
Stijn Holzhauer 60b3d47efd Add kibana-upload --space option (#251)
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-10-08 12:21:54 -06:00
Justin Ibarra bd680a2bd4 Re-organize commands under more specific click groups (#356)
* Restructure commands under more specific click groups
* standardize CLI error handling
* add global debug options
* move es and kibana clients into their click groups
* move commands and groups to dedicated files 
* distinguish variable names for better env/config parsing
2020-10-07 12:15:33 -08:00
Justin Ibarra bf202b6b6c [New Rule] Initial converted EQL rules (#304)
* 18 converted eql rules (not all prod)
2020-09-30 21:40:55 -08:00
Justin Ibarra 7c1e9c1ed5 Update package summary extras produced during package generation (#341)
* update summary.txt
* add summary.xlsx
* add changelog entry autogeneration
2020-09-30 14:43:45 -08:00
Justin Ibarra a212008f8c [Rule Tuning] Remove event.module from rules for compatibility with agent integrations (#342) 2020-09-30 09:41:33 -08:00
shravaka fa12340ff0 [Bug fix] Add missing parenthesis for -kibana-url 2020-09-30 09:32:43 -06:00
Justin Ibarra 065bcd8018 Refresh ATT&CK data to v7.2 and expand threat validation (#330)
* refresh to latest ATT&CK 7.2
* add new unit test to further validate threat mappings
* updated threat mappings in rules to reflect changes
* new func to download and refresh mitre data based on version
2020-09-23 22:03:29 -08:00
Justin Ibarra 6ad3344af3 Collect unique query fields per rule (#296) 2020-09-23 14:36:34 -08:00
Ross Wolf 453553f685 Change the way we get environment variables (#280)
* Change the way we get environment variables
* Change environ to getenv
* Read from envvar, then config file
* Switch to get_path
* Lint: Remove unused import
* Add --cloud-id/--elasticsearch-url
* Fix comment copy-pasta
2020-09-16 10:23:22 -06:00
Ross Wolf 9d22970e21 Add EQL rules and schema validation (#297)
* Add EQL rules and schema validation
* Lint nitpick
* Rename get_schema_from_eql
* Add EQL default language
* Rename parsed_kql to parsed_query
* Fix parsed_kql method call in loader
* Autopopulate dependent values
2020-09-16 08:36:48 -06:00
Justin Ibarra 6b7ea7e66c Fix kibana-diff command (#198) 2020-09-02 12:19:17 -05:00
Ross Wolf 464d5e645a Fix kibana-upload and remove cumbersome dataclasses (#216)
* Fix kibana-upload and remove cumbersom dataclasses

* Linting fixes
2020-09-01 05:47:27 -06:00
Justin Ibarra 79a0dfefbe Add ECS 1.6.0 schema for validation testing (#220)
* Add ecs 1.6.0 and refresh master ecs (2.0.0)
* update rule metadata to use ecs_version 1.6.0
2020-08-27 11:54:49 -05:00
Justin Ibarra 28c869fb5f Expand documentation on CLI and workflows (#130) 2020-08-18 14:27:51 -05:00
Ross Wolf cb1c401e27 Merge branch '7.9' into main 2020-08-03 15:20:36 -06:00
Brent Murphy 01b1e8be26 [Rule Tuning] Update Tags for Cloud Rules (#99)
* [Rule Tuning] Update Tags for Cloud Rules

* commenting out specifying alphabetical tag order in rule formatter

* Update rule_formatter.py

* py lint

* Lint fix comments

* update modified dates

* Update credential_access_secretsmanager_getsecretvalue.toml

* adding Continuous Monitoring tag

* update tags

* fixed and in tags

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-08-03 17:15:15 -04:00
Ross Wolf a99b7c96fe Merge branch '7.9' into main 2020-08-03 14:03:15 -06:00
Ross Wolf 0455307577 Downgrade rule version before uploading to Kibana (#97)
* Downgrade version before uploading to Kibana
* Update downgrade exception format
* Update s/siem/detection

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-07-28 11:03:47 -06:00
Justin Ibarra 8f5ddbb121 Add better CLI support for handling Kibana exported rules (#83) 2020-07-27 23:31:19 -05:00
Ross Wolf d15da0ada1 Add versioned schemas with a downgrade path (#84)
* Add versioned schemas with a downgrade path
* Remove and move unused variables
* Add missing license
* Skip NotField for output_index
* Add strip_additional_properties for kibana import
* Remove stray comment
* Apply suggestions from code review

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-07-23 11:39:35 -06:00
Ross Wolf 978a8d9df8 [Bug] Set threshold.field to empty string instead of null (#87) 2020-07-22 19:31:09 -04:00
Ross Wolf 16fb306254 Add command to upload to kibana (#58)
* Add upload command to kibana
* Restore skipped fields
* Change prefix to DR_
* Add note to manage_versions call
* Reorder requirements.txt to trigger build
2020-07-20 15:58:28 -06:00
Justin Ibarra 1cfb8f92bb Windows DNS server vulnerability (CVE-2020-1350) rules (#69) 2020-07-17 14:32:52 -05:00
Justin Ibarra 7647699e2b Add support for threshold rules (#65) 2020-07-16 19:06:34 -05:00
Justin Ibarra 916917a619 Update rule.py 2020-07-15 09:40:07 -05:00
Ross Wolf db4f50d4b8 Improve the validation and testing time (#61)
* Improve the validation and testing time
* Lint fix
* Cache schema validation
2020-07-15 08:05:55 -06:00
Ross Wolf e96eabaa2e Generate linted .ts in package (#49)
* Generate linted .ts in package
* (Lin|ni)t changes

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-07-09 17:33:28 -06:00
Garrett Spong c28795c25e [New Rule] Elastic Endpoint and External Alerts (#42)
* Adds the Elastic Endpoint and External Alerts rules and required schema updates
* Optimizing queries to fix tests
* Apply PEP257 changes
* Apply suggestions from code review
* Update rules/cross-platform/external_alerts.toml
* Last fixes from review
* Fixing test for unrequired default
* Adding increased default max_signals to not interfere with testing
* Make promotions folder
* Refining Elastic Endpoint rule index

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-07-09 15:24:36 -06:00
Ross Wolf 8a561b3817 Add kibana-push command (#38)
* Add kibana-push command
* Add ctx.exit instead of return
* Make the base branch configurable
2020-07-08 18:02:12 -06:00
Justin Ibarra 119c98f05f Package kibana index file with release rules (#40) 2020-07-08 18:58:00 -05:00
Andrew Pease e0f2e8b4a9 Add dataset and index to network rules (#15)
* Add dataset and index to network rules
* Restore iptables changes
* Fix beats parsing logic
* Updated date and ECS version
* Only update modules if empty

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-07-08 13:19:35 -06:00
Justin Ibarra 29a92f8976 Package notice file with release (#32) 2020-07-08 13:17:42 -05:00
Ross Wolf e2d97b0a74 Remove unreachable and legacy code
Co-Authored-By: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-06-30 10:12:23 -06:00
Ross Wolf 0ddb8ee798 Switch to click.echo() for the banner 2020-06-29 23:58:20 -06:00
Ross Wolf 3b305d3003 Add rule loader and dependencies
Co-Authored-By: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-06-29 23:17:42 -06:00