* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
* add support for self-signed certs in es and kibana
* allow Kibana to auth against any providerType
* fix export-rules command
* fix kibana upload-rule command
* fix view-rule command
* fix validate-rule command
* fix search-rules command
* fix dev kibana-diff command
* fix dev package-stats command
* fix dev search-rule-prs command
* fix dev deprecate-rule command
* replace toml with pytoml to fix import-rules command
* use no_verify in get_kibana_client
* use Path for rule-file type in view-rule
* update schemas to resolve additionalProperties type bug
* fix missing unique_fields in package rule filter
* fix github pr loader
* Load gh rules as TOMLRule instead of dict
* remove unnecessary version insertion
* create new cli commands
* add kibana object to create_dnstwist_rule
* Adding code for index-dnstwist-results
* Changed es to es_client
* Tested. it works!
* flake8-ed
* Adding timestamps
* use eql.utils.load_dump to load json file
* rename data to dnstwist_data
* start working on create-dnstwist-rule command
* add print statements for user
* tweak formatting for line length
* add template threat match rule file
* continue working on threat match rule creation
* create rule using TomlRuleContents
* save rule to toml file
* Moving rule creation to eswrap.py
* Moving create dnstwist rule stuff to eswrap
* Fixed imports
* flake8 fixes
* More flake8 fixes
* fix usage of @add_client('kibana')
* use ctx.invoke to upload rule
* cleanup record assembly and use bulk api
* swap order of notes in `note` for sample rule
* small modifications
* move command to root click group
* remove unused click group
* Update detection_rules/main.py
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
* remove rule upload and convert template to ndjson
* Adding docs for typosquatting rule
* renaming the file
* Adding a note
* separate index and rule prep commands
* Final changes
Co-authored-by: Apoorva <appujo@gmail.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Apoorva Joshi <30438249+ajosh0504@users.noreply.github.com>
* Add DeprecatedCollection to RuleCollection to bypass validation
* use DeprecatedRule properties in RuleCollection
* use RuleCollection filter for max/min filtering in Package
* Port historical schemas to jsonschema
* Add marshmallow-json dependency
* Mark etc/api_schemas as binary
* Remove gitattributes attempt
* Lint fix
* Apply PR feedback
* Additional PR feedback
* Extract stack version from packages.yml
* Fix the backport schemas
* Cache the schema reads
* Add migration for #1167
* Make a separate 'migration not found' error
* Add a RuleCollection object instead of a "loader" module
* Remove legacy loader code
* Remove more legacy loader
* Freeze the default collection
* Change RULE_LOADER default
* Rename to _toml_load_cache
* Use rglob magic
* Typo should've been a string
* Remove no longer needed glob import
* Fix pycharm import bad ordering
* Restore the detection_rules/schemas imports
* Put more imports back for a smaller diff
* Check cache in _deserialize_toml
* Add multi collection and single collection decorators
* Reorder RuleCollection methods
* Move filter method up
* WIP: Convert Rule to a dataclass
* Fix make release
* Lint fixes
* Remove dead code
* Fix lint and tests
* Use Python 3.8 in GitHub actions
* Update README to 3.8+
* Add Python 3.8 assertion
* Fix is_dirty property
* Remove incorrect pop from contents
* Add mixin with from_dict() and to_dict() methods
* Bypass validation for deprecated rules
* Fix rule_prompt
* Fix dict_hash usage
* Fix rule_event_search
* Switch to definitions.Date
* Fix toml-lint command, ignoring 'unneeded defaults'
* Moved severity Literal to definitions.Severity
* Remove BaseMarshmallowDataclass
* Fix lint and tests
* Add maturity to metadata for rule prompt loop
* Fix typo in devtools
* Use rule loader to load single rule in toml-lint
* Add Schema hint to __schema method
* Add MITREAttackURL definition
* Fix is_dirty to compare sha<-->sha
* Normalize the autoformatted rule output for API and toml-lint
* Make the package hash match
* Make the rule object mutable but not rule contents
* Restore the rules
* Add commands to generate index rules files or index them directly to elasticsearch
* files generated at package creation
* add readme explaining these index files
* add load_gh_pr_rules function
* add dev package-stats command
* add dev search-rule-prs command, which extends the same functionality in rule-search to rules in PR
* [Rule Tuning] Add empty arrays in place of tactic only threat mappings
* dynamically insert empty technique array in payload
* use replace_id as function parameter
* Restructure commands under more specific click groups
* standardize CLI error handling
* add global debug options
* move es and kibana clients into their click groups
* move commands and groups to dedicated files
* distinguish variable names for better env/config parsing
Terrance DeJesus