* adjusting inheritance of ESQL rule data
* update tests to handle missing index from QueryRuleData
* removed test es|ql rule
---------
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
* Adding support for elastic package version 3
* replaced OS with Pathlib where applicable
* added sub-dataclasses for V3
* fixed flake errors
* adjusted registry dataclasses to inherit base
* adding solution for historical rules in release package
* addressing flake errors
* format changes
* REVERT CHANGES - testing release-fleet workflow
* REVERTING CHANGES
* added historical flag for packaging to account for older branches
* addressing flake errors
* updated build for CI
* REMOVE: This is temporary to run a workflow from this branch
* updates to address requirements for contents
* reverting packages.yml
* Update detection_rules/integrations.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* Update detection_rules/integrations.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* addressed feedback and added click echo comments
* addressed flake errors and added some comments
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* Generate attack layer files and build with package
* add update-navigator-gists command
* add workflow to update navigator gists on pushes to main
* Add coverage readme
* fix keys for links
* update navigator layer names
* purge gist files prior to update; add badge
* Update how the navigator links are displayed
* moved navigator code to dedicated and refactored to dataclasses
* convert gist links to permalink versions
* alphabetize; catch 404 for gist update
* add support for self-signed certs in es and kibana
* allow Kibana to auth against any providerType
* fix export-rules command
* fix kibana upload-rule command
* fix view-rule command
* fix validate-rule command
* fix search-rules command
* fix dev kibana-diff command
* fix dev package-stats command
* fix dev search-rule-prs command
* fix dev deprecate-rule command
* replace toml with pytoml to fix import-rules command
* use no_verify in get_kibana_client
* use Path for rule-file type in view-rule
* update schemas to resolve additionalProperties type bug
* fix missing unique_fields in package rule filter
* fix github pr loader
* Load gh rules as TOMLRule instead of dict
* remove unnecessary version insertion
* Add DeprecatedCollection to RuleCollection to bypass validation
* use DeprecatedRule properties in RuleCollection
* use RuleCollection filter for max/min filtering in Package
* Save the stack versions in the lock file
* Support tracking of multiple stacks in the lock
* Update the version locking logic
* Fix bugs and test lock file
* Restore version lock
* Fix lint errors
* Call both click.echo and verbose echo separately
* Change when the change_rules message is output
* Add command to publish integrations PR
* Add workflow_dispatch job to publish package
* Get working directory dynamically
* Fix the repo settings
* Get the absolute path for local-repo
* Filter out 'main' branch
* Update the description for target_branch
* Fix workflow definition
* Move 'if' into job
* Update ref format
* Remove unnecessary E501 suppression
* Add a link to the full commit hash
* s/partial_args/prefix_args
* Add min_stack_version to metadata of rule structure
* validate all "stack versions" between defined and current package
* Use master schemas if min_stack_version > current_package
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
* Update version lock overwrite command
* Fix tooling and restore old version lock
* Lint fix
* Fix tests
* Remove dead code
* Filter to prod+deprecated rules
* Cast set -> list
* Store deprecation info
* Add correct version.lock.json (finally)
* Fix "stack_version" typo
* Remove stack_version
* Back out main.py changes
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
* Port historical schemas to jsonschema
* Add marshmallow-json dependency
* Mark etc/api_schemas as binary
* Remove gitattributes attempt
* Lint fix
* Apply PR feedback
* Additional PR feedback
* Extract stack version from packages.yml
* Fix the backport schemas
* Cache the schema reads
* Add migration for #1167
* Make a separate 'migration not found' error
* Add a RuleCollection object instead of a "loader" module
* Remove legacy loader code
* Remove more legacy loader
* Freeze the default collection
* Change RULE_LOADER default
* Rename to _toml_load_cache
* Use rglob magic
* Typo should've been a string
* Remove no longer needed glob import
* Fix pycharm import bad ordering
* Restore the detection_rules/schemas imports
* Put more imports back for a smaller diff
* Check cache in _deserialize_toml
* Add multi collection and single collection decorators
* Reorder RuleCollection methods
* Move filter method up
* WIP: Convert Rule to a dataclass
* Fix make release
* Lint fixes
* Remove dead code
* Fix lint and tests
* Use Python 3.8 in GitHub actions
* Update README to 3.8+
* Add Python 3.8 assertion
* Fix is_dirty property
* Remove incorrect pop from contents
* Add mixin with from_dict() and to_dict() methods
* Bypass validation for deprecated rules
* Fix rule_prompt
* Fix dict_hash usage
* Fix rule_event_search
* Switch to definitions.Date
* Fix toml-lint command, ignoring 'unneeded defaults'
* Moved severity Literal to definitions.Severity
* Remove BaseMarshmallowDataclass
* Fix lint and tests
* Add maturity to metadata for rule prompt loop
* Fix typo in devtools
* Use rule loader to load single rule in toml-lint
* Add Schema hint to __schema method
* Add MITREAttackURL definition
* Fix is_dirty to compare sha<-->sha
* Normalize the autoformatted rule output for API and toml-lint
* Make the package hash match
* Make the rule object mutable but not rule contents
* Restore the rules
* Add commands to generate index rules files or index them directly to elasticsearch
* files generated at package creation
* add readme explaining these index files
* add load_gh_pr_rules function
* add dev package-stats command
* add dev search-rule-prs command, which extends the same functionality in rule-search to rules in PR
* [Rule Tuning] Add empty arrays in place of tactic only threat mappings
* dynamically insert empty technique array in payload
* use replace_id as function parameter
* Tweak Rules for 7.10
* Add endpoint index for packetbeat rules
* update unit test to account for Network tag as well
* update modified date, add endpoint tag
* use Host instead of Endpoint
* Update packaging.py
* add v back to changelog url
* Add "tag" comment to get_markdown_rule_info
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Terrance DeJesus