* Drop Rule Support for Outdated Stack Versions Less Than 8.3
* changed version lock key assignment logic and updated version lock file
* added comment to stack-schema-map file
* changed version lock key assignment logic to use custom Version method)
* Update detection_rules/devtools.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* reverting version lock file to original
* updated version lock from adjusted comparison logic of stack versions
* updated logic in devtools; removed < 8.3.0 in version lock file
* trimmed lock version before merge
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* adding solution for historical rules in release package
* addressing flake errors
* format changes
* REVERT CHANGES - testing release-fleet workflow
* REVERTING CHANGES
* added historical flag for packaging to account for older branches
* addressing flake errors
* updated build for CI
* REMOVE: This is temporary to run a workflow from this branch
* updates to address requirements for contents
* reverting packages.yml
* Update detection_rules/integrations.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* Update detection_rules/integrations.py
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* addressed feedback and added click echo comments
* addressed flake errors and added some comments
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* initial changes to release fleet workflow and CLI
* changed the default value of package version for 8.8
* changed how true/false is passed into CLI command
* reverted changes to packages.yml
* testing order of operations in workflow
* reverted testing order; adjusting secrets token
* adjusting secrets token
* changing checkout to v3
* removed token for testing workflow
* changed repo reference
* changing secret token
* reverting token changes
* removing master reference
* adjusted elastic-package installation
* changed path of integrations during install
* added integrations fetch run commands
* changed target branch to main, setup latest go
* changed token back to protections machine
* trying different secret for integrations PR creation
* created testing token for permission errors
* adjusted 'bump-pkg-versions' so minors are bumped if no previous pkg
* added bumping package versions as a step
* updated actions/upload-artifact to v3
* removed inaccurate comments; removed release-kibana workflow
* adjusted sequence of steps to bump packge version before build
* added a bump to major if it does not match packages.yml
* removed custom semver and replaced with pypi
* updated beats.py version references
* updated bump-versions CLI command to use semver and change logic
* updated schemas __init__, test_version_lock and unstage incompatible rules CLI
* updated test_stack_schema_map in TestVersions unittest
* updated test_all_rules unit testing Version() references
* updated stack_compat.py for get_restricted_field references)
* updated version_lock.py Version() references
* updated docs.py Version() reference for parse_registry
* updated devtools.py Version() reference for trim-version-lock
* updated mixins.py Version() reference in validate_field_compatibility
* adjusted schemas.__init__ Version() reference in get_stack_schemas
* adjusted ecs.py Version() references
* adjusted integrations.py Version() references
* adjusted rule.py Version() references
* sorted imports
* replaced custom semver with pypi semver in unit test files
* addressed unit test and flake errors
* changed semver strings casted to version_lock.py
* fixed sorting in integrations.py
* updated bump-pkgs-versions CLI command
* adjusted semantic version in unstage-incompatible-rules command
* adjusted semver import to VersionInfo
* added semver 3 and adjusted import names
* added option_minor_and_patch parameter where version is major.minor
* updated bump-pkg-versions to always save to packages.yml
* removed leftover split call & updated find latest compatible version command
* updated integrations.py, version_lock.py and schemas.__init__.py
* changed fstring reference in downgrade function
* reverted formatting changes for detection_rules __init__.py
* added newline to detection_rules __init__.py
* adjusted finding latest_release for attack package logic
* adjusted unstage-incompatible-rules command logic comparing versions
* removing changes from misc.py related to auto-formatting
* adding newline to misc.py
* fixed bug in downgrade function calling decorators
* added semantic version validation on migrate decorator function
* added expected type returned from find_latest_integration_version in integrations.py
* add comment about stripped versions for version lock file
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
---------
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* Add test command to verify version collisions do not occur
* add max_allowable_version to schema and lock flow
* add max_allowable_version to all entries in version.lock
* add test-version-lock command
* use min supported stack if > locked min stack
* share lock conversion code with rule and lock to fix M.m bug
* Cleanup rule survey code
* default to only unique-ing on process name for lucene rules
* fix bug in kibana url parsing by removing redundant port from domain
* update search-alerts columns and nest fields
* fix rule.contents.data.index
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
* Generate attack layer files and build with package
* add update-navigator-gists command
* add workflow to update navigator gists on pushes to main
* Add coverage readme
* fix keys for links
* update navigator layer names
* purge gist files prior to update; add badge
* Update how the navigator links are displayed
* moved navigator code to dedicated and refactored to dataclasses
* convert gist links to permalink versions
* alphabetize; catch 404 for gist update
* Ensure github module is installed before running PR commands
* move go and elastic-package assertions to top of command
* update error msg for missing pkg
* remove redundant github assertion
Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
* add support for self-signed certs in es and kibana
* allow Kibana to auth against any providerType
* fix export-rules command
* fix kibana upload-rule command
* fix view-rule command
* fix validate-rule command
* fix search-rules command
* fix dev kibana-diff command
* fix dev package-stats command
* fix dev search-rule-prs command
* fix dev deprecate-rule command
* replace toml with pytoml to fix import-rules command
* use no_verify in get_kibana_client
* use Path for rule-file type in view-rule
* update schemas to resolve additionalProperties type bug
* fix missing unique_fields in package rule filter
* fix github pr loader
* Load gh rules as TOMLRule instead of dict
* remove unnecessary version insertion
* Add DeprecatedCollection to RuleCollection to bypass validation
* use DeprecatedRule properties in RuleCollection
* use RuleCollection filter for max/min filtering in Package
* Save the stack versions in the lock file
* Support tracking of multiple stacks in the lock
* Update the version locking logic
* Fix bugs and test lock file
* Restore version lock
* Fix lint errors
* Call both click.echo and verbose echo separately
* Change when the change_rules message is output
* Add command to publish integrations PR
* Add workflow_dispatch job to publish package
* Get working directory dynamically
* Fix the repo settings
* Get the absolute path for local-repo
* Filter out 'main' branch
* Update the description for target_branch
* Fix workflow definition
* Move 'if' into job
* Update ref format
* Remove unnecessary E501 suppression
* Add a link to the full commit hash
* s/partial_args/prefix_args
* Update version lock overwrite command
* Fix tooling and restore old version lock
* Lint fix
* Fix tests
* Remove dead code
* Filter to prod+deprecated rules
* Cast set -> list
* Store deprecation info
* Add correct version.lock.json (finally)
* Fix "stack_version" typo
* Remove stack_version
* Back out main.py changes
Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
* move github and ml to their own files
* refactor release and ml commands
* update ML readmes
* add unzip_to_dict function
* prompt for model ID in remove-model
* update experimental rule upload process
* update remove-scripts-pipelines to take multiple options
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Co-authored-by: Apoorva <appujo@gmail.com>
* Port historical schemas to jsonschema
* Add marshmallow-json dependency
* Mark etc/api_schemas as binary
* Remove gitattributes attempt
* Lint fix
* Apply PR feedback
* Additional PR feedback
* Extract stack version from packages.yml
* Fix the backport schemas
* Cache the schema reads
* Add migration for #1167
* Make a separate 'migration not found' error
Terrance DeJesus