Commit Graph

94 Commits

Author SHA1 Message Date
Sergey Polzunov 3bdda091e1 chore: use docs-dev instead of docs dir for docs (#4522)
* chore: use `docs-dev` instead of `docs` folder

* patch version bump

* Rollback an incorrect rename

* Use exact docs dir in the helper comment

* Revert some overeager renamings

* Moving `docs` to `docs-dev`

* Update Docs Paths

---------

Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
2025-03-07 14:34:51 +01:00
Sergey Polzunov a650b028f3 Bumping number of versions per rule to 4 in total (#4451)
* Bumping number of versions per rule to 4 in total

* Add explicit caps

* Simpler comment

* Renaming constants

* Drop to 8.17 again

* Clearer constants

* Drop if condition and extend the comment

* Shorten the lines

* Version bump

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2025-02-07 16:28:36 +01:00
Terrance DeJesus ad180777cf [Maintenance] Repository Config Update (#4359)
* updating tokens

* bumped patch

* updated navigator gist ID

* updated naming

* Update .github/workflows/manual-backport.yml

* updated navigator url

* updated noreply email

* updated naming

* Update .github/workflows/manual-backport.yml

Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>

* updating README

* updated gist token

* replaced guidelines token with GITHUB_TOKEN

---------

Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
2025-01-09 16:35:18 -05:00
shashank-elastic d3c05a08cc Add all historical versions for v8.17.0 and above packages (#4279) 2024-12-03 23:36:32 +05:30
shashank-elastic c2e0a9315c Fix extra new line in ATT&CK-coverage.md (#4263) 2024-11-08 20:13:21 +05:30
shashank-elastic e2f1fcefa8 Add flag to update the docs/ATT&CK-coverage.md with markdown URL(s) (#4077) 2024-09-19 23:12:01 +05:30
shashank-elastic 0c25cfb82e Remove unused @click.pass_context (#3996) 2024-08-20 23:11:22 +05:30
Eric Forte 47d7a3acaa [DaC] Beta Release (#3889)
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
2024-08-06 18:07:12 -04:00
Mika Ayenson 357204e1c5 [FR] Limit historical rules to the latest 2 (#3842) 2024-06-28 06:42:10 -05:00
Eric Forte f43fbfba0d [FR] Update utility path computation to use pathlib (#3699)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object

* Fix typo
2024-05-23 17:36:51 -04:00
shashank-elastic 63e91c2f12 Back-porting Version Trimming (#3704) 2024-05-23 00:45:10 +05:30
Mika Ayenson 2c3dbfc039 Revert "Back-porting Version Trimming (#3681)"
This reverts commit 71d2c59b5c.
2024-05-22 13:51:46 -05:00
shashank-elastic 71d2c59b5c Back-porting Version Trimming (#3681) 2024-05-23 00:11:50 +05:30
Mika Ayenson 371e24b2ed Revert "[FR] Update Utility Path Computation to use Pathlib (#3659)"
This reverts commit 23567c1d0c.
2024-05-21 16:14:45 -05:00
Eric Forte 23567c1d0c [FR] Update Utility Path Computation to use Pathlib (#3659)
* update

* Updated to pathlib

* Linting

* Add string cast where needed

* Add additional string conversion as needed

* Str conversions to support eql lib

* Attack typo

* Typo in test script

* Updated for more pathlib

* Linting

* Update to convert string to path object
2024-05-21 14:19:20 -04:00
Jonhnathan d023ad66b1 [Rule Tuning] Add Initial SentinelOne Compatibility to Windows DRs (#3627)
* [Rule Tuning] Add Initial SentinelOne Compatibility

* updated definitions.py; updated tags; fixed unit tests

* added prerelease versions for s1 integration; updated build CLI commands to allow prerelease; bumped min-stacks

* updating manifests and integrations

* fixing flake errors

* min_stack

---------

Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Samirbous <64742097+Samirbous@users.noreply.github.com>
2024-05-20 09:50:57 -03:00
Eric Forte 707ca32ab1 [FR] Add --force flag to update-lock-versions (#3693)
* Add --force flag to update-lock-versions

* Add type hinting
2024-05-17 20:25:08 -04:00
Mika Ayenson 79f575b33c [FR] Normalize yml ext to yaml (#3675) 2024-05-15 15:18:39 -05:00
shashank-elastic 3fbffa24ed Deprecate Releasing to a patch kibana version workflow (#3552) 2024-04-03 08:34:45 +05:30
shashank-elastic 8b215eac41 Fix create PR in release workflow (#3528) 2024-04-01 21:17:10 +05:30
Mika Ayenson d26981f712 [FR] Independently package kql / kibana and bump to py3.12 (#3514) 2024-03-14 20:18:32 -05:00
Mika Ayenson 3d2a36be32 Revert "[FR] Independently package kql / kibana and bump to py3.12 (#3492)"
This reverts commit fc139fc3c2.
2024-03-14 19:48:50 -05:00
Mika Ayenson fc139fc3c2 [FR] Independently package kql / kibana and bump to py3.12 (#3492) 2024-03-14 19:14:25 -05:00
Terrance DeJesus 06b97ec79b [Bug] Adjust build-release CLI and fix links when generating security docs (#3434)
* removed historical argument; added setup string; fixed links

* fixing flake errors

* added types for command arguments

* adjusted get_release_diff to append strings for release tags

* set fetch-depth to 0 for integrations checkout in workflow

* changed the name of the workflow

* removed TODOs

* adjusted release docs workflow to remove prefix for release tags

* adjusted URL replacement only if pointed to docs site

* added elastic website to regex pattern

* add docstrings; adjusted regex; add note for stopgap

* added a note about the regex pattern for elastic URLs
2024-02-12 10:08:06 -05:00
Terrance DeJesus 7df7ab5101 [Bug] Update Prebuilt Detection Rules Release Process (#3403)
* release fleet workflow updates; build package integration reference changes

* updated commit hash extraction to output to env

* adjusted bump-pkg-versions to only include release if necessary

* fixed flake errors

* add historical argument for build-release set to yes by default

* Update detection_rules/devtools.py

* fixed fleet workflow; updated registry data references

* updated job names

* removed extract commit hash job and consolidated into fleet pr job

* added echo statement for current branch before checkout

* removed id from extract commit hash
2024-02-06 08:59:06 -05:00
Terrance DeJesus cdeb398ab3 [FR] Adjust Prebuilt Rules Packaging to Use Elastic Package v3 (#3252)
* Adding support for elastic package version 3

* replaced OS with Pathlib where applicable

* added sub-dataclasses for V3

* fixed flake errors

* adjusted registry dataclasses to inherit base
2023-11-01 12:47:40 -04:00
Apoorva Joshi 747ee7d593 [New Rule] Adding Lateral Movement Rules from Advanced Analytic LMD Package (#3119)
* Adding Lateral Movement Detection rules

* added tags; adjusted tests; updated manifests and schemas

* added default value to build_integrations_schema

* combined analytic and non-dataset packages for related integrations

* adjusted machine learning definitions

* adjusted machine learning definitions

* removed splat for machine learning list due to 3.8 constraints

---------

Co-authored-by: terrancedejesus <terrance.dejesus@elastic.co>
Co-authored-by: Terrance DeJesus <99630311+terrancedejesus@users.noreply.github.com>
2023-09-27 14:53:38 -04:00
Terrance DeJesus 73970eb2f2 [FR] Add Support for Multi-Fields and Validation in Rules (#2882) 2023-06-28 20:35:33 -04:00
Terrance DeJesus 8db42da040 Limit backports to 8.3+ (#2450)
* Drop Rule Support for Outdated Stack Versions Less Than 8.3

* changed version lock key assignment logic and updated version lock file

* added comment to stack-schema-map file

* changed version lock key assignment logic to use custom Version method)

* Update detection_rules/devtools.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* reverting version lock file to original

* updated version lock from adjusted comparison logic of stack versions

* updated logic in devtools; removed < 8.3.0 in version lock file

* trimmed lock version before merge

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-06-12 12:51:40 -04:00
Terrance DeJesus e0ceb5a434 adjust integrations file; add option for single integration update (#2816) 2023-05-31 11:00:58 -04:00
Mika Ayenson 81bef59236 [FR] Generate mdx docs (#2718) 2023-05-03 16:27:30 -04:00
Mika Ayenson 6ecd65721d [FR] Add release-docs workflow and automation (#2745) 2023-04-27 11:44:05 -04:00
Terrance DeJesus 597e6e2de1 [Bug] Add --add-historical argument to lock versions workflow (#2739)
* bug fix for lock version workflow

* updated all use cases with build-release

* added default to add historical

* fixed flake errors
2023-04-24 12:12:49 -04:00
Terrance DeJesus b5ef2f5f02 [FR] Generate Historical Rule Files in Build Release Packages (#2715)
* adding solution for historical rules in release package

* addressing flake errors

* format changes

* REVERT CHANGES - testing release-fleet workflow

* REVERTING CHANGES

* added historical flag for packaging to account for older branches

* addressing flake errors

* updated build for CI

* REMOVE: This is temporary to run a workflow from this branch

* updates to address requirements for contents

* reverting packages.yml

* Update detection_rules/integrations.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* addressed feedback and added click echo comments

* addressed flake errors and added some comments

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-04-21 11:03:29 -04:00
Terrance DeJesus 894e34f82c [Bug] Add new-package argument to bump-pkg-versions CLI (#2703)
* initial changes to release fleet workflow and CLI

* changed the default value of package version for 8.8

* changed how true/false is passed into CLI command

* reverted changes to packages.yml
2023-04-12 13:48:58 -04:00
Justin Ibarra 411ec36ff0 Validate markdown plugin fields (#2602) 2023-03-28 09:17:50 -04:00
Terrance DeJesus 66a0cbb5de [Bug] Fix release-* Github Workflows and Review integrations-pr command (#2605)
* testing order of operations in workflow

* reverted testing order; adjusting secrets token

* adjusting secrets token

* changing checkout to v3

* removed token for testing workflow

* changed repo reference

* changing secret token

* reverting token changes

* removing master reference

* adjusted elastic-package installation

* changed path of integrations during install

* added integrations fetch run commands

* changed target branch to main, setup latest go

* changed token back to protections machine

* trying different secret for integrations PR creation

* created testing token for permission errors

* adjusted 'bump-pkg-versions' so minors are bumped if no previous pkg

* added bumping package versions as a step

* updated actions/upload-artifact to v3

* removed inaccurate comments; removed release-kibana workflow

* adjusted sequence of steps to bump packge version before build

* added a bump to major if it does not match packages.yml
2023-03-01 10:43:16 -05:00
Terrance DeJesus 73d581500c [Bug] Change YAML Dump Parameters for Integrations Changelog (#2545)
* changed yamp.dump parameters to have correct order for changelog

* adjusted note in changelog
2023-02-14 12:10:41 -05:00
Terrance DeJesus f8d26f4ce0 [Bug] Removed Strip Calls in Favor of F-Strings with Major and Minor Versions (#2541)
* removed strip calls in favor of f-strings with major and minor versions

* changed variable reference in minor_release of bump-pkg-versions
2023-02-10 13:18:53 -05:00
Terrance DeJesus fb2b4529c5 [FR] Adapt PyPi semver Library and Remove Custom (#2503)
* removed custom semver and replaced with pypi

* updated beats.py version references

* updated bump-versions CLI command to use semver and change logic

* updated schemas __init__, test_version_lock and unstage incompatible rules CLI

* updated test_stack_schema_map in TestVersions unittest

* updated test_all_rules unit testing Version() references

* updated stack_compat.py for get_restricted_field references)

* updated version_lock.py Version() references

* updated docs.py Version() reference for parse_registry

* updated devtools.py Version() reference for trim-version-lock

* updated mixins.py Version() reference in validate_field_compatibility

* adjusted schemas.__init__ Version() reference in get_stack_schemas

* adjusted ecs.py Version() references

* adjusted integrations.py Version() references

* adjusted rule.py Version() references

* sorted imports

* replaced custom semver with pypi semver in unit test files

* addressed unit test and flake errors

* changed semver strings casted to version_lock.py

* fixed sorting in integrations.py

* updated bump-pkgs-versions CLI command

* adjusted semantic version in unstage-incompatible-rules command

* adjusted semver import to VersionInfo

* added semver 3 and adjusted import names

* added option_minor_and_patch parameter where version is major.minor

* updated bump-pkg-versions to always save to packages.yml

* removed leftover split call & updated find latest compatible version command

* updated integrations.py, version_lock.py and schemas.__init__.py

* changed fstring reference in downgrade function

* reverted formatting changes for detection_rules __init__.py

* added newline to detection_rules __init__.py

* adjusted finding latest_release for attack package logic

* adjusted unstage-incompatible-rules command logic comparing versions

* removing changes from misc.py related to auto-formatting

* adding newline to misc.py

* fixed bug in downgrade function calling decorators

* added semantic version validation on migrate decorator function

* added expected type returned from find_latest_integration_version in integrations.py

* add comment about stripped versions for version lock file

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-02-07 14:26:29 -05:00
Mika Ayenson 1784429aa7 [FR] Add Integration Schema Query Validation (#2470) 2023-02-02 16:22:44 -05:00
Terrance DeJesus 953e8d98ae [Bug] Adjust Kibana Path for File System Rules (#2397)
* adjusted kibana rules path

* addressed flake errors for long string

* added missing / to directory path
2023-01-03 14:54:24 -05:00
Terrance DeJesus ae4e59ec7d [FR] Update ATT&CK Package to v12.1 (#2422)
* initial update to v12.1 attack package

* added additional click echo output

* addressed flake errors

* updated rules with refreshed att&ck data

* Update detection_rules/devtools.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-12-16 12:04:20 -05:00
Mika Ayenson 5bf69b7967 Update package and install process (#1948) 2022-12-08 15:49:49 -05:00
Justin Ibarra c1dd3c57ad Adds commands to manage ATT&CK mappings (#2343)
* add att&ck commands; fix 2 rule mappings

* update message to stdout

* updated date for rule changes

* unrelated click bug fix

* add type hinting
2022-11-01 13:14:40 -06:00
Mika Ayenson aa8239652d [FR] Add endgame schema validation to detection-rule query (#2257) 2022-10-19 09:54:47 -04:00
Terrance DeJesus b31a1b761c [FR] Re-factor Build Integrations Manifest (#2274)
* adjusted how integrations list is created

* removed unused import and addressed linting errors

* adjusted integration_manifest dictionary to only load latest major

* adjusted manifests sourcing from GH to EPR CDN

* addressed flake errors

* added some additional comments and formatting

* updaing integration-manifests file

* adjusted test_integration testing

* addressed flake errors

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Update detection_rules/integrations.py

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* added folder unit tests

* updated unit test to remove network calls

* Update tests/test_all_rules.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-09-28 09:33:49 -04:00
Justin Ibarra 2ee5a185c7 Add test command to verify version collisions do not occur (#2272)
* Add test command to verify version collisions do not occur
* add max_allowable_version to schema and lock flow
* add max_allowable_version to all entries in version.lock
* add test-version-lock command
* use min supported stack if > locked min stack
* share lock conversion code with rule and lock to fix M.m bug
2022-09-19 09:53:30 -06:00
Mika Ayenson c2e7011ec6 break out the logic to a script and manual workflow (#1908)
* Break out the logic to a script and manual workflow with an option to skip staging files
2022-09-16 13:34:04 -04:00
Justin Ibarra 332ea40100 Cleanup rule survey code (#1923)
* Cleanup rule survey code

* default to only unique-ing on process name for lucene rules

* fix bug in kibana url parsing by removing redundant port from domain

* update search-alerts columns and nest fields

* fix rule.contents.data.index

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2022-09-06 15:53:47 -06:00