24 Commits

Author SHA1 Message Date
Mika Ayenson, PhD f0f7d217c0 [FR] Refactor Schema Validation & Support Multi-Dataset Sequence Validation (#5059) 2025-09-10 13:11:04 -05:00
Eric Forte cbb892b4bc [Bug] Incorrect Integrations Schema Parsing for Nested Fields (#5058)
* Add proper handling for nested fields

* Updated schemas

* bump patch

---------

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
2025-09-04 14:12:33 -04:00
Mika Ayenson, PhD 1dc3926203 [New Rules] External Promotion Alerts (#4903) 2025-07-31 11:00:50 -05:00
Sergey Polzunov cdb346cb77 fix: Skip invalid YAML files in Beats dist (#4865)
* Skip invalid YAML files but keep them in the branch

* Typo fix

* Patch version bump

* Adding a schema generation command to `test_cli.bash` flow
2025-07-02 13:39:35 +02:00
Sergey Polzunov 1fb60d6475 fix: type hinting fixes and additional code checks (#4790)
* first pass

* Adding a dedicated code checking workflow

* Type fixes

* linting config and python version bump

* Type hints

* Drop incorrect config option

* More fixes

* Style fixes

* CI adjustments

* Pyproject fixes

* CI & pyproject fixes

* Proper version bump

* Tests formatting

* Resolve cirtular dependency

* Test fixes

* Make sure the tests are formatted correctly

* Check tweaks

* Bumping python version in CI images

* Pin marshmallow do 3.x because 4.x is not supported

* License fix

* Convert path to str

* Making myself a codeowner

* Missing kwargs param

* Adding a missing kwargs to `set_score`

* Update .github/CODEOWNERS

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>

* Dropping unnecessary raise

* Dropping skipped test

* Drop unnecessary var

* Drop unused commented-out func

* Disable typehinting for the whole func

* Update linting command

* Invalid type hist on the input param

* Incorrect field type

* Incorrect value used fix

* Stricter values check

* Simpler function call

* Type condition fix

* TOML formatter fix

* Simpligy output conditions

* Formatting

* Use proper types instead of aliases

* MITRE attack fixes

* Using pathlib.Path for an argument

* Use proper method to update a set from a dict

* First round of `ruff` fixes

* More fixes

* More fixes

* Hack against cyclic dependency

* Ignore `PLC0415`

* Remove unused markers

* Cleanup

* Fixing the incorrect condition

* Update .github/CODEOWNERS

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>

* Set explicit default values for optional fields

* Update the guidelines

* Adding None Defaults

---------

Co-authored-by: Mika Ayenson, PhD <Mikaayenson@users.noreply.github.com>
Co-authored-by: eric-forte-elastic <eric.forte@elastic.co>
2025-07-01 08:20:55 -05:00
shashank-elastic 04e1fc1436 Account for CCS '::' index pattern (#4258) 2024-11-13 11:17:08 +05:30
Eric Forte 47d7a3acaa [DaC] Beta Release (#3889)
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
2024-08-06 18:07:12 -04:00
Mika Ayenson face95058f [Bug] Use integration schemas for required_field types (#3303) 2023-12-11 11:32:38 -06:00
Terrance DeJesus fb2b4529c5 [FR] Adapt PyPi semver Library and Remove Custom (#2503)
* removed custom semver and replaced with pypi

* updated beats.py version references

* updated bump-versions CLI command to use semver and change logic

* updated schemas __init__, test_version_lock and unstage incompatible rules CLI

* updated test_stack_schema_map in TestVersions unittest

* updated test_all_rules unit testing Version() references

* updated stack_compat.py for get_restricted_field references)

* updated version_lock.py Version() references

* updated docs.py Version() reference for parse_registry

* updated devtools.py Version() reference for trim-version-lock

* updated mixins.py Version() reference in validate_field_compatibility

* adjusted schemas.__init__ Version() reference in get_stack_schemas

* adjusted ecs.py Version() references

* adjusted integrations.py Version() references

* adjusted rule.py Version() references

* sorted imports

* replaced custom semver with pypi semver in unit test files

* addressed unit test and flake errors

* changed semver strings casted to version_lock.py

* fixed sorting in integrations.py

* updated bump-pkgs-versions CLI command

* adjusted semantic version in unstage-incompatible-rules command

* adjusted semver import to VersionInfo

* added semver 3 and adjusted import names

* added option_minor_and_patch parameter where version is major.minor

* updated bump-pkg-versions to always save to packages.yml

* removed leftover split call & updated find latest compatible version command

* updated integrations.py, version_lock.py and schemas.__init__.py

* changed fstring reference in downgrade function

* reverted formatting changes for detection_rules __init__.py

* added newline to detection_rules __init__.py

* adjusted finding latest_release for attack package logic

* adjusted unstage-incompatible-rules command logic comparing versions

* removing changes from misc.py related to auto-formatting

* adding newline to misc.py

* fixed bug in downgrade function calling decorators

* added semantic version validation on migrate decorator function

* added expected type returned from find_latest_integration_version in integrations.py

* add comment about stripped versions for version lock file

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

---------

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-02-07 14:26:29 -05:00
Mika Ayenson 1784429aa7 [FR] Add Integration Schema Query Validation (#2470) 2023-02-02 16:22:44 -05:00
Mika Ayenson 6219fc06b9 Move etc under detection_rules (#1885)
* Move etc directory under detection_rules
* Prepend original `etc` path with `detection_rules`
* Update docstrings in util and CODEOWNERS
* Add resiliency to tags to account for the old directory structure
* Bug fix: remove unused param caused by commit 6ed1a39efe

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-05-02 10:11:21 -04:00
Mika Ayenson 84b7ce6582 update beats master branch ref to main (#1853)
* update beats master branch ref to main

* update filename of master beat schema to main

* delete old main beats schema

* rebuilt main beats archive
2022-03-18 10:06:34 -04:00
Christian Clauss ddec37b731 Fix typos discovered by codespell (#1430) 2021-08-14 20:29:10 -08:00
Justin Ibarra 781953a0a0 Add min_stack_version to rule metadata (#1173)
* Add min_stack_version to metadata of rule structure
* validate all "stack versions" between defined and current package
* Use master schemas if min_stack_version > current_package

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2021-06-30 13:26:27 -08:00
Andrew Pease 92eaa5b18a [New Rule] Threat intel indicator match rule (#1133) 2021-04-26 07:07:04 -05:00
Justin Ibarra a0a3143a52 Refresh beats and ecs schemas (#1140)
* download new beats and ecs schemas
* add beats download func by version and download v7.11.2
2021-04-22 09:49:06 -08:00
Justin Ibarra 3fc34b86f2 Update License to Elastic v2 (#944) 2021-03-03 22:12:11 -09:00
Justin Ibarra 0ed1e1df71 Add support to validate against dev ECS and beats schemas (#691) 2020-12-08 13:29:56 -09:00
Brent Murphy 6a296c64c5 [New Rule] Microsoft 365 Exchange DKIM Signing Configuration Disabled (#578)
* [New Rule] O365 Exchange DKIM Signing Configuration Disabled

* rebrand to m365

* still req non ecs schema

* Remove the ECS override

* Update _flatten_schema logic

* Allow fields with * in the path

* Allow explicit fields to overwrite implicit * fields

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-12-08 16:38:00 -05:00
Justin Ibarra 97ee8cc9ac Refresh beats and ecs schemas and default to use latest to validate (#570)
* Refresh beats and ecs schemas and default to use latest to validate
* remove incorrect ecs_version from zoom rule
* remove stale ecs_version from rules
2020-12-01 13:24:20 -09:00
Ross Wolf 9d22970e21 Add EQL rules and schema validation (#297)
* Add EQL rules and schema validation
* Lint nitpick
* Rename get_schema_from_eql
* Add EQL default language
* Rename parsed_kql to parsed_query
* Fix parsed_kql method call in loader
* Autopopulate dependent values
2020-09-16 08:36:48 -06:00
Ross Wolf db4f50d4b8 Improve the validation and testing time (#61)
* Improve the validation and testing time
* Lint fix
* Cache schema validation
2020-07-15 08:05:55 -06:00
Andrew Pease e0f2e8b4a9 Add dataset and index to network rules (#15)
* Add dataset and index to network rules
* Restore iptables changes
* Fix beats parsing logic
* Updated date and ECS version
* Only update modules if empty

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-07-08 13:19:35 -06:00
Ross Wolf 3b305d3003 Add rule loader and dependencies
Co-Authored-By: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-06-29 23:17:42 -06:00