[Hunting] Re-factor Hunting Library Code (#4085)

* updating python code for hunting library

* fixed okta queries; added MITRE search capability

* fixed hunting unit test imports

* fixed duplicate UUID; fixed duplicate index entry bug

* fixed technique finding sub-technique in search

* added more unit tests

* linted

* flake errors addressed; fixed unit test import; fixed markdown generate bug

* added description for generate-markdown command

* updated README

* adjusted YAML index, adjusted code for index changes

* adjusted relative imports; updated CODEOWNERS

* adding updates; moving to different branch for main dependencies

* finished run-query command; made some code adjustments

* removed some comments

* revised makefile; fixed unit tests; adjusted detection rules pyproject

* updated README

* updated README

* adjusted unit tests; adjusted hunt guidelines; updated makefile; adjusted several commands

* adjusted package to be more object-oriented

* removed unused variable

* Add simple breakdown stats

* addressed feedback; added keyword option for search

* Update hunting/README.md

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* Update detection_rules/etc/test_hunting_cli.bash

Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>

* addressing feedback

* addressed feedback

* added message for unknown index; fixed function call

* fixed search command

* fixed flake error

---------

Co-authored-by: Mika Ayenson <Mika.ayenson@elastic.co>
Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
Co-authored-by: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
This commit is contained in:
Terrance DeJesus
2024-10-03 12:47:40 -04:00
committed by GitHub
parent 45a347580c
commit 50e23ba242
48 changed files with 1659 additions and 528 deletions
+2 -2
View File
@@ -28,7 +28,7 @@ Detection Rules contains more than just static rule files. This repository also
|------------------------------------------------ |------------------------------------------------------------------------------------ |
| [`detection_rules/`](detection_rules) | Python module for rule parsing, validating and packaging |
| [`etc/`](detection_rules/etc) | Miscellaneous files, such as ECS and Beats schemas |
| [`hunting`](./hunting/) | Root directory where threat hunting queries are stored |
| [`hunting/`](./hunting/) | Root directory where threat hunting package and queries are stored |
| [`kibana/`](lib/kibana) | Python library for handling the API calls to Kibana and the Detection Engine |
| [`kql/`](lib/kql) | Python library for parsing and validating Kibana Query Language |
| [`rta/`](rta) | Red Team Automation code used to emulate attacker techniques, used for rule testing |
@@ -78,7 +78,7 @@ Collecting Click==7.0
...
```
Note: The `kibana` and `kql` packages are not available on PyPI and must be installed from the `lib` directory.
Note: The `kibana` and `kql` packages are not available on PyPI and must be installed from the `lib` directory. The `hunting` package has optional dependencies to be installed with `pip3 install ".[hunting]`.
```console