[FR] Detection Rule PR Guidelines and Issue Forms (#3850)

This commit is contained in:
Mika Ayenson
2024-07-10 17:18:45 -05:00
committed by GitHub
parent 59a10be7c8
commit c62321f810
28 changed files with 861 additions and 267 deletions
-31
View File
@@ -1,31 +0,0 @@
---
name: Bug report
about: Report a bug to report for the python/testing parts of Detection Rules
title: "[Bug]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS:
- Version:
**Additional context**
Add any other context about the problem here.
+59
View File
@@ -0,0 +1,59 @@
name: Bug Report
description: Report a bug for the Python/testing components of detection-rules
title: "[Bug] "
labels: ["bug", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: textarea
id: bug_description
attributes:
label: Describe the Bug
description: "A clear and concise description of what the bug is."
placeholder: "Describe the bug..."
- type: textarea
id: reproduce_steps
attributes:
label: To Reproduce
description: "Steps to reproduce the behavior:"
placeholder: "1. Go to '...'\n2. Click on '....'\n3. Scroll down to '....'\n4. See error"
- type: textarea
id: expected_behavior
attributes:
label: Expected Behavior
description: "A clear and concise description of what you expected to happen."
placeholder: "Expected behavior..."
- type: textarea
id: screenshots
attributes:
label: Screenshots
description: "If applicable, add screenshots to help explain your problem."
placeholder: "Upload screenshots..."
- type: dropdown
id: os
attributes:
label: Desktop - OS
options:
- Windows
- macOS
- Linux
- other - explain
- type: input
id: version
attributes:
label: Desktop - Version
description: "The version of the operating system."
placeholder: "e.g., 10, 11, Big Sur, Ubuntu 20.04"
- type: textarea
id: additional_context
attributes:
label: Additional Context
description: "Add any other context or explanations about the problem here."
placeholder: "Additional context..."
-20
View File
@@ -1,20 +0,0 @@
---
name: Feature request
about: 'Suggest an idea for this project (Note: this does not include rule logic)'
title: "[FR]"
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
@@ -0,0 +1,45 @@
name: Feature Request
description: 'Suggest an idea for this repository (Note: this does not include rule logic).'
title: "[FR] "
labels: ["enhancement", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: dropdown
id: feature
attributes:
label: Repository Feature
options:
- Core Repo - (rule management, validation, testing, lib, cicd, etc.)
- Detections-as-Code (DaC) - (primarily custom rule management)
- Hunting Library - (hunt query and markdown generation)
- other - explain
- type: textarea
id: problem_description
attributes:
label: Problem Description
description: "Is your feature request related to a problem? Please describe it clearly and concisely. Ex. I'm always frustrated when [...]"
placeholder: "Describe the problem..."
- type: textarea
id: desired_solution
attributes:
label: Desired Solution
description: "A clear and concise description of what you want to happen."
placeholder: "Describe the solution you want..."
- type: textarea
id: considered_alternatives
attributes:
label: Considered Alternatives
description: "A clear and concise description of any alternative solutions or features you've considered."
placeholder: "Describe any alternatives you've considered..."
- type: textarea
id: additional_context
attributes:
label: Additional Context
description: "Add any other context, explanations or screenshots about the feature request here."
placeholder: "Additional context, explanations, or screenshots..."
+43
View File
@@ -0,0 +1,43 @@
name: Tune Existing Hunt
description: Suggestion for logic changes to an existing hunt
title: "[Hunt Tuning] Name of Hunt"
labels: ["Hunt: Tuning", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: input
id: hunt_link
attributes:
label: Link to hunt
description: "Provide a link to the hunt being recommended."
placeholder: "https://github.com/elastic/detection-hunts/tree/main/hunting/..."
- type: dropdown
id: tuning_type
attributes:
label: Hunt Tuning Type
options:
- False Positives - Reducing benign events mistakenly identified as threats.
- False Negatives - Enhancing detection of true threats that were previously missed.
- Performance - Optimizing resource consumption and execution time of detection hunts.
- Contextual Tuning - Customizing hunts based on specific environment factors.
- Threshold Adjustments - Modifying sensitivity by changing alert triggering thresholds.
- Behavioral Tuning - Refining hunts to better detect deviations from typical behavior.
- Temporal Tuning - Adjusting hunts based on time-based patterns.
- Severity Tuning - Adjusting priority or severity levels of alerts.
- Data Quality - Ensuring integrity and quality of data used by detection hunts.
- type: textarea
id: description
attributes:
label: Description
description: "Provide a detailed description of the suggested changes."
placeholder: "Detailed description..."
- type: textarea
id: example_data
attributes:
label: Example Data
description: "If the query is to be changed, include example JSON data or a screenshot."
placeholder: "Example JSON data or screenshot..."
@@ -1,39 +0,0 @@
---
name: Integration (OOB) updates
about: Template used by Elastic team to release updates to fleet integration package
title: "[Integration Release] <x.x.x>"
labels: fleet-release
assignees: ''
---
# OOB Fleet integration release
### Release branch
<!-- this will dictate which stacks get the updates (>= up to major)
the title should reflect this version; ex: releasing from 8.0 is 1.0.x
the patch version represents the iteration of the release, so the 3rd release for 8.0 is 1.0.3
-->
*
## Checklist
<!-- each root level checklist item should have accompanying pr link -->
<!-- always push from latest (main) and merge, before proceeding
link the completed "kibana updates" issue here
-->
### Prep
- [ ] complete `updates to kibana` <issue link>
- [ ] tag the locked commit (ex `integration-vx.x.x`) <tag link>
### Release package
- [ ] integrations PR <link>
- [ ] package-storage promotion to `production` PR <link>
- [ ] `Pipeline Release Package Distribution` job <job link>
- [ ] production `epr` <link>
### Updates
- [ ] security-docs PR <link>
- [ ] newsfeed PR <link>
-28
View File
@@ -1,28 +0,0 @@
---
name: Kibana updates
about: Template used by Elastic team to push rule updates to Kibana
title: "[Kibana Updates] <source-branch> to kibana:<target-banch>"
labels: kibana-updates
assignees: ''
---
# Kibana updates
- [ ] check if this the final push to the respective Kibana release branch
### Which Kibana branches will this backport to?
<!-- bullet per branch, if none, add 'none' as a bullet. Also link to each backport PR -->
*
## Checklist
<!-- each root level checklist item should have accompanying pr link -->
- [ ] lock versions
- [ ] PR rules updates to Kibana
## Additional if this is the final push targeting a respective Kibana release branch
- [ ] create a tag for the branch from the locked versions commit (ex: `v7.15.0`)
- [ ] update security-docs with rule changes
+93
View File
@@ -0,0 +1,93 @@
name: New Hunt
description: Suggestions and ideas for new hunts
title: "[New hunt] Name of hunt"
labels: ["Hunt: New", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: textarea
id: description
attributes:
label: Description
description: "Provide a detailed description of the activity to be detected."
placeholder: "Detailed description..."
- type: dropdown
id: target_huntset
attributes:
label: Target Huntset
description: "Select the target rulset."
options:
- apm
- cross-platform
- aws
- aws_bedrock
- azure
- azure_openai
- beaconing
- cloud_defend
- cyberparkpas
- ded
- dga
- endpoint
- fim
- gcp
- github
- google_workspace
- kubernetes
- lmd
- o365
- okta
- problemchild
- linux
- macos
- ml
- network
- promotions
- threat_intel
- windows
- other
- type: dropdown
id: hunt_type
attributes:
label: Target hunt Type
description: "Select the target type."
options:
- Custom (KQL or Lucene)
- Machine Learning
- Threshold
- Event Correlation (EQL)
- Indicator Match
- New Terms
- ES|QL
- OSQuery
- type: textarea
id: query
attributes:
label: Query
description: "Provide the query for the hunt (optional)."
placeholder: "Query..."
- type: textarea
id: related_issues_prs
attributes:
label: Related issues or PRs
description: "Link any related issues or PRs (optional)."
placeholder: "Related issues or PRs..."
- type: textarea
id: references
attributes:
label: References
description: "List any references (optional)."
placeholder: "References..."
- type: textarea
id: example_data
attributes:
label: Redacted Example Data
description: "Provide a redacted example JSON data from the actual activity."
placeholder: "Example JSON data..."
@@ -1,37 +0,0 @@
---
name: Kibana schema update feature request
about: 'New Kibana schema feature request.'
title: "[FR] Update schemas to support <name of new feature>"
labels: "Area: DED,Team: TRADE,schema,python,enhancement"
assignees: ''
projects: "elastic/1268,elastic/1271"
---
## Summary
<!-- A clear and concise statement summarizing the goal and success criteria of the new feature If there is a parent link it here. -->
## Tasks
<!-- Outline the Meta tasks that fall under this Epic, each with a brief description. These should guide the creation of separate Meta issues. METAs should be detailed enough to capture key deliverables. -->
```[tasklist]
#### PR Checklist
- [ ] Link to the relevant Kibana PR or issue provided
- [ ] Exported detection rule(s) from Kibana to showcase the feature(s)
- [ ] Converted the exported ndjson file(s) to toml in the detection-rules repo
- [ ] Re-exported the toml rule(s) to ndjson and re-imported into Kibana
- [ ] Updated necessary unit tests to accommodate the feature
- [ ] Applied min_compat restrictions to limit the feature to a specified minimum stack version
- [ ] Executed all unit tests locally with a test toml rule to confirm passing
- [ ] Included Kibana PR implementer as an optional reviewer for insights on the feature
- [ ] Implemented requisite downgrade functionality
- [ ] Cross-referenced the feature with product documentation for consistency
- [ ] Incorporated a comprehensive test rule in unit tests for full schema coverage
- [ ] Conducted system testing, including fleet, import, and create APIs
```
## Dependencies and Constraints
<!-- Identify any dependencies that could impact the progress of this issue, including external resources, team availability, or technology constraints. Detail the resources needed to complete the task, such as access to specific platforms, tools, or expertise. For example, we may not want to merge this feature until the produce is ga.-->
...
-28
View File
@@ -1,28 +0,0 @@
---
name: New Meta Issue
about: Meta Template for Sprint work
title: "[Meta] Name of Meta Issue"
labels: "Area: RAD,Area: DED,Meta,Team: TRADE"
assignees: ''
projects: "elastic/1270"
---
## Parent Epic (If Applicable)
<!-- Please link parent epic issue here if one exists and if not you can leave it blank. -->
## Meta Summary
<!-- Please provide a detailed explanation with what you are planning on doing, what you hope to accomplish and why this is important. -->
## Estimated Time to Complete
## Potential Blockers
## Tasklist
```[tasklist]
### Meta Tasks
- [ ] Provide Week 1 Update Comment
- [ ] Provide Week 2 Update or Closeout Comment
```
## Resources / References
+48
View File
@@ -0,0 +1,48 @@
name: New Meta Issue
description: Meta Template for Sprint work
title: "[Meta] Name of Meta Issue"
labels: ["Meta", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: input
id: parent_epic
attributes:
label: Epic Link
description: "Please link parent epic issue here if one exists and if not you can leave it blank."
placeholder: "https://github.com/elastic/detection-rules/issues/1234"
- type: textarea
id: meta_summary
attributes:
label: Meta Summary
description: Please provide a detailed explanation with what you are planning on doing, what you hope to accomplish and why this is important."
placeholder: "Detailed explanation..."
- type: input
id: estimated_time
attributes:
label: Estimated Time to Complete
description: "Provide an estimate of the time required to complete the tasks."
placeholder: "e.g., 2 weeks"
- type: textarea
id: potential_blockers
attributes:
label: Potential Blockers
description: "List any potential blockers that might impede progress."
placeholder: "- Potential blockers..."
- type: textarea
attributes:
label: Tasking
value: "```[tasklist]\n### Meta Tasks\n- [ ] Provide Week 1 Update Comment\n- [ ] Provide Week 2 Update or Closeout Comment\n```"
render:
- type: textarea
id: potential_references
attributes:
label: Potential References
description: "List any references or resources used."
placeholder: "- Reference url..."
-47
View File
@@ -1,47 +0,0 @@
---
name: New rule
about: Suggestions and ideas for new rules
title: "[New Rule] Name of rule"
labels: 'Rule: New'
assignees: ''
---
<!-- Before submitting an issue to tune a rule, be sure to reference CONTRIBUTING.md --->
## Description
<!-- Provide a detailed description of the activity to be detected -->
## Required Info
### Target indexes
<!-- filebeat-*, logs-windows.*, etc. -->
### Additional requirements
<!-- sysmon, beats config modification, etc. -->
### Target Operating Systems
<!-- windows, linux, macOS, etc -->
### Platforms
<!-- okta, cloudtrail, etc -->
### Tested ECS Version
x.x.x
## Optional Info
### Query
### New fields required in ECS/data sources for this rule?
### Related issues or PRs
### References
## Example Data
<!-- Example JSON data from the actual detonated activity makes this process much quicker -->
+106
View File
@@ -0,0 +1,106 @@
name: New Rule
description: Suggestions and ideas for new rules
title: "[New Rule] Name of rule"
labels: ["Rule: New", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: textarea
id: description
attributes:
label: Description
description: "Provide a detailed description of the activity to be detected."
placeholder: "Detailed description..."
- type: dropdown
id: target_ruleset
attributes:
label: Target Ruleset
description: "Select the target rulset."
options:
- apm
- cross-platform
- aws
- aws_bedrock
- azure
- azure_openai
- beaconing
- cloud_defend
- cyberparkpas
- ded
- dga
- endpoint
- fim
- gcp
- github
- google_workspace
- kubernetes
- lmd
- o365
- okta
- problemchild
- linux
- macos
- ml
- network
- promotions
- threat_intel
- windows
- other
- type: dropdown
id: rule_type
attributes:
label: Target Rule Type
description: "Select the target type."
options:
- Custom (KQL or Lucene)
- Machine Learning
- Threshold
- Event Correlation (EQL)
- Indicator Match
- New Terms
- ES|QL
- type: input
id: ecs_version
attributes:
label: Tested ECS Version
description: "Specify the tested ECS version."
placeholder: "x.x.x"
- type: textarea
id: query
attributes:
label: Query
description: "Provide the query for the rule (optional)."
placeholder: "Query..."
- type: textarea
id: new_fields
attributes:
label: New fields required in ECS/data sources for this rule?
description: "List any new fields required in ECS or data sources for this rule (optional)."
placeholder: "New fields..."
- type: textarea
id: related_issues_prs
attributes:
label: Related issues or PRs
description: "Link any related issues or PRs (optional)."
placeholder: "Related issues or PRs..."
- type: textarea
id: references
attributes:
label: References
description: "List any references (optional)."
placeholder: "References..."
- type: textarea
id: example_data
attributes:
label: Redacted Example Data
description: "Provide a redacted example JSON data from the actual activity."
placeholder: "Example JSON data..."
@@ -1,15 +0,0 @@
---
name: Rule deprecation
about: Recommendation to deprecate a rule
title: "[Deprecation] Name of the rule"
labels: 'Rule: Deprecation'
assignees: ''
---
## Link to rule
## Description
Provide a detailed description of why the rule should be deprecated
@@ -0,0 +1,21 @@
name: Rule Deprecation
description: Recommendation to deprecate a rule
title: "[Deprecation] Name of the rule"
labels: ["Rule: Deprecation", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: input
id: rule_link
attributes:
label: Link to Rule
description: "Provide a link to the rule being recommended for deprecation."
placeholder: "https://github.com/elastic/detection-rules/tree/main/rules/..."
- type: textarea
id: description
attributes:
label: Description
description: "Provide a detailed description of why the rule should be deprecated."
placeholder: "Detailed description..."
-19
View File
@@ -1,19 +0,0 @@
---
name: Tune existing rule
about: Suggestion for logic changes to an existing rule
title: "[Rule Tuning] Name of rule"
labels: 'Rule: Tuning'
assignees: ''
---
<!-- Before submitting an issue to tune a rule, be sure to reference CONTRIBUTING.md --->
## Link to rule
## Description
<!-- Provide a detailed description of the suggested changes -->
## Example Data
<!-- If the query is to be changed, include example JSON data or a screenshot -->
+43
View File
@@ -0,0 +1,43 @@
name: Tune Existing Rule
description: Suggestion for logic changes to an existing rule
title: "[Rule Tuning] Name of rule"
labels: ["Rule: Tuning", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: input
id: rule_link
attributes:
label: Link to Rule
description: "Provide a link to the rule being recommended."
placeholder: "https://github.com/elastic/detection-rules/tree/main/rules/..."
- type: dropdown
id: tuning_type
attributes:
label: Rule Tuning Type
options:
- False Positives - Reducing benign events mistakenly identified as threats.
- False Negatives - Enhancing detection of true threats that were previously missed.
- Performance - Optimizing resource consumption and execution time of detection rules.
- Contextual Tuning - Customizing rules based on specific environment factors.
- Threshold Adjustments - Modifying sensitivity by changing alert triggering thresholds.
- Behavioral Tuning - Refining rules to better detect deviations from typical behavior.
- Temporal Tuning - Adjusting rules based on time-based patterns.
- Severity Tuning - Adjusting priority or severity levels of alerts.
- Data Quality - Ensuring integrity and quality of data used by detection rules.
- type: textarea
id: description
attributes:
label: Description
description: "Provide a detailed description of the suggested changes."
placeholder: "Detailed description..."
- type: textarea
id: example_data
attributes:
label: Example Data
description: "If the query is to be changed, include example JSON data or a screenshot."
placeholder: "Example JSON data or screenshot..."
@@ -0,0 +1,42 @@
name: Kibana schema update feature request
description: 'Suggest a rule schema change related to Kibana schemas'
title: "[FR] Update schemas to support <name of new feature>"
labels: ["schema", "python", "Team: TRADE"]
assignees: []
projects: ["elastic/1268"]
body:
- type: input
id: kibana_pr_link
attributes:
label: Link to Kibana PR
description: "Provide a link to the Kibana PR with the relevant schema changes."
placeholder: "https://github.com/elastic/kibana/pull/..."
- type: textarea
id: feature_description
attributes:
label: Feature Description
description: "Which Kibana feature needs be supported within our schemas?"
placeholder: "Describe the feature or provide a link..."
- type: textarea
id: desired_solution
attributes:
label: Desired Solution
description: "A clear and concise description of what you want to happen."
placeholder: "Describe the solution you want..."
- type: textarea
id: considered_alternatives
attributes:
label: Considered Alternatives
description: "A clear and concise description of any alternative solutions or features you've considered."
placeholder: "Describe any alternatives you've considered..."
- type: textarea
id: additional_context
attributes:
label: Additional Context
description: "Add any other context, explanations, or screenshots about the feature request here."
placeholder: "Additional context, explanations, or screenshots..."
@@ -0,0 +1,34 @@
## Bug - Guidelines
These guidelines serve as a reminder set of considerations when addressing a bug in the code.
### Documentation and Context
- [ ] Provide detailed documentation (description, screenshots, reproducing the bug, etc.) of the bug if not already documented in an issue.
- [ ] Include additional context or details about the problem.
- [ ] Ensure the fix includes necessary updates to the release documentation and versioning.
### Code Standards and Practices
- [ ] Code follows established design patterns within the repo and avoids duplication.
- [ ] Code changes do not introduce new warnings or errors.
- [ ] Variables and functions are well-named and descriptive.
- [ ] Any unnecessary / commented-out code is removed.
- [ ] Ensure that the code is modular and reusable where applicable.
- [ ] Check for proper exception handling and messaging.
### Testing
- [ ] New unit tests have been added to cover the bug fix or edge cases.
- [ ] Existing unit tests have been updated to reflect the changes.
- [ ] Provide evidence of testing and detecting the bug fix (e.g., test logs, screenshots).
- [ ] Validate that any rules affected by the bug are correctly updated.
- [ ] Ensure that performance is not negatively impacted by the changes.
- [ ] Verify that any release artifacts are properly generated and tested.
### Additional Checks
- [ ] Ensure that the bug fix does not break existing functionality.
- [ ] Review the bug fix with a peer or team member for additional insights.
- [ ] Verify that the bug fix works across all relevant environments (e.g., different OS versions).
- [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
@@ -0,0 +1,34 @@
## Enhancement - Guidelines
These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.
### Documentation and Context
- [ ] Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
- [ ] Include additional context or screenshots.
- [ ] Ensure the enhancement includes necessary updates to the documentation and versioning.
### Code Standards and Practices
- [ ] Code follows established design patterns within the repo and avoids duplication.
- [ ] Code changes do not introduce new warnings or errors.
- [ ] Variables and functions are well-named and descriptive.
- [ ] Any unnecessary / commented-out code is removed.
- [ ] Ensure that the code is modular and reusable where applicable.
- [ ] Check for proper exception handling and messaging.
### Testing
- [ ] New unit tests have been added to cover the enhancement.
- [ ] Existing unit tests have been updated to reflect the changes.
- [ ] Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
- [ ] Validate that any rules affected by the enhancement are correctly updated.
- [ ] Ensure that performance is not negatively impacted by the changes.
- [ ] Verify that any release artifacts are properly generated and tested.
### Additional Checks
- [ ] Ensure that the enhancement does not break existing functionality.
- [ ] Review the enhancement with a peer or team member for additional insights.
- [ ] Verify that the enhancement works across all relevant environments (e.g., different OS versions).
- [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
@@ -0,0 +1,31 @@
## Hunt: New - Guidelines
Welcome to the `hunting` folder within the `detection-rules` repository! This directory houses a curated collection of threat hunting queries designed to enhance security monitoring and threat detection capabilities using the Elastic Stack.
### Documentation and Context
- [ ] Detailed description of the Hunt.
- [ ] List any new fields required in ECS/data sources.
- [ ] Link related issues or PRs.
- [ ] Include references.
- [ ] Field Usage: Ensure standardized fields for compatibility across different data environments and sources.
### Hunt Metadata Checks
- [ ] `author`: The name of the individual or organization authoring the rule.
- [ ] `creation_date` matches the date of creation PR initially merged.
- [ ] `min_stack_version` supports the widest stack versions.
- [ ] `name` and `description` are descriptive and typo-free.
- [ ] `language`: The query language(s) used in the rule, such as `KQL`, `EQL`, `ES|QL`, `OsQuery`, or `YARA`.
- [ ] `query` is inclusive, not overly exclusive, considering performance for diverse environments.
- [ ] `integration` aligns with the `index`. Ensure updates if the integration is newly introduced.
- [ ] `setup` includes necessary steps to configure the integration.
- [ ] `note` includes additional information (e.g., Triage and analysis investigation guides, timeline templates).
- [ ] `tags` are relevant to the threat and align with `EXPECTED_HUNT_TAGS` in `definitions.py`.
- [ ] `threat`, `techniques`, and `subtechniques` map to ATT&CK whenever possible.
### Testing and Validation
- [ ] Evidence of testing and detecting the expected threat.
- [ ] Check for the existence of coverage to prevent duplication.
- [ ] Generate Markdown: Run `python generate_markdown.py` to update the documentation.
@@ -0,0 +1,39 @@
## Hunt: Tuning - Guidelines
These guidelines serve as a reminder set of considerations when tuning an existing Hunt.
### Documentation and Context
- [ ] Detailed description of the suggested changes.
- [ ] Provide example JSON data or screenshots.
- [ ] Evidence of reducing benign events mistakenly identified as threats (False Positives).
- [ ] Evidence of enhancing detection of true threats that were previously missed (False Negatives).
- [ ] Evidence of optimizing resource consumption and execution time of detection rules (Performance).
- [ ] Evidence of specific environment factors influencing customized hunt tuning (Contextual Tuning).
- [ ] Evidence of improvements by modifying sensitivity (Threshold Adjustments).
- [ ] Evidence of refining hunts to better detect deviations from typical behavior (Behavioral Tuning).
- [ ] Evidence of improvements based on time-based patterns (Temporal Tuning).
- [ ] Reasoning for adjusting priority or severity levels of alerts (Severity Tuning).
- [ ] Evidence of improving the quality integrity of data used by hunts (Data Quality).
- [ ] Ensure necessary updates to release documentation and versioning.
- [ ] Field Usage: Ensure standardized fields for compatibility across different data environments and sources.
### Hunt Metadata Checks
- [ ] `author`: The name of the individual or organization authoring the rule.
- [ ] `updated_date` matches the date of tuning PR merged.
- [ ] `min_stack_version` supports the widest stack versions.
- [ ] `name` and `description` are descriptive and typo-free.
- [ ] `language`: The query language(s) used in the rule, such as `KQL`, `EQL`, `ES|QL`, `OsQuery`, or `YARA`.
- [ ] `query` is inclusive, not overly exclusive. Review to ensure the original intent of the hunt is maintained.
- [ ] `integration` aligns with the `index`. Ensure updates if the integration is newly introduced.
- [ ] `setup` includes necessary steps to configure the integration.
- [ ] `note` includes additional information (e.g., Triage and analysis investigation guides, timeline templates).
- [ ] `tags` are relevant to the threat and align with `EXPECTED_HUNT_TAGS` in `definitions.py`.
- [ ] `threat`, `techniques`, and `subtechniques` map to ATT&CK whenever possible.
### Testing and Validation
- [ ] Generate Markdown: Run `python generate_markdown.py` to update the documentation.
- [ ] Validate the tuned hunt's performance and ensure it does not negatively impact the stack.
- [ ] Ensure the tuned hunt has a low false positive rate.
@@ -0,0 +1,20 @@
## Rule: Deprecation - Guidelines
These guidelines serve as a reminder set of considerations when recommending the deprecation of a rule.
### Documentation and Context
- [ ] Description of the reason for deprecation.
- [ ] Include any context or historical data supporting the deprecation decision.
### Rule Metadata Checks
- [ ] `deprecated = true` added to the rule metadata.
- [ ] `updated_date` should be the date of the PR.
### Testing and Validation
- [ ] A prior rule tuning occurred for the rule where `Deprecated - ` is prepended to the rule name, and the rule has already been released.
- [ ] Rule has be moved to the `_deprecated` directory.
- [ ] Double check gaps potentially or inadvertently introduced.
- [ ] Provide evidence that the rule is no longer needed or has been replaced (e.g., alternative rules, updated detection methods).
@@ -0,0 +1,33 @@
## Rule: New - Guidelines
These guidelines serve as a reminder set of considerations when proposing a new rule.
### Documentation and Context
- [ ] Detailed description of the rule.
- [ ] List any new fields required in ECS/data sources.
- [ ] Link related issues or PRs.
- [ ] Include references.
### Rule Metadata Checks
- [ ] `creation_date` matches the date of creation PR initially merged.
- [ ] `min_stack_version` should support the widest stack versions.
- [ ] `name` and `description` should be descriptive and not include typos.
- [ ] `query` should be inclusive, not overly exclusive, considering performance for diverse environments. Non ecs fields should be added to `non-ecs-schema.json` if not available in an integration.
- [ ] `min_stack_comments` and `min_stack_version` should be included if the rule is only compatible starting from a specific stack version.
- [ ] `index` pattern should be neither too specific nor too vague, ensuring it accurately matches the relevant data stream (e.g., use logs-endpoint.process-* for process data).
- [ ] `integration` should align with the `index`. If the integration is newly introduced, ensure the manifest, schemas, and `new_rule.yaml` template are updated.
- [ ] `setup` should include the necessary steps to configure the integration.
- [ ] `note` should include any additional information (e.g. Triage and analysis investigation guides, timeline templates).
- [ ] `tags` should be relevant to the threat and align/added to the `EXPECTED_RULE_TAGS` in the definitions.py file.
- [ ] `threat`, `techniques`, and `subtechniques` should map to ATT&CK always if possible.
#### New BBR Rules
- [ ] `building_block_type` should be included if the rule is a building block and the rule should be located in the `rules_building_block` folder.
- [ ] `bypass_bbr_timing` should be included if adding custom lookback timing to the rule.
### Testing and Validation
- [ ] Provide evidence of testing and detecting the expected threat.
- [ ] Check for existence of coverage to prevent duplication.
@@ -0,0 +1,30 @@
## Rule: Tuning - Guidelines
These guidelines serve as a reminder set of considerations when tuning an existing rule.
### Documentation and Context
- [ ] Detailed description of the suggested changes.
- [ ] Provide example JSON data or screenshots.
- [ ] Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
- [ ] Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
- [ ] Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
- [ ] Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
- [ ] Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
- [ ] Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
- [ ] Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
- [ ] Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
- [ ] Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
- [ ] Ensure the tuning includes necessary updates to the release documentation and versioning.
### Rule Metadata Checks
- [ ] `updated_date` matches the date of tuning PR merged.
- [ ] `min_stack_version` should support the widest stack versions.
- [ ] `name` and `description` should be descriptive and not include typos.
- [ ] `query` should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.
### Testing and Validation
- [ ] Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
- [ ] Ensure that the tuned rule has a low false positive rate.
@@ -0,0 +1,46 @@
## Enhancement - Guidelines
These guidelines serve as a reminder set of considerations when addressing adding a new schema feature to the code.
### Documentation and Context
- [ ] Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
- [ ] Include additional context or screenshots.
- [ ] Ensure the enhancement includes necessary updates to the documentation and versioning.
### Code Standards and Practices
- [ ] Code follows established design patterns within the repo and avoids duplication.
- [ ] Code changes do not introduce new warnings or errors.
- [ ] Variables and functions are well-named and descriptive.
- [ ] Any unnecessary / commented-out code is removed.
- [ ] Ensure that the code is modular and reusable where applicable.
- [ ] Check for proper exception handling and messaging.
### Testing
- [ ] New unit tests have been added to cover the enhancement.
- [ ] Existing unit tests have been updated to reflect the changes.
- [ ] Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
- [ ] Validate that any rules affected by the enhancement are correctly updated.
- [ ] Ensure that performance is not negatively impacted by the changes.
- [ ] Verify that any release artifacts are properly generated and tested.
### Additional Schema Related Checks
- [ ] Ensure that the enhancement does not break existing functionality. (e.g., run `make test-cli`)
- [ ] Review the enhancement with a peer or team member for additional insights.
- [ ] Verify that the enhancement works across all relevant environments (e.g., different OS versions).
- [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
- [ ] Link to the relevant Kibana PR or issue provided
- [ ] Exported detection rule(s) from Kibana to showcase the feature(s)
- [ ] Converted the exported ndjson file(s) to toml in the detection-rules repo
- [ ] Re-exported the toml rule(s) to ndjson and re-imported into Kibana
- [ ] Updated necessary unit tests to accommodate the feature
- [ ] Applied min_compat restrictions to limit the feature to a specified minimum stack version
- [ ] Executed all unit tests locally with a test toml rule to confirm passing
- [ ] Included Kibana PR implementer as an optional reviewer for insights on the feature
- [ ] Implemented requisite downgrade functionality
- [ ] Cross-referenced the feature with product documentation for consistency
- [ ] Incorporated a comprehensive test rule in unit tests for full schema coverage
- [ ] Conducted system testing, including fleet, import, and create APIs (e.g., run `make test-remote-cli`)
+33 -3
View File
@@ -4,13 +4,43 @@ There are a few simple things to check before submitting your pull request
that can help with the review process. You should delete these items
from your submission, but they are here to help bring them to your attention.
-->
# Pull Request
## Issues
<!-- Link to related issues. Use closing keywords when appropriate -->
*Issue link(s)*:
## Summary
<!--
Add Related Issues / PRs for context. Eg:
Related to elastic/repo#999
Resolves #123
If there is no issue link, take extra care to write a clear summary and label the PR just as you would label an issue to give additional context to reviewers.
-->
## Summary - What I changed
<!--
Summarize your PR. Animated gifs are 💯. Code snippets are ⚡️. Examples & screenshots are 🔥
-->
## How To Test
<!--
Some examples of what you could include here are:
* Links to GitHub action results for CI test improvements
* Sample data before/after screenshots (or short videos showing how something works)
* Copy/pasted commands and output from the testing you did in your local terminal window
* If tests run in GitHub, you can 🪁or 🔱, respectively, to indicate tests will run in CI
* Query used in your stack to verify the change
-->
## Checklist
<!-- Delete any items that are not applicable to this PR. -->
- [ ] Added a label for the type of pr: `bug`, `enhancement`, `Rule: New`, `Rule: Deprecation`, `Rule: Promote`, `Rule: Tuning`, `Hunt: New`, or `Hunt: Tuning` so guidelines can be generated
- [ ] Added the `meta:rapid-merge` label if planning to merge within 24 hours
- [ ] Secret and sensitive material has been managed correctly
- [ ] Automated testing was updated or added to match the most common scenarios
- [ ] Documentation and comments were added for features that require explanation
## Contributor checklist
+61
View File
@@ -0,0 +1,61 @@
name: Add PR Guidelines Comment
on:
pull_request_target:
types: [opened, labeled]
jobs:
add-comment:
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v2
- name: Set environment variable for early exit control
id: check_label
run: |
echo "CONTINUE_JOB=true" >> $GITHUB_ENV
if [[ "${{ github.event.action }}" == "labeled" || "${{ github.event.action }}" == "opened" ]]; then
RELEVANT_LABELS=("bug" "enhancement" "Rule: New" "Rule: Tuning" "Rule: Promotion" "Rule: Deprecation")
TRIGGERED_LABEL="${{ github.event.label.name }}"
if [[ ! " ${RELEVANT_LABELS[@]} " =~ " ${TRIGGERED_LABEL} " ]]; then
echo "CONTINUE_JOB=false" >> $GITHUB_ENV
fi
fi
- name: Determine Guidelines Label
run: |
# Initialize GUIDELINES_FILE to empty
echo "GUIDELINES_FILE=" >> $GITHUB_ENV
if [[ "${{ contains(github.event.pull_request.labels.*.name, 'bug') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/bug_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'enhancement') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/enhancement_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'schema') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/schema_enhancement_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'Rule: New') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/rule_new_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'Rule: Tuning') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/rule_tuning_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'Rule: Deprecation') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/rule_deprecation_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'Hunt: New') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/hunt_new_guidelines.md" >> $GITHUB_ENV
elif [[ "${{ contains(github.event.pull_request.labels.*.name, 'Hunt: Tuning') }}" == "true" ]]; then
echo "GUIDELINES_FILE=.github/PULL_REQUEST_GUIDELINES/hunt_tuning_guidelines.md" >> $GITHUB_ENV
fi
- name: Fail if no relevant labels are found
if: env.GUIDELINES_FILE == ''
uses: actions/github-script@v7
with:
script: |
core.setFailed('No appropriate GitHub label found in the PR. Failing the job.')
- name: Add Guidelines Comment
if: env.CONTINUE_JOB == 'true' && (github.event.action == 'opened' || github.event.action == 'labeled')
uses: mshick/add-pr-comment@v2
with:
message-path: ${{ env.GUIDELINES_FILE }}
repo-token: ${{ secrets.PROTECTIONS_MACHINE_TOKEN }}
message-id: "guidelines-comment"