Commit Graph

34 Commits

Author SHA1 Message Date
Terrance DeJesus b1a689b6fd Revert "[Security Content] Investigation Guides Line breaks refactor (#2412)" (#2453)
This reverts commit d1481e1a88.
2023-01-09 10:44:54 -05:00
Jonhnathan d1481e1a88 [Security Content] Investigation Guides Line breaks refactor (#2412)
* [Security Content] Investigation Guides Line break refactor

* undo updated_date bump on deprecated rules

* Remove duplicated key
2023-01-09 11:56:39 -03:00
Terrance DeJesus 4312d8c958 [FR] Add Endpoint, APM and Windows Integration Tags to Rules and Supportability (#2429)
* initial commit

* addressing flake errors

* added apm to _get_packagted_integrations logic

* addressed flake errors

* adjusted integration schema and updated rules to be a list

* updated several rules and removed a unit test

* updated rules with logs-* only index patterns

* Update tests/test_all_rules.py

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>

* addressed flake errors

* integration is none is windows, endpoint or apm

* adding rules with accepted incoming changes from main

* fixed tag and tactic alignment errors from unit testing

* adjusted unit testing logic for integration tags; added more exclusion rules

* adjusted test_integration logic to be rule resistent and skip if -8.3

* adjusted comments for unit test skip

* fixed merge conflicts from main

* changing test_integration_tag to remove logic for rule version comparisons

* added integration tag to new rule

* adjusted rules updated_date value

* ignore guided onboarding rule in unit tests

* added integration tag to new rule

Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com>
2023-01-04 09:30:07 -05:00
Jonhnathan ac01718bb6 [Rule Tuning] Add tags to flag Sysmon-only rules & Modify Investigation Guide-related tag (#2352)
* [Rule Tuning] Add tags to flag Sysmon-only rules

* Modify tags

* Revert "Modify tags"

This reverts commit 3d9267d171a41f727bb499501d71d5c4db4f0434.

* Modify tags

* Update test_all_rules.py

* Update test_all_rules.py

* Update test_all_rules.py

* Update test_all_rules.py

* Update test_all_rules.py
2022-11-18 12:32:27 -03:00
Jonhnathan 6055d0db60 [Security Content] Introduce Osquery Markdown Plugin Queries in Investigation Guides (#2387)
* [Security Content] Introduce Osquery Markdown Plugin Queries in Investigation Guides

* Remove min_stack and add Note

* Fix Typo and preffix

* Update command_and_control_certutil_network_connection.toml

* Add unit test to check Note about Osquery Markdown plugin and Version limitations

* Update test_all_rules.py

* Update test_all_rules.py

* Change Note Verbiage
2022-11-17 18:38:34 -03:00
Jonhnathan 183b1ffdd3 [Rule Tuning] Add endgame support for Windows Rules (#2285)
* [Rule Tuning] Add endgame support for Windows Rules

* Update collection_email_powershell_exchange_mailbox.toml

* Supported Rules - First Half

* bum updated_date

* Add tag

* Revert compat

* missing tags
2022-10-19 08:27:44 -07:00
Jonhnathan f02ffbbe13 [Security Content] Add Investigation Guides - 8.5 (#2305)
* [Security Content] Add Investigation Guides - 8.5

* Update persistence_run_key_and_startup_broad.toml

* Apply suggestions from security-docs review review

* Update execution_suspicious_jar_child_process.toml

* Apply suggestions from review
2022-09-23 18:44:24 -03:00
Jonhnathan ec04a39413 [Security Content] Tag rules with robust Investigation Guides (#2297) 2022-09-23 14:20:32 -03:00
Justin Ibarra 46d5e37b76 min_stack all rules to 8.3 (#2259)
* min_stack all rules to 8.3

* bump date

Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
2022-08-24 10:38:49 -06:00
Samirbous b89d6185b2 [Rule Tuning] Reduce FPs (#2223)
9 rules tuned to exclude common noisy FP patterns.

Co-authored-by: Colson Wilhoit <48036388+DefSecSentinel@users.noreply.github.com>
2022-08-15 09:15:48 -05:00
Mika Ayenson a52751494e 2058 add setup field to metadata (#2061)
* Convert config header to setup in note field
* Parse note field into separate setup and note field with marko gfm
* only validate and parse note on elastic authored rules and add CLI description for new DR_BYPASS_NOTE_VALIDATION_AND_PARSE environment variable

Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
2022-07-18 15:41:32 -04:00
Jonhnathan 3a8efc8183 [Security Content] 8.4 - Add Investigation Guides (#2069)
* [Security Content] 8.4 - Add Investigation Guides

* Apply suggestions from code review

Co-authored-by: Joe Peeples <joe.peeples@elastic.co>

* Update rules/windows/credential_access_cmdline_dump_tool.toml

Co-authored-by: Joe Peeples <joe.peeples@elastic.co>

* Apply suggestions from code review

Co-authored-by: nastasha-solomon <79124755+nastasha-solomon@users.noreply.github.com>

* Update rules/windows/credential_access_credential_dumping_msbuild.toml

Co-authored-by: nastasha-solomon <79124755+nastasha-solomon@users.noreply.github.com>

Co-authored-by: Joe Peeples <joe.peeples@elastic.co>
Co-authored-by: nastasha-solomon <79124755+nastasha-solomon@users.noreply.github.com>
2022-07-13 11:28:34 -03:00
Justin Ibarra 6bdfddac8e Expand timestamp override tests (#1907)
* Expand timestamp_override tests
* removed timestamp_override from eql sequence rules
* add config entry for eql rules with beats index and t_o
* add timestamp_override to missing fields
2022-04-01 15:27:08 -08:00
Terrance DeJesus 93edc44284 [Rule Tuning] Timeline Templates For Windows and Linux (#1892)
* added comprehensive file timeline to Hosts File Modified rule

* added Comprehensive Process Timeline to Interactive Terminal Spawned via Python rule

* updated rules to have generic instead of comprehensive

* updated several rules with timeline ID and timeline title values

* changed updated_date for threat intel fleet integrations

* added missing templates to timeline_templates dict in definitions.py

* added comprehensive timeline templates to alerts after definitions.py was updated

* updated rules with comprehensive timeline templates and added min stack comments and versions

* removing timeline template changes which is tracked in #1904

* Update rules/linux/execution_python_tty_shell.toml

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

* Delete Pipfile

Removing pipfile

* Delete Pipfile.lock

deleting pipfile.lock

* Update rules/windows/execution_command_shell_started_by_svchost.toml

updating title

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
2022-04-01 13:44:35 -04:00
Jonhnathan e1b4a0d87c Svchost spawning Cmd - False Positives Tuning (#1894) 2022-03-31 19:28:46 -03:00
Justin Ibarra 82ec6ac1ee Convert windows rules from KQL to EQL (#1114) 2021-04-30 11:21:12 -08:00
Justin Ibarra 3fc34b86f2 Update License to Elastic v2 (#944) 2021-03-03 22:12:11 -09:00
brokensound77 a77bd6178f Merge remote-tracking branch 'upstream/7.11' into merge-7.11-to-7.12
# Conflicts:
#	rules/linux/privilege_escalation_setgid_bit_set_via_chmod.toml
2021-02-17 14:11:50 -09:00
Justin Ibarra 90a9320f93 [Rule Tuning] Remove timestamp_override for endgame-* promotion rules (#951)
* remove timestamp_override from endgame promotion rules
* updated version.lock to previous state for endgame promotion rule changes
* fix incorrect year in updated_date
2021-02-17 13:48:57 -09:00
brokensound77 6ce418877f Merge remote-tracking branch 'upstream/7.12' into merge-7.11-to-7.12
# Conflicts:
#	etc/version.lock.json
#	rules/cross-platform/defense_evasion_deleting_websvr_access_logs.toml
#	rules/cross-platform/impact_hosts_file_modified.toml
#	rules/cross-platform/privilege_escalation_setuid_setgid_bit_set_via_chmod.toml
#	rules/cross-platform/privilege_escalation_sudoers_file_mod.toml
#	rules/linux/defense_evasion_deletion_of_bash_command_line_history.toml
#	rules/linux/defense_evasion_timestomp_touch.toml
#	rules/linux/privilege_escalation_setgid_bit_set_via_chmod.toml
#	rules/macos/credential_access_credentials_keychains.toml
#	rules/macos/credential_access_promt_for_pwd_via_osascript.toml
#	rules/macos/defense_evasion_attempt_del_quarantine_attrib.toml
#	rules/network/command_and_control_cobalt_strike_default_teamserver_cert.toml
#	rules/promotions/external_alerts.toml
#	rules/windows/collection_email_powershell_exchange_mailbox.toml
#	rules/windows/collection_persistence_powershell_exch_mailbox_activesync_add_device.toml
#	rules/windows/collection_winrar_encryption.toml
#	rules/windows/command_and_control_common_webservices.toml
#	rules/windows/command_and_control_encrypted_channel_freesslcert.toml
#	rules/windows/command_and_control_remote_file_copy_desktopimgdownldr.toml
#	rules/windows/command_and_control_remote_file_copy_mpcmdrun.toml
#	rules/windows/command_and_control_teamviewer_remote_file_copy.toml
#	rules/windows/credential_access_cmdline_dump_tool.toml
#	rules/windows/credential_access_copy_ntds_sam_volshadowcp_cmdline.toml
#	rules/windows/credential_access_credential_dumping_msbuild.toml
#	rules/windows/credential_access_domain_backup_dpapi_private_keys.toml
#	rules/windows/credential_access_dump_registry_hives.toml
#	rules/windows/credential_access_iis_apppoolsa_pwd_appcmd.toml
#	rules/windows/credential_access_iis_connectionstrings_dumping.toml
#	rules/windows/credential_access_kerberoasting_unusual_process.toml
#	rules/windows/credential_access_lsass_memdump_file_created.toml
#	rules/windows/credential_access_mimikatz_memssp_default_logs.toml
#	rules/windows/defense_evasion_adding_the_hidden_file_attribute_with_via_attribexe.toml
#	rules/windows/defense_evasion_clearing_windows_event_logs.toml
#	rules/windows/defense_evasion_code_injection_conhost.toml
#	rules/windows/defense_evasion_cve_2020_0601.toml
#	rules/windows/defense_evasion_delete_volume_usn_journal_with_fsutil.toml
#	rules/windows/defense_evasion_deleting_backup_catalogs_with_wbadmin.toml
#	rules/windows/defense_evasion_disable_windows_firewall_rules_with_netsh.toml
#	rules/windows/defense_evasion_dotnet_compiler_parent_process.toml
#	rules/windows/defense_evasion_enable_inbound_rdp_with_netsh.toml
#	rules/windows/defense_evasion_encoding_or_decoding_files_via_certutil.toml
#	rules/windows/defense_evasion_execution_lolbas_wuauclt.toml
#	rules/windows/defense_evasion_execution_msbuild_started_by_office_app.toml
#	rules/windows/defense_evasion_execution_msbuild_started_by_script.toml
#	rules/windows/defense_evasion_execution_msbuild_started_by_system_process.toml
#	rules/windows/defense_evasion_execution_msbuild_started_renamed.toml
#	rules/windows/defense_evasion_execution_msbuild_started_unusal_process.toml
#	rules/windows/defense_evasion_execution_suspicious_explorer_winword.toml
#	rules/windows/defense_evasion_execution_via_trusted_developer_utilities.toml
#	rules/windows/defense_evasion_hide_encoded_executable_registry.toml
#	rules/windows/defense_evasion_iis_httplogging_disabled.toml
#	rules/windows/defense_evasion_injection_msbuild.toml
#	rules/windows/defense_evasion_masquerading_as_elastic_endpoint_process.toml
#	rules/windows/defense_evasion_masquerading_renamed_autoit.toml
#	rules/windows/defense_evasion_masquerading_suspicious_werfault_childproc.toml
#	rules/windows/defense_evasion_masquerading_trusted_directory.toml
#	rules/windows/defense_evasion_modification_of_boot_config.toml
#	rules/windows/defense_evasion_port_forwarding_added_registry.toml
#	rules/windows/defense_evasion_scheduledjobs_at_protocol_enabled.toml
#	rules/windows/defense_evasion_sdelete_like_filename_rename.toml
#	rules/windows/defense_evasion_solarwinds_backdoor_service_disabled_via_registry.toml
#	rules/windows/defense_evasion_suspicious_managedcode_host_process.toml
#	rules/windows/defense_evasion_suspicious_zoom_child_process.toml
#	rules/windows/defense_evasion_system_critical_proc_abnormal_file_activity.toml
#	rules/windows/defense_evasion_unusual_dir_ads.toml
#	rules/windows/defense_evasion_unusual_system_vp_child_program.toml
#	rules/windows/defense_evasion_via_filter_manager.toml
#	rules/windows/defense_evasion_volume_shadow_copy_deletion_via_wmic.toml
#	rules/windows/discovery_adfind_command_activity.toml
#	rules/windows/discovery_admin_recon.toml
#	rules/windows/discovery_file_dir_discovery.toml
#	rules/windows/discovery_net_command_system_account.toml
#	rules/windows/discovery_net_view.toml
#	rules/windows/discovery_peripheral_device.toml
#	rules/windows/discovery_process_discovery_via_tasklist_command.toml
#	rules/windows/discovery_query_registry_via_reg.toml
#	rules/windows/discovery_remote_system_discovery_commands_windows.toml
#	rules/windows/discovery_security_software_wmic.toml
#	rules/windows/discovery_whoami_command_activity.toml
#	rules/windows/execution_apt_solarwinds_backdoor_child_cmd_powershell.toml
#	rules/windows/execution_apt_solarwinds_backdoor_unusual_child_processes.toml
#	rules/windows/execution_command_shell_started_by_powershell.toml
#	rules/windows/execution_command_shell_started_by_svchost.toml
#	rules/windows/execution_command_shell_started_by_unusual_process.toml
#	rules/windows/execution_command_shell_via_rundll32.toml
#	rules/windows/execution_from_unusual_directory.toml
#	rules/windows/execution_from_unusual_path_cmdline.toml
#	rules/windows/execution_shared_modules_local_sxs_dll.toml
#	rules/windows/execution_suspicious_cmd_wmi.toml
#	rules/windows/execution_suspicious_image_load_wmi_ms_office.toml
#	rules/windows/execution_suspicious_pdf_reader.toml
#	rules/windows/execution_suspicious_powershell_imgload.toml
#	rules/windows/execution_suspicious_psexesvc.toml
#	rules/windows/execution_suspicious_short_program_name.toml
#	rules/windows/execution_via_compiled_html_file.toml
#	rules/windows/execution_via_hidden_shell_conhost.toml
#	rules/windows/execution_via_net_com_assemblies.toml
#	rules/windows/execution_via_xp_cmdshell_mssql_stored_procedure.toml
#	rules/windows/impact_volume_shadow_copy_deletion_via_vssadmin.toml
#	rules/windows/initial_access_script_executing_powershell.toml
#	rules/windows/initial_access_suspicious_ms_office_child_process.toml
#	rules/windows/initial_access_suspicious_ms_outlook_child_process.toml
#	rules/windows/initial_access_unusual_dns_service_children.toml
#	rules/windows/initial_access_unusual_dns_service_file_writes.toml
#	rules/windows/initial_access_via_explorer_suspicious_child_parent_args.toml
#	rules/windows/lateral_movement_execution_from_tsclient_mup.toml
#	rules/windows/lateral_movement_local_service_commands.toml
#	rules/windows/lateral_movement_mount_hidden_or_webdav_share_net.toml
#	rules/windows/lateral_movement_rdp_enabled_registry.toml
#	rules/windows/lateral_movement_rdp_tunnel_plink.toml
#	rules/windows/lateral_movement_remote_file_copy_hidden_share.toml
#	rules/windows/lateral_movement_suspicious_rdp_client_imageload.toml
#	rules/windows/lateral_movement_via_startup_folder_rdp_smb.toml
#	rules/windows/persistence_adobe_hijack_persistence.toml
#	rules/windows/persistence_appcertdlls_registry.toml
#	rules/windows/persistence_appinitdlls_registry.toml
#	rules/windows/persistence_evasion_registry_ifeo_injection.toml
#	rules/windows/persistence_gpo_schtask_service_creation.toml
#	rules/windows/persistence_local_scheduled_task_commands.toml
#	rules/windows/persistence_ms_office_addins_file.toml
#	rules/windows/persistence_ms_outlook_vba_template.toml
#	rules/windows/persistence_priv_escalation_via_accessibility_features.toml
#	rules/windows/persistence_registry_uncommon.toml
#	rules/windows/persistence_run_key_and_startup_broad.toml
#	rules/windows/persistence_services_registry.toml
#	rules/windows/persistence_startup_folder_file_written_by_suspicious_process.toml
#	rules/windows/persistence_startup_folder_scripts.toml
#	rules/windows/persistence_suspicious_com_hijack_registry.toml
#	rules/windows/persistence_suspicious_image_load_scheduled_task_ms_office.toml
#	rules/windows/persistence_suspicious_scheduled_task_runtime.toml
#	rules/windows/persistence_suspicious_service_created_registry.toml
#	rules/windows/persistence_system_shells_via_services.toml
#	rules/windows/persistence_user_account_creation.toml
#	rules/windows/persistence_via_application_shimming.toml
#	rules/windows/persistence_via_hidden_run_key_valuename.toml
#	rules/windows/persistence_via_lsa_security_support_provider_registry.toml
#	rules/windows/persistence_via_telemetrycontroller_scheduledtask_hijack.toml
#	rules/windows/persistence_via_update_orchestrator_service_hijack.toml
#	rules/windows/persistence_via_windows_management_instrumentation_event_subscription.toml
#	rules/windows/privilege_escalation_named_pipe_impersonation.toml
#	rules/windows/privilege_escalation_printspooler_service_suspicious_file.toml
#	rules/windows/privilege_escalation_printspooler_suspicious_spl_file.toml
#	rules/windows/privilege_escalation_rogue_windir_environment_var.toml
#	rules/windows/privilege_escalation_uac_bypass_com_clipup.toml
#	rules/windows/privilege_escalation_uac_bypass_com_ieinstal.toml
#	rules/windows/privilege_escalation_uac_bypass_com_interface_icmluautil.toml
#	rules/windows/privilege_escalation_uac_bypass_diskcleanup_hijack.toml
#	rules/windows/privilege_escalation_uac_bypass_dll_sideloading.toml
#	rules/windows/privilege_escalation_uac_bypass_event_viewer.toml
#	rules/windows/privilege_escalation_uac_bypass_mock_windir.toml
#	rules/windows/privilege_escalation_uac_bypass_winfw_mmc_hijack.toml
#	rules/windows/privilege_escalation_unusual_parentchild_relationship.toml
#	rules/windows/privilege_escalation_unusual_svchost_childproc_childless.toml
2021-02-17 12:18:06 -09:00
Justin Ibarra 61deed3fd2 [Rule Tuning] 7.11.2: Add timestamp_override to all query and non-sequence EQL rules (#948)
* [Rule Tuning] Add timestamp_override field to 7.11.0 rules
* Lock versions for 7.11.2 rules
2021-02-16 10:52:48 -09:00
Justin Ibarra a0e86e20d6 [Rule Tuning] Add windows integration index to rules (#923) 2021-01-28 20:53:57 -09:00
Justin Ibarra 97ee8cc9ac Refresh beats and ecs schemas and default to use latest to validate (#570)
* Refresh beats and ecs schemas and default to use latest to validate
* remove incorrect ecs_version from zoom rule
* remove stale ecs_version from rules
2020-12-01 13:24:20 -09:00
Justin Ibarra f87f2a46f4 [Rule Tuning] Remove all rule timelines (#466) 2020-11-03 09:51:53 -09:00
Justin Ibarra da64bacac1 [Rule Tuning] Add timeline_title to rules with timeline IDs defined (#452) 2020-11-02 14:12:20 -09:00
Derek Ditch 580db2c13e Add timeline_id to detection rules (#95)
* Adds timeline_id to all network rules
- Uses the ID for the 'Generic Network Timeline' from Elastic
* Adds timeline_id to all endpoint rules
- Uses the ID for the 'Generic Endpoint Timeline' from Elastic
* Adds timeline_id to all process-oriented rules
    - Uses the ID for the 'Generic Process Timeline' from Elastic
* Ran tests and toml-lint
* Bumped 'updated_date'
2020-10-27 13:34:16 -05:00
seth-goodwin 2065af89b1 [Rule Tuning] Tag Categorization Updates (#380)
* Add new categorization tags

* Change updated_date to 2020/10/26

Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>, @bm11100
2020-10-26 13:50:45 -05:00
Justin Ibarra 065bcd8018 Refresh ATT&CK data to v7.2 and expand threat validation (#330)
* refresh to latest ATT&CK 7.2
* add new unit test to further validate threat mappings
* updated threat mappings in rules to reflect changes
* new func to download and refresh mitre data based on version
2020-09-23 22:03:29 -08:00
brokensound77 aec3ec31b9 Merge branch '7.9' into main 2020-08-27 15:54:44 -08:00
Justin Ibarra 79a0dfefbe Add ECS 1.6.0 schema for validation testing (#220)
* Add ecs 1.6.0 and refresh master ecs (2.0.0)
* update rule metadata to use ecs_version 1.6.0
2020-08-27 11:54:49 -05:00
Justin Ibarra be08536880 Increase lookback for endpoint rules (#200) 2020-08-21 12:23:43 -05:00
Brent Murphy 7efe33e01d [Rule Tuning] Update Index Pattern for Detection Engine Rules (#101)
* [Rule Tuning] Update Index Pattern for Detection Engine Rules

* update indices
2020-08-03 15:46:57 -04:00
Justin Ibarra 95908c22a4 Improve ECS compatibility for endpoint rules 2020-07-07 15:41:23 -06:00
Ross Wolf 5fcece8416 Populate rules/ directory.
Co-Authored-By: Brent Murphy <56412096+bm11100@users.noreply.github.com>
Co-Authored-By: Craig Chamberlain <randomuserid@users.noreply.github.com>
Co-Authored-By: David French <56409778+threat-punter@users.noreply.github.com>
Co-Authored-By: Derek Ditch <dcode@users.noreply.github.com>
Co-Authored-By: Justin Ibarra <brokensound77@users.noreply.github.com>
2020-06-29 22:57:03 -06:00