Commit Graph

43 Commits

Author SHA1 Message Date
Mei Liu cbda88fcbb Example:
-O:
attackMapFile: It's used to set subFunction in XML rule. It's a map of subFunction and tags.attack in YML file.
ruleIndex: It's used to set rule id in XML rule. The format of rule id is PH_Rule_{ruleType}_SIGMA_{ruleIndex}
ruleType: It's used to set rule id in XML rule.

1. Generate rule for one YML file
    a. tools/sigmac -t fortisiem -c fortisiem-windows rules/windows/network_connection/win_net_python.yml
    b. tools/sigmac -t fortisiem -c fortisiem-windows -O attackMapFile=tools/config/fortisiem/MITRE-Attack-matrix.csv -O ruleIndex=0 -O ruleType=Windows rules/windows/network_connection/win_net_python.yml
   Output:
      <Rules>
      <Rule group="PH_SYS_RULE_THREAT_HUNTING" natural_id="PH_Rule_Windows_SIGMA_0"  phIncidentCategory="Server" function="Security" subFunction="Discovery" technique="T1046">
         <Name>Python Initiated Connection </Name>
         <IncidentTitle>Python Initiated Connection</IncidentTitle>
         <active>true</active>
         <Description> Adversaries may attempt to get a listing of services running on remote hosts, including those that may be vulnerable to remote software exploitation </Description>
         <SigmaFileName> rules/windows/network_connection/win_net_python.yml </SigmaFileName>
         <CustomerScope groupByEachCustomer="true">
            <Include all="true"/>
           <Exclude/>
         </CustomerScope>
         <IncidentDef eventType="PH_RULE_Python_Initiated_Connection" severity="7">
           <ArgList> compEventType = Filter.eventType,hostName = Filter.hostName,isInitialed = Filter.isInitialed,procName = Filter.procName </ArgList>
         </IncidentDef>
         <PatternClause window="300">
           <SubPattern displayName="Filter" name="Filter">
               <SingleEvtConstr> eventType REGEXP ( "Win-Sysmon-3-Network-Connect.*" ) AND isInitialed="true" AND procName REGEXP ( ".*python.*" ) </SingleEvtConstr>
               <GroupByAttr> eventType,hostName,isInitialed,procName </GroupByAttr>
               <GroupEvtConstr> COUNT(*) &gt;= 1 </GroupEvtConstr>
           </SubPattern>
         </PatternClause>
         <TriggerEventDisplay>
           <AttrList> phRecvTime,hostName,isInitialed,procName,rawEventMsg </AttrList>
         </TriggerEventDisplay>
       </Rule>
       </Rules>

2. Generate rules for YML files under rules/windows
   a. tools/sigmac -t fortisiem -c fortisiem-windows -r rules/windows -o rule.xml
   b. tools/sigmac -t fortisiem -c fortisiem-windows -r rules/windows -O attackMapFile=tools/config/fortisiem/MITRE-Attack-matrix.csv -O ruleIndex=0 -O ruleType=Windows -o rule.xml
   Generate rules for YML files under rules/windows

3. Find files that is modified after some date.
  a. tools/sigmac --lists-files-after-date 2020/06/04 rules/windows/wmi_event/sysmon_wmi_susp_scripting.yml
  b. tools/sigmac --lists-files-after-date 2020/06/04 -r rules/windows/
  Output:
     rules/windows/wmi_event/sysmon_wmi_susp_scripting.yml, Updated
     rules/windows/wmi_event/TestFile.yml, No date
2022-03-08 17:16:08 -08:00
Mei Liu c5e72a9446 Example:
-O:
attackMapFile: It's used to set subFunction in XML rule. It's a map of subFunction and tags.attack in YML file.
ruleIndex: It's used to set rule id in XML rule. The format of rule id is PH_Rule_{ruleType}_SIGMA_{ruleIndex}
ruleType: It's used to set rule id in XML rule.

1. Generate rule for one YML file
    a. tools/sigmac -t fortisiem -c fortisiem-windows rules/windows/sysmon/sysmon_config_modification_error.yml
    b. tools/sigmac -t fortisiem -c fortisiem-windows -O attackMapFile=/opt/phoenix/data-definition/MITRE-Attack-matrix.csv -O ruleIndex=0 -O ruleType=Windows rules/windows/sysmon/sysmon_config_modification_error.yml
   Output:
         <Rule group="PH_SYS_RULE_THREAT_HUNTING" id="PH_Rule_SIGMA_1"  phIncidentCategory="Server" function="Security" subFunction="Persistence" technique="T1564">
           <Name>Sysmon Configuration Error </Name>
           <IncidentTitle>Sysmon Configuration Error</IncidentTitle>
           <active>true</active>
           <Description> Someone try to hide from Sysmon </Description>
           <SigmaFileName> rules/windows/sysmon/sysmon_config_modification_error.yml </SigmaFileName>
           <CustomerScope groupByEachCustomer="true">
               <Include all="true"/>
               <Exclude/>
           </CustomerScope>
           <IncidentDef eventType="PH_RULE_Sysmon_Configuration_Error" severity="7">
               <ArgList> description = Filter.description,hostName = Filter.hostName </ArgList>
           </IncidentDef>
           <PatternClause window="300">
              <SubPattern displayName="Filter" name="Filter">
                  <SingleEvtConstr> description REGEXP ( ".*Failed to connect to the driver to update configuration.*|.*Failed to open service configuration with error.*" ) AND (description NOT REGEXP ( ".*Failed to open service configuration with error.*" ) OR description NOT REGEXP ( ".*Last error: The media is write protected\\..*" )) </SingleEvtConstr>
                  <GroupByAttr> description,hostName </GroupByAttr>
                  <GroupEvtConstr> COUNT(*) &gt;= 1 </GroupEvtConstr>
              </SubPattern>
          </PatternClause>
          <TriggerEventDisplay>
              <AttrList> phRecvTime,hostName,description,rawEventMsg </AttrList>
          </TriggerEventDisplay>
      </Rule>

2. Generate rules for YML files under rules/windows
   a. tools/sigmac -t fortisiem -c fortisiem-windows -r rules/windows
   b. tools/sigmac -t fortisiem -c fortisiem-windows -r rules/windows -O attackMapFile=/opt/phoenix/data-definition/MITRE-Attack-matrix.csv -O ruleIndex=0 -O ruleType=Windows
   Generate rules for YML files under rules/windows
2022-02-14 15:51:45 -08:00
Thomas Patzke 76c02a14b2 Merge pull request #1558 from maketsi/splunk-search-ext
Added ability to define free-text searches in the logsource mapping
2021-10-16 20:49:14 +02:00
frack113 4e895da471 fix error "has no len()" 2021-08-20 09:20:56 +02:00
Wietze 7ba375dea0 Optimising lists/subexpressions with length 1
Should reduce brackets on some output targets
2021-08-11 18:00:09 +01:00
Markku Parviainen 900263315a Added support for free-text search in logsources configuration, enabling usage of splunk macros and ability to optimize the resulting searches. 2021-06-16 14:52:45 +03:00
Joshua Roys 7923852cc3 Elastic: raise an error from the base backend if a rule has multiple conditions 2021-03-31 16:01:05 -04:00
Thomas Patzke daf7ab5ff7 Cleanup: removal of corelight_* backends 2020-05-24 22:41:38 +02:00
Thomas Patzke d45f8e19fe Fixes 2020-05-24 21:46:55 +02:00
Thomas Patzke 24b08bbf30 Merge branch 'master' of https://github.com/socprime/sigma into socprime-master 2020-05-24 17:06:32 +02:00
vh fb9c5841f4 Added Humio, Crowdstrike, Corelight 2020-05-08 13:41:52 +03:00
pdr9rc aa175a7d5b wip
wip
2020-05-04 18:02:27 +01:00
pdr9rc dd9e128a15 kibana target update
kibana target now compatible with overrides
2020-05-04 17:35:12 +01:00
pdr9rc b3194e66c4 Update base.py 2020-05-04 16:37:36 +01:00
pdr9rc 98391f985a wip
wip
2020-04-30 15:19:38 +01:00
pdr9rc 9ce84a38e5 overrides section support + one example rule + cloudtrail config
ditto
2020-04-29 20:36:45 +01:00
Thomas Patzke 54c75167ce Default configurations for backends 2019-11-03 23:32:50 +01:00
Thomas Patzke 30948b9c1a Added sigma-similarity tool
Fixed also bug in backend base class that was triggered by the way
backends are used by this tool.
2019-10-25 21:59:03 +02:00
Thomas Patzke 805c739611 Merge branch 'devel-modifiers' 2019-07-31 23:44:10 +02:00
Thomas Patzke 31c6ffcb61 No escaping for typed values 2019-07-31 23:43:29 +02:00
Thomas Patzke 1bb29dca26 Implemented type modifiers and regular expressions 2019-07-15 22:52:10 +02:00
Florian GAULTIER 6bf010fb4b introduce elastalert-dsl
(cherry picked from commit 0235ec23200e62766d9f21fbd26ed834991a0b61)
2019-05-27 17:18:19 +02:00
Thomas Patzke af0bd1b082 Removed debug code from backend option handling
Additionally: code simplification
2019-05-21 00:21:52 +02:00
Thomas Patzke eb022f3908 Conditional field mapping for null values
Fixes #326
2019-04-25 23:24:05 +02:00
Thomas Patzke d0bd8a2a41 Mandatory configuration for most backends 2019-04-22 23:40:21 +02:00
Thomas Patzke 5e973a6321 Fixes and CI testing of --backend-config 2019-03-15 23:46:38 +01:00
Thomas Patzke 0864d05aa5 Merge branch 'backend-config-file' of https://github.com/christophetd/sigma into christophetd-backend-config-file 2019-03-15 23:35:11 +01:00
christophetd 3a7160d52b Accept backend options from a configuration file (closes #213) 2019-02-23 13:20:20 +01:00
Thomas Patzke a9cf14438c Merge branch 'master' into project-1 2019-01-14 22:36:15 +01:00
Thomas Patzke ffd43823cf Fixed wildcard issue in es-qs backend and depending
See GitHub issue #194. Fix for es-dsl is pending.
2018-12-19 00:33:12 +01:00
Daniel Roethlisberger 87aa1b5521 Move optimizer to sigma.parser.condition to enable it for all backends 2018-10-03 00:24:31 +02:00
Daniel Roethlisberger cd3661b60c Fix optimization of NOT corner cases 2018-10-02 22:48:33 +02:00
Daniel Roethlisberger bed88cf813 Make uniq work for lists within definitions 2018-10-02 22:12:54 +02:00
Daniel Roethlisberger 7165128fa5 Remove None from AST - fixes None-related test failures 2018-10-02 21:44:37 +02:00
Daniel Roethlisberger 2242fc5ac8 Optimize the boolean expressions in the AST before generating output
Add code optimizing the boolean expressions in the abstract syntax tree
before generating output using the backend.

The main idea behind optimizing the AST is that less repeated terms is
generally better for backend performance.  This is especially relevant
to backends that do not perform any query language optimization down
the road, such as those that generate code.

The following optimizations are currently performed:

-   Removal of empty OR(), AND()
-   OR(X), AND(X)                 =>  X
-   OR(X, X, ...), AND(X, X, ...) =>  OR(X, ...), AND(X, ...)
-   OR(X, OR(Y))                  =>  OR(X, Y)
-   OR(AND(X, ...), AND(X, ...))  =>  AND(X, OR(AND(...), AND(...)))
-   NOT(NOT(X))                   =>  X

A common example for when these suboptimal rules actually occur in
practice is when a rule has multiple alternative detections that are
OR'ed together in the condition, and all of the detections include a
common element, such as the same EventID.

This implementation is not optimized for performance and will perform
poorly on very large expressions.
2018-10-02 21:14:25 +02:00
Thomas Patzke 1d7722c1cb Added configuration and field mapping chains
Missing: field name mapping of log source conditions.
2018-08-27 00:17:27 +02:00
Thomas Patzke af9f636199 Removal of backend output classes
Breaking change: Instead of feeding the output class with the results,
they are now returned as strings (*Backend.generate()) or list
(SigmaCollectionParser.generate()). Users of the library must now take
care of the output to the terminal, files or wherever Sigma rules should
be pushed to.
2018-08-02 22:41:32 +02:00
Thomas Patzke df74460629 Fixed imports after config split 2018-07-27 23:54:18 +02:00
Thomas Patzke 1c4c67053c Fixes for parser split
* Fixed imports
* Rename
2018-07-27 00:02:07 +02:00
Thomas Patzke b76fa884ec Changed copyright notices accordingly 2018-07-24 00:01:16 +02:00
Thomas Patzke c8e21b3f24 Fixing after split
* Fixing imports
* Discovery in new sub modules
2018-07-21 01:09:02 +02:00
Thomas Patzke d340487e94 Removal from sigma.backends.base 2018-07-10 23:44:14 +02:00
Thomas Patzke 0c93040da5 Splitting backends - Copy base.py 2018-07-10 23:15:04 +02:00