sigma-rules/docs/experimental-machine-learning/host-risk-score.md

The setup instructions in this document have been deprecated. Please follow the steps outlined here, to enable Host Risk Score in your environment.

Host Risk Score

Host Risk Score is an experimental feature that assigns risk scores to hosts in a given Kibana space. Risk scores are calculated for each host by utilizing transforms on the alerting indices. The transform runs hourly to update the score as new alerts are generated. The Host Risk Score package contains all of the required artifacts for setup. The Host Risk Score feature provides drilldown Lens dashboards and additional Kibana features such as the Host Risk Score Card on the Overview page of the Elastic Security app, and the Host Risk Keyword on the Alert details flyout for an enhanced experience.

Notes

• Host name collision: Hosts are identified by the host.name field in alerts. There may be some edge cases where different hosts use the same name. details

Setup Instructions

1. Obtain artifacts
2. Upload scripts
3. Upload ingest pipeline
4. Upload and start the pivot transform
5. Create the Host Risk Score index
6. Upload and start the latest transform
7. Import dashboards
8. Enable Kibana features

1. Obtain artifacts

The Host Risk Score functionality is space aware for privacy. Downloaded artifacts must be modified with the desired space before they can be used.

• Download the release bundle from here. The Host Risk Score releases can be identified by the tag ML-HostRiskScore-YYYYMMDD-N. Check the release description to make sure it is compatible with the Elastic Stack version you are running.
• Unzip the contents of ML-HostRiskScore-YYYYMMDD-N.zip.
• Run ml_hostriskscore_generate_scripts.py script in the unzipped directory with your Kibana space as the argument.

Example of modifying artifacts for the default space

python ml_hostriskscore_generate_scripts.py --space default

• Find a new folder named after your space in the unzipped directory. You will be using the scripts within this directory for the next steps.

Note: Host Risk Score artifacts should be updated if/when you update to a newer Elastic Stack version. To do this, simply download a release bundle that is compatible with your new Stack version and repeat all the steps. Backwards compatibility of release bundles is not guaranteed.

2. Upload scripts

• Navigate to Management / Dev Tools in Kibana.
• Upload the contents of ml_hostriskscore_levels_script.json, ml_hostriskscore_map_script.json, ml_hostriskscore_reduce_script.json and ml_hostriskscore_init_script.json (for Elastic Stack version 8.1+ only) using the Script API with the following syntax.
• Ensure that your space name (such as default) replaces <your-space-name> in the script names below.

uploading scripts

PUT _scripts/ml_hostriskscore_levels_script_<your-space-name>
{contents of ml_hostriskscore_levels_script.json file}

PUT _scripts/ml_hostriskscore_map_script_<your-space-name>
{contents of ml_hostriskscore_map_script.json file}

PUT _scripts/ml_hostriskscore_reduce_script_<your-space-name>
{contents of ml_hostriskscore_reduce_script.json file}

For Elastic Stack version 8.1+ only

PUT _scripts/ml_hostriskscore_init_script_<your-space-name>
{contents of ml_hostriskscore_init_script.json file}

3. Upload ingest pipeline

• Upload the contents of ml_hostriskscore_ingest_pipeline.json using the Ingest API with the following syntax.
• Ensure that your space name (such as default) replaces <your-space-name> below.

uploading ingest pipeline

PUT _ingest/pipeline/ml_hostriskscore_ingest_pipeline_<your-space-name>
{contents of ml_hostriskscore_ingest_pipeline.json file}

4. Upload and start the pivot transform

This transform calculates the risk level every hour for each host in the Kibana space specified.

• Upload the contents of ml_hostriskscore_pivot_transform.json using the Transform API with the following syntax.
• Ensure that your space name (such as default) replaces <your-space-name> below.

uploading pivot transform

PUT _transform/ml_hostriskscore_pivot_transform_<your-space-name>
{contents of ml_hostriskscore_pivot_transform.json file}

• Navigate to Transforms under Management / Stack Management in Kibana. Find the transform with the ID ml_hostriskscore_pivot_transform_<your-space-name>. Open the Actions menu on the right side of the row, then click Start.
• Confirm the transform is working as expected by navigating to Management / Dev Tools and ensuring the target index exists.

sample test query

GET ml_host_risk_score_<your-space-name>/_search

5. Create the Host Risk Score index

• Navigate to Management / Dev Tools in Kibana.
• Create the Host Risk Score index (ml_host_risk_score_latest_<your-space-name>) with the following mappings.
• Ensure that your space name (such as default) replaces <your-space-name> below.

creating the Host Risk Score index

PUT ml_host_risk_score_latest_<your-space-name>
{
  "mappings":{
    "properties":{
      "host.name":{
        "type":"keyword"
      }
    }
  }
}

6. Upload and start the latest transform

This transform recurringly calculates risk levels for all hosts in the Kibana space specified.

• Upload the contents of ml_hostriskscore_latest_transform.json using the Transform API with the following syntax.
• Ensure that your space name (such as default) replaces <your-space-name> below.

uploading latest transform

PUT _transform/ml_hostriskscore_latest_transform_<your-space-name>
{contents of ml_hostriskscore_latest_transform.json file}

• Navigate to Transforms under Management / Stack Management in Kibana. Find the transform with the ID ml_hostriskscore_latest_transform_<your-space-name>. Open the Actions menu on the right side of the row, and click Start.
• Confirm the transform is working as expected by navigating to Management / Dev Tools and ensuring the target index exists. You should see documents starting to appear in the index if there is ongoing alerting activity associated with hosts.

sample test query

GET ml_host_risk_score_latest_<your-space-name>/_search

7. Import dashboards

• Navigate to Management / Stack Management / Kibana / Saved Objects in Kibana.
• Click on Import and import the ml_hostriskscore_dashboards.ndjson file.
• Navigate to Analytics / Dashboard.
• Confirm you can see a dashboard named Current Risk Scores for Hosts, which displays the current list (Top 20) of suspicious hosts in your environment.
• Confirm you can see a dashboard named Drilldown of Host Risk Score, which allows you to further drill down into details of the risk associated with a particular host of interest.

8. Enable Kibana features

To enable the Kibana features for Host Risk Score, you will first need to add the following configuration to kibana.yml.

xpack.securitySolution.enableExperimental: ['riskyHostsEnabled']

Instructions to modify kibana.yml on Elastic Cloud

1. Navigate to your deployment on the cloud

2. Click on Kibana on the sidebar and click on Edit configuration on your Kibana instance

3. Click on Edit user settings

4. Modify Kibana configuration by adding xpack.securitySolution.enableExperimental: ['riskyHostsEnabled']

5. Save updated Kibana settings

6. Confirm activity finished

7. View Host Risk Score Card on the Overview page

Once you have modified the kibana.yml file, you will find Host Risk Scoring features in the following Kibana locations:

Host Risk Score card on the Overview page

Host Risk Keyword on Alert Details Flyout

For Elastic Stack version 8.1+ only:

Host risk classification column in the All hosts table on the Hosts page

Hosts by risk tab on the Hosts page

The host risk table in the above tab is not affected by the KQL time range. The table shows the latest recorded risk score for each host.

Host risk overview on the Host details page

Hosts by risk tab on the Host details page

---

Host name collision details

Physical Windows clients - desktops and laptops - in an Active Directory forest are unlikely to have name collisions, as their computer accounts and distinguished names should be unique. Non-domain member servers, desktops and laptops, in a Windows workgroup, may occasionally have name collisions. Macs are often not managed by a directory service and may have name collisions. Virtual servers, that are created from templates or cloning processes may have hostname collisions.