Justin Ibarra
0ec8d67e78
* move github and ml to their own files * refactor release and ml commands * update ML readmes * add unzip_to_dict function * prompt for model ID in remove-model * update experimental rule upload process * update remove-scripts-pipelines to take multiple options Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com> Co-authored-by: Apoorva <appujo@gmail.com>
2.8 KiB
Machine Learning on Domain Generation Algorithm (DGA)
To create and use supervised DGA ML models to enrich data within the stack, check out these Elastic blogs:
- Part 1: Machine learning in cybersecurity: Training supervised models to detect DGA activity
- Part 2: Machine learning in cybersecurity: Detecting DGA activity in network data
You can also find some supplementary material and examples here
We also released a blog about getting started with DGA using the CLI and Kibana, which also includes a case study of the process applied to the 2020 SolarWinds supply chain attack:
For questions, please reach out to the ML team in the #machine-learning channel of the Elastic community Slack workspace
The team can also be reached by using the stack-machine-learning tag in the discuss forums
Note: in order to use these ML features, you must have a platinum or higher subscription Note: the ML features are considered experimental in Kibana as well as this rules CLI
Detailed steps
1. Upload and setup the model file and dependencies
Run python -m detection_rules es <args_or_config> experimental ml setup -t <release-tag>
If updating a new model, you should first uninstall any existing models using remove-model
You can also upload files locally using the -d option, so long as the naming convention of the files match the
expected pattern for the filenames.
2. Update packetbeat configuration
You will need to update your packetbeat.yml config file to point to the enrichment pipeline
Under Elasticsearch Output add the following:
output.elasticsearch:
hosts: ["your-hostname:your-port"]
pipeline: dns_enrich_pipeline
3. Refresh your packetbeat index
You can optionally choose to refresh your packetbeat index mapping from within Kibana:
- Navigate to
Stack Management > (Kibana) Index Patterns - Select the appropriate packetbeat index
- Click
refresh field list
4. Verify enrichment fields
Any packetbeat documents with the field dns.question.registered_domain should now be enriched with ml_is_dga.*