move bin scripts into bin, apis into atomic-red-team

CONTRIBUTIONS.md

@@ -1,35 +1,39 @@
-## How to contribute to Atomic Red Team
+# How to contribute to Atomic Red Team
-
-#### **Atomic Contribution**
 
+## Atomic Philosophy
 Atomic Red Team welcomes all types of contributions as long as it is mapped to [MITRE ATT&CK](https://attack.mitre.org/wiki/Main_Page).
 
-The Framework is also meant to be "easy". If your Atomic test is complicated and requires multiple external utilities/packages/Kali, we may dismiss it.
+- Tests are made to be "easy". If your Atomic test is complicated and requires multiple external utilities/packages/Kali, we may dismiss it.
 
-TEST YOUR Atomic Test! Be sure to run it from a few OS platforms before submitting a pull to ensure everything is working correctly.
+- TEST YOUR Atomic Test! Be sure to run it from a few OS platforms before submitting a pull to ensure everything is working correctly.
 
-If sourcing from another tool/product (ex. generated command), be sure to cite it in your .md file.
+- If sourcing from another tool/product (ex. generated command), be sure to cite it in the test's description.
 
-Any and all Payloads need to be placed in the respective Windows|Mac|Linux Payload directory.
+## How to contribute
+Pick the technique you want to add a test for and run the generator:
 
-Be sure you update the ATT&CK url, Txxxx number, and the title (ex. InstallUtil).
+```
+bin/new-atomic.rb T1234
+```
 
+This makes a new test for the technique with a bunch of TBDs you'll fill in and opens up your editor
+so you can get to work.
 
-#### Atomic Template Example
+Fill in the TBDs with the information for your test. Read the [Atomic Red Team YAML Spec](atomic-red-team/spec.yaml)
+for complete details about what each field means and a list of possible values.
 
+Validate that your Atomic Test is up to code!
 
-    ## InstallUtil
+```
+bin/validate-atomics.rb
+```
 
-    MITRE ATT&CK Technique: [T1118](https://attack.mitre.org/wiki/Technique/T1118)
+Submit a pull request once your test is complete and everything validates.
 
-    ### Execution Examples:
+## Generating Atomic docs yourself (optional)
+If you want to see what the pretty Markdown version of your Atomic Test is going to look like, 
+you can generate the Atomic Docs yourself:
 
-    Input:
+```
-
+bin/generate-atomic-docs.rb
-        x86 - C:\Windows\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe /logfile= /LogToConsole=false /U AllTheThings.dll
+```
-
-        x64 - C:\Windows\Microsoft.NET\Framework64\v4.0.30319\InstallUtil.exe /logfile= /LogToConsole=false /U AllTheThings.dll
-
-    ## Test Script
-
-    [InstallUtilBypass.cs](https://github.com/redcanaryco/atomic-red-team/blob/master/Windows/Payloads/InstallUtilBypass.cs)

LICENSE.txt

@@ -1,7 +1,7 @@
 
 The MIT License
 
-Copyright (c) 2016 Red Canary, Inc.
+Copyright (c) 2018 Red Canary, Inc.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

atomic-red-team/atomic_red_team.rb

@@ -0,0 +1,130 @@
+#! /usr/bin/env ruby
+require 'yaml'
+require 'erb'
+require 'attack_api'
+
+class AtomicRedTeam
+  ATTACK_API = Attack.new
+
+  ATOMICS_DIRECTORY = "#{File.dirname(File.dirname(__FILE__))}/atomics"
+
+  # TODO- should these all be relative URLs?
+  ROOT_GITHUB_URL = "https://github.com/redcanaryco/atomic-red-team"
+  
+  #
+  # Returns a list of paths that contain Atomic Tests
+  #
+  def atomic_test_paths
+    Dir["#{ATOMICS_DIRECTORY}/t*/t*.yaml"].sort
+  end
+
+  #
+  # Returns a list of Atomic Tests in Atomic Red Team (as Hashes from source YAML) 
+  #
+  def atomic_tests
+    @atomic_tests ||= atomic_test_paths.collect do |path| 
+      atomic_yaml = YAML.load(File.read path)
+      atomic_yaml['atomic_yaml_path'] = path
+      atomic_yaml
+    end
+  end
+
+  #
+  # Returns the individual Atomic Tests for a given identifer, passed as either a string (T1234) or an ATT&CK technique object
+  #
+  def atomic_tests_for_technique(technique_or_technique_identifier)
+    technique_identifier = if technique_or_technique_identifier.is_a? Hash
+      ATTACK_API.technique_identifier_for_technique technique_or_technique_identifier
+    else
+      technique_or_technique_identifier
+    end
+
+    atomic_tests.find do |atomic_yaml| 
+      atomic_yaml.fetch('attack_technique').downcase == technique_identifier.downcase
+    end.to_h.fetch('atomic_tests', [])
+  end
+
+  #
+  # Returns a Markdown formatted Github link to a technique. This will be to the edit page for 
+  # techniques that already have one or more Atomic Red Team tests, or the create page for
+  # techniques that have no existing tests.
+  #
+  def github_link_to_technique(technique, include_identifier=false)
+    technique_identifier = ATTACK_API.technique_identifier_for_technique(technique).downcase
+    link_display = "#{"#{technique_identifier.upcase} " if include_identifier}#{technique['name']}"
+
+    if File.exists? "#{ATOMICS_DIRECTORY}/#{technique_identifier}/#{technique_identifier}.md"
+      # we have a file for this technique, so link to it's Markdown file
+      "[#{link_display}](#{ROOT_GITHUB_URL}/tree/master/atomics/#{technique_identifier}/#{technique_identifier}.md)"
+    else
+      # we don't have a file for this technique, so link to an edit page
+      "[#{link_display}](#{ROOT_GITHUB_URL}/edit/master/atomics/#{technique_identifier}/#{technique_identifier}.md)"
+    end
+  end
+
+  def validate_atomic_yaml!(yaml)
+    raise("YAML file has no elements") if yaml.nil?
+  
+    raise('`attack_technique` element is required') unless yaml.has_key?('attack_technique')
+    raise('`attack_technique` element must be an array') unless yaml['attack_technique'].is_a?(String)
+  
+    raise('`display_name` element is required') unless yaml.has_key?('display_name')
+    raise('`display_name` element must be an array') unless yaml['display_name'].is_a?(String)
+  
+    raise('`atomic_tests` element is required') unless yaml.has_key?('atomic_tests')
+    raise('`atomic_tests` element must be an array') unless yaml['atomic_tests'].is_a?(Array)
+    raise('`atomic_tests` element is empty - you have no tests') unless yaml['atomic_tests'].count > 0
+  
+    yaml['atomic_tests'].each_with_index do |atomic, i|
+      raise("`atomic_tests[#{i}].name` element is required") unless atomic.has_key?('name')
+      raise("`atomic_tests[#{i}].name` element must be a string") unless atomic['name'].is_a?(String)
+  
+      raise("`atomic_tests[#{i}].description` element is required") unless atomic.has_key?('description')
+      raise("`atomic_tests[#{i}].description` element must be a string") unless atomic['description'].is_a?(String)
+  
+      raise("`atomic_tests[#{i}].supported_platforms` element is required") unless atomic.has_key?('supported_platforms')
+      raise("`atomic_tests[#{i}].supported_platforms` element must be an Array (was a #{atomic['supported_platforms'].class.name})") unless atomic['supported_platforms'].is_a?(Array)
+  
+      valid_supported_platforms = ['windows', 'centos', 'ubuntu', 'macos', 'linux']
+      atomic['supported_platforms'].each do |platform|
+        if !valid_supported_platforms.include?(platform)
+          raise("`atomic_tests[#{i}].supported_platforms` '#{platform}' must be one of #{valid_supported_platforms.join(', ')}")
+        end
+      end
+  
+      (atomic['input_arguments'] || {}).each_with_index do |arg_kvp, iai|
+        arg_name, arg = arg_kvp
+        raise("`atomic_tests[#{i}].input_arguments[#{iai}].description` element is required") unless arg.has_key?('description')
+        raise("`atomic_tests[#{i}].input_arguments[#{iai}].description` element must be a string") unless arg['description'].is_a?(String)
+  
+        raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element is required") unless arg.has_key?('type')
+        raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element must be a string") unless arg['type'].is_a?(String)
+        raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element must be lowercased and underscored (was #{arg['type']})") unless arg['type'] =~ /[a-z_]+/
+  
+        # TODO: determine if we think default values are required for EVERY input argument
+        # raise("`atomic_tests[#{i}].input_arguments[#{iai}].default` element is required") unless arg.has_key?('default')
+        # raise("`atomic_tests[#{i}].input_arguments[#{iai}].default` element must be a string (was a #{arg['default'].class.name})") unless arg['default'].is_a?(String)
+      end
+  
+      raise("`atomic_tests[#{i}].executor` element is required") unless atomic.has_key?('executor')
+      executor = atomic['executor']
+      raise("`atomic_tests[#{i}].executor.name` element is required") unless executor.has_key?('name')
+      raise("`atomic_tests[#{i}].executor.name` element must be a string") unless executor['name'].is_a?(String)
+      raise("`atomic_tests[#{i}].executor.name` element must be lowercased and underscored (was #{executor['name']})") unless executor['name'] =~ /[a-z_]+/
+  
+      valid_executor_types = ['command_prompt', 'sh', 'bash', 'powershell', 'manual']
+      case executor['name']
+        when 'manual'
+          raise("`atomic_tests[#{i}].executor.steps` element is required") unless executor.has_key?('steps')
+          raise("`atomic_tests[#{i}].executor.steps` element must be a string") unless executor['steps'].is_a?(String)
+  
+        when 'command_prompt', 'sh', 'bash', 'powershell'
+          raise("`atomic_tests[#{i}].executor.command` element is required") unless executor.has_key?('command')
+          raise("`atomic_tests[#{i}].executor.command` element must be a string") unless executor['command'].is_a?(String)
+  
+        else
+          raise("`atomic_tests[#{i}].executor.name` '#{executor['name']}' must be one of #{valid_executor_types.join(', ')}")
+      end
+    end
+  end
+end

atomic-red-team/atomic_test_template.yaml

@@ -20,7 +20,7 @@ atomic_tests:
       type: todo
       default: TODO
 
-  executors:
+  executor:
-    name: TODO
+    name: command_prompt
     command: |
       TODO

atomic_red_team.rb

@@ -1,55 +0,0 @@
-#! /usr/bin/env ruby
-require 'yaml'
-require 'erb'
-require './attack_api'
-
-class AtomicRedTeam
-  ATTACK_API = Attack.new
-
-  # TODO- should these all be relative URLs?
-  ROOT_GITHUB_URL = "https://github.com/redcanaryco/atomic-red-team"
-  
-  #
-  # Returns a list of Atomic Tests in Atomic Red Team (as Hashes from source YAML) 
-  #
-  def atomic_tests
-    @atomic_tests ||= Dir["#{File.dirname(__FILE__)}/atomics/t*/t*.yaml"].sort.collect do |path| 
-      atomic_yaml = YAML.load(File.read path)
-      atomic_yaml['atomic_yaml_path'] = path
-      atomic_yaml
-    end
-  end
-
-  #
-  # Returns the individual Atomic Tests for a given identifer, passed as either a string (T1234) or an ATT&CK technique object
-  #
-  def atomic_tests_for_technique(technique_or_technique_identifier)
-    technique_identifier = if technique_or_technique_identifier.is_a? Hash
-      ATTACK_API.technique_identifier_for_technique technique_or_technique_identifier
-    else
-      technique_or_technique_identifier
-    end
-
-    atomic_tests.find do |atomic_yaml| 
-      atomic_yaml.fetch('attack_technique').downcase == technique_identifier.downcase
-    end.to_h.fetch('atomic_tests', [])
-  end
-
-  #
-  # Returns a Markdown formatted Github link to a technique. This will be to the edit page for 
-  # techniques that already have one or more Atomic Red Team tests, or the create page for
-  # techniques that have no existing tests.
-  #
-  def github_link_to_technique(technique, include_identifier=false)
-    technique_identifier = ATTACK_API.technique_identifier_for_technique(technique).downcase
-    link_display = "#{"#{technique_identifier.upcase} " if include_identifier}#{technique['name']}"
-
-    if File.exists? "#{File.dirname(__FILE__)}/atomics/#{technique_identifier}/#{technique_identifier}.md"
-      # we have a file for this technique, so link to it's Markdown file
-      "[#{link_display}](#{ROOT_GITHUB_URL}/tree/master/atomics/#{technique_identifier}/#{technique_identifier}.md)"
-    else
-      # we don't have a file for this technique, so link to an edit page
-      "[#{link_display}](#{ROOT_GITHUB_URL}/edit/master/atomics/#{technique_identifier}/#{technique_identifier}.md)"
-    end
-  end
-end

bin/generate-atomic-docs.rb

@@ -1,7 +1,8 @@
 #! /usr/bin/env ruby
+$LOAD_PATH << "#{File.dirname(File.dirname(__FILE__))}/atomic-red-team"
 require 'erb'
-require './attack_api'
+require 'attack_api'
-require './atomic_red_team'
+require 'atomic_red_team'
 
 class AtomicRedTeamDocs
   ATTACK_API = Attack.new
@@ -26,9 +27,10 @@ class AtomicRedTeamDocs
         puts "FAIL\n#{ex}\n#{ex.backtrace.join("\n")}"
       end
     end
-
+    puts
-    generate_attack_matrix! "#{File.dirname(__FILE__)}/atomics/matrix.md"
+    puts "Generated docs for #{oks.count} techniques, #{fails.count} failures"
-    generate_index! "#{File.dirname(__FILE__)}/atomics/index.md"
+    generate_attack_matrix! "#{File.dirname(File.dirname(__FILE__))}/atomics/matrix.md"
+    generate_index! "#{File.dirname(File.dirname(__FILE__))}/atomics/index.md"
     
     return oks, fails
   end
@@ -40,7 +42,7 @@ class AtomicRedTeamDocs
     technique = ATTACK_API.technique_info(atomic_yaml.fetch('attack_technique'))
     technique['identifier'] = atomic_yaml.fetch('attack_technique').upcase
 
-    template = ERB.new File.read("#{File.dirname(__FILE__)}/atomics/atomic_doc_template.md.erb"), nil, "-"
+    template = ERB.new File.read("#{File.dirname(File.dirname(__FILE__))}/atomic-red-team/atomic_doc_template.md.erb"), nil, "-"
     generated_doc = template.result(binding)
 
     print " => #{output_doc_path} => "
@@ -63,6 +65,8 @@ class AtomicRedTeamDocs
       result += "| #{row_values.join(' | ')} |\n"
     end
     File.write output_doc_path, result
+
+    puts "Generated ATT&CK matrix at #{output_doc_path}"
   end
 
   #
@@ -83,6 +87,8 @@ class AtomicRedTeamDocs
     end
 
     File.write output_doc_path, result
+
+    puts "Generated Atomic Red Team index at #{output_doc_path}"
   end
 end
 
@@ -90,7 +96,5 @@ end
 # MAIN
 #
 oks, fails = AtomicRedTeamDocs.new.generate_all_the_docs!
-puts
-puts "Generated docs for #{oks.count} techniques, #{fails.count} failures"
 
 exit fails.count

bin/generate-docs

@@ -1,8 +0,0 @@
-#!/usr/bin/env ruby
-require 'ostruct'
-require 'yaml'
-
-Dir["#{File.dirname __FILE__}/../atomics/**/t*.yaml"].each do |technique_file|
-  technique = OpenStruct.new YAML.load(File.read(technique_file))
-  p technique.display_name
-end

bin/validate-atomics.rb

@@ -0,0 +1,29 @@
+#! /usr/bin/env ruby
+$LOAD_PATH << "#{File.dirname(File.dirname(__FILE__))}/atomic-red-team"
+require 'yaml'
+require 'atomic_red_team'
+
+ATOMIC_RED_TEAM = AtomicRedTeam.new
+ATOMIC_TEST_TEMPLATE = "#{File.dirname(File.dirname(__FILE__))}/atomic-red-team/atomic_test_template.yaml"
+
+oks = []
+fails = []
+
+(ATOMIC_RED_TEAM.atomic_test_paths + [ATOMIC_TEST_TEMPLATE]).each do |path|
+  begin
+    print "Validating #{path}..."
+    YAML.load_file(path) rescue raise 'Invalid YAML'
+    AtomicRedTeam.new.validate_atomic_yaml! YAML.load_file(path)
+
+    oks << path
+    puts "OK"
+  rescue => ex
+    fails << path
+    puts "FAIL\n#{ex}\n#{ex.backtrace.join("\n")})"
+  end
+end
+
+puts
+puts "#{oks.count + fails.count} techniques, #{fails.count} failures"
+
+exit fails.count

validate_atomics.rb

@@ -1,101 +0,0 @@
-#! /usr/bin/env ruby
-require 'yaml'
-
-def validate_is_yaml!(path)
-  YAML.load_file(path)
-rescue
-  raise 'Invalid YAML'
-end
-
-def validate_is_atomic!(path)
-  yaml = YAML.load_file(path)
-  raise("YAML file has no elements") if yaml.nil?
-
-  raise('`attack_technique` element is required') unless yaml.has_key?('attack_technique')
-  raise('`attack_technique` element must be an array') unless yaml['attack_technique'].is_a?(String)
-
-  raise('`display_name` element is required') unless yaml.has_key?('display_name')
-  raise('`display_name` element must be an array') unless yaml['display_name'].is_a?(String)
-
-  raise('`atomic_tests` element is required') unless yaml.has_key?('atomic_tests')
-  raise('`atomic_tests` element must be an array') unless yaml['atomic_tests'].is_a?(Array)
-  raise('`atomic_tests` element is empty - you have no tests') unless yaml['atomic_tests'].count > 0
-
-  yaml['atomic_tests'].each_with_index do |atomic, i|
-    raise("`atomic_tests[#{i}].name` element is required") unless atomic.has_key?('name')
-    raise("`atomic_tests[#{i}].name` element must be a string") unless atomic['name'].is_a?(String)
-
-    raise("`atomic_tests[#{i}].description` element is required") unless atomic.has_key?('description')
-    raise("`atomic_tests[#{i}].description` element must be a string") unless atomic['description'].is_a?(String)
-
-    raise("`atomic_tests[#{i}].supported_platforms` element is required") unless atomic.has_key?('supported_platforms')
-    raise("`atomic_tests[#{i}].supported_platforms` element must be an Array (was a #{atomic['supported_platforms'].class.name})") unless atomic['supported_platforms'].is_a?(Array)
-
-    valid_supported_platforms = ['windows', 'centos', 'ubuntu', 'macos', 'linux']
-    atomic['supported_platforms'].each do |platform|
-      if !valid_supported_platforms.include?(platform)
-        raise("`atomic_tests[#{i}].supported_platforms` '#{platform}' must be one of #{valid_supported_platforms.join(', ')}")
-      end
-    end
-
-    (atomic['input_arguments'] || {}).each_with_index do |arg_kvp, iai|
-      arg_name, arg = arg_kvp
-      raise("`atomic_tests[#{i}].input_arguments[#{iai}].description` element is required") unless arg.has_key?('description')
-      raise("`atomic_tests[#{i}].input_arguments[#{iai}].description` element must be a string") unless arg['description'].is_a?(String)
-
-      raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element is required") unless arg.has_key?('type')
-      raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element must be a string") unless arg['type'].is_a?(String)
-      raise("`atomic_tests[#{i}].input_arguments[#{iai}].type` element must be lowercased and underscored (was #{arg['type']})") unless arg['type'] =~ /[a-z_]+/
-
-      # TODO: determine if we think default values are required for EVERY input argument
-      # raise("`atomic_tests[#{i}].input_arguments[#{iai}].default` element is required") unless arg.has_key?('default')
-      # raise("`atomic_tests[#{i}].input_arguments[#{iai}].default` element must be a string (was a #{arg['default'].class.name})") unless arg['default'].is_a?(String)
-    end
-
-    raise("`atomic_tests[#{i}].executor` element is required") unless atomic.has_key?('executor')
-    executor = atomic['executor']
-    raise("`atomic_tests[#{i}].executor.name` element is required") unless executor.has_key?('name')
-    raise("`atomic_tests[#{i}].executor.name` element must be a string") unless executor['name'].is_a?(String)
-    raise("`atomic_tests[#{i}].executor.name` element must be lowercased and underscored (was #{executor['name']})") unless executor['name'] =~ /[a-z_]+/
-
-    valid_executor_types = ['command_prompt', 'sh', 'bash', 'powershell', 'manual']
-    case executor['name']
-      when 'manual'
-        raise("`atomic_tests[#{i}].executor.steps` element is required") unless executor.has_key?('steps')
-        raise("`atomic_tests[#{i}].executor.steps` element must be a string") unless executor['steps'].is_a?(String)
-
-      when 'command_prompt', 'sh', 'bash', 'powershell'
-        raise("`atomic_tests[#{i}].executor.command` element is required") unless executor.has_key?('command')
-        raise("`atomic_tests[#{i}].executor.command` element must be a string") unless executor['command'].is_a?(String)
-
-      else
-        raise("`atomic_tests[#{i}].executor.name` '#{executor['name']}' must be one of #{valid_executor_types.join(', ')}")
-    end
-  end
-end
-
-oks = []
-fails = []
-
-(Dir["#{File.dirname(__FILE__)}/atomics/t*/t*.yaml"] + 
- Dir["#{File.dirname(__FILE__)}/atomics/template.yaml"]).sort.each do |path|
-  begin
-    print "Validating #{path}..."
-    validate_is_yaml! path
-    validate_is_atomic! path
-
-    puts "OK"
-  rescue => ex
-    fails << path
-    if ENV['DEBUG'] == 'true'
-      puts "FAIL (#{ex} #{ex.backtrace.join("\n")})"
-    else
-      puts "FAIL (#{ex})"
-    end
-  end
-end
-
-puts
-puts "#{oks.count + fails.count} techniques, #{fails.count} failures"
-
-exit fails.count