Initial commit: prowler-ops
This commit is contained in:
ghstshdw
@@ -0,0 +1,193 @@
|
||||
# GreySec Prowler Ops
|
||||
|
||||
Scan → Report → Remediate pipeline for AWS cloud security.
|
||||
|
||||
## What this does
|
||||
|
||||
```
|
||||
AWS Environment
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Prowler v5 (Docker) │ ← Runs on a cron schedule (EventBridge, cron, or manual)
|
||||
│ Read-only IAM scan │
|
||||
└──────────┬──────────┘
|
||||
│ JSON findings
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Python Consumer │ ← Parses findings, stores in SQLite, enqueues remediation
|
||||
│ (prowler_consumer) │
|
||||
└──────────┬──────────┘
|
||||
│ FAIL findings
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Remediation Engine │ ← Maps findings → AWS API calls (dry-run by default)
|
||||
│ (remediator) │
|
||||
└──────────┬──────────┘
|
||||
│ Applied changes
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ HTML/JSON Reports │ ← Compliance summaries, finding details
|
||||
│ (report.py) │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Prerequisites
|
||||
|
||||
```bash
|
||||
# Install Python deps
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install boto3 pyyaml watchdog jinja2
|
||||
|
||||
# Make scripts executable
|
||||
chmod +x prowler-runner.sh
|
||||
```
|
||||
|
||||
### 2. AWS credentials
|
||||
|
||||
Prowler and the consumer read from `~/.aws/` mounted read-only inside the container.
|
||||
Make sure you have valid AWS credentials in `~/.aws/credentials`:
|
||||
|
||||
```
|
||||
[default]
|
||||
aws_access_key_id = AKIA...
|
||||
aws_secret_access_key = ...
|
||||
```
|
||||
|
||||
To scan a specific IAM role (cross-account):
|
||||
```bash
|
||||
aws sts assume-role --role-arn arn:aws:iam::123456789012:role/ProwlerScanningRole
|
||||
```
|
||||
Set the resulting `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`
|
||||
in `~/.aws/credentials` under a named profile, then reference it in `.env`.
|
||||
|
||||
### 3. First scan
|
||||
|
||||
```bash
|
||||
# One-shot scan, dry-run (read-only)
|
||||
./prowler-runner.sh
|
||||
|
||||
# Or via Docker directly:
|
||||
docker run --rm \
|
||||
-v ~/.aws:/root/.aws:ro \
|
||||
-v ./output:/output \
|
||||
prowler/cloud:5 \
|
||||
prowler aws -f json -o /output/prowler-findings.json
|
||||
```
|
||||
|
||||
### 4. Consumer
|
||||
|
||||
```bash
|
||||
# Process all existing findings (dry-run remediation)
|
||||
python consumer/prowler_consumer.py --once
|
||||
|
||||
# Run as daemon (watches output dir, processes new files as they appear)
|
||||
python consumer/prowler_consumer.py --daemon
|
||||
|
||||
# Print stats
|
||||
python consumer/prowler_consumer.py --stats
|
||||
|
||||
# Actually apply remediation changes (DANGEROUS)
|
||||
python consumer/prowler_consumer.py --daemon --apply
|
||||
```
|
||||
|
||||
### 5. Remediation Daemon
|
||||
|
||||
```bash
|
||||
# Log what WOULD change (default)
|
||||
python remediation_service/remediation_daemon.py --dry-run
|
||||
|
||||
# Actually make AWS API calls
|
||||
python remediation_service/remediation_daemon.py --apply
|
||||
```
|
||||
|
||||
### 6. Reports
|
||||
|
||||
```bash
|
||||
# Generate HTML report from all findings in DB
|
||||
python -c "
|
||||
from consumer.prowler_consumer import FindingsDB
|
||||
from consumer.output.report import generate_html_report
|
||||
db = FindingsDB()
|
||||
import sqlite3
|
||||
# load all findings
|
||||
rows = sqlite3.connect(db.db_path).execute('SELECT * FROM findings').fetchall()
|
||||
print('stats:', db.count_by_status())
|
||||
print('severity:', db.count_by_severity())
|
||||
"
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
| Component | File | Role |
|
||||
|---|---|---|
|
||||
| Scanner | `prowler-runner.sh` | Docker wrapper for Prowler CLI |
|
||||
| Models | `consumer/models.py` | `Finding` and `RemediationAction` dataclasses |
|
||||
| Consumer | `consumer/prowler_consumer.py` | Parses JSON, stores in SQLite, enqueues actions |
|
||||
| Remediation engine | `consumer/remediation/remediator.py` | Maps finding → action, executes |
|
||||
| S3 actions | `consumer/remediation/actions/s3.py` | `block_public_access`, `enable_encryption`, etc. |
|
||||
| IAM actions | `consumer/remediation/actions/iam.py` | `remove_access_keys`, `attach_mfa_policy`, etc. |
|
||||
| RDS actions | `consumer/remediation/actions/rds.py` | `enable_encryption`, `enable_backup`, etc. |
|
||||
| Lambda actions | `consumer/remediation/actions/lambda_.py` | `remove_wildcard_permissions` |
|
||||
| Compliance map | `consumer/output/compliance_map.py` | Maps findings → SOC2, ISO27001, NIST CSF, etc. |
|
||||
| Reports | `consumer/output/report.py` | HTML and JSON report generators |
|
||||
| Remediation daemon | `remediation_service/remediation_daemon.py` | Long-running action applier |
|
||||
|
||||
## Action Map (check_id → remediation)
|
||||
|
||||
| Prowler check prefix | Action | AWS API |
|
||||
|---|---|---|
|
||||
| `s3_bucket_public*` | `s3_block_public_access` | `s3.put_public_access_block` |
|
||||
| `s3` (encryption) | `s3_enable_encryption` | `s3.put_bucket_encryption` |
|
||||
| `iam_user_access_key` | `iam_remove_access_keys` | `iam.delete_access_key` |
|
||||
| `iam_user_mfa` | `iam_attach_mfa_policy` | `iam.put_user_policy` |
|
||||
| `iam_role_inline` | `iam_remove_inline_policies` | `iam.delete_role_policy` |
|
||||
| `rds_instance_storage` | `rds_enable_encryption` | `rds.modify_db_instance` |
|
||||
| `rds_backup` | `rds_enable_backup` | `rds.modify_db_instance` |
|
||||
| `rds_public` | `rds_restrict_public_access` | `ec2.revoke_security_group_ingress` |
|
||||
| `lambda_permission_wildcard` | `lambda_remove_wildcards` | `lambda.remove_permission` |
|
||||
|
||||
## Safety
|
||||
|
||||
- **Dry-run by default.** Remediation actions log what they WOULD do without calling AWS.
|
||||
- Pass `--apply` to actually make changes.
|
||||
- `~/.aws` is mounted read-only inside the container.
|
||||
- All finding state is stored in SQLite (`consumer/findings.db`), so we track what's new vs remediated vs ignored.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
greysec/prowler-ops/
|
||||
├── README.md
|
||||
├── docker-compose.yml # Full stack (scanner + consumer + remediator)
|
||||
├── .env.example # Template for environment variables
|
||||
├── prowler-runner.sh # Bash wrapper for Prowler Docker container
|
||||
├── config/
|
||||
│ └── prowler-config.yaml # Prowler CLI flags (regions, services, severity)
|
||||
├── aws/
|
||||
│ └── sample-iam-readonly-role.json # Sample IAM role for cross-account scanning
|
||||
├── consumer/
|
||||
│ ├── Dockerfile
|
||||
│ ├── requirements.txt
|
||||
│ ├── models.py # Finding, RemediationAction dataclasses
|
||||
│ ├── prowler_consumer.py # Main consumer daemon
|
||||
│ ├── remediation/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── remediator.py # Mapping engine + action dispatcher
|
||||
│ │ └── actions/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── utils.py # boto3 client helpers
|
||||
│ │ ├── s3.py
|
||||
│ │ ├── iam.py
|
||||
│ │ ├── rds.py
|
||||
│ │ └── lambda_.py
|
||||
│ └── output/
|
||||
│ ├── __init__.py
|
||||
│ ├── compliance_map.py # Compliance framework mapping
|
||||
│ └── report.py # HTML/JSON report generators
|
||||
└── remediation_service/
|
||||
└── remediation_daemon.py # Long-running daemon that applies fixes
|
||||
```
|
||||
Reference in New Issue
Block a user