Sifter — Automated Reconnaissance Scanner

A Bash recon pipeline that chains nmap, gobuster, and feroxbuster into a single workflow — scan a target, find HTTP services, enumerate directories, and get a clean, risk-classified report, without jumping between tools manually.

Built for CTF labs, OSCP-style practice, and authorized penetration testing engagements.

---

Why I built this

Every box meant running the same three tools in sequence, copy-pasting ports between them, then manually sifting through feroxbuster's raw JSON to find the files that actually mattered. Sifter automates that pipeline and adds a report layer that highlights high-interest findings — config files, backups, exposed scripts — without making you scroll through hundreds of lines of static assets.

---

What it does

1. Port scan — runs nmap -sCV against the target, accepting either an IP address or a hostname/FQDN, and saves results as .txt and .xml
2. Service detection — parses nmap output to extract HTTP and HTTPS ports automatically
3. Directory enumeration — runs gobuster and feroxbuster against every discovered web service, with an optional Nikto pass for additional web server vulnerability checks
4. Smart reporting — feroxbuster's JSON output is parsed, paths grouped by parent directory, findings classified by file type and risk level
5. Summary — a clean count of total URLs, directories, open directory listings, and high/medium-interest files

All output is saved to a timestamped folder (sifter_<IP>_<YYYYMMDD_HHMMSS>/) so results never overwrite between runs. A rate-limiting mode is available for slower, less detectable scans when stealth matters more than speed.

---

Dependencies

Tool	Purpose	Install
nmap	Port scanning & service detection	sudo apt install nmap
gobuster	Fast directory brute-forcing	sudo apt install gobuster
feroxbuster	Recursive content discovery	GitHub releases
jq	JSON parsing for feroxbuster output	sudo apt install jq

The script checks for all four on startup and exits cleanly if anything is missing.

Default wordlist: /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt — ships with Kali Linux and ParrotOS. Pass a different list via a command-line argument to override it for a specific run.

---

Setup

git clone https://github.com/Harshil015/sifter.git
cd sifter
chmod +x sifter.sh

No install step, no Python environment, no config file. Mark it executable and run it.

---

Usage

sudo ./sifter.sh

The script prompts for a target — an IP address or a hostname/FQDN both work — then runs everything automatically: live nmap output, then gobuster, then feroxbuster's categorized report.

nmap -sCV requires root privileges — run with sudo.

---

Output

Terminal output — the feroxbuster report groups discovered paths by parent directory, with status codes color-coded (green = 200, cyan = 3xx, yellow = 401/403, red = 5xx), and files tagged by risk:

Tag	Examples	Risk
Script/Config/Sensitive	.php, .env, .bak, .sql, .conf	High
Data/Config	.json, .yaml, .log, .xml	Medium
HTML Page	.html, .htm	Low
Static Asset	.css, .js, .png	None

Saved files — each run creates a folder containing human-readable and machine-readable nmap output, per-port gobuster results, and a clean feroxbuster report. An HTML version of the report can be generated alongside the plain-text output for easier sharing.

---

Example run (condensed)

[*] Starting nmap -sCV scan on 10.10.11.45...

PORT     STATE SERVICE VERSION
22/tcp   open  ssh     OpenSSH 8.9p1
80/tcp   open  http    Apache httpd 2.4.52

[+] Found HTTP service -> Port 80 (http)

[*] Running gobuster on http://10.10.11.45:80...
[*] Running feroxbuster on http://10.10.11.45:80 (recursive)...

  DISCOVERED PATHS
  ────────────────────────────────
  /admin/  [OPEN DIRECTORY LISTING]
      200  /admin/config.php  [Script/Config/Sensitive]
      200  /admin/db.bak      [Script/Config/Sensitive]

  FINDINGS SUMMARY
  ────────────────────────────────
  Total URLs found  : 4
  High-interest      : 2

---

How it works under the hood

• nmap output is parsed line-by-line with regex to extract port numbers and detect HTTP vs SSL/HTTP
• feroxbuster runs with --json and --silent; jq filters and strips the config block
• Paths are grouped by parent directory using dirname logic
• Open directory listings are flagged by reading feroxbuster's heuristics messages from the same JSON stream

---

Limitations

• Requires root for nmap's service version detection (-sCV)
• IPv6 targets are not yet supported
• Tested on Kali Linux and Ubuntu 22.04; not tested on macOS (BSD date syntax differs)

---

Legal disclaimer

Use only on systems you own or have explicit written permission to test. Built for authorized penetration testing, CTF competitions, and security research in controlled lab environments.

---

Author

Harshil Makwana — ECE graduate from SVNIT Surat, building security tools and looking for a first role in penetration testing, VAPT, or SOC.

linkedin.com/in/harshilmakwana · github.com/Harshil015