Automated OSINT Intelligence Pipeline
One-time purchase · $50 · Lifetime access
What You Get
A fully automated, ten-stage intelligence pipeline that runs on GitHub Actions — no servers, no databases, no DevOps. Add three API keys, push to a repository, and the system runs itself twice a day. Every run queries live web data, extracts entities, detects convergence patterns, verifies facts, and publishes a live nine-tab analytical dashboard to your domain.
Everything ships as a single .tar.xz archive: 10 Python pipeline scripts, a shell orchestrator, a context sync utility, three GitHub Actions workflows, a static HTML dashboard, and complete setup documentation. Extract, push, add secrets, done.
The Pipeline — Ten Stages, Fully Automated
| Stage | Name | What It Does | API |
|-------|------|--------------|-----|
| 1 | Node Tracker | Queries live status of configurable intelligence nodes via Brave Search, then sends results to Anthropic Claude for structured analysis. Classifies events as FRICTION, COMPLIANCE, ESCALATION, DE_ESCALATION, or NEUTRAL. Identifies leverage type: LEGAL, FINANCIAL, REPUTATIONAL, KINETIC, or REGULATORY. | Brave Search + Claude |
| 2 | Entity Extraction | Dual-engine extraction — GLiNER2 runs locally at zero cost, optional Llama Scout 17B adds structured relationship mapping with cross-node links. | GLiNER2 (local) + Llama Scout |
| 3 | Convergence Detection | Identifies multi-node convergence windows from historical data — flags when 3+ nodes show simultaneous activity. Uses adaptive time windows (1-day for kinetic nodes, 7-day for financial/information nodes). Detects friction↔compliance pairs automatically. | Local — zero cost |
| 4 | Daily Intelligence | Three-pass investigation: signal sweep, cross-node convergence scan, and under-radar probe. Tracks breaking news across entities and verifies pending predictions against live data. | Brave Search + Claude |
| 5 | Fact Checker | Inline date/dollar triage, then web-grounded verification — searches Brave for live evidence and sends enriched claims to Claude. Deduplicates, verifies, and corrects errors in-place before publication. | Brave Search + Claude |
| 6 | Rhetoric vs. Reality | Three-column gap analysis for each active development: what was said (rhetoric), what documents show (reality), and the domestic cost. Tracks institutional bypass with confidence levels. | Claude |
| 6.5 | Knowledge Base Builder | Generates structured knowledge base files from entity, convergence, and rhetoric/reality data — ready for AI agent integration. | Local — zero cost |
| 7 | Pipeline Validation | Cross-pipeline consistency check — verifies critical nodes appear in top developments, convergence events have entity overlap, and source diversity is maintained. | Local — zero cost |
| 7.5 | Chart Data Builder | Aggregates full run history into chart-ready JSON for the dashboard's trend visualizations. | Local — zero cost |
| 8 | Entity Graph | NetworkX builds a directed relationship graph. PyVis renders it as an interactive HTML visualization embedded in the dashboard. | Local — zero cost |
The Dashboard — Nine Tabs, Zero Backend
A static single-page HTML dashboard served from GitHub Pages. No backend, no database — reads JSON files directly in the browser.
- Node Status — Live status cards with classification badges, confidence levels, and entity tags
- Intelligence — Daily summary with signal status, breaking news, top developments, and priority watchlist
- Convergence — Multi-node convergence windows and friction/compliance pairs
- Predictions — Pending and resolved prediction tracking with verification results
- Entities — Structured entity and relationship extraction
- Entity Graph — Interactive PyVis HTML graph with color-coded entity types
- Charts — Historical trend charts (thermostat timeline, stacked area, node activation heatmap)
- Rhetoric vs. Reality — Three-column gap analysis table
- History — Timeline of pipeline runs and convergence events
What You Need
| Requirement | Details |
|-------------|---------|
| GitHub Account | Free tier works |
| Brave Search API Key | Required — web search for Stages 1, 4, 5 |
| Anthropic API Key | Required — Claude for Stages 1, 4, 5, 6 |
| Llama Scout API Key | Optional — enhances entity extraction in Stage 2 |
| Setup Time | Under 15 minutes |
| Monthly Cost | Under $5/month in API fees |
How Setup Works
- Extract the archive and push it to a new GitHub repository
- Add your API keys as GitHub Secrets (Brave Search + Anthropic, optionally Llama Scout)
- Enable GitHub Pages (Settings → Pages → Deploy from branch →
main / docs)
- Trigger the first run from the Actions tab
- The pipeline now runs automatically at 08:00 and 20:00 UTC every day
No servers. No databases. No cron jobs to maintain. GitHub Actions handles everything.
What's in the Package
- 11 Python scripts (10 pipeline stage scripts and a context sync utility)
- Shell orchestrator (
run_pipeline.sh)
- 3 GitHub Actions workflows (pipeline runner, deploy/validation, knowledge base sync)
- Static HTML dashboard (
docs/index.html)
- Unified configuration file (
tracker_config.json)
- Requirements files for dependency installation
- Complete setup documentation (SETUP.md, Pipeline_Setup.md, SECRETS_REPORT.md)
.env.example for local development
Key Features
- Fully Serverless — Runs entirely on GitHub Actions free tier. No infrastructure to manage.
- Self-Healing — Pipeline stages are non-blocking (except Stage 1). If a stage fails, the rest continue.
- Three-Pass Investigation — Signal sweep, cross-node convergence scan, and under-radar probe on every run. Searches that other tools skip.
- Web-Grounded Verification — Fact checker searches the live web before verifying claims — not just AI knowledge. Corrections applied in-place so the dashboard always shows verified content.
- Prediction Tracking — Logs forecasts and automatically verifies them against new data.
- Dual-Engine Entity Extraction — Zero-cost local AI (GLiNER2) with optional cloud enhancement (Llama Scout).
- Configurable for Any Topic — Shipped as a template. Edit
tracker_config.json to track any set of leverage nodes, signals, and entities.
- Data-Driven Corrections — Add standing entity corrections to the config file and the fact checker applies them automatically.
- API Budget Controls — Built-in daily call limits prevent runaway costs.
- Full Data Sovereignty — All data stays in your GitHub repository. No vendor lock-in, no third-party dashboards.
Terms
- One-time purchase — $50, no subscription, no recurring fees
- Lifetime access — You own the delivered package
- Personal license — One license per user, non-transferable
- Refund policy — If the product fails to function as described and we cannot resolve within 7 days, full refund
- Support — Email support at austin@dvmgservices.com
Built by Austin Smith. Independent OSINT researcher and developer.
