Documentation
Everything you need to build, deploy, and scale your agency on SpiderIQ — the AI-Native Agency OS with 157 MCP tools, 128 workers, and 12 integrated products.
Quick Install
npm install -g @spideriq/cli
# Authenticate
spideriq auth request --email admin@your-agency.com
# Bind your project directory
spideriq use your-project
# Verify — you're ready
spideriq auth whoami
MCP Servers
Drop these into Claude Desktop, Cursor, or any MCP-compatible client:
| Server | Package | Tools | Purpose |
|---|---|---|---|
| mcp-publish | @spideriq/mcp-publish | 105 | Pages, components, templates, deploy |
| mcp-mail | @spideriq/mcp-mail | 18 | Outbound email, warmup, rotation |
| mcp-gate | @spideriq/mcp-gate | 12 | LLM routing, key vault, rate limits |
| mcp-leads | @spideriq/mcp-leads | 15 | Maps, people, enrichment |
| mcp-admin | @spideriq/mcp-admin | 7 | System admin, tenant management |
Agent Trace Example
One prompt. Five products. Four minutes.
→ mcp:spidermaps.search_places(query="dentists austin tx", limit=100) ✓ 100
→ mcp:spiderverify.verify_batch(emails=[...]) ✓ 73 verified
→ mcp:spidermail.draft_campaign(leads=73, template="dental-cold-v3") ✓ 73 drafts
→ mcp:spiderpublish.component_update({slug: "dental-landing"}) ✓ 1 page
→ mcp:spiderbook.flow_publish({flow_id: "bf_dental"}) ✓ flow live
Completed in 4m 12s. Awaiting your review.
confirm_token.The Twelve Products
| Product | What it does | Workers |
|---|---|---|
| SpiderMaps | Google Maps scraper · 19 fields · 44,691 cities | 45 |
| SpiderSite | AI web crawler · markdown compendiums | 70 |
| SpiderVerify | SMTP email verification · 10 warmed IPs | 10 |
| SpiderBrowser | Anti-detect browsers · 50 fingerprints | 50 |
| SpiderPeople | LinkedIn + people data + AI reports | — |
| SpiderMail | Outbound email · warmup · 50 senders | 50 |
| SpiderGate | LLM key vault · model eval · rate gov | — |
| SpiderPublish | Multi-tenant publishing · edge Liquid | — |
| SpiderBook | JSON booking flows · self-hosted cal.com | — |
| SpiderPost | Social publishing · Meta, LinkedIn, X, TikTok | — |
| OpenClaw | Lead workspace · Clay-style enrichment | — |
| FuzzIQ | Dedup engine · hash + fuzzy matching | — |
Quickstart
Go from zero to your first agent-driven campaign in under 5 minutes.
Prerequisites
Node.js 18+, an active SpiderIQ account, and an MCP-compatible client (Claude Desktop, Cursor, or Claude Code).
Step 1 — Install
Step 2 — Authenticate
# Admin receives approval email → click approve
spideriq auth whoami # verify
Step 3 — Bind Project
Step 4 — Run Your First Agent
Architecture
How twelve products share one CRM, one auth system, and one MCP surface.
System Overview
SpiderIQ is a vertically integrated platform. Every product shares a PostgreSQL CRM, Better Auth authentication, and a FastAPI gateway exposing 235+ REST endpoints. The MCP surface wraps these into 157 agent-callable tools across 5 servers.
Infrastructure
| Layer | Technology | Scale |
|---|---|---|
| Gateway | FastAPI + Uvicorn | 235+ endpoints |
| Workers | Celery + Redis | 128 distributed |
| Database | PostgreSQL 16 | Multi-tenant |
| Cache | Redis Cluster | 26 nodes |
| CDN | Cloudflare R2 | Edge-rendered |
Multi-Tenancy
Every SpiderIQ instance supports complete brand isolation. One platform, unlimited client workspaces.
How It Works
Each brand (tenant) gets its own isolated namespace: separate CRM data, API keys, email senders, domains, and billing. Agents operating under one brand cannot access another brand's data.
Tenant Boundaries
| Resource | Isolation Level |
|---|---|
| CRM Contacts | Full — per-brand PostgreSQL row-level security |
| API Keys | Full — scoped to brand + environment |
| Email Senders | Full — separate warmup pools per brand |
| SpiderPublish Sites | Full — separate domains, templates, components |
| SpiderGate Keys | Full — per-brand LLM budgets and vault |
| Workers | Shared pool — priority-queued by plan tier |
MCP Tools
157 agent-callable tools across 5 MCP servers. Drop into Claude Desktop, Cursor, or any MCP-compatible client.
Server Overview
| Server | Package | Tools | Purpose |
|---|---|---|---|
| mcp-publish | @spideriq/mcp-publish | 105 | Pages, components, templates, deploy, blog, navigation |
| mcp-mail | @spideriq/mcp-mail | 18 | Outbound email, warmup, rotation, campaign management |
| mcp-gate | @spideriq/mcp-gate | 12 | LLM routing, key vault, rate limits, model evaluation |
| mcp-leads | @spideriq/mcp-leads | 15 | Maps scraping, people search, email enrichment |
| mcp-admin | @spideriq/mcp-admin | 7 | System admin, tenant management, billing |
Installation
{
"mcpServers": {
"spideriq": {
"command": "npx",
"args": ["@spideriq/mcp-publish@latest"],
"env": { "SPIDERIQ_TOKEN": "your-pat-token" }
}
}
}
Safety: Dry-Run by Default
All destructive operations (publish, delete, deploy) default to dry_run=true. The agent receives a preview envelope with a confirm_token. Only after explicit human review does the operation execute.
Agent-Native Design
SpiderIQ is built for AI agents first. Every product exposes structured APIs that agents can discover, call, and chain autonomously.
Design Principles
Discovery-first. Agents can query /v1/mcp/tools to discover all available operations, their parameters, and return types — no documentation parsing needed.
Structured I/O. Every tool returns typed JSON with consistent error envelopes. No free-text parsing, no HTML scraping.
Composable workflows. Tools are designed to chain: SpiderMaps output feeds directly into SpiderVerify, which feeds into SpiderMail. Agents orchestrate multi-product workflows in a single conversation.
Agent Trace Example
→ mcp:spidermaps.search_places(query="dentists austin tx", limit=100) ✓ 100
→ mcp:spiderverify.verify_batch(emails=[...]) ✓ 73 verified
→ mcp:spidermail.draft_campaign(leads=73, template="dental-cold-v3") ✓ 73 drafts
→ mcp:spiderpublish.component_update({slug: "dental-landing"}) ✓ 1 page
→ mcp:spiderbook.flow_publish({flow_id: "bf_dental"}) ✓ flow live
Completed in 4m 12s. Awaiting your review.
SpiderMaps
Google Maps scraper with 19 extracted fields across 44,691 indexed cities. 45 distributed workers.
What It Does
SpiderMaps scrapes Google Maps search results at scale. Give it a query ("dentists in Austin TX") and it returns structured business data: name, address, phone, email, website, rating, reviews, hours, and 10+ more fields.
Key MCP Tools
| Tool | Description |
|---|---|
| search_places | Search Google Maps by query + location. Returns up to 500 results per request. |
| get_place_details | Enrich a single place with full 19-field extraction. |
| search_cities | Browse the 44,691-city index. Filter by state, country, population. |
| export_results | Export results as CSV or push directly into the CRM. |
Extracted Fields
Name, address, phone, email, website, rating, review count, category, hours, latitude, longitude, place ID, photos, price level, business status, owner-verified, plus_code, CID, and Google Maps URL.
SpiderSite
AI web crawler that converts entire websites into structured markdown compendiums. 70 distributed workers.
What It Does
Point SpiderSite at any URL and it crawls the entire domain, extracting clean markdown content, metadata, internal link structure, and page hierarchy. Used for competitive research, content auditing, and training data preparation.
Key MCP Tools
| Tool | Description |
|---|---|
| crawl_site | Full-site crawl. Respects robots.txt, rate limits, and depth controls. |
| crawl_page | Single-page extraction with markdown conversion. |
| get_sitemap | Parse and return structured sitemap data. |
SpiderVerify
SMTP email verification with 10 warmed sender IPs. Real-time and batch modes.
What It Does
Verifies email deliverability via direct SMTP handshake — no email is actually sent. Checks MX records, mailbox existence, catch-all detection, and disposable domain filtering.
Verification Results
| Status | Meaning |
|---|---|
| valid | Mailbox exists and accepts mail |
| invalid | Mailbox does not exist |
| catch_all | Domain accepts all addresses — risky |
| disposable | Temporary/throwaway email service |
| unknown | Server didn't respond — retry later |
SpiderMail
Outbound email infrastructure with warmup, rotation, and 50 sender accounts. 50 workers.
What It Does
SpiderMail manages your entire cold outreach pipeline: sender warmup, domain rotation, campaign scheduling, open/click tracking, and reply detection. All senders are warmed automatically to maintain deliverability.
Key MCP Tools
| Tool | Description |
|---|---|
| draft_campaign | Create a campaign from leads + template. Supports A/B variants. |
| send_campaign | Schedule and send. Auto-rotates senders to stay under limits. |
| warmup_status | Check warmup progress for all sender accounts. |
| get_analytics | Opens, clicks, replies, bounces per campaign. |
SpiderGate
LLM gateway with key vault, model evaluation, and rate governance. OpenAI-compatible API.
What It Does
SpiderGate routes all LLM calls through a single endpoint. It handles key management, cost tracking, provider failover, and per-agent observability. Supports 100+ LLM providers via LiteLLM Core.
Base URL
Drop-in replacement for any OpenAI-compatible client. Pass X-Spider-Agent header to attribute costs to specific agents.
SpiderPublish
Multi-tenant publishing engine with edge-rendered Liquid templates, Shadow DOM components, and instant deploys.
What It Does
SpiderPublish powers all SpiderIQ websites. It manages pages, components (Shadow DOM-scoped HTML/CSS/JS), blog posts, navigation, and deployment — all via API and MCP tools.
Key MCP Tools (105 total)
| Tool | Description |
|---|---|
| content_create_component | Create a Shadow DOM component with HTML/CSS/JS. |
| page_update | Update page blocks, SEO metadata, and override CSS. |
| deploy_preview | Generate a preview URL. Returns confirm_token for production push. |
| deploy_production | Push preview to production CDN. Requires confirm_token. |
| blog_create_post | Create blog posts with Liquid templating and author attribution. |
SpiderBook
JSON-driven booking flows with self-hosted cal.com integration.
What It Does
SpiderBook manages appointment scheduling for your agency's clients. Define booking flows as JSON, embed them on any SpiderPublish page, and let leads self-schedule. Built on a self-hosted cal.com instance with full API control.
Key Features
Customizable availability rules, timezone detection, calendar sync (Google/Outlook), confirmation emails via SpiderMail, and CRM contact auto-creation on booking.
OpenClaw
Lead workspace with Clay-style enrichment pipelines. Visual data transformation for your CRM.
What It Does
OpenClaw provides a spreadsheet-like interface for managing and enriching lead data. Chain enrichment steps: SpiderMaps → SpiderVerify → SpiderPeople → custom AI enrichment. Each column can pull from a different data source.
Enrichment Sources
Google Maps (SpiderMaps), SMTP verification (SpiderVerify), LinkedIn profiles (SpiderPeople), website scraping (SpiderSite), and custom LLM-powered enrichment via SpiderGate.
FuzzIQ
Deduplication engine with hash-based exact matching and fuzzy string similarity.
What It Does
FuzzIQ identifies and merges duplicate records in your CRM. It combines exact-hash matching (email, phone, domain) with fuzzy matching (company name similarity, address normalization) to catch duplicates that simple equality checks miss.
Match Strategies
| Strategy | Method | Use Case |
|---|---|---|
| exact | SHA-256 hash | Email, phone, domain |
| fuzzy | Levenshtein + Jaro-Winkler | Company names, addresses |
| composite | Weighted multi-field | Cross-field dedup (name + city + domain) |
Authentication
PAT-based authentication with five independent tenant locks.
Token Format
SpiderIQ uses Personal Access Tokens (PATs) in the format client_id:api_key:api_secret. Pass as a Bearer token in the Authorization header.
-H "Authorization: Bearer cli_xxx:key_xxx:secret_xxx" \
-H "Content-Type: application/json"
Five Tenant Locks
Every API call is validated against five independent checks:
| # | Lock | Purpose |
|---|---|---|
| 1 | Project ID | Identifies the brand/workspace |
| 2 | API Key | Authenticates the caller |
| 3 | API Secret | Proves key ownership |
| 4 | Session Binding | Ties to the active MCP session |
| 5 | IP Allowlist | Optional — restrict by source IP |
Endpoints
235+ REST endpoints across the SpiderIQ platform. All follow a consistent pattern.
Base URL
Common Patterns
| Pattern | Method | Example |
|---|---|---|
| List resources | GET | /content/pages |
| Get by ID | GET | /content/pages/{id} |
| Get by slug | GET | /content/components/by-slug/{slug} |
| Create | POST | /content/components |
| Update | PATCH | /content/components/{id} |
| Publish | POST | /content/components/{id}/publish |
| Deploy preview | POST | /content/deploy/preview |
| Deploy production | POST | /content/deploy/production?confirm_token=... |
Webhooks
Real-time event notifications for campaign activity, lead updates, and deployment status.
Supported Events
| Event | Trigger |
|---|---|
| email.sent | Campaign email successfully sent |
| email.opened | Recipient opened the email |
| email.replied | Recipient replied to the email |
| email.bounced | Email bounced (hard or soft) |
| lead.created | New contact added to CRM |
| lead.enriched | Contact enrichment completed |
| deploy.completed | Site deployment pushed to production |
| booking.created | New appointment booked via SpiderBook |
Error Codes
Standard HTTP status codes with structured JSON error envelopes.
Error Format
"error": {
"code": "TENANT_MISMATCH",
"message": "API key does not belong to this project",
"status": 403
}
}
Common Codes
| Status | Code | Meaning |
|---|---|---|
| 400 | VALIDATION_ERROR | Request body failed schema validation |
| 401 | UNAUTHORIZED | Missing or invalid Bearer token |
| 403 | TENANT_MISMATCH | Key doesn't match the project |
| 404 | NOT_FOUND | Resource doesn't exist |
| 409 | SLUG_CONFLICT | Component/page slug already exists |
| 429 | RATE_LIMITED | Too many requests — back off |
| 500 | INTERNAL_ERROR | Server error — contact support |
Self-Hosting
Deploy SpiderIQ on your own infrastructure. Docker Compose for single-VPS, Ansible for multi-node.
Docker Compose (Single VPS)
Minimum requirements: 4 vCPU, 16 GB RAM, 100 GB SSD. Recommended for up to 3 brands and 50k contacts.
git clone https://github.com/spideriq/self-hosted.git
cd self-hosted
cp .env.example .env # edit with your settings
# Start all services
docker compose up -d
# Verify
docker compose ps # all services should be 'Up'
curl http://localhost:8000/health # → {"status":"ok"}
Ansible (Multi-Node)
For production deployments with 10+ brands and high availability. Separates gateway, workers, database, and cache across dedicated nodes.