Bulk Lead Sourcing API

Source leads in bulk, then run every lead through the same enrichment pipeline a campaign uses. The records can be bought from a data provider in one job covering many search terms across many locations, or uploaded from a file you already have.

Use this instead of a campaign when you want breadth in a single purchase. A campaign runs one Google Maps search per location; bulk buys every query x location combination at once.

POST /bulk-lead-sourcing/submit  ->  202 (manifest written, provider NOT called)
        |  bulk worker
   submit -> poll -> fetch -> parse -> dedup -> fan-out
        |  one job per lead
   SpiderSite -> SpiderVerify -> VayaPin       (identical to a campaign)

The provider is not contacted during this request. A manifest row is created and 202 Accepted returns immediately with status: "pending". The bulk worker drives the run. A 202 means accepted and gated, not bought.

Where records come from

Four sources are registered. Two buy records; two read a file you upload.

Do not hardcode this list. GET /sources resolves it from the adapter registry at request time, so a source added on the backend appears without a client change.

An upload is free at the source, which is not the same as free. You still pay for each enrichment stage you switch on, once per lead. Branch your cost messaging on source_is_free, never on has_cost — an unpriced provider like outscraper looks identical to a free upload on every money field.

Two traps that cost real money

Neither of these errors. Both look like a clean success. They apply to the provider sources only; an upload has no queries and no geo.

1. Omitting limits.max_records_per_query buys 500 records per search. Two queries across three cities reads as "6 searches" and is a 3,000-record purchase. The platform default is 500 records per expanded query. Quote yourself records, never searches.

2. country_code does not place a search. It is a locale hint. Only geo[].label, appended as "{query}, {label}", or an explicit latitude and longitude steers where the provider looks. A target carrying only {"country_code": "US"} buys a nationwide set of genuinely valid businesses that nobody asked for.

GET /api/v1/dashboard/bulk-lead-sourcing/sources

Every source, with availability resolved live from the adapter registry.

curl "https://spideriq.ai/api/v1/dashboard/bulk-lead-sourcing/sources" \
  -H "Authorization: Bearer $CLIENT_ID:$API_KEY:$API_SECRET"

Response

{
  "sources": [
    {
      "provider": "csv",
      "label": "CSV upload",
      "description": "Bring your own records. No provider, no spend at the source.",
      "source_kinds": ["google_maps"],
      "available": true,
      "unavailable_reason": null,
      "requires_upload": true
    }
  ]
}

Errors:

This route has no other failure mode. An adapter that vanishes between listing and lookup is skipped rather than erroring, so the list is always answerable.

POST /api/v1/dashboard/bulk-lead-sourcing/upload

Store a records file and read back what is in it. This is not a submit. Nothing is enriched, nothing is queued, nothing is spent. The file sits in storage for 7 days and you can abandon it.

multipart/form-data with three parts: provider (csv or json), source_kind, and file.

curl -X POST "https://spideriq.ai/api/v1/dashboard/bulk-lead-sourcing/upload" \
  -H "Authorization: Bearer $CLIENT_ID:$API_KEY:$API_SECRET" \
  -F "provider=csv" \
  -F "source_kind=google_maps" \
  -F "file=@my-leads.csv"

Response 201 Created

{
  "upload_id": "97a3669f-3c2e-4a1b-9d55-0f2b7c8e4d10",
  "provider": "csv",
  "source_kind": "google_maps",
  "filename": "my-leads.csv",
  "size_bytes": 4182,
  "sha256": "bae968a5...f3712",
  "record_count": 3,
  "expires_at": "2026-08-20T09:14:00Z",
  "columns": {
    "columns": ["Firma", "Str.", "Ort", "PLZ", "Tel", "Homepage"],
    "proposed_mapping": { "phone": "Tel", "website": "Homepage" },
    "unmapped_columns": ["Firma", "Str.", "Ort", "PLZ"],
    "delimiter": ";",
    "encoding": "cp1252"
  },
  "message": "Stored bulk upload"
}

Errors:

Reading the column proposal, and why the mapping is yours to confirm

A CSV has arbitrary headers, so the pipeline cannot assume a shape. The upload response reports the header row in file order, the detected delimiter and text encoding, and a proposed mapping.

Everything in columns is advisory. The proposal is a guess shown for confirmation; the mapping that actually runs is the one you send back at submit time in source.filters.column_mapping.

A run needs at least one identifying column mapped: name, place_id, google_place_id, website, or phone. With none of them every row is an anonymous bag of attributes, deduplication collapses the file to a single lead, and the enrichment stages have nothing to look up. The mapping is refused at the door, before anything is spent.

There is deliberately no email or contact target to map onto. A client can name a column anything, so a deny-list cannot work; the guarantee is that no mapping target for contact data exists to select. Contact data is produced by the verification stages, never accepted from the file.

POST /api/v1/dashboard/bulk-lead-sourcing/estimate

Size a run without committing to it. Writes nothing, contacts no provider, consumes no quota. Asking is not buying, which is exactly why this is a separate route from submit: submit returns its estimate only after the manifest is written.

Takes the same body as submit.

curl -X POST "https://spideriq.ai/api/v1/dashboard/bulk-lead-sourcing/estimate" \
  -H "Authorization: Bearer $CLIENT_ID:$API_KEY:$API_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "source": {
      "provider": "csv",
      "source_kind": "google_maps",
      "queries": [],
      "upload": { "upload_id": "97a3669f-3c2e-4a1b-9d55-0f2b7c8e4d10" }
    },
    "settings": { "workflow": { "spidersite": { "enabled": true }, "spiderverify": { "enabled": true } } }
  }'

Response

{
  "provider": "csv",
  "source_kind": "google_maps",
  "estimated_queries": 0,
  "estimated_records": 3,
  "estimated_cost_usd": null,
  "has_cost": false,
  "records_cap": 25000,
  "would_exceed_cap": false,
  "is_upload": true,
  "source_is_free": true,
  "record_count": 3,
  "record_count_known": true,
  "enrichment_stages": ["spidersite", "spiderverify"],
  "enrichment_jobs": 6,
  "upload_filename": "my-leads.csv"
}

Errors:

There is no 429 here. Nothing is reserved and nothing is bought, so there is nothing to refuse. would_exceed_cap reports what submit would do rather than refusing on its behalf.

POST /api/v1/bulk-lead-sourcing/submit

Submit one bulk buy order. Returns 202 Accepted with a manifest handle; the purchase happens afterwards, in the worker.

Authenticate with Authorization: Bearer <client_id>:<api_key>:<api_secret>.

A dashboard-scoped twin lives at POST /api/v1/dashboard/bulk-lead-sourcing/submit. It accepts a session cookie or a PAT and runs the same gate through the same execute_submission call. Two doors onto one irreversible spend do not mean two gates.

Parameters

Example — a provider job

import { SpiderIQClient } from "@spideriq/core";

const client = new SpiderIQClient({ token: process.env.SPIDERIQ_PAT });

const run = await client.bulkLeadSourcing({
  source: {
    provider: "apify",
    source_kind: "google_maps",
    queries: ["restaurants", "cafes"],
    geo: [
      { label: "Atlanta, Georgia, USA", country_code: "US" },
      { label: "Savannah, Georgia, USA", country_code: "US" },
    ],
    limits: { max_records_per_query: 100 },
  },
  settings: {
    workflow: {
      spidersite: { enabled: true },
      spiderverify: { enabled: true },
      vayapin: { enabled: false },
    },
  },
});

console.log(run.estimated_records);   // → 400, across 4 expanded queries
console.log(run.estimated_cost_usd);  // → 1.6 on apify; null on outscraper
// a 429 here means a guard refused the run BEFORE anything was bought

Example — an uploaded CSV

Upload first, confirm the mapping, then submit the upload_id with the mapping you confirmed.

curl -X POST "https://spideriq.ai/api/v1/bulk-lead-sourcing/submit" \
  -H "Authorization: Bearer $CLIENT_ID:$API_KEY:$API_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "source": {
      "provider": "csv",
      "source_kind": "google_maps",
      "queries": [],
      "upload": { "upload_id": "97a3669f-3c2e-4a1b-9d55-0f2b7c8e4d10" },
      "filters": {
        "column_mapping": {
          "name": "Firma",
          "address": "Str.",
          "city": "Ort",
          "postal_code": "PLZ",
          "phone": "Tel",
          "website": "Homepage"
        }
      }
    },
    "settings": { "workflow": { "spidersite": { "enabled": true } } }
  }'

The mapping keys are our field names; the values are your headers, byte for byte as the upload reported them. Do not normalise a header before sending it back.

From the CLI:

spideriq bulk-source submit \
  -q restaurants -q cafes \
  -g "Atlanta, Georgia, USA" -g "Savannah, Georgia, USA" \
  --max-records 100

From an MCP client:

submit_bulk_lead_sourcing(
  provider="apify",
  source_kind="google_maps",
  queries=["restaurants", "cafes"],
  geo=[{"label": "Atlanta, Georgia, USA"}, {"label": "Savannah, Georgia, USA"}],
  max_records_per_query=100
)

The CLI and MCP surfaces cover the provider sources. An upload needs the multipart call above first, which the dashboard drives.

Response

{
  "bulk_job_id": "051c62ab-ef51-470a-a8f0-4abdd6b14f90",
  "job_id": "7ae3b20d-a9ee-4601-a6ed-317885f6a3aa",
  "provider": "apify",
  "source_kind": "google_maps",
  "status": "pending",
  "estimated_queries": 4,
  "estimated_records": 400,
  "estimated_cost_usd": 1.6,
  "message": "Bulk lead sourcing accepted"
}

Errors:

Both 429s are evaluated before the manifest is written and before anything is purchased, so a denial leaves nothing behind and charges nothing. The record ceiling applies to an upload exactly as it does to a purchase, because an upload's records are counted at upload time.

Checking progress

There is no bulk-specific status route. Poll the parent job with the job_id the submit returned:

curl "https://spideriq.ai/api/v1/jobs/7ae3b20d-a9ee-4601-a6ed-317885f6a3aa/status" \
  -H "Authorization: Bearer $CLIENT_ID:$API_KEY:$API_SECRET"

The manifest walks these states:

pending -> submitted -> polling -> ready -> fetching -> parsing -> fanning_out
        -> enriching -> completed
                     (or partial, or failed, or cancelled)

A polling state lasting hours is normal: the provider is running your searches.

enriching means the leads are fanned out and the downstream pipeline is still working. It is not terminal, and this is the point: sourcing successfully is not the same as enriching successfully. The four terminal states are completed, partial, failed and cancelled.

The results envelope carries enriched_count and not_enriched_count. A run where some leads enriched and some did not is partial; a run where none did is failed, not completed.

Results come back in the same envelope as a campaign, under a synthetic campaign id of bulk_ followed by the bulk_job_id as 32 hex characters with no dashes. For the response above that is bulk_051c62abef51470aa8f04abdd6b14f90.

Stage eligibility depends on the source kind

A LinkedIn company has no street address and no place_id, so a map pin would be garbage. Enabling an ineligible stage is a 422 at submit, deliberately not a silent skip: a run that quietly did less looks like a success.

linkedin_company appears in the eligibility table although no adapter serves it yet. That is deliberate. The rule is a property of the source kind, not of the adapter, so a future LinkedIn adapter inherits the vayapin exclusion instead of rediscovering it against live data.

How deduplication works

The flat result set is deduplicated on an exact canonical key, in this precedence: place_id, then website domain, then phone, then name plus locality. There is no fuzzy matching.

For an uploaded file the key is derived from the columns you mapped, so mapping a website or phone column gives you meaningfully distinct leads and mapping none of them collapses the file.

The key is scoped to the pair of bulk_job_id and canonical key, so deduplication is per run and does not span runs. A business you sourced last week will be sourced again this week.

How bulk differs from a campaign

Because the envelope matches, every parser, export and dashboard that already reads campaign output reads bulk output unchanged. An uploaded record arrives as its own result rather than being re-searched, so its metadata.query is null.

Related

• POST /api/v1/campaigns/submit — the per-location alternative.

• GET /api/v1/jobs/{job_id}/status and GET /api/v1/jobs/{job_id}/results.