Skip to main content

Geocode a large address list with the batch endpoint

POST /api/v1/geocode/batch takes up to 100 addresses per call and returns one result per address, in order, each the same shape as a single /api/v1/geocode result. A 400,000-row HMDA file is about four thousand calls instead of four hundred thousand.

The endpoint does the per-call work. The job around it still needs three things: stop geocoding the same address twice, slow down when the API says so, and be able to stop and restart without losing the work. This post builds that in about 70 lines.

What you'll need

  • An API key (X-API-Key: gck_...).
  • A monthly quota that covers the file. Every distinct address counts once: the free Developer tier is 5,000 a month, Property Pro 50,000, Growth 500,000.
  • Python 3.9+ with requests.
  • A CSV with an address column.

Step 1: deduplicate first

This is the single largest saving and almost everyone skips it. Address lists built from transaction records repeat heavily — the same branch, the same servicer address, the same handful of corporate mailing addresses, thousands of times.

import csv
from collections import OrderedDict

def norm(address):
    """The cache key. Must be used everywhere, or variants miss the cache."""
    return " ".join(address.split()).upper()

def unique_addresses(path):
    """{normalised key: one original spelling to send to the API}."""
    seen = OrderedDict()
    with open(path, newline="", encoding="utf-8") as fh:
        for row in csv.DictReader(fh):
            seen.setdefault(norm(row["address"]), row["address"])
    return seen

Normalising whitespace and case before the comparison is what makes this work: 123 Main St and 123 MAIN ST become one lookup, not two.

The batch endpoint also resolves a repeated address once and charges it once, but only within a single call. Deduplicating the whole file first is what stops the same address from being paid for in batch 3 and again in batch 900.

Keep that norm() in one place and use it for both the dedupe and the cache lookup. Deduplicating on the normalised form but caching under the raw string is an easy mistake, and it quietly reintroduces every duplicate you just removed.

Step 2: cache to disk, so a crash is not a restart

Write every result as you get it. SQLite is enough, and it makes the job resumable — a rerun skips everything already stored.

import json
import sqlite3

def open_cache(path="geocodes.db"):
    db = sqlite3.connect(path)
    db.execute("""
        CREATE TABLE IF NOT EXISTS geocode (
            address TEXT PRIMARY KEY,
            payload TEXT NOT NULL
        )
    """)
    db.commit()
    return db

def cached(db, key):
    row = db.execute(
        "SELECT payload FROM geocode WHERE address = ?", (key,)
    ).fetchone()
    return json.loads(row[0]) if row else None

def store(db, key, payload):
    db.execute(
        "INSERT OR REPLACE INTO geocode (address, payload) VALUES (?, ?)",
        (key, json.dumps(payload)),
    )
    db.commit()

Storing results is worth saying out loud: you are allowed to. There is no caching clause here and no expiry on what you geocoded. That is not true of every provider — see how this compares if you are migrating from one that restricts it.

Step 3: send 100 at a time, and read the 429 you get back

Two different limits answer with HTTP 429, and they need opposite responses:

  • The burst limit: 300 calls a minute per key. A batch counts as one call however many addresses it carries. The body says "error": "burst_rate_limit" with a retry_after in seconds. Wait and retry.
  • The monthly quota: a batch is all-or-nothing against it. If the whole batch does not fit in what is left this month, nothing is consumed and the body carries remaining and reset_date. Waiting a minute will not help, so stop the job.
import time
from concurrent.futures import ThreadPoolExecutor, as_completed

import requests

API = "https://addresscanvas.com/api/v1/geocode/batch"
BATCH = 100   # the per-call maximum
WORKERS = 4

class QuotaExhausted(Exception):
    """The month's quota cannot cover the next batch."""

def chunks(items, size):
    for i in range(0, len(items), size):
        yield items[i:i + size]

def geocode_batch(session, api_key, batch, attempts=5):
    """One call of up to 100 (key, original) pairs. Bounded retries matter: an
    unbounded loop turns a sustained limit into a hang you cannot interrupt."""
    for attempt in range(attempts):
        r = session.post(
            API,
            headers={"X-API-Key": api_key, "Content-Type": "application/json"},
            json={"addresses": [original for _, original in batch]},
            timeout=60,
        )
        if r.status_code != 429:
            r.raise_for_status()
            return r.json()["results"]
        detail = r.json().get("detail") or {}
        if detail.get("error") == "burst_rate_limit":
            time.sleep(detail.get("retry_after") or 2 ** attempt)
            continue
        raise QuotaExhausted(
            f"{detail.get('remaining', 0)} geocodes left this month, "
            f"resets {detail.get('reset_date')}"
        )
    raise RuntimeError(f"still rate limited after {attempts} attempts")

def run(addresses, api_key, db):
    """`addresses` is the {key: original} mapping from unique_addresses()."""
    todo = [(k, v) for k, v in addresses.items() if cached(db, k) is None]
    print(f"{len(addresses) - len(todo)} already cached, {len(todo)} to fetch")
    done = 0

    with requests.Session() as session, ThreadPoolExecutor(WORKERS) as pool:
        futures = {
            pool.submit(geocode_batch, session, api_key, batch): batch
            for batch in chunks(todo, BATCH)
        }
        for future in as_completed(futures):
            batch = futures[future]
            try:
                results = future.result()
            except QuotaExhausted as exc:
                print(f"stopping: {exc}; rerun after the reset to resume")
                for pending in futures:
                    pending.cancel()
                break
            except Exception as exc:
                print(f"batch failed, will retry on the next run: {exc}")
                continue
            # Results come back one per submitted address, in order.
            for (key, _), result in zip(batch, results):
                store(db, key, result)
            done += len(batch)
            if done % 5000 < BATCH:
                print(f"  {done}/{len(todo)}")

Four workers keep a few calls in flight while each one is answered. If they outrun the burst limit, the burst_rate_limit branch slows them down; that branch is a signal to wait, not an error to log and move past. A failed batch is left out of the cache rather than written as empty, so the next run picks it up. Dropping it instead is how you end up with a 3% hole in your output that nobody notices until someone asks why a county is under-represented.

Step 4: write the enriched file

def enrich(in_path, out_path, db):
    with open(in_path, newline="", encoding="utf-8") as src, \
         open(out_path, "w", newline="", encoding="utf-8") as dst:
        reader = csv.DictReader(src)
        writer = csv.DictWriter(
            dst,
            fieldnames=reader.fieldnames + [
                "lat", "lon", "census_block", "tract", "fips",
                "location_code", "match_code",
            ],
        )
        writer.writeheader()
        for row in reader:
            g = cached(db, norm(row["address"])) or {}
            block = g.get("censusBlock") or ""
            row.update({
                "lat": g.get("lat", ""),
                "lon": g.get("lon", ""),
                "census_block": block,
                # State(2) + county(3) + tract(6) = the 11-digit tract GEOID.
                "tract": block[:11],
                "fips": g.get("fips", ""),
                "location_code": g.get("locationCode", ""),
                "match_code": g.get("matchCode", ""),
            })
            writer.writerow(row)

Note the dedupe key is rebuilt here from the original row, so the output has one line per input row even though the API saw each distinct address once.

Checking the result before you trust it

locationCode tells you what the coordinate actually is. For anything regulatory, count them before you file:

from collections import Counter

counts = Counter(
    (cached(db, key) or {}).get("locationCode") or "NO_MATCH" for key in addresses
)
print(counts.most_common())

ROOFTOP and ADDRESS_POINT are structure-level. PARCEL is the parcel centroid. SEGMENT is interpolated along a street and is the one to look at twice — it is a position on a block face, not a building. If your SEGMENT share is above a percent or two, check whether those addresses are rural or simply malformed.

Why not just use the Census Geocoder?

For a one-off file under 10,000 rows, do. It is free and authoritative, and we say so on the comparison page. The 10,000-record cap per batch file is what pushes people to something else once a job stops being a one-off.

Where to go next

Run this on your own addresses

Start free with 5,000 geocodes a month, Census block included.

Get your API key — 150 free credits

More guides

Enrich a CSV of addresses with Census demographics (Python) Cache-first geocoding, and when to re-geocode Add rooftop geocoding to a Django app