Custom HTTP

The HTTP adapter connects to any JSON REST API. A mapping file tells VisiGrid how to extract the 9 canonical columns from each JSON response item.

Setup

No fixed authentication — configure the auth method that matches your API. Credentials are always resolved from environment variables, never inline.

Usage

vgrid fetch http \
  --url https://api.example.com/v1/transactions \
  --auth bearer-env:EXAMPLE_API_KEY \
  --map mapping.json \
  --from 2026-01-01 --to 2026-02-01 \
  --out transactions.csv

Use --sample to inspect the raw API response before writing a mapping:

vgrid fetch http \
  --url https://api.example.com/v1/transactions \
  --auth bearer-env:EXAMPLE_API_KEY \
  --from 2026-01-01 --to 2026-01-02 \
  --sample

Options

Flag	Description	Default
--url	API endpoint (HTTPS only)	required
--auth	Auth method	none
--map	Path to mapping JSON	required
--from / --to	Date range (YYYY-MM-DD)	required
--out	Output CSV path	stdout
--sample	Print raw JSON response and exit	off
--save-raw	Save raw JSON to file for audit	off
--timeout	Request timeout in seconds	15
--max-items	Safety cap on total items	10,000
--max-pages	Safety cap on pages fetched	100
--fingerprint	Write signed request fingerprint JSON	off
-q / --quiet	Suppress progress messages	off

Auth methods

Method	Flag	Example
None	--auth none	Public APIs
Bearer token	--auth bearer-env:VAR	bearer-env:FORTE_TOKEN
Custom header	--auth header-env:NAME:VAR	header-env:X-API-Key:MY_KEY
Basic auth	--auth basic-env:USER:PASS	basic-env:API_USER:API_PASS

Type mapping

User-defined via the mapping file. Use map dictionaries with a * wildcard for unmapped values:

"type": { "path": "$.category", "map": { "payment": "charge", "refund": "refund", "*": "adjustment" } }

Date handling

Dates are extracted from arbitrary fields via the columns section. The params section maps --from/--to to API query parameters with format options: iso (YYYY-MM-DD), unix_s, or unix_ms.

Notes

See vgrid fetch for the complete mapping file reference, including pagination configuration, transforms, value mapping, and request fingerprinting.

HTTPS enforced. HTTP URLs are rejected.

Response size cap. 10 MB per page.

Default sort. group_id, effective_date, source_id (overridable via sort_by in the mapping file).