Sources
ODBC
Load tables and query results from any database with an ODBC driver: connections, choosing tables, query tables, value sizes and types.
From sqlload 0.27.0, an odbc source loads tables, views and query
results from any database that has an ODBC driver. SQLLoad reads the rows as Arrow through the
driver and loads them into the source’s target.
From 0.37.0 (not released yet), the sqlload and sqlload-cli wheels include ODBC on every
platform. Before that, and in a build from source (cargo install, or pip building an sdist where
no wheel fits), ODBC is built only with the odbc feature (cargo install sqlload --features odbc);
without it, plan refuses an odbc source.
[connections.erp]kind = "odbc"connection_string = "${ENV:ERP_ODBC}"
[sources.erp]kind = "odbc"connection = "erp"description = "Tables of the ERP database."include = ["sales.*"]exclude = ["sales.tmp_*"]max_text_size = 8000
[sources.erp.tables."sales.items"]description = "Items, one row per item."cursor_column = "updated_at"unique_key = ["id"]write_strategy = "merge"
[sources.erp.tables.recent]description = "Items added recently."sql = "SELECT id, name FROM sales.items WHERE id > 2400"$ sqlload planPlan ready 2 selected, 2 objects to create, 2 to load
Objects (2) sqlload apply├── raw.erp__items create table│ ├── id int -> BIGINT│ ├── name text -> VARCHAR│ ├── price decimal(10,2) -> DECIMAL(10,2)│ ├── created date -> DATE│ ├── updated_at timestamp_ntz -> TIMESTAMP│ └── + _sqlload_load_id, _loaded_at└── raw.erp__recent create table ├── id int -> BIGINT ├── name text -> VARCHAR └── + _sqlload_load_id, _loaded_at
Loads (2) sqlload load├── erp__items merge (unique_key: id)│ ├── source odbc erp sales.items│ ├── route read as Arrow -> DuckDB takes Arrow│ └── cursor updated_at first run, everything└── erp__recent replace ├── source odbc erp query recent └── route read as Arrow -> DuckDB takes ArrowConnection
Section titled “Connection”| Key | Meaning |
|---|---|
connection_string |
The driver’s connection string, such as Driver={PostgreSQL Unicode};Server=db.example.com;Database=erp;Uid=loader;Pwd=.... Use ${ENV:NAME}, since it usually holds a password. |
dsn |
A data source name from odbc.ini, instead of connection_string. It and user can’t contain ;, { or }. |
user, password |
With dsn, when the DSN doesn’t hold them. |
Set connection_string or dsn, not both. Install the database’s ODBC driver on the machine
SQLLoad runs on; SQLLoad reaches it through a driver manager:
- Windows: the driver manager built into Windows; register drivers and DSNs as usual (ODBC Data Sources).
- Linux and macOS wheels (from 0.37.0, not released yet): the unixODBC 2.3.14 driver manager,
bundled as a separate shared library, so no system unixODBC is needed. It reads drivers and
system DSNs from
odbcinst.iniandodbc.iniin/etcon Linux, where distribution driver packages register, and in Homebrew’setcon macOS (/opt/homebrew/etcon Apple silicon,/usr/local/etcon Intel), or in the folderODBCSYSINInames; user DSNs from~/.odbc.ini. A driver registered by file name alone, as Debian and Ubuntu packages register theirs (Driver=psqlodbcw.so), is looked for in/usr/lib/<arch>-linux-gnu/odbcand/usr/lib/<arch>-linux-gnu(then/usr/lib64/odbcand/usr/lib64) on Linux, and Homebrew’slibon macOS. unixODBC is LGPL-2.1: its notice and licence are in each wheel’s.dist-info/licenses, and its source is in the package’s sdist on PyPI. cargo install --features odbc: the system’s unixODBC, which it links against:unixodbc-devon Debian and Ubuntu,unixodbcwith Homebrew.
Drivers
Section titled “Drivers”SQLLoad writes names in ANSI double quotes ("sales"."items"). A database that doesn’t read them
as names is refused when the source is read; switch it to ANSI quoting. For MySQL, add
INITSTMT=SET SESSION sql_mode=CONCAT(@@SESSION.sql_mode,',ANSI_QUOTES'); to the connection string.
| Driver | Tested |
|---|---|
psqlODBC (PostgreSQL Unicode) |
Yes, against Postgres 17 in containers on Linux and on Windows. Add BoolsAsChar=0 to the connection string so boolean columns load as bool rather than text. |
| Microsoft ODBC Driver 18 for SQL Server | Yes, against SQL Server 2022 in containers on Linux and against SQL Server LocalDB on Windows, datetime2 and legacy datetime cursors included. |
MySQL Connector/ODBC (MySQL ODBC Unicode Driver) |
Yes, against MySQL 8.4 in containers on Linux and MySQL 8.4 on Windows, in an ANSI_QUOTES session (above). On Linux, install MySQL’s package for your distribution rather than the generic glibc tarball: the tarball’s own OpenSSL fails to load once another driver, such as SQL Server’s or psqlODBC, has loaded the system one in the same run. |
| OpenEdge (Progress) | Tested for timestamp handling (parsing {ts} values and binding timestamp bounds), not against a live server. |
| Others | Not yet. A driver needs to describe prepared statements and list tables; run sqlload plan to see the types it reports. |
From 0.37.0 (not released yet), a cursor filter compares the cursor column with bound parameters,
such as "updated_at" > ?, each passed as the cursor’s own type: a timestamp without a time zone as
an ODBC TIMESTAMP, with every fractional digit; a date as a DATE; text as text. No database’s
literal rules can change a bound, and a millisecond maximum such as 23:59:59.997 resumes exactly.
SQL Server’s legacy datetime is the one exception: it stores 1/300 s ticks, which its driver reads
rounded to milliseconds, so its bounds go as ISO 8601 text (2026-01-31T23:59:59.997) in the
column’s own precision, which SQL Server converts to the column’s type alike under every login
language. A cursor column pinned to a type (columns = { ... }) compares with a quoted literal, as
the pinned type may not be the column’s own, such as a varchar read as a timestamp. A numeric
cursor compares with a bare number, such as "id" > 41.
Any other cursor, a timestamp with a time zone for one, still compares with a quoted literal,
which the database converts to the column’s type; check that yours does before relying on it.
For a timestamp or date cursor, --start-cursor, --end-cursor and cursor_start also take ODBC
escape values, such as
{ts '2026-01-31 23:59:59.997'} or {d '2026-01-31'}.
From 0.37.0 (not released yet), a timestamp column is read in at least microseconds whatever precision
the driver describes: MySQL Connector/ODBC describes every DATETIME(n) with none, and its fractional
seconds were dropped before.
Choosing tables
Section titled “Choosing tables”include and exclude are schema.table patterns, matched without regard to case (* matches
any text). SQLLoad lists the tables and views the driver reports. For a database without schemas,
the patterns match the table name alone. A [sources.erp.tables."schema.table"] entry sets options
for one table; on its own, it doesn’t add the table.
A source needs at least one include pattern or one query table. The primary key the driver
reports becomes the table’s default unique_key.
Query tables
Section titled “Query tables”A table entry with sql is a query table: its rows are the query’s result, and the entry’s key is
its name. SQLLoad prepares the query at plan to read its columns, without running it.
cursor_column names a column of the query’s result. SQLLoad wraps the query and filters on the
cursor outside it, so the query needs no placeholder. Write the query in the database’s own SQL.
Value sizes
Section titled “Value sizes”Drivers read values into buffers of a fixed size per column, from the length the driver reports.
max_text_size (bytes) caps the buffer of every text column, and max_binary_size that of every
binary column, so set them when the driver reports a huge length for unbounded columns such as
TEXT or VARCHAR(MAX), or none at all, which the read then refuses, naming the setting. A value
longer than its buffer, even in a VARCHAR(10000) column capped at 8000 bytes, fails the read,
naming the setting to raise; it’s never cut short.
SQLLoad reads each column’s type as the driver describes it:
| ODBC type | Loads as |
|---|---|
TINYINT, SMALLINT |
smallint |
INTEGER, BIGINT |
int |
NUMERIC(p,s), DECIMAL(p,s), p up to 38 |
decimal(p,s), read as text so no digit is lost |
REAL, FLOAT, DOUBLE |
float |
BIT |
bool |
DATE, TIME |
date, time |
TIMESTAMP |
timestamp_ntz; a timestamp with a time zone, in the session’s time zone |
CHAR, VARCHAR and their wide and long forms |
text |
BINARY, VARBINARY, LONGVARBINARY |
binary |
Anything else, and NUMERIC above 38 digits |
text, as the driver writes it |
A decimal value with more places than its column’s scale fails the read rather than being rounded.
psqlODBC describes an unbounded numeric as NUMERIC(28,6), so pin such a column, for example
columns = { amount = "decimal(38,10)" } or "text".
Pin a type to load a column another way, such as columns = { total = "decimal(18,2)" }. A value that
doesn’t convert to the pinned type fails the read, naming the column. See Types.
Permissions
Section titled “Permissions”The database user needs to read the catalog (to list tables and their keys) and SELECT on the
tables and views.