Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
703862b
feat: add SQL Server 2025 support
joshmarkovic Jun 24, 2026
391080c
docs: document the as_columnstore table config
joshmarkovic Jun 24, 2026
7d8e90b
test: keep empty audit table on passing store-failures run (#601)
joshmarkovic Jun 24, 2026
66634a1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 24, 2026
c0298e2
Add CHANGELOG entries for v1.10.1 release
axellpadilla Jun 25, 2026
8298b7b
Merge pull request #724 from dbt-msft/feat/v1.10.1-changelog-review
axellpadilla Jun 25, 2026
bddbfe8
chore(deps): bump actions/checkout in the github-actions-updates group
dependabot[bot] Jun 25, 2026
c137d80
Update README.md
joshmarkovic Jun 25, 2026
4938cf2
Merge branch 'master' into feat/sql-server-2025
joshmarkovic Jun 25, 2026
ad89962
ci: promote SQL Server 2025 to baseline and fix README support section
joshmarkovic Jun 25, 2026
8d9d883
fix: gate 2019+ CREATE INDEX options on detected engine version
joshmarkovic Jun 27, 2026
6b0c70b
[pre-commit.ci] pre-commit autoupdate
pre-commit-ci[bot] Jun 29, 2026
216e00a
docs: correct stale version, release, and CI references
joshmarkovic Jun 29, 2026
10e1c77
Merge pull request #729 from joshmarkovic/fix/docs-drift
axellpadilla Jun 30, 2026
f04d801
Merge branch 'master' into feat/sql-server-2025
axellpadilla Jun 30, 2026
1c060e7
Merge pull request #728 from joshmarkovic/feat/sql-server-2025
axellpadilla Jun 30, 2026
901d4f4
fix: set default value for column_type_expansion_max_rows to 1000000 …
axellpadilla Jun 30, 2026
8034124
Merge pull request #727 from dbt-msft/dependabot/github_actions/githu…
axellpadilla Jun 30, 2026
cfa5086
Merge pull request #720 from dbt-msft/pre-commit-ci-update-config
axellpadilla Jul 1, 2026
cc7ceb0
Merge pull request #731 from dbt-msft/fix/safe-type-expansion
axellpadilla Jul 1, 2026
541ce1a
fix: escape embedded closing braces in pyodbc connection string values
joshmarkovic Jul 2, 2026
44f9281
fix: honor configured query retries and stop retry-event crash
joshmarkovic Jun 29, 2026
88ad636
docs: keep only reader-facing retry comments
joshmarkovic Jul 2, 2026
4a1e661
test: parametrize add_query retry failure cases
joshmarkovic Jul 2, 2026
a2e6196
Merge pull request #730 from joshmarkovic/fix/query-retry-loop
axellpadilla Jul 3, 2026
45d5c4b
Merge pull request #734 from joshmarkovic/fix/pyodbc-brace-escaping
axellpadilla Jul 3, 2026
2de4507
fix: make cancel() actually cancel the in-flight query
joshmarkovic Jun 29, 2026
894c217
Merge pull request #733 from joshmarkovic/fix/cancel-noop
axellpadilla Jul 6, 2026
e1c8bb9
fix: cast describe_indexes STRING_AGG input to nvarchar(max) (#735)
joshmarkovic Jul 10, 2026
f5d00a9
Merge pull request #739 from joshmarkovic/fix/string-agg-8000-byte-ov…
axellpadilla Jul 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions .github/workflows/integration-tests-sqlserver.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,31 +43,31 @@ jobs:
matrix:
python_version: ["3.10", "3.11", "3.12", "3.13"]
backend: [pyodbc, mssql-python]
# Baseline on 2022
sqlserver_version: ["2022"]
# Baseline on 2025 (newest)
sqlserver_version: ["2025"]
msodbc_version: ["18"]
collation: [SQL_Latin1_General_CP1_CI_AS]
exclude:
- backend: mssql-python
python_version: "3.10"
sqlserver_version: "2022"
sqlserver_version: "2025"
- backend: mssql-python
python_version: "3.11"
sqlserver_version: "2022"
sqlserver_version: "2025"
- backend: mssql-python
python_version: "3.12"
sqlserver_version: "2022"
sqlserver_version: "2025"
include:
# Keep pyodbc on every supported Python version, but retain
# SQL Server ODBC 17 coverage for the oldest and newest Python.
- backend: pyodbc
python_version: "3.10"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Older SQL Server versions stay on pyodbc only, with a single
Expand All @@ -82,17 +82,22 @@ jobs:
sqlserver_version: "2019"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Add the case-sensitive collation only on the latest SQL Server
# and latest Python/backend rows.
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2022"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Add the case-sensitive collation on the SQL Server 2025 baseline
# and latest Python/backend rows.
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CS_AS
# mssql-python stays on the latest Python only.
- backend: mssql-python
python_version: "3.13"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "18"
collation: SQL_Latin1_General_CP1_CS_AS
runs-on: ubuntu-latest
Expand All @@ -112,7 +117,7 @@ jobs:
DBT_TEST_USER_3: DBT_TEST_USER_3
COLLATION: ${{ matrix.collation }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7

- name: Install uv
run: pip install uv
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
packages: write
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Log in to the Container registry
uses: docker/login-action@v4.2.0
Expand All @@ -44,14 +44,14 @@ jobs:
publish-docker-server:
strategy:
matrix:
mssql_version: ["2017", "2019", "2022"]
mssql_version: ["2017", "2019", "2022", "2025"]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Log in to the Container registry
uses: docker/login-action@v4.2.0
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
name: Release new version
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7

- uses: actions/setup-python@v6
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
password: ${{ secrets.github_token }}
steps:

- uses: actions/checkout@v6
- uses: actions/checkout@v7

- name: Install uv
run: pip install uv
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ repos:
- '-d {extends: default, rules: {line-length: disable, document-start: disable}}'
- '-s'
- repo: 'https://github.com/astral-sh/ruff-pre-commit'
rev: v0.15.17
rev: v0.15.20
hooks:
- id: ruff-check
args:
Expand Down
38 changes: 29 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,47 @@

#### Features

- Add `dbt_sqlserver_enable_safe_type_expansion` behaviour flag to allow safe column type widening during schema expansion: `varchar` → `nvarchar`, integer family promotions (`bit` → `tinyint` → `smallint` → `int` → `bigint`), and `numeric`/`decimal` precision/scale upgrades. Gated by the per-model `column_type_expansion_max_rows` config (default 1,000,000 rows). See [#699](https://github.com/dbt-msft/dbt-sqlserver/issues/699).
- Add Postgres-style `indexes` model config for tables, incrementals, seeds and snapshots, covering most `CREATE INDEX` options. [#535](https://github.com/dbt-msft/dbt-sqlserver/issues/535)
- Index names are deterministic definition hashes (`dbt_idx_` prefix); creation is idempotent and unchanged definitions are never rebuilt.
- Reconcile indexes against the config on incremental, DML-refresh and snapshot runs, applied as one atomic batch. Constraint-backing, legacy post-hook and `as_columnstore` indexes are never dropped.
- Index introspection reads the catalog lock-free throughout (`NOLOCK` on every `sys` view, not just `sys.indexes`), so it no longer queues behind concurrent index DDL.
- Add `drop_unmanaged_indexes` config (`false` (default) / `warn` / `true`) for indexes dbt didn't create.
- Validate cross-index config conflicts (multiple clustered indexes, clustered vs `as_columnstore`).
- Document the minimum supported SQL Server version (2017). Partitioning, `XML_COMPRESSION` and ordered columnstore are not yet expressible in the `indexes` config.
- Add SQL Server 2025 to the integration-test matrix (pyodbc and `mssql-python` backends, ODBC Driver 18) and document it as a supported version.
- Add `dbt_sqlserver_enable_safe_type_expansion` behaviour flag to allow safe column type widening during schema expansion: `varchar` → `nvarchar`, integer family promotions (`bit` → `tinyint` → `smallint` → `int` → `bigint`), and `numeric`/`decimal` precision/scale upgrades. Gated by the per-model `column_type_expansion_max_rows` config (default 1,000,000 rows). [#699](https://github.com/dbt-msft/dbt-sqlserver/issues/699).
- Add `prefer_single_alter_column` model config to use a single `ALTER COLUMN` statement instead of the add+update+drop+rename pattern when altering column types on tables.
- Add `string_type_instance()` to preserve the NVARCHAR/NCHAR type family during column expansion, fixing incorrect promotion of NVARCHAR/NCHAR to VARCHAR.
- Add `tinyint` and `bit` to the `is_integer()` type list for correct type detection.
- Add `dbt_sqlserver_use_dbt_transactions` flag for proper T-SQL `BEGIN TRAN`/`COMMIT`/`ROLLBACK` transaction handling. When enabled, operations use real server-side transactions instead of the default autocommit mode. [#708](https://github.com/dbt-msft/dbt-sqlserver/issues/708)
- Implement relation and column comment persistence via `persist_docs`. [#289](https://github.com/dbt-msft/dbt-sqlserver/issues/289)
- Add optional `mssql-python` connection backend alongside the default `pyodbc` backend. [#681](https://github.com/dbt-msft/dbt-sqlserver/issues/681)

#### Bugfixes

- Fix unit tests with empty fixtures (`rows: []`) generating invalid `limit 0` syntax; emit `top 0` instead. Also fix `get_columns_in_query()` for queries starting with a CTE, which broke unit tests with an empty `expect` block; such queries are now described via `sp_describe_first_result_set` instead of being executed. [#698](https://github.com/dbt-msft/dbt-sqlserver/issues/698)
- Fix catalog generation for NVARCHAR/NCHAR columns: use `user_type_id` instead of `system_type_id` in catalog.sql, preventing them from appearing as `SYSNAME` in `dbt docs`. [#637](https://github.com/dbt-msft/dbt-sqlserver/issues/637)
- Fix `varchar(max)` / `nvarchar(max)` columns being incorrectly treated as size `-1` during type expansion, preventing `varchar(max)` → `varchar(100)` narrowing and properly allowing `varchar(100)` → `varchar(max)` expansion.
- Fix seed table ingestion of empty numeric cells by inlining `null` literals instead of binding parameters. [#425](https://github.com/dbt-msft/dbt-sqlserver/issues/425)
- Guard `run_hooks` commit with `@@trancount` check for autocommit safety when running post-hooks. [#444](https://github.com/dbt-msft/dbt-sqlserver/issues/444)
- Fix `columnstore IF EXISTS` guard to check `object_id('schema.table')` correctly.
- Gate the `optimize_for_sequential_key` and `resumable` index options on the detected engine version: they require SQL Server 2019+, so on 2017/2016 the index is now built without them (with a warning) instead of failing with "is not a recognized CREATE INDEX option".
- Cast the aggregated column names in `sqlserver__describe_indexes` to `nvarchar(max)` so index reconciliation no longer fails with `STRING_AGG` error 9829 on wide tables (e.g. a clustered columnstore index spanning enough columns to exceed the 8000-byte cap). [#735](https://github.com/dbt-msft/dbt-sqlserver/issues/735)
- Escape single quotes in `query_tag` before building `OPTION (LABEL)` clause.
- Map Python `float` to SQL Server `float`, not `bigint`.
- Set default port to `1433` (instead of Postgres `5432`) in `dbt init` profile template.
- Make `drop_schema_name` usable again. [#563](https://github.com/dbt-msft/dbt-sqlserver/issues/563)
- Raise `DbtRuntimeError` instead of `ValueError` for missing access token.

#### Features
#### Under the hood

- Add Postgres-style `indexes` model config for tables, incrementals, seeds and snapshots, covering most `CREATE INDEX` options. [#535](https://github.com/dbt-msft/dbt-sqlserver/issues/535)
- Index names are deterministic definition hashes (`dbt_idx_` prefix); creation is idempotent and unchanged definitions are never rebuilt.
- Reconcile indexes against the config on incremental, DML-refresh and snapshot runs, applied as one atomic batch. Constraint-backing, legacy post-hook and `as_columnstore` indexes are never dropped.
- Index introspection reads the catalog lock-free throughout (`NOLOCK` on every `sys` view, not just `sys.indexes`), so it no longer queues behind concurrent index DDL.
- Add `drop_unmanaged_indexes` config (`false` (default) / `warn` / `true`) for indexes dbt didn't create.
- Validate cross-index config conflicts (multiple clustered indexes, clustered vs `as_columnstore`).
- Document the minimum supported SQL Server version (2017). Partitioning, `XML_COMPRESSION` and ordered columnstore are not yet expressible in the `indexes` config.
- Consolidate linting tooling (isort, flake8, pycln, absolufy-imports) into Ruff. [#707](https://github.com/dbt-msft/dbt-sqlserver/issues/707)
- Drop Python 3.9 from publish-docker CI matrix.
- Remove dead clone macro and unused view scaffolding variable.
- Standardise devcontainer ODBC environment setup. [#722](https://github.com/dbt-msft/dbt-sqlserver/issues/722)
- Disable dbt telemetry and version check in test environment.
- Add `pytest-cov` and fix dead `from_dict` override found by coverage.
- Bump pip and GitHub Actions dependencies.

### v1.10.0

Expand Down
7 changes: 3 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,6 @@ All CI/CD pipelines are using GitHub Actions. The following pipelines are availa

* `publish-docker`: publishes the image we use in all other pipelines.
* `unit-tests`: runs the unit tests for each supported Python version.
* `integration-tests-azure`: runs the integration tests for Azure SQL Server.
* `integration-tests-sqlserver`: runs the integration tests for SQL Server.
* `release-version`: publishes the adapter to PyPI.

Expand All @@ -128,7 +127,7 @@ The following environment variables are available:

## Releasing a new version

Make sure the version number is bumped in `__version__.py`. Then, create a git tag named `v<version>` and push it to GitHub.
A GitHub Actions workflow will be triggered to build the package and push it to PyPI.
Make sure the version number is bumped in `dbt/adapters/sqlserver/__version__.py`. Then publish a GitHub Release with a tag named `v<version>`.
A GitHub Actions workflow will be triggered to build the package and push it to PyPI.

If you're releasing support for a new version of `dbt-core`, also bump the `dbt_version` in `setup.py`.
If you're releasing support for a new version of `dbt-core`, also bump the `dbt-core` constraint in `dependencies` in `pyproject.toml`.
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,23 @@

[dbt](https://www.getdbt.com) adapter for Microsoft SQL Server and Azure SQL services.

The adapter supports dbt-core 0.14 or newer and follows the same versioning scheme.
E.g. version 1.1.x of the adapter will be compatible with dbt-core 1.1.x.
The adapter supports dbt-core 1.10 or newer and follows the same versioning scheme.
E.g. version 1.10.x of the adapter is compatible with dbt-core 1.10.x.

The minimum supported SQL Server version is SQL Server 2017.
## Supported SQL Server versions

The adapter is tested against the following SQL Server versions:

| SQL Server version | Supported |
|---|---|
| SQL Server 2017 | ✅ (minimum supported version) |
| SQL Server 2019 | ✅ |
| SQL Server 2022 | ✅ |
| SQL Server 2025 | ✅ |

The minimum supported SQL Server version is SQL Server 2017; older versions are not supported.

SQL Server 2017, 2019, 2022, and 2025 are covered by the integration test suite. Azure SQL Database and Azure SQL Managed Instance are not covered by the integration test suite, but are expected to be compatible.

## Documentation

Expand Down Expand Up @@ -180,6 +193,29 @@ flags:

**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.

### `as_columnstore`

*(default: `true`)* When building a table, the adapter creates a [clustered columnstore index](https://learn.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview) (CCI) on it. Set `as_columnstore: false` to build a plain rowstore table instead.

This matters for any table containing a `(n)varchar(max)` or other LOB column, because SQL Server does not allow those data types to participate in a columnstore index. The table build fails with:

> Column '...' has a data type that cannot participate in a columnstore index.

A common case is dbt's [test failure storage](https://docs.getdbt.com/reference/resource-configs/store_failures): the audit tables can contain `VARCHAR(MAX)` columns (dbt's `STRING` type maps to `VARCHAR(MAX)`), so disable the CCI on those resources:

```yaml
# dbt_project.yml
data_tests:
+store_failures: true
+as_columnstore: false # avoids CCI on (n)varchar(max) audit columns
```

You can also set it per model:

```sql
{{ config(materialized="table", as_columnstore=false) }}
```

## Contributing

[![Unit tests](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml)
Expand Down
2 changes: 1 addition & 1 deletion dbt/adapters/sqlserver/sqlserver_configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
class SQLServerConfigs(AdapterConfig):
auto_provision_aad_principals: Optional[bool] = False
prefer_single_alter_column: Optional[bool] = False
column_type_expansion_max_rows: Optional[int] = None
column_type_expansion_max_rows: int = 1000000
indexes: Optional[Tuple[SQLServerIndexConfig, ...]] = None
# false (default) | warn | true - how index reconciliation treats
# droppable indexes dbt didn't create (YAML may supply bool or str)
Expand Down
67 changes: 56 additions & 11 deletions dbt/adapters/sqlserver/sqlserver_connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@

logger = AdapterLogger("sqlserver")

# Attribute used to stash the in-flight pyodbc / mssql-python cursor on a
# Connection so cancel() can reach it from another thread. See cancel().
_IN_FLIGHT_CURSOR_ATTR = "_dbt_sqlserver_in_flight_cursor"


class SQLServerConnectionManager(SQLConnectionManager):
TYPE = "sqlserver"
Expand Down Expand Up @@ -142,8 +146,40 @@ def connect() -> Any:

return conn

def cancel(self, connection: Connection):
logger.debug("Cancel query")
def cancel(self, connection: Connection) -> None:
"""Cancel the in-flight query on ``connection``, if any.

dbt-core's ``cancel_open`` calls this for sibling connections when a
run is interrupted (Ctrl-C) or another thread errors. We cancel by
calling ``Cursor.cancel()`` on the connection's in-flight cursor:
pyodbc exposes it and it is explicitly designed to be called from
another thread (it issues ``SQLCancel``); mssql-python's cursor is
used the same way when it supports it. Cancellation targets statement
execution. If no statement is in flight, the cursor is gone, or the
backend cursor does not support cancellation, this is a best-effort
no-op.
"""

cursor = getattr(connection, _IN_FLIGHT_CURSOR_ATTR, None)
if cursor is None:
logger.debug(f"No in-flight query to cancel for connection {connection.name}.")
return

cancel_cursor = getattr(cursor, "cancel", None)
if not callable(cancel_cursor):
logger.debug(
f"Backend cursor for connection {connection.name} does not "
"support cancellation; skipping."
)
return

try:
logger.debug(f"Cancelling in-flight query for connection {connection.name}.")
cancel_cursor()
except Exception as exc:
# The statement may have completed between the lookup and the
# cancel; cancellation is best-effort, so swallow and log.
logger.debug(f"Failed to cancel query for connection {connection.name}: {exc}")

def add_begin_query(self):
if self._dbt_sqlserver_use_dbt_transactions:
Expand Down Expand Up @@ -225,7 +261,7 @@ def _execute_query_with_retry(

fire_event(
AdapterEventDebug(
message=(
base_msg=(
f"Got a retryable error {type(e)}. {retry_limit - attempt} "
"retries left. Retrying in 1 second.\n"
f"Error:\n{e}"
Expand Down Expand Up @@ -270,16 +306,25 @@ def _execute_query_with_retry(
pre = time.time()

cursor = connection.handle.cursor()
# Track the in-flight cursor so cancel() / cancel_open() can stop it
# from another thread (e.g. on Ctrl-C); cleared once execution
# finishes. See cancel().
setattr(connection, _IN_FLIGHT_CURSOR_ATTR, cursor)
credentials = self.get_credentials(connection.credentials)

_execute_query_with_retry(
cursor=cursor,
sql=sql,
bindings=bindings,
retryable_exceptions=retryable_exceptions,
retry_limit=(credentials.retries if credentials.retries > 3 else retry_limit),
attempt=1,
)
try:
_execute_query_with_retry(
cursor=cursor,
sql=sql,
bindings=bindings,
retryable_exceptions=retryable_exceptions,
# ``retries`` caps total execute attempts, so ``retries: 1``
# means a single attempt with no retry.
retry_limit=credentials.retries,
attempt=1,
)
finally:
setattr(connection, _IN_FLIGHT_CURSOR_ATTR, None)

if is_pyodbc_handle(connection.handle):
connection.handle.add_output_converter(-155, byte_array_to_datetime)
Expand Down
Loading
Loading