Skip to content

IO: Windows paths crash with 'Unrecognized filesystem type in URI: c' #3720

Description

@qzyu999

Apache Iceberg version

Latest main (also affects all released versions)

Please describe the bug

On Windows, passing a local absolute path (e.g. C:\Users\data\warehouse) to any PyIceberg IO operation crashes with:
from pyiceberg.io.pyarrow import PyArrowFileIO

io = PyArrowFileIO()
f = io.new_input(r"C:\Users\me\warehouse\table\metadata.json")
# ValueError: Unrecognized filesystem type in URI: c

an example from #1005 is:

from pyiceberg.catalog import load_catalog

catalog = load_catalog(
    "my_catalog",
    **{
        "type": "sql",
        "uri": "sqlite:///catalog.db",
        "warehouse": r"C:\Users\me\iceberg_warehouse",
    },
)
# UserWarning: No preferred file implementation for scheme: c

The root cause: Python's urlparse treats Windows drive letters as URI schemes (C:\... scheme='c'). Since 'c' is not in PyIceberg's scheme-to-FileIO mapping, all local file operations fail.

This affects three independent parse sites:

  1. _infer_file_io_from_scheme() in pyiceberg/io/__init__.py
  2. PyArrowFileIO.parse_location() in pyiceberg/io/pyarrow.py
  3. FsspecFileIO._get_fs_from_uri() in pyiceberg/io/fsspec.py

Related issues

Proposed fix

A platform-guarded predicate that detects when urlparse has misidentified a Windows drive letter as a URI scheme, and remaps it to 'file':

import sys

def _is_windows_drive_letter(scheme: str) -> bool:
    return sys.platform == 'win32' and len(scheme) == 1 and scheme.isalpha()

Applied at all three parse sites. Zero behavior change on non-Windows (the platform check short-circuits). Includes platform-conditional tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions