Skip to content

Fix MySQL metadata database failing when only PyMySQL is installed - #71251

Open
1fanwang wants to merge 1 commit into
apache:mainfrom
1fanwang:mysql-pymysql-driver-fix
Open

Fix MySQL metadata database failing when only PyMySQL is installed#71251
1fanwang wants to merge 1 commit into
apache:mainfrom
1fanwang:mysql-pymysql-driver-fix

Conversation

@1fanwang

@1fanwang 1fanwang commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

related: #70235

configure_adapters() raises a RuntimeError telling the user to install
mysqlclient whenever mysqlclient is absent — even when PyMySQL is installed
and is the driver named in the connection string. The error is raised inside a
try that only catches ImportError, so it escapes instead of falling through
to the PyMySQL branch a few lines below, which the surrounding code clearly
intends as the alternative:

try:
    try:
        import MySQLdb.converters
    except ImportError:
        raise RuntimeError("You do not have `mysqlclient` package installed. ...")
    MySQLdb.converters.conversions[Pendulum] = MySQLdb.converters.DateTime2literal
except ImportError:          # cannot catch the RuntimeError above
    pass
try:
    import pymysql.converters      # never reached when mysqlclient is missing

So mysql+pymysql:// fails during import airflow, before any command runs. The
docs recommend mysqlclient and note it is the only driver covered by CI, but
they also point users at the SQLAlchemy MySQL dialect page if they want a
different one, so hard-failing here is not intended.

Each driver is now configured independently, and the error is raised only when
neither is importable — the condition the message actually describes.

#70235 fixes a separate PyMySQL failure further along the same path (dynamic
PREPARE/EXECUTE SQL in migration 0017_2_9_2). That one blocks a migration;
this one blocks import airflow. The two do not overlap.

Testing Done

Raw logs

Before, PyMySQL installed, mysqlclient absent:

$ airflow db migrate
  File ".../airflow/settings.py", line 665, in configure_adapters
    import MySQLdb.converters
ModuleNotFoundError: No module named 'MySQLdb'

During handling of the above exception, another exception occurred:
  File ".../airflow/settings.py", line 667, in configure_adapters
    raise RuntimeError(
RuntimeError: You do not have `mysqlclient` package installed. ...

After, same environment:

[info] Performing upgrade to the metadata database
[info] Running stamp_revision  -> 7a98f1b7dbd3
[info] Database migration done!

followed by a real DAG run on that database:

sum of squares = 55
DagRun Finished: dag_id=tidb_smoke, run_duration=4.42, state=success

With neither driver installed, the error still fires and now names both:

RuntimeError: You have a MySQL connection string but neither `mysqlclient` nor
`PyMySQL` is installed. Install one of them, e.g. `pip install mysqlclient` ...
or `pip install PyMySQL`.

Was generative AI tooling used to co-author this PR?
  • Yes — GitHub Copilot CLI (Claude Opus 5)

Generated-by: GitHub Copilot CLI (Claude Opus 5) following the guidelines

configure_adapters() raised a RuntimeError telling the user to install
mysqlclient whenever mysqlclient was absent, even though PyMySQL was
present and is configured a few lines below. The RuntimeError was raised
inside a try block that only catches ImportError, so it escaped instead of
falling through to the PyMySQL branch as the surrounding code intended.

Signed-off-by: 1fanwang <1fannnw@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant