Skip to content

allow passing kwargs to sftp client - #41

Merged
shcheklein merged 4 commits into
fsspec:mainfrom
ttomasz:feature/allow-passing-params-to-sftp-client
Aug 5, 2026
Merged

allow passing kwargs to sftp client#41
shcheklein merged 4 commits into
fsspec:mainfrom
ttomasz:feature/allow-passing-params-to-sftp-client

Conversation

@ttomasz

@ttomasz ttomasz commented Feb 17, 2024

Copy link
Copy Markdown
Contributor

Fixes #39

@codecov-commenter

codecov-commenter commented Feb 17, 2024

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.64%. Comparing base (15aac74) to head (032c69e).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #41      +/-   ##
==========================================
- Coverage   94.30%   93.64%   -0.67%     
==========================================
  Files          13       12       -1     
  Lines         738      803      +65     
==========================================
+ Hits          696      752      +56     
- Misses         42       51       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ttomasz

ttomasz commented Mar 11, 2024

Copy link
Copy Markdown
Contributor Author

@efiop Would you mind reviewing this PR?

Comment thread sshfs/spec.py Outdated
@efiop

efiop commented Mar 18, 2024

Copy link
Copy Markdown
Member

Hey, @ttomasz , thanks for the PR! Please see some suggestions above.

@shcheklein

Copy link
Copy Markdown
Collaborator

@ttomasz a gentle reminder :) or can we close this PR?

@ttomasz

ttomasz commented Jun 1, 2024

Copy link
Copy Markdown
Contributor Author

Sorry, was quite busy past few months 😅 I should be able to finish this up in June.

@ttomasz
ttomasz force-pushed the feature/allow-passing-params-to-sftp-client branch from 2a39ec4 to 0a8a32d Compare June 16, 2024 16:57
@ttomasz

ttomasz commented Jun 16, 2024

Copy link
Copy Markdown
Contributor Author

Changed popping kwargs to separate parameter for stuff to pass to underlying library.

@ttomasz

ttomasz commented Jun 16, 2024

Copy link
Copy Markdown
Contributor Author

Sorry for the mess with the commits. Do you want me to revert client_args/connect_args to being kwargs instead of parameter? Or is the current state ok?

@shcheklein
shcheklein requested a review from efiop June 16, 2024 18:03
@erikvanzijst

Copy link
Copy Markdown

Can I ask for this PR to get revived? I too need to pass options to asyncssh.SSHClientConnection.start_sftp_client().

I need to override the default sftp_version due to hitting ronf/asyncssh#587

@efiop
efiop removed their request for review December 23, 2025 02:35
@shcheklein
shcheklein requested a lite review from Copilot August 4, 2026 22:12
Add sftp_client_kwargs parameter to SSHFileSystem and the channel
pools, forwarded to asyncssh.SSHClientConnection.start_sftp_client
(e.g. env, send_env, path_encoding, path_errors, sftp_version).

Squashed and rebased onto main from fsspec#41.

Fixes fsspec#39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

As written it likely doesn’t fully resolve Issue #39 because path_encoding passed as a top-level kwarg would still be forwarded to asyncssh.connect(), and the new behavior also needs targeted tests.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR adds a way to pass asyncssh.SSHClientConnection.start_sftp_client() keyword arguments through SSHFileSystem, enabling configuration such as path_encoding (referenced in Issue #39) without overloading asyncssh.connect() options.

Changes:

  • Add sftp_client_kwargs to SSHFileSystem and plumb it through the connection/pool setup.
  • Update the SFTP channel pool base class to store sftp_client_kwargs and forward them into start_sftp_client(**...).
  • Refactor connection argument passing to distinguish connect_args from SFTP-client args.
File summaries
File Description
sshfs/spec.py Adds sftp_client_kwargs and separates connect_args from SFTP-client kwargs during connection/pool creation.
sshfs/pools/base.py Stores sftp_client_kwargs on the pool and forwards them to client.start_sftp_client(**...).
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 3
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread sshfs/spec.py
Comment thread sshfs/spec.py Outdated
Comment thread sshfs/pools/base.py
Assert the options reach the channel pool and that a filesystem
constructed with sftp_version=3 works end to end. Trim the call-site
comments and wrap the docstring within the line limit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@shcheklein
shcheklein force-pushed the feature/allow-passing-params-to-sftp-client branch from 2292727 to d02015d Compare August 4, 2026 22:16
@shcheklein

Copy link
Copy Markdown
Collaborator

Rebased this onto main (squashed the branch history into a single commit, keeping your authorship) and pushed:

  • d7190a2 — your change, rebased
  • d02015d — a test covering sftp_client_kwargs end to end, plus minor doc polish

To answer the open question: the explicit sftp_client_kwargs / connect_args parameters are the right shape — no need to revert to kwargs-popping. Thanks for the patience on this one, and thanks @erikvanzijst for the nudge.

… annotation

Both raised by Copilot review: the pool-level test captures the kwargs
actually passed to start_sftp_client (the fs-level test only proves
they are stored), and the timeout annotation now allows None as the
docstring describes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

It currently introduces a backward-incompatible API behavior by always passing sftp_client_kwargs into pool_type(...), which will break existing custom pool implementations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

sshfs/spec.py:126

  • sftp_client_kwargs is always normalized to {} and then unconditionally passed into pool_type(...). That makes every custom pool_type (which previously only needed to accept client and max_channels) start failing with TypeError: __init__() got an unexpected keyword argument 'sftp_client_kwargs' even when the caller doesn’t use this new feature. Consider only passing sftp_client_kwargs to the pool when it’s actually provided/non-empty, so existing custom pool implementations remain compatible by default.
        pool = pool_type(
            client,
            max_channels=max_sftp_channels,
            sftp_client_kwargs=sftp_client_kwargs,
        )
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

path_encoding=None flows through sftp_client_kwargs to a real channel
and asyncssh delivers paths as raw bytes, which is the workaround for
servers with non-UTF-8 file names that motivated the issue.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@shcheklein

Copy link
Copy Markdown
Collaborator

Re Copilot's "not ready to approve" note about always passing sftp_client_kwargs into pool_type(...): intentional, keeping as-is. pool_type is an sshfs-specific extension point — custom pools are expected to subclass BaseSFTPChannelPool, whose __init__ already accepts the option (plus **kwargs), so the natural implementation keeps working; both shipped pools do. Passing the option only when non-empty would just trade an obvious construction-time TypeError for a delayed one that fires the first time someone actually uses sftp_client_kwargs with a legacy pool. Not part of the common fsspec public API surface, so not treated as a compatibility blocker.

@shcheklein
shcheklein merged commit 29d3f6c into fsspec:main Aug 5, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

how to pass path_encoding parameter?

6 participants