SFTP¶

Connect Foundry to SFTP servers to sync data between folders and Foundry datasets.

Supported capabilities¶

Capability	Status
Exploration	🟢 Generally available
Bulk import	🟢 Generally available
Incremental	🟢 Generally available
Export tasks	🟡 Sunset (legacy SFTP source only)
File exports	🟢 Generally available

Data model¶

The connector can transfer files of any type into Foundry datasets. File formats are preserved and no schemas are applied during or after the transfer. Apply any necessary schema to the output dataset, or write a downstream transformation to access the data.

Performance and limitations¶

There is no limit to the size of transferable files. However, network issues can result in failures of large-scale transfers. In particular, Foundry syncs that take more than two days to run will be interrupted. To avoid network issues, we recommend using smaller file sizes and limiting the number of files that are ingested in every execution of the sync. Syncs can be scheduled to run frequently.

Setup¶

Open the Data Connection application and select + New Source in the upper right corner of the screen.
Select SFTP from the available connector types.
Follow the additional configuration prompts to continue the setup of your connector using the information in the sections below.

:::callout{theme="warning"} To access on-premises SFTP servers, we recommend using an agent proxy connection. :::

Learn more about setting up a connector in Foundry.

Authentication¶

Connections can be established to the SFTP server using either a password or an SSL private key. If you experience any issues when connecting to the server, verify that the authentication details are correct by attempting connections via an SFTP client outside of Foundry. The user for this connection must have access to the root directory as well as permissions to read and list all files and directories inside the root directory.

Option	Required?	Description
`Username`	Yes	The SFTP login username.
`Password`	No	The SFTP login password.
`Private key`	No	A SSL private key in RSA or OpenSSH format. The SFTP server must have the public key correctly configured as an authorized key for the username provided.

Networking¶

If your SFTP connection runs in Foundry, you must add a network egress policy to allowlist the connection.

Egress policies should be created for the SFTP server hostname (if connecting via a domain) as well as the IPs to which the SFTP server host name resolves. After correctly configuring the egress policy, a non-standard connection port (22) still may not work due to host name validation. If this happens, report an issue to Palantir support with a list of policies applied to this connection.

:::callout{theme="warning"} If the domain for the server resolves to multiple domains and/or servers, all of the associated domains and their related IPs need to be whitelisted. To verify whether a server resolves to multiple domains and/or servers, run the command dig <domain> from your terminal for the server you are trying to connect to and review the answer section. :::

On a UNIX machine, find the IP address that your server domain resolves to by running:

dig <domain> +short

:::callout{theme="neutral"} If an agent is running your connector, ensure that the agent's server can establish network connections to the SFTP servers and that firewalls are configured appropriately. We recommend verifying network connections using netcat ↗ or a similar utility when needed. :::

Certificates and private keys¶

Server SSH key¶

SFTP servers identify themselves using a public key. This key can be obtained either from the server's administrator (preferable) or by running the command below from any Linux server that has network access to the SFTP server in question:

ssh-keyscan -t rsa -p {port} {hostname}  | awk '{print $3}'

This key must be configured in the Host key section in the Connection details page. If it is not possible to obtain this key, or if the key changes frequently, key verification can be disabled using the Accept any host key toggle. Note that disabling key verification is unsafe and discouraged.

Hostname validation¶

Foundry attempts hostname validation for all egress routes. Network traffic outside of port 22 are sometimes not verified, resulting in hanging connections and/or timeout errors. If errors continue to occur despite proper egress policy configuration, report an issue to Palantir support with a list of policies for which you want to disable hostname validation.

Configuration options¶

Option	Required?	Default	Description
`Hostname`	Yes		The domain name pointing to the server or the IP address of the server.
`Port`	Yes		The port on which the SFTP server is running.
`Root directory`	Yes		The directory on the server that will be used as the starting directory for all requests via this connection.
`Username`	Yes		The SFTP login username.
`Password`	No		The SFTP login password.
`Private key`	No		A SSL private key in RSA or OpenSSH format. The SFTP server must have the public key correctly configured as an authorized key for the username provided.
`Host key`	Yes		See Server SSH key above for more information.
`Accept any host key`	No	`false`	See Server SSH key above for more information.
`Proxy`	No	Direct	Enable to allow a proxy connection to SFTP.
`Timeout`	No	0	See Timeouts below for more information.
`Maximum concurrent connections`	No	The maximum parallel uploads supported by the runtime	A higher value typically results in faster transfer speeds. Setting this value to higher than the maximum parallel uploads that the runtime supports will not have any affect. Use this setting if your SFTP server can limit the number of concurrent connections to the server to manage the load.
Connection settings	No		See the Connection settings section below for more information.
`Logs`	No	`Info`	The level of logs to be recorded. Selecting a level will record all logs at that level or above; for example, a level of `Error` will record logs at both `Error` and `Fatal` levels. Refer to the Logs section below for an example.
`Extension Negotiation`	No	N/A	These settings control the extension negotiation messages sent by the connector to the SFTP server. Disabling these feature can sometimes resolve issues with establishing connections. See below for more guidance.

Timeouts¶

A value of 0 indicates that connections will wait indefinitely for every response from the server. This timeout controls how long the connector waits while establishing a connection with the server and for every command run on the server. The timeout is not triggered if the server continues to respond. Setting this to a low value will not prevent large file transfers but can help with debugging hanging connections.

Connection settings¶

When establishing an SFTP connection, both the server and client (Foundry) must negotiate necessary details. These details include information to help the client and server decide which cryptographic algorithms to use. Due to compatibility issues, you may need to manually configure these settings instead of allowing them to be automatically negotiated. The table below lists the available configuration options you can use to adjust the negotiation as required. For more information, review the FAQ ↗.

Setting	Description
`Key exchange algorithms`	An explicit list of algorithms that can be used for key exchange. At least one algorithm must be selected.
`Ciphers`	A list of ciphers that can be used for encryption.
`Message Authentication Code (MAC)`	A list of MAC types.
`Host key types`	The types of host keys that can be used for this connection.
`Public key types`	The algorithms that will be used for public key authentication.

Sync data from SFTP¶

:::callout{theme="neutral"} For more complex scenarios, use pro-code alternatives. :::

The SFTP connector uses the file-based sync interface.

Troubleshooting¶

Borrowing connection for ls failed¶

If a build fails with the error Borrowing connection for ls failed, and the error contains SocketException: Connection reset, check build logs to see if a network connection to the SFTP server has been established. If the logs do not contain the lines Connection established and Remote version string, firewalls are blocking egressing traffic from the connection runtime, or the target SFTP server is not allowing the incoming connection. Check egress policies and firewall rules, and contact the SFTP server administrators to resolve network connectivity issues. Below is an example of successful connection logs:

jschLogMessage: Connecting to <HOSTNAME> port 2232
jschLogMessage: Connection established
jschLogMessage: Remote version string: <value identifying the type of server>

Hanging connections¶

If connections hang without any progress or obvious failures, set the timeout to a small value (1000, for example) to help identify which call is hanging. For example, it may be taking longer to list a particular directory, or parts of a file are taking a long time to read.

Hanging connections can occur due to hostname validation, and setting a small timeout will allow failure logs to be generated that can help support teams identify root causes for connection issues.

Agent worker connections [Legacy]¶

:::callout{theme="warning" title="Legacy"} This section applies only to existing legacy agent worker sources. :::

If you are using an egress proxy load balancer, note that FTP is a stateful protocol. Using a load balancer can cause the sync to fail (non-deterministically) if sequential requests do not originate from the same IP.

Export data to SFTP¶

:::callout{theme="neutral"} For more complex scenarios, use pro-code alternatives. :::

The connector can copy files from a Foundry dataset to any location on the SFTP server.

To export to a SFTP server, first enable exports for your SFTP connector. Then, create a new export.

:::callout{theme="neutral"} In many cases, Foundry prefixes the underlying filenames of datasets with spark/ by default. When trying to export a filename with that prefix to an SFTP instance, the connector will try to create a spark folder by default, which will fail if the SFTP user used by Foundry does not have permission to create folders on the SFTP instance. To prevent this, rename the files before exporting. Review the controlling output file structure documentation for an example of how to rewrite file paths before export. :::

Use SFTP sources in code¶

Pro-code alternatives can be used to connect to SFTP sources for more complex scenarios.

The examples below demonstrate how to connect to a SFTP source using the Paramiko ↗ Python client in an external transform.

Write files to SFTP¶

This example shows how to write a list of XLSX files to an SFTP server using OpenSSH private key authentication.

from transforms.api import Output, Input, TransformOutput, TransformInput, transform
from transforms.external.systems import external_systems, Source, ResolvedSource
import pandas as pd
import paramiko
import io

@external_systems(
    sftp_source=Source("<source_rid>")
)
@transform(
    output_file=Output(
        "<output_dataset_rid>"
    ),
    xlsx_files=Input("<input_dataset_rid>"),
)
def compute(
    sftp_source: ResolvedSource,
    output_file: TransformOutput,
    xlsx_files: TransformInput, # input dataset containing a list of xlsx files
) -> None:

    # 1. SFTP connection setup
    hostname = "<hostname>"
    port = <port> # (usually 22)
    username = "<username>"
    private_key_string = sftp_source.get_secret("privateKeyString")

    # 2. Transform private key string to proper input parameter of paramiko
    # Use StringIO to treat the string as a file-like object
    private_key_stream = io.StringIO(private_key_string)
    private_key = paramiko.Ed25519Key.from_private_key(private_key_stream)

    # 3. Define the SFTP client
    transport = paramiko.Transport((hostname, port))
    transport.connect(username=username, pkey=private_key)
    sftp = paramiko.SFTPClient.from_transport(transport)

    # 4. Iterate over input files and upload each to SFTP
    remote_paths = []
    fs = xlsx_files.filesystem()
    input_files = fs.ls()
    for f in input_files:
        with fs.open(f.path, "rb") as fileobj:  # Open in binary mode
            remote_path = get_remote_path(f.path)
            with sftp.file(remote_path, "wb") as remote_file:
                remote_file.write(fileobj.read())  # Write bytes directly
            remote_paths.append({"file_name": f.path, "remote_path": remote_path}) # record mapping of file name to uploaded path

    # 3. Return dataset of remote paths
    output_file.write_pandas(pd.DataFrame.from_records(remote_path))


def get_remote_path(file_path):
    """
    Define the remote path where the file will be uploaded.
    """
    remote_target_folder = "<example_target_folder>"
    clean_file_path = file_path.replace(" ", "_")
    return f"{remote_target_folder}/{clean_file_path}"

Legacy SFTP connector¶

:::callout{theme="warning"} The following sections cover the legacy SFTP connector and should only be referenced when maintaining existing uses of this connector. Any new creation of SFTP sources should follow the steps described in the sections above. :::

Migrate to the new SFTP connector¶

Migration from the legacy to new SFTP connector is not automated and must be done manually. The new connector supports almost identical configuration options, and a new source should be created to replace the old source.

Note that:

Passwords cannot be retrieved from the old source but must be requested from the server administrators if you no longer have access to them.
privateKeyFile and privateKeyPassphrase are no longer supported, and the contents of the private key should be entered directly in the source settings if using private key authentication.
knownHostsFile is no longer supported, and the base64HostKey must be configured correctly in the source settings.

Set up a legacy SFTP connector¶

To set up the legacy SFTP connector, navigate to the Data Connection application, then select + New source. On the source selection screen, scroll down to Advanced, then select Custom source. Follow the prompts in the configuration screens to continue setting up your SFTP connector.

A complete example of the SFTP YAML configuration can look like the following:

:::callout{theme="warning"} Note that the type of the legacy SFTP source must be magritte-sftp. :::

type: magritte-sftp
hostname: my.host.name
port: 22
username: username
password: '{{pasword}}'
rootDirectory: /home/palantir/sftp
base64Hostkey: >-
  FULL CONTENTS OF THE HOST KEY
unsafeAcceptAllHostKeys: false
privateKeyFile: location/of/private/key/file
privateKeyPassphrase: '{{passphrase}}'
proxyConfiguration:
    host: hostname.of.proxy
    port: port.of.proxy
    type: HTTP (can be HTTP/HTTPS/SOCKS)
    credentials:
        username: proxyUsername
        password: `{{proxyPassword}}`
timeout: 0
maxConcurrentConnections: 10
sessionParams:
    customKex:
        - "ecdh-sha2-nistp256"
        - "ecdh-sha2-nistp384"
knownHostsFile: /path/to/known_hosts_file

Export with the legacy SFTP connector¶

:::callout{theme="warning"} We generally do not recommend using export tasks to write data back to external sources, and the following documentation is only meant to support existing legacy export tasks. Any new exports must be created with the updated, first-class SFTP connector. :::

To export data, you must configure an export task. Navigate to the Project folder that contains the connector to which you want to export. Right select on the connector name, then select Create Data Connection Task.

In the left panel of the Data Connection view:

Verify the Source name matches the connector you want to use.
Add an Input named inputDataset. The input dataset is the Foundry dataset being exported.
Add an Output named outputDataset. The output dataset is used to run, schedule, and monitor the task.
Finally, add a YAML block in the text field to define the task configuration.

:::callout{theme="neutral"} The labels for the connector and input dataset that appear in the left side panel do not reflect the names defined in the YAML. :::

Use the following options when creating the export task YAML:

Option	Required?	Description
`directoryPath`	Yes	The directory where files will be written. The path must end with a trailing `/`.
`excludePaths`	No	A list of regular expressions; files with names matching these expressions will not be exported.
`rewritePaths`	No	See section below for more information.
`uploadConfirmation`	No	When the value is `exportedFiles`, the output dataset will contain a list of files that were exported.
`createTransactionFolders`	No	When enabled, data will be written to a subfolder within the specified `directoryPath`. Every subfolder will have a unique name for every exported transaction in Foundry and is based on the time the transaction was committed in Foundry.
`incrementalType`	No	For datasets that are built incrementally, set to `incremental` to only export transactions that occurred since the previous export.
`flagFile`	No	See Flag file section for more information.
`spanMultipleViews`	No	If `true`, multiple transactions in Foundry will be exported at once. If `false`, a single build will export only one transaction at a time. If incremental is enabled, the files from the oldest transaction will be exported first.

`rewritePaths`¶

If the first key matches the filename, the capture groups in the key will be replaced with the value. The value itself can have extra sections to add metadata to the filename.

If the value contains:

${dt:javaDateExpression}: This part of the value will be replaced by the timestamp of when the file is being exported. The javaDateExpression follows the DateTimeFormatter ↗ pattern.
${transaction}: This part of the value will be replaced with the Foundry transaction ID of the transaction that contains this file.
${dataset}: This part of the value will be replaced with the Foundry dataset ID of the dataset that contains this file.

Example:

Consider a file in a Foundry dataset called "spark/file_name", in a transaction with ID transaction_id and dataset ID dataset_id. If you use the expression fi.*ame as the key and file_${dt:DD-MM-YYYY}-${transaction}-${dataset}_end as a value, when the file is written to SFTP, it will be stored as spark/file_79-03-2023-transaction_id-dataset_id_end.

Flag file¶

The connector can write an empty flag file to the SFTP server once all data is copied for a given build. The empty file signifies that the contents are ready for consumption and will no longer be modified. The flag file will be written to the directoryPath. However, if createTransactionFolders is enabled, a flag file will be made for every folder to which content was written. If flag files are enabled, and the flag file is called confirmation.txt, all flag files will be written at once after files being exported in the build have been written.

:::callout{theme="neutral"} Flag files are written at the end of a build, not when a subfolder has been exported. :::

If the files in the SFTP server are newer than the flag file, this normally indicates that the previous export was not successful or an export is in progress for that folder.

An example of a simple export configuration is:

type: export-sftp-task
directoryPath: export-directory/subdirectory

After you configure the export task, select Save in the upper right corner.

:::callout{theme="neutral"} The directory specified in the directoryPath must already exist on the SFTP server. :::

:::callout{theme="neutral"} The directory specified in the directoryPath is relative to the root directory. For example, if the connection is configured with rootDirectory: /home/palantir/sftp and directoryPath is export-directory, then the files will be exported to /home/palantir/sftp/export-directory/. :::

Logs¶

Select the Logs tab in Data Connections to view logs from explorations and builds. These logs are useful for debugging connection issues. Search for jschLogMessage to view details.

The log details below show a successful login attempt:

jschLogMessage: Connecting to <HOSTNAME> port 2232
jschLogMessage: Connection established
jschLogMessage: Remote version string: SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u1
jschLogMessage: Local version string: SSH-2.0-JSCH_0.2.12
jschLogMessage: CheckCiphers: chacha20-poly1305@openssh.com
jschLogMessage: CheckKexes: curve25519-sha256,curve25519-sha256@libssh.org,curve448-sha512
jschLogMessage: CheckSignatures: ssh-ed25519,ssh-ed448
jschLogMessage: server_host_key proposal before known_host reordering is: ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-dss,ssh-rsa
jschLogMessage: server_host_key proposal after known_host reordering is: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-dss
jschLogMessage: SSH_MSG_KEXINIT sent
jschLogMessage: SSH_MSG_KEXINIT received
jschLogMessage: server proposal: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
jschLogMessage: server proposal: host key algorithms: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
jschLogMessage: server proposal: ciphers c2s: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
jschLogMessage: server proposal: ciphers s2c: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
jschLogMessage: server proposal: MACs c2s: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
jschLogMessage: server proposal: MACs s2c: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
jschLogMessage: server proposal: compression c2s: none,zlib@openssh.com
jschLogMessage: server proposal: compression s2c: none,zlib@openssh.com
jschLogMessage: server proposal: languages c2s:
jschLogMessage: server proposal: languages s2c:
jschLogMessage: client proposal: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,ext-info-c
jschLogMessage: client proposal: host key algorithms: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-dss
jschLogMessage: client proposal: ciphers c2s: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-cbc,aes256-cbc
jschLogMessage: client proposal: ciphers s2c: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-cbc,aes256-cbc
jschLogMessage: client proposal: MACs c2s: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
jschLogMessage: client proposal: MACs s2c: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
jschLogMessage: client proposal: compression c2s: none
jschLogMessage: client proposal: compression s2c: none
jschLogMessage: client proposal: languages c2s:
jschLogMessage: client proposal: languages s2c:
jschLogMessage: kex: algorithm: curve25519-sha256
jschLogMessage: kex: host key algorithm: ssh-ed25519
jschLogMessage: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
jschLogMessage: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
jschLogMessage: SSH_MSG_KEX_ECDH_INIT sent
jschLogMessage: expecting SSH_MSG_KEX_ECDH_REPLY
jschLogMessage: ssh_eddsa_verify: ssh-ed25519 signature true
jschLogMessage: Host '[<HOSTNAME>]:2232' is known and matches the EDDSA host key
jschLogMessage: SSH_MSG_NEWKEYS sent
jschLogMessage: SSH_MSG_NEWKEYS received
jschLogMessage: SSH_MSG_SERVICE_REQUEST sent
jschLogMessage: SSH_MSG_EXT_INFO received
jschLogMessage: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
jschLogMessage: SSH_MSG_SERVICE_ACCEPT received
jschLogMessage: Authentications that can continue: publickey,keyboard-interactive,password
jschLogMessage: Next authentication method: publickey
jschLogMessage: PubkeyAcceptedAlgorithms = ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa
jschLogMessage: PubkeyAcceptedAlgorithms in server-sig-algs = [ssh-ed25519, ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521, rsa-sha2-512, rsa-sha2-256, ssh-rsa]
jschLogMessage: rsa-sha2-512 preauth success
jschLogMessage: rsa-sha2-512 auth success
jschLogMessage: Authentication succeeded (publickey).

Note that in the example above, the server supports host key algorithms of ssh-ed25519, rsa-sha2-512, rsa-sha2-256, and ssh-rsa. If any connection issues occurred using ssh-ed25519, you could set the Host Key Algorithms setting to rsa-sha2-512 instead.

Extension negotiation¶

The SFTP protocol allows clients to use advanced features of the protocol if they are supported by both the client and the servers. For most industry standard implementations of SFTP, we recommend keeping extensions enabled. However, extensions can cause connection failures if there are compatibility issues. The Foundry SFTP connector allows the extensions to be disabled as required. In general, the best practice is to disable as few extensions as necessary rather than disabling all extensions.

中文翻译¶

SFTP¶

将 Foundry 连接到 SFTP 服务器，以便在文件夹和 Foundry 数据集之间同步数据。

支持的功能¶

功能	状态
探索(Exploration)	🟢 正式发布(Generally available)
批量导入(Bulk import)	🟢 正式发布(Generally available)
增量同步(Incremental)	🟢 正式发布(Generally available)
导出任务(Export tasks)	🟡 日落(Sunset)（仅限旧版 SFTP 源）
文件导出(File exports)	🟢 正式发布(Generally available)

数据模型¶

该连接器可以将任何类型的文件传输到 Foundry 数据集中。文件格式保持不变，在传输期间或之后不会应用任何模式。对输出数据集应用任何必要的模式，或编写下游转换(downstream transformation)来访问数据。

性能与限制¶

可传输文件的大小没有限制。然而，网络问题可能导致大规模传输失败。特别是，运行时间超过两天的 Foundry 同步将被中断。为避免网络问题，我们建议使用较小的文件大小，并限制每次同步执行时摄取的文件数量。同步可以调度(scheduled)为频繁运行。

设置¶

打开数据连接(Data Connection)应用程序，并在屏幕右上角选择 + 新建源(New Source)。
从可用的连接器类型中选择 SFTP。
按照额外的配置提示，使用以下部分中的信息继续设置您的连接器。

:::callout{theme="warning"} 要访问本地 SFTP 服务器，我们建议使用代理代理连接(agent proxy connection)。 :::

了解更多关于在 Foundry 中设置连接器(setting up a connector)的信息。

身份验证(Authentication)¶

可以使用密码或 SSL 私钥建立到 SFTP 服务器的连接。如果在连接到服务器时遇到任何问题，请尝试通过 Foundry 外部的 SFTP 客户端进行连接，以验证身份验证详细信息是否正确。此连接的用户必须具有对根目录的访问权限，以及读取和列出根目录内所有文件和目录的权限。

选项	是否必需？	描述
`Username`	是	SFTP 登录用户名。
`Password`	否	SFTP 登录密码。
`Private key`	否	RSA 或 OpenSSH 格式的 SSL 私钥。SFTP 服务器必须已将公钥正确配置为所提供用户名的授权密钥。

网络(Networking)¶

如果您的 SFTP 连接在 Foundry 中运行(runs in Foundry)，则必须添加网络出口策略(network egress policy)以将连接列入白名单。

应为 SFTP 服务器主机名（如果通过域名连接）以及 SFTP 服务器主机名解析到的 IP 创建出口策略。正确配置出口策略后，非标准连接端口（22）可能仍因主机名验证(host name validation)而无法工作。如果发生这种情况，请向 Palantir 支持报告问题，并附上应用于此连接的策略列表。

:::callout{theme="warning"} 如果服务器的域名解析到多个域名和/或服务器，则所有关联的域名及其相关 IP 都需要被列入白名单。要验证服务器是否解析到多个域名和/或服务器，请在终端中为您尝试连接的服务器运行命令 dig <domain>，并查看答案部分。 :::

在 UNIX 机器上，通过运行以下命令查找服务器域名解析到的 IP 地址：

dig <domain> +short

:::callout{theme="neutral"} 如果代理正在运行您的连接器，请确保代理的服务器能够建立到 SFTP 服务器的网络连接，并且防火墙已正确配置。我们建议在需要时使用 netcat ↗ 或类似实用程序验证网络连接。 :::

证书和私钥(Certificates and private keys)¶

服务器 SSH 密钥(Server SSH key)¶

SFTP 服务器使用公钥标识自身。此密钥可以从服务器管理员处获取（首选），或者通过在任何可以访问该 SFTP 服务器的 Linux 服务器上运行以下命令来获取：

ssh-keyscan -t rsa -p {port} {hostname}  | awk '{print $3}'

此密钥必须在连接详情(Connection details)页面的主机密钥(Host key)部分中进行配置。如果无法获取此密钥，或者密钥频繁更改，可以使用接受任何主机密钥(Accept any host key)开关禁用密钥验证。请注意，禁用密钥验证是不安全的，不鼓励这样做。

主机名验证(Hostname validation)¶

Foundry 对所有出口路由尝试主机名验证。端口 22 以外的网络流量有时不会被验证，导致连接挂起和/或超时错误。如果尽管正确配置了出口策略，错误仍然发生，请向 Palantir 支持报告问题，并附上您希望禁用主机名验证的策略列表。

配置选项¶

选项	是否必需？	默认值	描述
`Hostname`	是		指向服务器的域名或服务器的 IP 地址。
`Port`	是		SFTP 服务器运行的端口。
`Root directory`	是		服务器上用作通过此连接的所有请求的起始目录的目录。
`Username`	是		SFTP 登录用户名。
`Password`	否		SFTP 登录密码。
`Private key`	否		RSA 或 OpenSSH 格式的 SSL 私钥。SFTP 服务器必须已将公钥正确配置为所提供用户名的授权密钥。
`Host key`	是		有关更多信息，请参见上面的服务器 SSH 密钥(Server SSH key)。
`Accept any host key`	否	`false`	有关更多信息，请参见上面的服务器 SSH 密钥(Server SSH key)。
`Proxy`	否	直接	启用以允许代理连接到 SFTP。
`Timeout`	否	0	有关更多信息，请参见下面的超时(Timeouts)。
`Maximum concurrent connections`	否	运行时支持的最大并行上传数	较高的值通常会导致更快的传输速度。将此值设置为高于运行时支持的最大并行上传数不会产生任何影响。如果您的 SFTP 服务器可以限制到服务器的并发连接数以管理负载，请使用此设置。
连接设置(Connection settings)	否		有关更多信息，请参见下面的连接设置(Connection settings)部分。
`Logs`	否	`Info`	要记录的日志级别。选择一个级别将记录该级别及以上的所有日志；例如，`Error` 级别将记录 `Error` 和 `Fatal` 级别的日志。有关示例，请参见下面的日志(Logs)部分。
`Extension Negotiation`	否	不适用	这些设置控制连接器发送到 SFTP 服务器的扩展协商(extension negotiation)消息。禁用这些功能有时可以解决建立连接的问题。有关更多指导，请参见下文。

超时(Timeouts)¶

值为 0 表示连接将无限期等待服务器的每次响应。此超时控制连接器在建立与服务器的连接以及每次在服务器上运行命令时的等待时间。如果服务器继续响应，则不会触发超时。将此值设置为较低的值不会阻止大文件传输，但有助于调试挂起的连接。

连接设置(Connection settings)¶

建立 SFTP 连接时，服务器和客户端（Foundry）必须协商必要的详细信息。这些详细信息包括帮助客户端和服务器决定使用哪些加密算法的信息。由于兼容性问题，您可能需要手动配置这些设置，而不是允许它们自动协商。下表列出了可用于根据需要调整协商的可用配置选项。有关更多信息，请查看 FAQ ↗。

设置	描述
`Key exchange algorithms`	可用于密钥交换的算法的显式列表。必须至少选择一个算法。
`Ciphers`	可用于加密的密码列表。
`Message Authentication Code (MAC)`	MAC 类型列表。
`Host key types`	可用于此连接的主机密钥类型。
`Public key types`	将用于公钥身份验证的算法。

从 SFTP 同步数据¶

:::callout{theme="neutral"} 对于更复杂的场景，请使用专业代码替代方案(pro-code alternatives)。 :::

SFTP 连接器使用基于文件的同步(file-based sync)接口。

故障排除(Troubleshooting)¶

借用于 ls 的连接失败(Borrowing connection for ls failed)¶

如果构建失败并出现错误 Borrowing connection for ls failed，并且错误包含 SocketException: Connection reset，请检查构建日志以查看是否已建立到 SFTP 服务器的网络连接。如果日志不包含行 Connection established 和 Remote version string，则防火墙正在阻止来自连接运行时的出站流量，或者目标 SFTP 服务器不允许传入连接。检查出口策略和防火墙规则，并联系 SFTP 服务器管理员以解决网络连接问题。以下是成功连接日志的示例：

jschLogMessage: Connecting to <HOSTNAME> port 2232
jschLogMessage: Connection established
jschLogMessage: Remote version string: <value identifying the type of server>

连接挂起(Hanging connections)¶

如果连接挂起而没有任何进展或明显失败，请将超时设置为较小的值（例如 1000），以帮助识别哪个调用正在挂起。例如，列出特定目录可能需要更长时间，或者文件的某些部分读取时间过长。

连接挂起可能由于主机名验证(hostname validation)而发生，设置较小的超时将允许生成失败日志，这可以帮助支持团队识别连接问题的根本原因。

代理工作器连接(Agent worker connections) [旧版(Legacy)]¶

:::callout{theme="warning" title="旧版(Legacy)"} 本节仅适用于现有的旧版代理工作器(agent worker)源。 :::

如果您正在使用出口代理负载均衡器，请注意 FTP 是一种有状态协议。如果顺序请求并非源自同一 IP，使用负载均衡器可能导致同步失败（非确定性）。

将数据导出到 SFTP¶

:::callout{theme="neutral"} 对于更复杂的场景，请使用专业代码替代方案(pro-code alternatives)。 :::

该连接器可以将文件从 Foundry 数据集复制到 SFTP 服务器上的任何位置。

要导出到 SFTP 服务器，首先为您的 SFTP 连接器启用导出(enable exports)。然后，创建一个新的导出(create a new export)。

:::callout{theme="neutral"} 在许多情况下，Foundry 默认在数据集的底层文件名前加上 spark/。当尝试将带有此前缀的文件名导出到 SFTP 实例时，连接器将默认尝试创建一个 spark 文件夹，如果 Foundry 使用的 SFTP 用户没有在 SFTP 实例上创建文件夹的权限，这将失败。为防止这种情况，请在导出前重命名文件。查看控制输出文件结构文档(controlling output file structure documentation)以获取如何在导出前重写文件路径的示例。 :::

在代码中使用 SFTP 源¶

专业代码替代方案(Pro-code alternatives)可用于连接 SFTP 源以处理更复杂的场景。

以下示例演示了如何在外部转换(external transform)中使用 Paramiko ↗ Python 客户端连接到 SFTP 源。

将文件写入 SFTP¶

此示例演示了如何使用 OpenSSH 私钥身份验证将 XLSX 文件列表写入 SFTP 服务器。

from transforms.api import Output, Input, TransformOutput, TransformInput, transform
from transforms.external.systems import external_systems, Source, ResolvedSource
import pandas as pd
import paramiko
import io

@external_systems(
    sftp_source=Source("<source_rid>")
)
@transform(
    output_file=Output(
        "<output_dataset_rid>"
    ),
    xlsx_files=Input("<input_dataset_rid>"),
)
def compute(
    sftp_source: ResolvedSource,
    output_file: TransformOutput,
    xlsx_files: TransformInput, # input dataset containing a list of xlsx files
) -> None:

    # 1. SFTP connection setup
    hostname = "<hostname>"
    port = <port> # (usually 22)
    username = "<username>"
    private_key_string = sftp_source.get_secret("privateKeyString")

    # 2. Transform private key string to proper input parameter of paramiko
    # Use StringIO to treat the string as a file-like object
    private_key_stream = io.StringIO(private_key_string)
    private_key = paramiko.Ed25519Key.from_private_key(private_key_stream)

    # 3. Define the SFTP client
    transport = paramiko.Transport((hostname, port))
    transport.connect(username=username, pkey=private_key)
    sftp = paramiko.SFTPClient.from_transport(transport)

    # 4. Iterate over input files and upload each to SFTP
    remote_paths = []
    fs = xlsx_files.filesystem()
    input_files = fs.ls()
    for f in input_files:
        with fs.open(f.path, "rb") as fileobj:  # Open in binary mode
            remote_path = get_remote_path(f.path)
            with sftp.file(remote_path, "wb") as remote_file:
                remote_file.write(fileobj.read())  # Write bytes directly
            remote_paths.append({"file_name": f.path, "remote_path": remote_path}) # record mapping of file name to uploaded path

    # 3. Return dataset of remote paths
    output_file.write_pandas(pd.DataFrame.from_records(remote_path))


def get_remote_path(file_path):
    """
    Define the remote path where the file will be uploaded.
    """
    remote_target_folder = "<example_target_folder>"
    clean_file_path = file_path.replace(" ", "_")
    return f"{remote_target_folder}/{clean_file_path}"

旧版 SFTP 连接器(Legacy SFTP connector)¶

:::callout{theme="warning"} 以下部分涵盖旧版 SFTP 连接器，仅在维护此连接器的现有用途时应参考。任何新创建的 SFTP 源都应遵循上述部分中描述的步骤。 :::

迁移到新的 SFTP 连接器¶

从旧版 SFTP 连接器迁移到新版不是自动的，必须手动完成。新连接器支持几乎相同的配置选项，应创建一个新源来替换旧源。

请注意：

无法从旧源检索密码，如果您不再拥有访问权限，必须向服务器管理员请求。
privateKeyFile 和 privateKeyPassphrase 不再受支持，如果使用私钥身份验证，私钥的内容应直接在源设置中输入。
knownHostsFile 不再受支持，必须在源设置中正确配置 base64HostKey。

设置旧版 SFTP 连接器¶

要设置旧版 SFTP 连接器，请导航到数据连接(Data Connection)应用程序，然后选择 + 新建源(New source)。在源选择屏幕上，向下滚动到高级(Advanced)，然后选择自定义源(Custom source)。按照配置屏幕中的提示继续设置您的 SFTP 连接器。

SFTP YAML 配置的完整示例如下所示：

:::callout{theme="warning"} 请注意，旧版 SFTP 源的 type 必须是 magritte-sftp。 :::

type: magritte-sftp
hostname: my.host.name
port: 22
username: username
password: '{{pasword}}'
rootDirectory: /home/palantir/sftp
base64Hostkey: >-
  FULL CONTENTS OF THE HOST KEY
unsafeAcceptAllHostKeys: false
privateKeyFile: location/of/private/key/file
privateKeyPassphrase: '{{passphrase}}'
proxyConfiguration:
    host: hostname.of.proxy
    port: port.of.proxy
    type: HTTP (can be HTTP/HTTPS/SOCKS)
    credentials:
        username: proxyUsername
        password: `{{proxyPassword}}`
timeout: 0
maxConcurrentConnections: 10
sessionParams:
    customKex:
        - "ecdh-sha2-nistp256"
        - "ecdh-sha2-nistp384"
knownHostsFile: /path/to/known_hosts_file

使用旧版 SFTP 连接器导出¶

:::callout{theme="warning"} 我们通常不建议使用导出任务(export tasks)将数据写回外部源，以下文档仅用于支持现有的旧版导出任务。任何新的导出都必须使用更新的、一流的 SFTP 连接器创建。 :::

要导出数据，您必须配置一个导出任务(export task)。导航到包含您要导出的连接器的项目文件夹。右键单击连接器名称，然后选择创建数据连接任务(Create Data Connection Task)。

在数据连接(Data Connection)视图的左侧面板中：

验证 Source 名称与您要使用的连接器匹配。
添加一个名为 inputDataset 的输入(Input)。输入数据集(input dataset) 是要导出的 Foundry 数据集。
添加一个名为 outputDataset 的输出(Output)。输出数据集(output dataset) 用于运行、调度和监控任务。
最后，在文本字段中添加一个 YAML 块来定义任务配置。

:::callout{theme="neutral"} 左侧面板中显示的连接器和输入数据集的标签不反映 YAML 中定义的名称。 :::

创建导出任务 YAML 时使用以下选项：

选项	是否必需？	描述
`directoryPath`	是	文件将被写入的目录。路径必须以尾随的 `/` 结尾。
`excludePaths`	否	正则表达式列表；名称与这些表达式匹配的文件将不会被导出。
`rewritePaths`	否	有关更多信息，请参见下面的部分。
`uploadConfirmation`	否	当值为 `exportedFiles` 时，输出数据集将包含已导出文件的列表。
`createTransactionFolders`	否	启用后，数据将被写入指定 `directoryPath` 内的子文件夹。每个子文件夹将有一个基于 Foundry 中提交事务时间的唯一名称。
`incrementalType`	否	对于增量构建的数据集，设置为 `incremental` 以仅导出自上次导出以来发生的事务。
`flagFile`	否	有关更多信息，请参见标志文件部分(Flag file section)。
`spanMultipleViews`	否	如果为 `true`，Foundry 中的多个事务将一次性导出。如果为 `false`，单个构建将一次只导出一个事务。如果启用了增量，将首先导出最旧事务中的文件。

`rewritePaths`¶

如果第一个键与文件名匹配，键中的捕获组将被替换为值。值本身可以有额外的部分来向文件名添加元数据。

如果值包含：

${dt:javaDateExpression}：值的这一部分将被替换为文件导出时的时间戳。javaDateExpression 遵循 DateTimeFormatter ↗ 模式。
${transaction}：值的这一部分将被替换为包含此文件的 Foundry 事务 ID。
${dataset}：值的这一部分将被替换为包含此文件的 Foundry 数据集 ID。

示例：

考虑 Foundry 数据集中一个名为 "spark/file_name" 的文件，其事务 ID 为 transaction_id，数据集 ID 为 dataset_id。如果您使用表达式 fi.*ame 作为键，file_${dt:DD-MM-YYYY}-${transaction}-${dataset}_end 作为值，则当文件写入 SFTP 时，它将存储为 spark/file_79-03-2023-transaction_id-dataset_id_end。

标志文件(Flag file)¶

连接器可以在给定构建的所有数据复制完成后，向 SFTP 服务器写入一个空的标志文件。空文件表示内容已准备好供消费，并且将不再被修改。标志文件将被写入 directoryPath。但是，如果启用了 createTransactionFolders，将为每个写入内容的文件夹创建一个标志文件。如果启用了标志文件，并且标志文件名为 confirmation.txt，则在构建中导出的文件写入后，所有标志文件将一次性写入。

:::callout{theme="neutral"} 标志文件在构建结束时写入，而不是在子文件夹导出时写入。 :::

如果 SFTP 服务器上的文件比标志文件新，这通常表示先前的导出未成功，或者该文件夹的导出正在进行中。

一个简单的导出配置示例如下：

type: export-sftp-task
directoryPath: export-directory/subdirectory

配置导出任务后，选择右上角的保存(Save)。

:::callout{theme="neutral"} directoryPath 中指定的目录必须已存在于 SFTP 服务器上。 :::

:::callout{theme="neutral"} directoryPath 中指定的目录是相对于根目录的。例如，如果连接配置了 rootDirectory: /home/palantir/sftp 且 directoryPath 为 export-directory，则文件将被导出到 /home/palantir/sftp/export-directory/。 :::

日志(Logs)¶

在数据连接(Data Connections)中选择日志(Logs)选项卡，以查看来自探索(explorations)和构建的日志。这些日志对于调试连接问题非常有用。搜索 jschLogMessage 以查看详细信息。

下面的日志详细信息显示了一次成功的登录尝试：

jschLogMessage: Connecting to <HOSTNAME> port 2232
jschLogMessage: Connection established
jschLogMessage: Remote version string: SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u1
jschLogMessage: Local version string: SSH-2.0-JSCH_0.2.12
jschLogMessage: CheckCiphers: chacha20-poly1305@openssh.com
jschLogMessage: CheckKexes: curve25519-sha256,curve25519-sha256@libssh.org,curve448-sha512
jschLogMessage: CheckSignatures: ssh-ed25519,ssh-ed448
jschLogMessage: server_host_key proposal before known_host reordering is: ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-dss,ssh-rsa
jschLogMessage: server_host_key proposal after known_host reordering is: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-dss
jschLogMessage: SSH_MSG_KEXINIT sent
jschLogMessage: SSH_MSG_KEXINIT received
jschLogMessage: server proposal: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
jschLogMessage: server proposal: host key algorithms: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
jschLogMessage: server proposal: ciphers c2s: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
jschLogMessage: server proposal: ciphers s2c: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
jschLogMessage: server proposal: MACs c2s: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
jschLogMessage: server proposal: MACs s2c: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
jschLogMessage: server proposal: compression c2s: none,zlib@openssh.com
jschLogMessage: server proposal: compression s2c: none,zlib@openssh.com
jschLogMessage: server proposal: languages c2s:
jschLogMessage: server proposal: languages s2c:
jschLogMessage: client proposal: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,ext-info-c
jschLogMessage: client proposal: host key algorithms: ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-dss
jschLogMessage: client proposal: ciphers c2s: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-cbc,aes256-cbc
jschLogMessage: client proposal: ciphers s2c: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-cbc,aes256-cbc
jschLogMessage: client proposal: MACs c2s: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
jschLogMessage: client proposal: MACs s2c: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
jschLogMessage: client proposal: compression c2s: none
jschLogMessage: client proposal: compression s2c: none
jschLogMessage: client proposal: languages c2s:
jschLogMessage: client proposal: languages s2c:
jschLogMessage: kex: algorithm: curve25519-sha256
jschLogMessage: kex: host key algorithm: ssh-ed25519
jschLogMessage: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
jschLogMessage: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
jschLogMessage: SSH_MSG_KEX_ECDH_INIT sent
jschLogMessage: expecting SSH_MSG_KEX_ECDH_REPLY
jschLogMessage: ssh_eddsa_verify: ssh-ed25519 signature true
jschLogMessage: Host '[<HOSTNAME>]:2232' is known and matches the EDDSA host key
jschLogMessage: SSH_MSG_NEWKEYS sent
jschLogMessage: SSH_MSG_NEWKEYS received
jschLogMessage: SSH_MSG_SERVICE_REQUEST sent
jschLogMessage: SSH_MSG_EXT_INFO received
jschLogMessage: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
jschLogMessage: SSH_MSG_SERVICE_ACCEPT received
jschLogMessage: Authentications that can continue: publickey,keyboard-interactive,password
jschLogMessage: Next authentication method: publickey
jschLogMessage: PubkeyAcceptedAlgorithms = ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa
jschLogMessage: PubkeyAcceptedAlgorithms in server-sig-algs = [ssh-ed25519, ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521, rsa-sha2-512, rsa-sha2-256, ssh-rsa]
jschLogMessage: rsa-sha2-512 preauth success
jschLogMessage: rsa-sha2-512 auth success
jschLogMessage: Authentication succeeded (publickey).

请注意，在上面的示例中，服务器支持主机密钥算法 ssh-ed25519、rsa-sha2-512、rsa-sha2-256 和 ssh-rsa。如果使用 ssh-ed25519 出现任何连接问题，您可以将主机密钥算法设置(Host Key Algorithms setting)设置为 rsa-sha2-512。

扩展协商(Extension negotiation)¶

SFTP 协议允许客户端使用协议的高级功能，前提是客户端和服务器都支持这些功能。对于大多数行业标准的 SFTP 实现，我们建议保持扩展启用。但是，如果存在兼容性问题，扩展可能导致连接失败。Foundry SFTP 连接器允许根据需要禁用扩展。通常，最佳实践是尽可能少地禁用扩展，而不是禁用所有扩展。

SFTP¶

Supported capabilities¶

Data model¶

Performance and limitations¶

Setup¶

Authentication¶

Networking¶

Certificates and private keys¶

Server SSH key¶

Hostname validation¶

Configuration options¶

Timeouts¶

Connection settings¶

Sync data from SFTP¶

Troubleshooting¶

Borrowing connection for ls failed¶

Hanging connections¶

Agent worker connections [Legacy]¶

Export data to SFTP¶

Use SFTP sources in code¶

Write files to SFTP¶

Legacy SFTP connector¶

Migrate to the new SFTP connector¶

Set up a legacy SFTP connector¶

Export with the legacy SFTP connector¶

rewritePaths¶

Flag file¶

Logs¶

Extension negotiation¶

中文翻译¶

SFTP¶

支持的功能¶

数据模型¶

性能与限制¶

设置¶

身份验证(Authentication)¶

网络(Networking)¶

证书和私钥(Certificates and private keys)¶

服务器 SSH 密钥(Server SSH key)¶

主机名验证(Hostname validation)¶

配置选项¶

超时(Timeouts)¶

连接设置(Connection settings)¶

从 SFTP 同步数据¶

故障排除(Troubleshooting)¶

借用于 ls 的连接失败(Borrowing connection for ls failed)¶

连接挂起(Hanging connections)¶

代理工作器连接(Agent worker connections) [旧版(Legacy)]¶

将数据导出到 SFTP¶

在代码中使用 SFTP 源¶

将文件写入 SFTP¶

旧版 SFTP 连接器(Legacy SFTP connector)¶

迁移到新的 SFTP 连接器¶

设置旧版 SFTP 连接器¶

使用旧版 SFTP 连接器导出¶

rewritePaths¶

标志文件(Flag file)¶

日志(Logs)¶

扩展协商(Extension negotiation)¶

`rewritePaths`¶

`rewritePaths`¶