Agent proxy configuration reference(代理代理配置参考)¶
This page describes the configuration options available when using an agent as a network proxy via agent proxy egress policies on Foundry worker connections and assumes you are already familiar with data connection agents and their generic configuration options.
Agents used as a proxy provide a network path for services in Foundry to access systems within a separate network that are not directly accessible over the Foundry network. When Foundry is hosted in the cloud, a proxy is required to access systems which are not accessible over the Internet. When Foundry is hosted on a private network, a proxy is required to access systems in other networks, which may include the public Internet if otherwise not accessible.
Agent proxies allow applications in Foundry to operate as if they are connecting to a source directly over the Foundry network, and they can be configured with multiple agents to allow for alternating maintenance windows that prevent downtime. Follow the guide below to configure and manage an agent used as a proxy.
Use an agent as a proxy with a Data Connection source¶
You must first set up an agent before using it as a proxy. An existing agent already used for agent worker sources can be reused without additional configuration.
Once an agent has been set up, create an agent proxy egress policy using that agent. You can then create a source and assign that policy to it.
Security controls specific to agent proxy¶
To ensure that connections over the proxy only have access to required resources, a number of security controls are available.
Security controls are either mandatory and cannot be disabled, or optional and are not enabled by default.
The following security control configurations are available:
| Security control | Required? | Description |
|---|---|---|
| Source configuration enforcement | Yes | The agent proxy always restricts traffic to only the hostname(s) and port(s) configured on the assigned source. |
| Agent allowlist (Foundry) | Optional | Configure an allowlist in Foundry to filter traffic before it reaches the agent. Only agent resource owners can modify this setting. |
| Agent allowlist (local file) | Optional | Create an allowlist on the agent host for the local agent process to enforce within your network. Only users with SSH access to the agent host can modify this setting. |
| Agent host firewall | Optional | Set up host-level firewall controls to limit agents to communicating only with intended target systems (strongly recommended). These firewalls work independently of Foundry and add an extra security layer. |
Source configuration enforcement¶
The hostname and port in the URL defined on the source restrict access to only that hostname and port when connecting to the agent proxy. Attempts to connect to any other hostname or port will result in an HTTP 403 (Unauthorized) response code from the proxy.
This prevents unauthorized connections when importing and using the connection in code.
If only this security control is used, ensure that users who are able to assign an agent to a particular source are also trusted to connect to any system that is reachable from the agent host.
Agent allowlist (Foundry)¶
Configure an allowlist in Foundry to restrict connections to specific IP addresses or CIDR blocks for a specific agent, regardless of assigned sources.
To configure this, navigate to Agent settings, then toggle the Advanced option in the Manage Configuration section. In the YAML file, add a block for agentProxyConfiguration at the same indentation level as the security block.
List the CIDR blocks and ports where you want to allow agent connections. For example:
agentProxyConfiguration:
allowListedCidrs:
- cidrBlock: '192.168.1.1/32'
port: 7000
endPort: 9000
- cidrBlock: '192.168.2.2/24'
port: 443
Agent allowlist (local file)¶
The same configuration used for an agent allowlist in Foundry can also be set in a file on the agent host itself, preventing users in Foundry from editing the configuration.
To configure an agent allowlist locally, follow the steps below:
- SSH into the agent's host.
- Create a folder named
agentProxyConfigin the same directory that contains yourmagritte-bootvisor-<version>(the root folder). - In the newly-created
agentProxyConfigfolder, create a file namedagentProxyConfig.yml. - The
agentProxyConfig.ymlfile andagentProxyConfigfolder must be created with the root user, and the file permissions must be set to prevent the agent from writing to this file or folder. The agent will not run the agent proxy feature if theagentProxyConfig.ymlfile exists and is writeable, or if one of the parent directories is writeable by the agent. - The content of this file is the same as the configuration available in the agent settings interface.
For example:
allowListedCidrs:
- cidrBlock: '192.168.1.1/32'
port: 7000
endPort: 9000
- cidrBlock: '192.168.2.2/24'
port: 443
Agent host firewall¶
As for any agent, we recommend setting a firewall on the host to restrict connectivity to only what is necessary.
Connection pool configuration¶
The agent proxy maintains a pool of connections for handling requests. You can configure the connection pool size to optimize performance based on your workload.
To configure the connection pool, navigate to Agent settings, then toggle the Advanced option in the Manage Configuration section. In the YAML file, add the connection pool settings within the agentProxyConfiguration block:
agentProxyConfiguration:
maxConnections: 250
coreConnections: 100
| Option | Default | Description |
|---|---|---|
maxConnections |
300 | The maximum number of connections the agent proxy can open. |
coreConnections |
20 | The minimum number of connections maintained in the pool. |
:::callout{theme="neutral"}
Some connectors, such as Databricks, may open a large number of connections at the same time. If you observe connection pool exhaustion errors on your agent when using agent proxy policies, consider increasing the maxConnections and coreConnections values.
:::
中文翻译¶
代理代理配置参考¶
本页面介绍了在Foundry工作节点连接上通过代理代理出口策略将代理用作网络代理时的配置选项,并假设您已熟悉数据连接代理及其通用配置选项。
用作代理的代理为Foundry中的服务提供网络路径,使其能够访问位于独立网络中且无法通过Foundry网络直接访问的系统。当Foundry部署在云端时,需要代理来访问无法通过互联网访问的系统。当Foundry部署在私有网络中时,需要代理来访问其他网络中的系统,这些系统可能包括公共互联网(如果无法直接访问的话)。
代理代理允许Foundry中的应用程序像直接通过Foundry网络连接源系统一样运行,并且可以配置多个代理以实现交替维护窗口,从而防止停机。请按照以下指南配置和管理用作代理的代理。
将代理用作数据连接源的代理¶
在使用代理作为代理之前,您必须首先设置代理。已用于代理工作节点源的现有代理可以重复使用,无需额外配置。
设置代理后,使用该代理创建代理代理出口策略。然后您可以创建源并将该策略分配给它。
代理代理特有的安全控制¶
为确保通过代理的连接仅能访问所需资源,系统提供了多项安全控制。
安全控制分为强制(无法禁用)和可选(默认未启用)两种。
可用的安全控制配置如下:
| 安全控制 | 是否必需 | 描述 |
|---|---|---|
| 源配置强制 | 是 | 代理代理始终将流量限制为仅分配给源的指定主机名和端口。 |
| 代理允许列表(Foundry) | 可选 | 在Foundry中配置允许列表,在流量到达代理之前进行过滤。只有代理资源所有者可以修改此设置。 |
| 代理允许列表(本地文件) | 可选 | 在代理主机上创建允许列表,供本地代理进程在您的网络内强制执行。只有具有SSH访问代理主机权限的用户可以修改此设置。 |
| 代理主机防火墙 | 可选 | 设置主机级防火墙控制,限制代理仅与目标系统通信(强烈推荐)。这些防火墙独立于Foundry运行,增加了额外的安全层。 |
源配置强制¶
源上定义的URL中的主机名和端口将限制通过代理代理的连接仅能访问该主机名和端口。尝试连接任何其他主机名或端口将导致代理返回HTTP 403(未授权)响应代码。
这可以防止在代码中导入和使用连接时进行未经授权的连接。
如果仅使用此安全控制,请确保能够将代理分配给特定源的用户同样值得信任,可以连接到代理主机可访问的任何系统。
代理允许列表(Foundry)¶
在Foundry中配置允许列表,以限制特定代理仅能连接到指定的IP地址或CIDR块,无论分配了哪些源。
要配置此设置,请导航至代理设置,然后在管理配置部分切换高级选项。在YAML文件中,在与security块相同的缩进级别添加agentProxyConfiguration块。
列出允许代理连接的CIDR块和端口。例如:
agentProxyConfiguration:
allowListedCidrs:
- cidrBlock: '192.168.1.1/32'
port: 7000
endPort: 9000
- cidrBlock: '192.168.2.2/24'
port: 443
代理允许列表(本地文件)¶
与Foundry中的代理允许列表相同的配置也可以设置在代理主机本地的文件中,从而防止Foundry中的用户编辑该配置。
要本地配置代理允许列表,请按照以下步骤操作:
- 通过SSH登录代理主机。
- 在包含
magritte-bootvisor-<version>(根文件夹)的同一目录中创建一个名为agentProxyConfig的文件夹。 - 在新创建的
agentProxyConfig文件夹中,创建一个名为agentProxyConfig.yml的文件。 agentProxyConfig.yml文件和agentProxyConfig文件夹必须由root用户创建,并且文件权限必须设置为防止代理写入此文件或文件夹。如果agentProxyConfig.yml文件存在且可写,或者其中一个父目录可被代理写入,代理将不会运行代理代理功能。- 此文件的内容与代理设置界面中可用的配置相同。
例如:
allowListedCidrs:
- cidrBlock: '192.168.1.1/32'
port: 7000
endPort: 9000
- cidrBlock: '192.168.2.2/24'
port: 443
代理主机防火墙¶
与任何代理一样,我们建议在主机上设置防火墙,将连接限制为仅必要的连接。
连接池配置¶
代理代理维护一个连接池来处理请求。您可以根据工作负载配置连接池大小以优化性能。
要配置连接池,请导航至代理设置,然后在管理配置部分切换高级选项。在YAML文件中,在agentProxyConfiguration块内添加连接池设置:
agentProxyConfiguration:
maxConnections: 250
coreConnections: 100
| 选项 | 默认值 | 描述 |
|---|---|---|
maxConnections |
300 | 代理代理可以打开的最大连接数。 |
coreConnections |
20 | 连接池中维护的最小连接数。 |
:::callout{theme="neutral"}
某些连接器(例如Databricks)可能同时打开大量连接。如果在使用代理代理策略时观察到代理上的连接池耗尽错误,请考虑增加maxConnections和coreConnections的值。
:::