Relay Node + Reverse Proxy

post by Dayman on Jun 18, 2021

Hi Algorand team.

I have a fully synced relay node behind a Caddy server and I’ve configured it such that connections to the /relay/ route are forwarded to port 4161, but when I run ./goal node start -p 'example.com/relay/' from my local machine, my relay node metrics for algod_network_incoming_connections{} says 0 even though I don’t see any client errors reported in any of the local node or archive node logs. @tsachi, If there is documentation for properly running a relay node secured behind a reverse proxy, I’d greatly appreciate it.

Thank you!

post by fabrice on Jun 19, 2021

Just copying @tsachi’s answer from Discord:

A node can indeed connect through a reverse proxy, however, it cannot have any additional URI elements. Symmetrical to that, a relay can also be placed behind a firewall, but it would require few additional config.json settings.

I think that in your case the issue is the /relay/ part.

If you could have example.com directly it should work.

post by Dayman on Jun 19, 2021

I will give this a try and mark this as the solution here in the forum if it works.

post by Dayman on Jun 20, 2021

Removing the subroute worked. My node metrics now reflect that I have an inbound connection. Thanks for the help!

post by tsachi on Jun 21, 2021

Hi @Dayman,

I’m going to break my answer into two sections - the node as a server and the node as a client. The client role is when the node is trying to establish connection to a provided relay. A server node is a relay listening to an incoming request(s).

As a node client, the node can be configured to connect to an endpoint; an endpoint contains a host name or IP address, followed by an optional port number.

for example algod -d <data dir> -p example.com:1234 would connect to example.com at point 1234.

As for the node server, it’s slightly more complicated. A node server listens on its NetAddress (as specified in the config.json), while having the following configuration limits:

1. MaxConnectionsPerIP

Limit the number of TCP connections allowed from a single remote endpoint. 2. UseXForwardedForAddressField

Instruct the node server to ignore the remote endpoint IP address and use the X-Forwarded-For when present. A server configured as such should have all its incoming traffic have the above HTTP header, coming from the reverse proxy.

The interaction between #1 and #2 above are critical for a successful deployment of a node server, since the originating connection to the server would always be the reverse proxy, and therefore would not contain (by default) the originating IP address.

The node server would always listen at the “root” of the URI. It does not support listening for a “sub-URI”. If this configuration is needed, then the reverse proxy should perform these conversions and rewrite the URL as needed.

post by Dayman on Jun 21, 2021

Thank you for the detailed explanation @tsachi. I was able to connect via reverse proxy by putting the relay at the root of the URI as you explained.

I am still interested in enabling the relay to listen to a sub-URI as this is useful for enabling other features for my application. It makes sense that the request would need to be re-written internally, and this is quite easy for me to do with Caddy. However, I notice in my node.log file that if I try to connect to example.com/relay/ by executing ./goal node start -p " [example.com/relay/](http://example.com/relay/), the following message is present:

"msg":"ws connect(ws://example.com/v1/mainnet-v1.0/gossip) fail - bad handshake, Status code = 502

Thus, it seems algod is stripping my subpath, /relay/, from the connection call, and therefore my URL rewrite is never triggered. I’m I understanding this correctly? Should I be specifying the endpoint differently in this case? Thanks again.

post by tsachi on Jun 21, 2021

@Dayman,

Yes, I think that your observation is correct.

The -p parameter you’ve used isn’t expecting to get a full URL, and it’s stripping away anything after the host/port.

The above configuration is clearly an interesting, but so far, you’re the first person who asked for that “construct”. The majority of the usage for the -p is for either testing or local private networks.

I think that if you need to tunnel multiple HTTP connections, you should be able to accomplish that by either creating two reverse proxies (one that builds up the URL, and one that strips it down), or maybe try to use multiple port numbers.

post by Dayman on Jun 21, 2021

I can certainly see how this is a non-standard use case. I’ll experiment with your suggestion and if I come up with anything, I’ll put it here for future reference of other users.

post by Dayman on Jun 23, 2021

After experimenting and finding a pretty good solution, I’m putting this here for future reference for anyone else who is running multiple services (including a relay node) behind a proxy.

1. put the relay node port on the root path / (reverse_proxy /* localhost:4161)
2. If you want a browser to redirect on the URL to something else, you should match a GET on the / path and redirect to the desired sub URI. i.e redir { method GET, path /} /myuri/