solovyov.net

ngrok for the wicked, or expose your ports comfortably

7 min read · programming

I've started using ngrok a lot lately (I know, I know, late to the party). But then last week, Homebrew has updated it to a version where it wants some $25 to supply custom domain names. I mean, I could pay that, but I'm paying Hetzner like $8 or $9 for a server, and then I'm paying for my domain and… it's still cheaper?

I understand that hosting is more commoditized than tunnels β€” I guess the market is wider β€” but still, it felt like I could spend a few hours and get something similar working. Why do I need custom domains? Because losing cookies makes me unhappy, and cookies require domain not to change. Plus, testing OAuth is really painful without custom domain since everybody wants to pin redirect URL to a stable domain.

There are many open source alternatives, the one I really liked is called SirTunnel. It's a small script which uses Caddy's JSON API to add and remove domains. But it got me thinking: why add and remove domains when I can just give another domain for some site forever? So if I started that site's process, it's working, and when not β€” well, you'll get 502 Gateway Timeout on that particular domain, no big deal.

The Plan

It's simple! I create a wildcard domain (something like *.xxx.solovyov.net, xxx is for "real domain is none of your business" 😁), and then reverse-proxy everything through an SSH tunnel from a server to my laptop.

I'm still going to use Caddy since automatic HTTPS and laconic config appeals to me. πŸ‘

Why do I need a local Caddy? Because SSH can proxy only single port and local processes occur on different ports. You can ignore that part if you don't need an ability to run multiple sites simultaneously. You know what, this case makes everything easier, since you require only one external domain for that rather than my wildcard setup.

Execution

So, I've got a local Caddy working with many domains mapped to various ports. My plan is to run every project on a separate port, so when I start a process, it's immediately available to the world. Feels a bit exhibitionist, but very convenient. ☺️

Next was permanent SSH tunnel. I could've done that myself, but I just found a recipe.

The first problem is that handling *.xxx.solovyov.net in Server-Caddy makes Caddy request a wildcard certificate. And this requires integration of Caddy with DNS provider, which is limited to a few big providers. So, I opted to just repeating site definitions. :)

Next problem was… that it all worked! I could not believe my eyes. 🀣

But the story does not end here. You know what is irritating about ngrok? Latency. Especially when you're on a bad connection.

I mean my site is right here on my laptop but those roundtrips just to test OAuth and what not… Argh. I did not invent anything better than just writing that development domain in /etc/hosts. And. It. Worked! Too bad /etc/hosts does not support wildcards, so I'll have to repeat domains there too.

Tutorial

You'll need a domain name you own and control (really, control matters more than ownership here 😜) and a VPS somewhere in the world. Please do not copy and paste stuff blindly, you'll have to change at least the domain name for this to work. :)

Domain

Add *.xxx entry of type A to your domain pointing at your VPS.

Local Caddy

brew install caddy β€” correct for your package manager β€” and start with this Caddyfile:

{
	auto_https disable_redirects # so remote caddy is happy
	email your@real.email # so you can debug problems with certs
}

(local) {
	{args.0}.xxx.solovyov.net {args.0}.xxx.solovyov.net:80 {
		encode zstd gzip
		reverse_proxy localhost:{args.1}

		handle_errors {
			respond "Local: {http.error.status_code} {http.error.status_text}"
		}

		log {
			level DEBUG
			output file /opt/homebrew/var/log/caddy/{args.0}.log
		}
	}
}

import local dev 5000
import local experiment 5001
import local blog 5002

This (local) thingie is called a snippet. Now I can just copy this import line as many times as I want, not having to repeat those lines.

We instruct Caddy to listen to port 80 so that basic HTTP works. We need HTTP since our SSH tunnel targets this port. But HTTPS (domain without :80) is also nice to have β€” it makes external and internal setup more similar.

brew services start caddy or equivalent to make Caddy run after system startup.

Persistent SSH Tunnel

Just follow a tutorial from Tyler, should be simple enough. Your /.ssh/config entry should look like this:

Host sshtun
	HostName your.remote.server
	RemoteForward 6800 127.0.0.1:80
    ServerAliveInterval 60

What Tyler doesn't tell is that you have launchctl load -w Library/LaunchAgents/your.plist.name.plist, this little -w marks it enabled so launchctl will start it after restart (sounds like a common theme ain't it?).

In case you're not on macOS, use your system's process manager or look at autossh.

Obviously, you can replace that part with some VPN solution, like Wireguard for an open-source solution or Tailscale like something more convenient.

Remote Caddy

No need to disable automatic redirects, so minimal version will look like this:

{
    email alexander@solovyov.net
}

(sshtun) {
    {args.0}.xxx.solovyov.net {
        encode zstd gzip
        reverse_proxy localhost:6800

		handle_errors {
			respond "Remote: {http.error.status_code} {http.error.status_text}"
		}

        log {
            output file /var/log/caddy/sshtun.log
        }
    }
}

import sshtun test
import sshtun experiment
import sshtun blog

You can see I'm using snippets here as well, but the port is the same every time, since this is our SSH tunnel.

Remote Caddy Wildcard

After some thinking, I decided to try out wildcard setup anyway. I moved my domain to Cloudflare, since this is one of the providers supported by Caddy (filter by caddy-dns here to see others), downloaded a custom Caddy build, added a diversion (Debian/Ubuntu-specific) so that the regular package is in place (though I'll have to upgrade to new versions manually) and changed config to this:

{
    email alexander@solovyov.net
}

*.xxx.solovyov.net {
	encode zstd gzip
	reverse_proxy localhost:5900
	handle_errors {
		respond "Server: {http.error.status_code} {http.error.status_text}"
	}
	tls {
		dns cloudflare <API TOKEN HERE>
	}
	log {
		output file /var/log/caddy/d.log
	}
}

You can get Cloudflare API token here. And it worked! I got wildcard certificate so no need to edit this config any more!

/etc/hosts

This is optional, but if you want the same glorious setup, add this to /etc/hosts:

127.0.0.1 test.xxx.solovyov.net

but do not forget to use your actual DNS address. :)

Adding a new domain

There are a few edit points:

In case something is not working, you will get different errors:

The End

And you know what? No need to start many ngrok processes occupying your terminal when you want a few of your sites running! Epic.

If you like what you read β€” subscribe to my Twitter, I always post links to new posts there. Or, in case you're an old school person longing for an ancient technology, put a link to my RSS feed in your feed reader (it's actually Atom feed, but who cares).

Other recent posts

PostgreSQL collation
History snapshotting in TwinSpark.js
Code streaming: hundred ounces of nuances
Useful shell prompt