Exposing to the Internet

Why do we want to expose things?

We want to expose services to the internet for people to use! For example, a gitlab instance such as https://git.yadunut.dev/ should be publicly available!

However, Yadunand might not want to expose his TopSecretService running on port 8080 to the internet, so we have to be careful of how we are exposing these services.

Cloudflare Tunnels

How does it work?

Cloudflare Tunnel uses an outbound-only connection model to enable bidirectional communication. When you install and run cloudflared, cloudflared initiates an outbound connection through your firewall from the origin to the Cloudflare global network.

Most firewalls (and ours!) allow outbound traffic. cloudflared takes advantage of this standard by connecting out to the Cloudflare network from the server you installed cloudflared on. You can then configure your firewall to allow only these outbound connections and block all inbound traffic, effectively blocking access to your origin from anything other than Cloudflare.

All traffic will then be securely routed through the tunnel.

We will not be setting up via the Cloudflare Dashboard

For some reason, it requires us to add a card, even though it is a free service :<

But no fear, we can do it on the cli for free! WAHOOo

Installation On Ubuntu 24.04

For other Linux Distros: https://pkg.cloudflare.com/index.html

# Add cloudflare gpg key
sudo mkdir -p --mode=0755 /usr/share/keyrings
curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null

# Add this repo to your apt repositories
# Stable
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared noble main' | sudo tee /etc/apt/sources.list.d/cloudflared.list
# Nightly
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://next.pkg.cloudflare.com/cloudflared noble main' | sudo tee /etc/apt/sources.list.d/cloudflared.list

# install cloudflared
sudo apt-get update && sudo apt-get install cloudflared

For the rest of the setup, we are following:

Create a locally-managed tunnelCloudflare Docs

If you did not manage to register a domain, you can skip the next few steps, instead just expose your services now with Cloudflare Temporary Tunnels

sudo su 
cloudflared tunnel login

It will then launch a window on cloudflare dashboard, where you can select a domain to authorize.

Create a Tunnel

cloudflared tunnel create <NAME>

This will create a tunnel with a unique UUID, and generate a tunnel credential file.

Take note of this UUID, as we will be using it later.

You can view your existing tunnels using: cloudflared tunnel list

If you need to delete a tunnel

cloudflared tunnel cleanup <UUID>
cloudflared tunnel delete <UUID>

If the tunnel is still running, you need to run these to kill the process:

systemctl stop cloudflared
pkill -f "cloudflared tunnel"

Creating Tunnel Configuration file

The configuration file should be created at /etc/cloudflared/config.yml

This is what the file can look like:

sudo mkdir -p /etc/cloudflared
sudo vim /etc/cloudflared/config.yml

config.yml

### THIS IS AN EXAMPLE CONFIG FILE ###

tunnel: 6ff42ae2-765d-4adf-8112-31c55c1551ef
credentials-file: /root/.cloudflared/6ff42ae2-765d-4adf-8112-31c55c1551ef.json
origincert: /root/.cloudflared/cert.pem

ingress:
  # Rules map traffic from a hostname to a local service:
  - hostname: example.com
    service: http://localhost:8000
  # Rules can match the request's path to a regular expression:
  - hostname: static.example.com
    path: \.(jpg|png|css|js)$
    service: http://localhost:8001
  # Rules can match the request's hostname to a wildcard character:
  - hostname: "*.example.com"
    service: http://localhost:8002
  # An example of a catch-all rule:
  - service: http://localhost:8003

In our case, let's set up a config which points the docker subdomain to https://localhost:9443

config.yml

tunnel: TUNNEL-UUID
credentials-file: /root/.cloudflared/<TUNNEL-UUID>.json
origincert: /root/.cloudflared/cert.pem

ingress:
  # Let's map the subdomain docker here.
  - hostname: docker.<domain-name>.<tld>
    service: https://localhost:9443
    originRequest:
      noTLSVerify: true
  
    
  # Catch all everything else to 404
  - service: http_status:404

Note that we had to add an additional config of noTLSVerify. This is because

Full Configuration File Docs (Cloudflare)

Configure DNS

In our example, since we have configured cloudflared to tunnel traffic from: docker.<domain-name>.tld -> https://localhost:9443 , we need to configure out cloudflare DNS to also match this

sudo cloudflared tunnel route dns <TUNNEL-NAME OR TUNNEL-UUID> docker.<domain-name>.<tld>

Start running Cloudflared

Install

sudo cloudflared service install

Start service

sudo systemctl start cloudflared

Check service

sudo systemctl status cloudflared

How to add new routes

1) Edit the config and add the new domain and which localhost its pointed to

sudo vim /etc/cloudflared/config.yml

2) Add new DNS route

sudo cloudflared tunnel route dns <TUNNEL-UUID or TUNNEL-NAME> docker.<domain-name>.<tld>

3) Reload cloudflared

sudo systemctl restart cloudflared

4) Check the status after

sudo systemctl status cloudflared

We should be able to view portainer now!

Nothing shows up? This might be the Portainer instance timed out for security purposes, to re-enable your Portainer instance, you will need to restart Portainer.

To do this, we can run:

docker restart portainer

This might not work if the container is not named portainer. In that case, you should run:

docker ps
docker logs <container-id>
docker restart <container-id>

We can also troubleshoot other problems on the Cloudflare Dashboard. Common culprits are DNS as well. (Are you sure you set that up?)

Quickly exposing services to the internet

Cloudflare provides a service called Quick Tunnels. This is usually meant for development environments, so it is not a good permanent solution!

It will generate a domain that you can use, which is pointed towards your ip address:

You will however, probably need to run these in separate tmux windows.. (It really isn't ideal!)

sudo cloudflared tunnel  --config /dev/null --url http://127.0.0.1:<port-number>

To tunnel to portainer specifically, you can run:

sudo cloudflared tunnel --config /dev/null --no-tls-verify --url https://127.0.0.1:9443

⚠️: The --config /dev/null flag is needed because cloudflared will sometimes use the signed in credentials rather than creating a quick tunnel.

Additional Docs:

Create a locally-managed tunnelCloudflare Docs

Configuration fileCloudflare Docs

Useful commandsCloudflare Docs

What are some other ways to do this?

This will not be covered in the scope of this workshop, but there are many other ways to do this as well.

VPNs: Wireguard/ Tailscale

Usually, VPNs are used when we only want our services exposed to authenticated users with a VPN client. This is the same principle that SoC uses, for us to connect into SoC internal network via SoC VPN.