Set up SSH and Tailscale for a GPU Container Job

Configure SSH and Tailscale in a running GPU Container Job so that you can connect to it securely from your local machine.

Configure SSH and Tailscale in a running GPU Container Job, and then connect to the container from your local machine over the Tailscale network.

Prerequisites

Before you start, make sure that you have the following.

A running GPU Container Job. See Create a GPU Container Job with the CLI or Create a GPU Container Job in the web interface.
Root access in the container, with sudo -s or a root shell.
Your SSH public key at ~/.ssh/id_ed25519.pub on your local machine.
Tailscale administrator access, if your Tailscale access control policies require you to approve new nodes.

Steps

Open the container shell

Open a shell in the GPU Container Job. See Connect to a GPU Container Job.

Become the root user. Run the remaining commands as root.

sudo -s

Install SSH and Tailscale

Update the package list, and install the required packages.

apt update -y
apt install -y sudo openssh-server nano curl

Install the Tailscale client.

curl -fsSL https://tailscale.com/install.sh | sh

Add your SSH public key

Create the SSH directory, and set its permissions.

mkdir -p /root/.ssh
chmod 700 /root/.ssh

Open the authorized_keys file.

nano /root/.ssh/authorized_keys

Paste the contents of your local ~/.ssh/id_ed25519.pub file, and save the file. Then set its permissions.

chmod 600 /root/.ssh/authorized_keys

Configure SSH for key-based root access

Allow root login with public key authentication, and turn off password authentication.

sed -i \
  -e 's/^#\?PermitRootLogin.*/PermitRootLogin yes/' \
  -e 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' \
  -e 's/^#\?PubkeyAuthentication.*/PubkeyAuthentication yes/' \
  -e 's/^#\?UsePAM.*/UsePAM no/' \
  /etc/ssh/sshd_config

Check the SSH settings.

grep -E 'PermitRootLogin|PasswordAuthentication|PubkeyAuthentication|UsePAM' /etc/ssh/sshd_config

The output includes the following lines.

PermitRootLogin yes
PubkeyAuthentication yes
PasswordAuthentication no
UsePAM no

Root login is allowed because SSH accepts only key-based authentication, over the Tailscale network. Don't turn on password authentication.

Start SSH and Tailscale

Create the runtime directories.

mkdir -p /var/run/sshd
mkdir -p /var/run/tailscale

Start tailscaled in the background.

nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown

Start the SSH server.

/usr/sbin/sshd

Check that both processes are running.

ps ax | grep -E 'tailscaled|sshd'

Connect the container to Tailscale

Start Tailscale with a unique hostname and any tags that your Tailscale access control policies require.

tailscale up --hostname=<container-name> --advertise-tags=tag:<tag-name>

Replace <container-name> with the container name, and <tag-name> with a tag that your Tailscale access control policies let you advertise.

If the node isn't authenticated, Tailscale shows a login URL. Open the URL, and approve the device.

Verify the connection

In the container, check the processes and the Tailscale status.

ps ax | grep -E 'tailscaled|sshd'
tailscale status
tailscale ip -4

On your local machine, connect to the container over Tailscale.

ssh root@<container-name>

SSH uses your key and doesn't ask for a password.

Help and troubleshooting

SSH fails with Permission denied (publickey)

Confirm the public key is in /root/.ssh/authorized_keys, with permissions 600 on the file and 700 on .ssh.

SSH connects but asks for a password

Re-run the SSH hardening sed command, then restart sshd.

tailscale up fails

Confirm tailscaled is running with ps ax | grep tailscaled, then check /var/log/tailscaled.log.

Node not visible in Tailscale

Run tailscale login, or approve the device in Tailscale.

sshd won't start

Confirm /var/run/sshd exists, then check journalctl or /var/log/auth.log.

Restart SSH or Tailscale

Restart SSH after configuration changes.

pkill sshd
/usr/sbin/sshd

Restart Tailscale.

pkill tailscaled
nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown
tailscale up --hostname=<container-name> --advertise-tags=tag:<tag-name>