# Automating Let's Encrypt with Certbot and TransIP using Docker

> Want to auto update/manage your SSL certificates inside your Docker with Certbot and hosting your domain with TransIP? Then look no further!

- URL: https://www.allsubjectsmatter.nl/automating-lets-encrypt-with-certbot-and-transip-using-docker-2/
- Language: en
- Published: 2024-11-28
- Updated: 2024-11-29
- Author: Remco Loup (https://www.allsubjectsmatter.nl/author/remco/)
- Tags: Under the hood
- Translation (nl, translated with AI): https://www.allsubjectsmatter.nl/nl/automating-lets-encrypt-with-certbot-and-transip-using-docker-2/

Managing SSL certificates for domains hosted on TransIP can be cumbersome without automation. TransIP luckily offers a robust API for DNS management, but to integrate it seamlessly with Certbot for `dns-01` validation, a custom solution is necessary. Since I couldn’t find a ready-made script for this purpose, I created one that securely handles authentication, DNS updates, and certificate issuance.

You can go to the full API documentation here:

[TransIP API](https://api.transip.nl/rest/docs.html): 

This guide explains the full setup using Docker, including how to generate the API key pair in TransIP’s web interface and use it for automated certificate management.

## How TransIP authentication works

TransIP uses an RSA key pair for API authentication. Here’s how to set it up:

1. **Generate the Key Pair in TransIP**:
    - Log in to your [TransIP control panel](https://www.transip.nl/).
    - Go to **API Settings** under your account and enable API access.
    - Generate a new API key pair.
    - **Download/Save the private key** (`transip_private_key.pem`) securely and create a secret from it to use in your Docker setup.

```bash
docker secret create transip_private_key transip_private_key.pem
```

1. **Keep the Private Key Secure**:
    - Store the private key somewhere safe after creating a secret from it.
    - This private key is needed to sign requests to TransIP’s API.
2. **How It Works**:
    - The script uses the private key to sign an authentication request.
    - TransIP’s API responds with a short-lived token (10 minutes by default), which is then used to manage the DNS records.
    - It will create a new token for convenience on every run. This made it more resilient then checking for an active token and using that which made the whole script less resilient and cumbersome.

## What this setup is going to do

1. **Automates SSL Certificates**:
    - Automates issuance and renewal of wildcard and domain-specific certificates.
2. **Handles DNS Challenges**:
    - Manages `dns-01` challenges by creating and deleting DNS TXT records via the TransIP API.
3. **Seamless Integration**:
    - Encapsulates everything in a Docker container for portability and simplicity.
    - You can remove or add domains in this script on the -d rule part. I put the examples in for a better explanation.

We will pipe a add or clean command to the bash script to take the necessary actions as defined in the script. So we use the --manual-auth-hook and the manual-cleanup-hook here. This makes sure that a fail in the add section still cleans up after itself for example.

You can also add a --force to this script. but your will run out of attempts to Letsencrypt quickly since they only allow you to do a few calls every 24 hours. But this will make sure it refreshes the certificates always when you need to test or verify something.

```yaml
version: "3.8"

services:
  certbot:
    image: certbot/certbot:latest
    container_name: certbot-transip
    secrets:
      - transip_private_key  # Reference the secret you created earlier
    # adjust these volumes to point to the right .pem and script. right now this dockerfile expects the files to be in the same folder as the docker-certbot-transip.yml
    volumes:
      - "./certbot-transip.sh:/usr/local/bin/certbot-transip.sh:ro"
      - "./letsencrypt:/etc/letsencrypt"
    environment:
    #These Variables are going to be piped in our bash script.
      LOGIN_NAME: "your-transip-login"        # Replace with your TransIP login name
      PRIVATE_KEY_PATH: "/etc/transip_private_key.pem"
      DOMAIN: "example.com"                  # Replace with your domain
      TTL: "60"                              # Optional: Time-to-live for DNS challenge
      CERTBOT_EMAIL: "you@example.com"       # Replace with your valid email as requested by letsencrypt.
    entrypoint:
      - sh
      - -c
      - |
        apk update && apk add --no-cache bash curl jq bind-tools && \
        chmod +x /usr/local/bin/certbot-transip.sh && \
        certbot certonly --manual --preferred-challenges dns \
          --manual-auth-hook "/usr/local/bin/certbot-transip.sh add" \
          --manual-cleanup-hook "/usr/local/bin/certbot-transip.sh clean" \
          --cert-name "$DOMAIN" \
          -d "*.$DOMAIN" \
          -d "example2.$DOMAIN" \
          -d "example3.$DOMAIN" \
          --expand \
          --non-interactive --agree-tos \
          --email "$CERTBOT_EMAIL"

secrets:
  transip_private_key:
    external: true  # Secret must already exist in Docker
```

## The Bash script

This is the "advanced" part that will do its magic and make everything work with TransIP. I tried to make it a bit more resilient then standard also. So this should hopefully help in creating a resilient and good working environment.

You can also use this TransIP script to run it locally creating certificates or in a Kubectl or Helm chart.

```bash
#!/bin/bash

# Configuration settings
LOGIN_NAME=${LOGIN_NAME}
TTL=${TTL:-60}  # Default value 60 if TTL is not set
PRIVATE_KEY_PATH=${PRIVATE_KEY_PATH}
DOMAIN=${DOMAIN}

AUTH_URL="https://api.transip.nl/v6/auth"
LOG_FILE="/var/log/transip_dyndns.log"

# Function to update a log file
#log_message() {
#    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
#}

# Function to log messages to stdout
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1"
}

# Function to log an error and stop the script
handle_error() {
    log_message "ERROR: $1"
    echo "ERROR: $1" >&2
    exit 1
}

# Function to obtain a new access token with a short validity period (10 minutes)
generate_new_token() {
    log_message "Generating a new token with a validity period of 10 minutes."

    NONCE=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16)
    TIMESTAMP=$(date +%s)
    LABEL="Certbot DNS update $TIMESTAMP"

    REQUEST_BODY=$(printf '{"login":"%s","nonce":"%s","read_only":false,"expiration_time":"10 minutes","label":"%s","global_key":true}' "$LOGIN_NAME" "$NONCE" "$LABEL")

    log_message "Request Body: $REQUEST_BODY"

    SIGNATURE=$(echo -n "$REQUEST_BODY" | openssl dgst -sha512 -sign "$PRIVATE_KEY_PATH" | base64 | tr -d '\n')

    log_message "Generated signature: $SIGNATURE"

    RESPONSE=$(curl -s -X POST "$AUTH_URL" \
        -H "Signature: $SIGNATURE" \
        -H "Content-Type: application/json" \
        -d "$REQUEST_BODY") || handle_error "Failed to connect to TransIP API"

    log_message "API-response received: $RESPONSE"

    ACCESS_TOKEN=$(echo "$RESPONSE" | jq -r '.token // empty')
    if [ -z "$ACCESS_TOKEN" ]; then
        handle_error "The received access token is empty or invalid. Check the API-response: $RESPONSE"
    fi

    log_message "Received token: $ACCESS_TOKEN"
}

# Function to add or update a TXT record
add_or_update_txt_record() {
    local full_domain="$1"
    local value="$2"

    # Extract the subdomain part from the full domain
    local name="_acme-challenge.${full_domain%%.*}"

    if [ -z "$name" ] || [ -z "$value" ]; then
        log_message "ERROR: Name or value for TXT record is missing."
        exit 1
    fi

    # Check if the DNS record already exists
    RESPONSE=$(curl -s -X GET \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    # Search for all existing records with the same name and type TXT
    MATCHING_RECORDS=$(echo "$RESPONSE" | jq -r --arg name "$name" '[.dnsEntries[] | select(.name==$name and .type=="TXT")]')

    # Check if there are multiple or no matching records
    MATCHING_COUNT=$(echo "$MATCHING_RECORDS" | jq 'length')

    # Ensure MATCHING_COUNT is always an integer
    if ! [[ "$MATCHING_COUNT" =~ ^[0-9]+$ ]]; then
        MATCHING_COUNT=0
    fi

    if [ "$MATCHING_COUNT" -eq 0 ]; then
        log_message "No existing TXT record found for $name. Creating a new record."
    elif [ "$MATCHING_COUNT" -ge 1 ]; then
        log_message "Multiple or one existing TXT record found for $name. Removing all existing records."

        # Remove all existing records using the full details of each record
        echo "$MATCHING_RECORDS" | jq -c '.[]' | while read -r record; do
            existing_name=$(echo "$record" | jq -r '.name')
            existing_expire=$(echo "$record" | jq -r '.expire')
            existing_content=$(echo "$record" | jq -r '.content')

            # Log the details of the record to be removed
            log_message "Remove record: name=$existing_name, expire=$existing_expire, content=$existing_content"

            # Make the DELETE call using the full DNS-entry details
            DELETE_ENTRY=$(jq -n \
            --arg name "$existing_name" \
            --arg expire "$existing_expire" \
            --arg type "TXT" \
            --arg content "$existing_content" \
            '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

            DELETE_RESPONSE=$(curl -s -X DELETE \
                -H "Authorization: Bearer $ACCESS_TOKEN" \
                -H "Content-Type: application/json" \
                -d "$DELETE_ENTRY" \
                "https://api.transip.nl/v6/domains/$DOMAIN/dns")

            # Log the response of the DELETE call
            log_message "Response after removing $existing_name: $DELETE_RESPONSE"

            # Check if the removal was successful
            if [[ "$DELETE_RESPONSE" == *"error"* ]]; then
                log_message "ERROR: The record $existing_name could not be removed."
            else
                log_message "Record $existing_name successfully removed."
            fi
        done
    fi

    # Create a new record
    DNS_ENTRY=$(jq -n \
    --arg name "$name" \
    --arg expire "$TTL" \
    --arg type "TXT" \
    --arg content "$value" \
    '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

    RESPONSE=$(curl -s -X POST \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        -H "Content-Type: application/json" \
        -d "$DNS_ENTRY" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    log_message "API-response for TXT record creation: $RESPONSE"

    if [[ "$RESPONSE" == *"error"* ]]; then
        handle_error "The TXT record could not be created. Check the API-response: $RESPONSE"
    else
        log_message "TXT record successfully added for $name with value $value"
    fi
}

# Function to update an existing TXT record
update_txt_record() {
    local name="$1"
    local value="$2"

    DNS_ENTRY=$(jq -n \
    --arg name "$name" \
    --arg expire "$TTL" \
    --arg type "TXT" \
    --arg content "$value" \
    '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

    RESPONSE=$(curl -s -X PATCH \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        -H "Content-Type: application/json" \
        -d "$DNS_ENTRY" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    log_message "API-response for TXT record update: $RESPONSE"

    if [[ "$RESPONSE" == *"error"* ]]; then
        handle_error "The TXT record could not be updated. Check the API-response: $RESPONSE"
    else
        log_message "TXT record successfully updated for $name with value $value"
    fi
}

# Function to remove a TXT record for `dns-01` validation
remove_txt_record() {
    local full_domain="$1"

    # Extract the subdomain part from the full domain
    local name="_acme-challenge.${full_domain%%.*}"

    # Check if the DNS record exists
    RESPONSE=$(curl -s -X GET \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    # Search for all records with the same name and type TXT
    MATCHING_RECORDS=$(echo "$RESPONSE" | jq -r --arg name "$name" '[.dnsEntries[] | select(.name==$name and .type=="TXT")]')

    # Check if there are existing records that need to be removed
    MATCHING_COUNT=$(echo "$MATCHING_RECORDS" | jq 'length')

    if [ "$MATCHING_COUNT" -eq 0 ]; then
        log_message "No existing TXT record found for $name to remove."
    else
        log_message "Removing all existing TXT records for $name."

        # Loop through all found records and remove them
        echo "$MATCHING_RECORDS" | jq -c '.[]' | while read -r record; do
            existing_name=$(echo "$record" | jq -r '.name')
            existing_expire=$(echo "$record" | jq -r '.expire')
            existing_content=$(echo "$record" | jq -r '.content')

            # Make the DELETE call using the full details
            DELETE_ENTRY=$(jq -n \
            --arg name "$existing_name" \
            --arg expire "$existing_expire" \
            --arg type "TXT" \
            --arg content "$existing_content" \
            '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

            DELETE_RESPONSE=$(curl -s -X DELETE \
                -H "Authorization: Bearer $ACCESS_TOKEN" \
                -H "Content-Type: application/json" \
                -d "$DELETE_ENTRY" \
                "https://api.transip.nl/v6/domains/$DOMAIN/dns")

            # Check if the removal was successful
            if [[ "$DELETE_RESPONSE" == *"error"* ]]; then
                log_message "ERROR: The TXT record $existing_name could not be removed."
            else
                log_message "TXT record $existing_name successfully removed."
            fi
        done
    fi
}

# Always generate a new token for each run
generate_new_token

# Certbot passes these values when adding and removing the record
if [ "$1" == "add" ]; then
    add_or_update_txt_record "$CERTBOT_DOMAIN" "$CERTBOT_VALIDATION"
    sleep 25  # Wait for DNS update to propagate
elif [ "$1" == "clean" ]; then
    remove_txt_record "$CERTBOT_DOMAIN"
fi
```

## Steps to run

1. **Download the Private Key**: Save the private key file (`transip_private_key.pem`) from TransIP’s API settings somewhere on your device you are going to run this from.
2. **Set Up Docker Compose**: Place the `docker-compose.yml` and `certbot-transip.sh` files in the same directory. (or edit the script)
3. **Run the Container**: Start the container (cd to the path where your yml is)

```bash
docker-compose up --build
```

Monitor the Log output: The log output of the docker container will show the progress of DNS challenges and certificate issuance.

Thats all there is to it!

I will also post a version with a working helm chart soon. This was a little bigger culprit to make sure it all works correctly and updates all certificates through different namespaces where i needed the certificates to be available on different domains.
