Multi-Cloud HA Static Site with CloudFront, GCP, and Home Kubernetes

The Problem with a Single Origin

A static site backed by a single S3 bucket is one IAM misconfiguration, accidental public-access block, or regional AWS incident away from going dark. My site ziteworld.com ran this way for years — CloudFront in front, S3 behind, one failure domain.

The fix is a multi-origin architecture: CloudFront routes to S3 as primary, fails over to GCP Cloud Storage, and a self-hosted nginx webserver on a home Kubernetes cluster is exposed publicly via Cloudflare Tunnel as an independent third origin. Additionally, a gated Bitbucket pipeline promotes changes through a staging environment before touching any production origin.

Architecture

Bitbucket (main push)
        │
        ▼
[Stage 1: Auto → <your-domain>-stg]
  hello.<your-domain>.com  (internal only)
        │
   Review on LAN
        │
  [Manual approval]
        │
        ▼
┌───────┼──────────────┐
▼       ▼              ▼
S3    GCP Cloud    Home K8s (<your-domain> ns)
      Storage      nginx + Cloudflare Tunnel
                   cf.<your-domain>.com

CloudFront uses an Origin Group for automatic failover:

CloudFront → Origin Group
               ├─ Primary:  S3 (<your-site>.com bucket)
               └─ Failover: GCP Cloud Storage (<your-site>-static-site)
                            triggers on: 500, 502, 503, 504

cf.<your-domain>.com (home K8s + Cloudflare Tunnel) is registered as a standalone CloudFront origin and serves as a public fallback reachable independently of the failover group.

Home Kubernetes Layout

Two namespaces handle staging and production on the same cluster:

Namespace	Hostname	Purpose	Access
<your-domain>-stg	hello.<your-domain>.com	Staging — preview before prod	Internal network only
<your-domain>	cf.<your-domain>.com	Prod webserver — CloudFront origin	Public via Cloudflare Tunnel

Both use an initContainer pattern: a generic nginx:latest image serves content that a alpine/git init container clones from the Bitbucket content repo at pod startup. Rolling restarts (triggered by a restartedAt annotation update) pull fresh content without rebuilding any Docker image.

initContainers:
- name: init-copy-files
  image: alpine/git:latest
  command: ["/bin/sh", "-c"]
  args:
  - |
    apk add openssh-client &&
    mkdir -p /root/.ssh &&
    cp /git-creds/sshPrivateKey /root/.ssh/id_rsa &&
    chmod 600 /root/.ssh/id_rsa &&
    ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts &&
    git clone git@bitbucket.org:<your-workspace>/<your-site>.git /tmp/<your-site> &&
    cp -r /tmp/<your-site>/_site/* /html-mount
  volumeMounts:
  - name: html-volume
    mountPath: /html-mount
  - name: git-creds
    mountPath: /git-creds

SSH credentials are stored as a SealedSecret — sealed separately per namespace since SealedSecrets are namespace-scoped.

Cloudflare Tunnel for cf..com

The home cluster has no public IP and sits behind NAT. Cloudflare Tunnel solves this by establishing an outbound-only connection from cloudflared (running as a K8s Deployment in the <your-domain> namespace) to Cloudflare’s edge, which then proxies traffic in.

# cloudflared ConfigMap
tunnel: <tunnel-id>
credentials-file: /etc/cloudflared/credentials.json
ingress:
  - hostname: cf.<your-domain>.com
    service: http://<your-site>-prod-svc:80
  - service: http_status:404

The tunnel credentials JSON is stored as a SealedSecret. The Cloudflare DNS record for cf.<your-domain>.com is set to DNS only (orange-cloud off) so CloudFront can connect to the tunnel endpoint as a custom origin without Cloudflare proxying in front of CloudFront.

GCP Cloud Storage via Terraform

The GCP bucket is provisioned with Terraform using a service account key authenticated provider:

resource "google_storage_bucket" "<your-site>" {
  name                        = "<your-site>-static-site"
  location                    = "US"
  storage_class               = "STANDARD"
  uniform_bucket_level_access = true

  website {
    main_page_suffix = "index.html"
    not_found_page   = "error.html"
  }
}

resource "google_storage_bucket_iam_member" "public_read" {
  bucket = google_storage_bucket.<your-site>.name
  role   = "roles/storage.objectViewer"
  member = "allUsers"
}

The service account key lives only on the deployment host — never committed to git. For CI/CD it goes into Bitbucket encrypted repository variables.

CloudFront Origin Group

With the GCP bucket and cf.<your-domain>.com registered as custom origins, the Origin Group is configured via the AWS CLI (or Terraform):

• Primary: <your-site>.com.s3.us-west-2.amazonaws.com
• Failover: storage.googleapis.com (origin path /<your-site>-static-site)
• Failover criteria: HTTP 500, 502, 503, 504

The default cache behavior’s TargetOriginId is updated from the S3 origin to the Origin Group ID. CloudFront attempts the primary on every request; on a 5xx response it immediately retries against the GCP origin.

Note: CloudFront Origin Groups support exactly two members (one primary, one failover). cf.<your-domain>.com is attached as a separate standalone origin — CloudFront does not chain three origins in a single group.

Bitbucket Pipeline

The pipeline in the content repo drives the full promotion flow. Two required Bitbucket repository variables beyond the existing AWS credentials:

Variable	Purpose
GITHUB_TOKEN	Fine-grained PAT — write access to the gitops repo
GCP_SA_KEY	Raw JSON of the GCP service account key

pipelines:
  branches:
    main:
      - step:
          name: "Stage 1 — Deploy to staging (<your-domain>-stg)"
          script:
            - git clone https://x-token-auth:${GITHUB_TOKEN}@github.com/<your-username>/<your-domain>.git /tmp/<your-domain>
            - RESTART_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
            - sed -i "s|restartedAt:.*|restartedAt: \"${RESTART_TIME}\"|" /tmp/<your-domain>/gitops/basic/webui-deployment.yaml
            - cd /tmp/<your-domain> && git add . && git commit -m "chore: trigger stg restart" && git push origin main

      - step:
          name: "Stage 2 — Deploy to AWS S3"
          trigger: manual
          deployment: production
          script:
            - pipe: atlassian/aws-s3-deploy:0.3.7
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                S3_BUCKET: <your-site>.com
                LOCAL_PATH: $BITBUCKET_CLONE_DIR/_site
                EXTRA_ARGS: --delete

      - step:
          name: "Stage 2 — Deploy to GCP Cloud Storage"
          image: google/cloud-sdk:slim
          script:
            - echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
            - gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
            - gsutil -m rsync -r -d $BITBUCKET_CLONE_DIR/_site gs://<your-site>-static-site

      - step:
          name: "Stage 2 — Restart prod webserver (<your-domain>)"
          script:
            - git clone https://x-token-auth:${GITHUB_TOKEN}@github.com/<your-username>/<your-domain>.git /tmp/<your-domain>
            - RESTART_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
            - sed -i "s|restartedAt:.*|restartedAt: \"${RESTART_TIME}\"|" /tmp/<your-domain>/gitops/prod/webui-deployment.yaml
            - cd /tmp/<your-domain> && git add . && git commit -m "chore: trigger prod restart" && git push origin main

Stage 1 runs automatically on every push to main. The pipeline pauses at Stage 2 until manually approved in the Bitbucket UI. Once approved, S3, GCP, and the K8s prod pod are updated in sequence.

The K8s restarts work via GitOps — the pipeline commits a timestamp annotation change to the gitops repo; ArgoCD detects the drift and rolls out a new pod, which re-clones the latest content at startup. No kubectl or KUBECONFIG is needed in the pipeline.

Verification

1. Push a change to main — confirm Stage 1 deploys to https://hello.<your-domain>.com (verify on LAN).
2. Approve the Bitbucket gate — confirm _site/ content appears on the S3 bucket, GCP bucket, and https://cf.<your-domain>.com.
3. To test CloudFront failover: temporarily block public access on the S3 bucket. CloudFront should serve the site from GCP within one or two cache misses.

Conclusion

This architecture eliminates the single-origin failure mode with no changes to how the site is authored. The pipeline gate separates staging confidence from production risk. The Cloudflare Tunnel removes the hard dependency on a static home IP. And the GitOps approach means no cluster credentials ever touch the CI pipeline — ArgoCD observes the repo and acts.

The main operational tradeoff is the initContainer git-clone pattern: pod startup is slower than serving a pre-baked image, and every restart creates a runtime dependency on Bitbucket availability. Baking _site/ into a Docker Hub image at pipeline time is the natural next step when startup latency or Bitbucket uptime become a concern.