PR Review Apps

Every non-draft pull request to main gets its own running copy of feature-app/, built from the PR's code, at:

https://pr-<number>.catalyst.platforms.commutatus.com

Reviewers can click through the change in a real app instead of pulling the branch locally. The app is redeployed on every push and destroyed when the PR is closed or merged.

At a Glance

Piece	What it is	Where it's defined
Workflow	GitHub Actions job that deploys and tears down review apps	.github/workflows/review-app.yml
Fly app	cm-admin-pr-<number>: Puma, Sidekiq and Redis on one machine	feature-app/fly.review.toml
Docker image	Built from the repo root so the gem source is this PR's checkout	feature-app/Dockerfile.review
Database	Neon branch pr-<number>, one per PR	Created by the workflow
File uploads	Tigris bucket cm-admin-pr-<number>, one per PR	Created by the workflow, config/storage.yml
Router	cm-admin-review-router: Caddy that maps each pr-* host to its app	feature-app/review-router/
Rails env	review, which behaves like staging	feature-app/config/environments/review.rb
Seeds	Shared seeds plus the reviewer accounts	feature-app/db/seeds/review.rb

Lifecycle

PR opened / reopened / pushed to / marked ready for review
  └─ deploy job
       ├─ Create Neon branch pr-<number>             (reused if it exists)
       ├─ Create Fly app cm-admin-pr-<number>        (reused if it exists)
       ├─ Create Tigris bucket                       (reused if it exists)
       ├─ Stage secrets: DATABASE_URL, POSTMARK_API_TOKEN, APP_HOST,
       │                 SECRET_KEY_BASE (first deploy only)
       ├─ flyctl deploy (remote build of Dockerfile.review)
       │    └─ release command: bin/review-app-release
       │         ├─ bin/rails db:prepare   (schema + seeds on first run, migrations after)
       │         └─ bin/rails db:seed      (fills in cm_admin permissions)
       ├─ Allocate a private Flycast address     (so the router can reach the app)
       └─ Post or update the "Review app:" comment on the PR

PR closed or merged
  └─ teardown job
       ├─ Destroy the Tigris bucket and the Fly app
       └─ Delete the Neon branch

Every step in the deploy job is safe to repeat, so a push to an existing PR simply redeploys the same app against the same database. Pushes that arrive while a deploy is running cancel it, so only the latest commit is deployed.

When a PR does not get a review app

The deploy job is skipped when:

• the PR is a draft (marking it ready for review triggers the deploy);
• the PR comes from a fork or from Dependabot, since GitHub doesn't give those runs the Actions secrets;
• the PR targets a branch other than main;
• every changed file is documentation: **/*.md, **/*.mdx, docs/** or docs-site/**.

Using a Review App

1. Open the link in the Review app comment on the PR. The comment is updated in place on each deploy and shows the commit SHA it was built from.
2. Sign in with your email address. It must be one of the users listed in feature-app/db/seeds/review.rb.
3. Enter the one-time code that is emailed to you.

Things to keep in mind:

• It sleeps when idle. Fly suspends the machine after a period without traffic, and the first request afterwards can take a few seconds while it wakes.
• Emails are really sent through Postmark. Don't trigger mail to addresses outside the team.
• Data persists across pushes. The Neon branch and Tigris bucket live as long as the PR, so records and uploads you create survive redeploys. Sessions do too, as SECRET_KEY_BASE is generated once per app.
• Background jobs run. Sidekiq runs inside Puma (EMBEDDED_SIDEKIQ=true), using the default and mailers queues.

Adding a reviewer

Add an entry to the users list in feature-app/db/seeds/review.rb and merge it to main. The seeds run on every deploy and only create users that don't exist yet, so open PRs pick the new user up on their next push once they include the change.

How It Fits Together

Routing

Browser ── https://pr-42.catalyst.platforms.commutatus.com
   │
   ▼
Fly edge (TLS, wildcard certificate *.catalyst.platforms.commutatus.com)
   │
   ▼
cm-admin-review-router (Caddy)
   │   Host ^pr-(\d+)\.catalyst… → cm-admin-pr-42.flycast:80
   ▼
cm-admin-pr-42 (Puma on :3000)

The router holds a single wildcard certificate, so review apps need no certificate or DNS record of their own and don't count towards Let's Encrypt's weekly limit for the domain. It reaches each app over Flycast (Fly's private network), which is HTTP-only, so HTTPS is forced at the router and Caddy passes X-Forwarded-Proto: https on to Rails. Requests through Flycast wake a suspended app just as public requests do. Any host that doesn't match the pattern gets a 404 saying "No review app here."

Each app is also reachable directly at https://cm-admin-pr-<number>.fly.dev.

The image

Dockerfile.review is built with the repo root as its context, so feature-app's gem 'cm-admin', path: '../' uses the PR's version of the gem. In particular:

• config/database.review.yml is copied over config/database.yml.
• bundle install is not frozen, because Gemfile.lock can lag behind the gem version in the checkout.
• assets:precompile also runs cm_admin:scss:build, which needs npm in the repo root.
• bin/review-app-start is the entrypoint. It starts Redis inside the machine before Puma, so Redis suspends and resumes along with the app.

Dockerfile.review.dockerignore keeps .git, node_modules, logs, master keys and the docs site out of the build context. Because .git is excluded, cm_admin.gemspec falls back to a file glob for its file list.

The review environment

config/environments/review.rb loads the staging config and then:

• points db:seed at db/seeds/review.rb;
• reads the Postmark token from the POSTMARK_API_TOKEN Fly secret, since only development credentials are committed;
• builds mailer links from APP_HOST;
• stores Active Storage uploads in the app's Tigris bucket.

The database

Each PR gets a Neon branch named pr-<number>, branched from the project's default branch. The app connects with the direct (unpooled) URL, because PgBouncer's transaction mode breaks prepared statements and migration advisory locks.

Configuration

Repository secrets and variables

Name	Type	Used for
FLY_API_TOKEN	Secret	Creating, deploying and destroying Fly apps and buckets
NEON_API_KEY	Secret	Creating and deleting Neon branches
POSTMARK_API_TOKEN	Secret	Sending OTP and other emails from review apps
FLY_ORG	Variable	Fly organisation that review apps are created in
NEON_PROJECT_ID	Variable	Neon project that PR branches are created in

One-off router setup

The router has to exist before any review app is reachable on the custom domain. From feature-app/review-router/:

fly apps create cm-admin-review-router --org <FLY_ORG>
fly deploy
fly certs add "*.catalyst.platforms.commutatus.com"

Then add two records in Route 53:

1. the _acme-challenge CNAME that fly certs setup "*.catalyst.platforms.commutatus.com" prints;
2. *.catalyst.platforms.commutatus.com CNAME cm-admin-review-router.fly.dev.

After changing the Caddyfile, run fly deploy in that directory again.

Troubleshooting

Symptom	Likely cause and fix
No Review app comment on the PR	The PR is a draft, from a fork or Dependabot, or only touches docs. Check the "Review app" workflow run in the Actions tab.
"No review app here." (404)	The Fly app doesn't exist yet or the host is mistyped. Wait for the deploy job to finish.
Deploy fails in the release command	A migration or seed failed. Check the release logs with fly logs --app cm-admin-pr-<number>.
The domain hangs or errors but fly.dev works	The app has no Flycast address, or the router is down. Re-run the workflow, or check fly status --app cm-admin-review-router.
No OTP email arrives	Your email isn't in db/seeds/review.rb, or the POSTMARK_API_TOKEN secret is wrong.
First request is slow	Expected: the machine was suspended and is waking up.
Need a clean database	Delete the pr-<number> branch in Neon and push to the PR (or re-run the workflow). The deploy creates a fresh branch and seeds it.
Resources left behind after a PR closed	The teardown job tolerates missing resources but can still fail. Re-run it from the Actions tab, or destroy cm-admin-pr-<number> and the Neon branch by hand.