v3.1 Production Runtime + Self-Healing Worker Cloud OS

Production Runtime

AutoBuilder Production Runtime is the reliability layer for the autonomous venture factory. It prepares worker queues, schedules, heartbeats, leases, retries, logs, recovery flows, quarantine, cost controls, alerts, backups, and runbooks.

DEMO / LOCAL MODE — workers not running live. DEMO RUNTIME STATE — LOCAL MODE

CONTROLLED RUNTIMEWORKER NOT RUNNINGNO SECRET VALUES SHOWNNO AUTO DEPLOY

CONTROLLED RUNTIMEWORKER NOT RUNNINGNO AUTO DEPLOYBACKUP NOT VERIFIED

Autonomous Factory (v3.0)Mission Control Operator Cloud Worker Fleet Public Launch (v3.2)

Next: Configure Redis + worker lease consumer with approval gates

AutoBuilder Production Runtime prepares worker queues, schedules, heartbeats, leases, retries, logs, recovery, quarantine, cost controls, alerts, backups, and runbooks. Runtime status is labeled honestly. No deploy, push, spend, contact, or publish without approval. Secret values are never displayed.

Command

Runtime command center

Runtime mode

READINESS ONLY — LOCAL MODE

Worker status

WORKER NOT RUNNING

Queue status

Queue schemas prepared — no live consumer

Scheduler

Schedules defined — not auto-running

Failed jobs

Stuck jobs

Quarantined

Approvals needed

Cost status

NO AUTO SPEND — limits configured

Server readiness

Hetzner plan prepared — NOT VERIFIED

Backup readiness

BACKUP NOT VERIFIED

Alert readiness

Telegram templates ready — NO AUTO CONTACT

Next runtime action: Configure Redis + worker lease consumer with approval gates

READINESS ONLY

Workers

Worker cloud dashboard

Lane	Name	Heartbeat	Last seen	Current job	Cost	Status
build	Build Worker	no heartbeat	—	—	$0.00	WORKER NOT RUNNINGWorker lane is represented in UI — no background process is executing this job.
eval	Eval Worker	no heartbeat	—	—	$0.00	WORKER NOT RUNNINGWorker lane is represented in UI — no background process is executing this job.
recovery	Recovery Worker	no heartbeat	—	—	$0.00	WORKER NOT RUNNINGWorker lane is represented in UI — no background process is executing this job.

Kill/pause actions: APPROVAL REQUIRED — not wired in v3.1

Schedule

Job scheduler

Job	Schedule	Type	Enabled	Approval	Status
Lightweight scan	/15 * * *	health_scan	no	no	PLANNEDOn the roadmap; not available in production yet.
Venture review	0 * * * *	venture_review	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Daily portfolio review	0 9 * * *	portfolio_daily	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Weekly portfolio report	0 10 * * 1	portfolio_weekly	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Monthly strategy report	0 8 1 * *	strategy_monthly	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Failed job retry scan	/30 * * *	retry_scan	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Quarantine review	0 /6 * *	quarantine_review	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.
Approval reminder	0 8,16 * * *	approval_reminder	no	no	PLANNEDOn the roadmap; not available in production yet.
Cost summary	0 23 * * *	cost_summary	no	no	PLANNEDOn the roadmap; not available in production yet.
Noaerth candidate scan	0 7 * * *	noaerth_scan	no	yes	APPROVAL REQUIREDAutoBuilder prepared this; you must approve before it executes.

Schedules disabled by default — not auto-running.

Queues

Queue system readiness

job

primary

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

priority

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

approval

pending 2 · running 0 · failed 0 · DLQ 0

active — demo items

DEMO

recovery

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

dead-letter

dead_letter

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

quarantine

pending 1 · running 0 · failed 0 · DLQ 0

active — demo item

DEMO

scheduled

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

report

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

alert

pending 0 · running 0 · failed 0 · DLQ 0

paused — schema ready

READINESS ONLY

Leases

Worker lease system

Job job_demo_001 · Worker w_build · Owner unassigned

Start — · Expires — · Timeout 300s · Heartbeat none

DEMO

Retry

Timeout and retry rules

small jobs

Max retries 3 · Backoff exponential · Quarantine after 3

Retry safe: yes

READINESS ONLY

medium jobs

Max retries 2 · Backoff exponential · Quarantine after 2

Retry safe: yes

READINESS ONLY

large jobs

Max retries 1 · Backoff linear · Quarantine after 1

Retry safe: no — approval required

APPROVAL REQUIRED

approval_required jobs

Max retries 0 · Backoff none · Quarantine after 1

Retry safe: no — approval required

NO AUTO SPEND

DLQ

Dead-letter queue

job_failed_demo · build · venture venture_demo

Example failure — no real job executed

Recovery: Inspect build logs, verify env, retry only after approval

DEMO

Recovery

Recovery automation readiness

build_timeout — timeout

Check worker lease expiry, increase timeout for large jobs, verify build command

Rollback: Revert to last known good build artifact

Human review: required

READINESS ONLY

Quarantine

Quarantine system

unsafe_public_claim

Demo item — marketing copy exceeded verified capability

landing page · severity medium

Action: Replace with READINESS ONLY label; operator review

QUARANTINED

Logs

Logs and observability

runtime · info · production-runtime

Runtime dashboard initialized — LOCAL MODE, no workers connected

Secrets redacted: yes · 2026-05-29T10:31:28.507Z

DEMO

No fake historical logs — only initialization events in demo.

Env

Environment variable checker

NO SECRET VALUES SHOWN — PRESENT or MISSING only.

Variable	Required for	Status	Client safe
NEXT_PUBLIC_SUPABASE_URL	Supabase client	PRESENT	yes
NEXT_PUBLIC_SUPABASE_ANON_KEY	Supabase auth	PRESENT	yes
SUPABASE_SERVICE_ROLE_KEY	Supabase server	PRESENT	server only
OPENROUTER_API_KEY	Model routing	PRESENT	server only
VERCEL_TOKEN	Deploy automation	MISSING	server only
GITHUB_TOKEN	GitHub integration	MISSING	server only
TELEGRAM_BOT_TOKEN	Telegram alerts	MISSING	server only
STRIPE_SECRET_KEY	Billing	MISSING	server only
NEXT_PUBLIC_APP_URL	App URL	PRESENT	yes
DATABASE_URL	Database	MISSING	server only
REDIS_URL	Queue backend	MISSING	server only

Secrets

Secret safety checker

env_exposure_risk

Verify NEXT_PUBLIC_* vars contain only public-safe keys

Client-side bundles · risk medium

Never expose service role or API keys in client bundles

READINESS ONLY

hardcoded_key_risk

Run secret scan before production; no automated scan in v3.1

Source scan — manual review · risk high

Use env vars; enable pre-commit secret detection

PLANNED

Deploy

GitHub / Vercel monitor readiness

Repo: NOT VERIFIED — connect token to check

Commit: —

Build: NOT VERIFIED

Deploy: NOT VERIFIED

Deploy approval required · Auto deploy disabled

NO AUTO DEPLOY

Database

Supabase runtime readiness

Tables: NOT VERIFIED

RLS: NOT VERIFIED

Migrations: NOT VERIFIED

Auth: NOT VERIFIED

Service role: Server-only — never client-side

Backup: BACKUP NOT VERIFIED

Missing: Live migration audit, RLS policy review, Backup schedule

NOT VERIFIED

Queue backend

Redis / queue backend readiness

Redis URL: MISSING CONFIG

Backend: MISSING CONFIG

Namespace: autobuilder:v31

Consumer: WORKER NOT RUNNING

MISSING CONFIG

Infrastructure

Hetzner worker runtime plan

Role: worker-fleet-primary

Process manager: systemd (planned)

Restart: on-failure, max 3 restarts/hour

Resources: 2 vCPU, 4GB RAM, 40GB disk per worker node

Deploy: Manual deploy with operator approval — NO AUTO DEPLOY

Rollback: Restore previous systemd unit + artifact tag

PLANNED

Alerts

Telegram operator alert readiness

MISSING CONFIG — NO AUTO CONTACT

Templates prepared — NO AUTO CONTACT until operator enables.

MISSING CONFIG

Cost

Cost and runaway controls

Daily spend

$0.00

Weekly spend

$0.00

Max jobs/hr

Max retries

Max workers

Approval threshold

$25/day

NO AUTO SPEND

Safety

Emergency stop system

Scope: global · Status: armed — not triggered

Workers paused: no · Paid jobs paused: no · External actions paused: no

CONTROLLED RUNTIME

Backup

Backup and restore readiness

Database: BACKUP NOT VERIFIED

Venture export: Local export available in workspace

Logs backup: NOT VERIFIED

Last backup: —

Verified: no

1) Verify backup 2) Test restore in staging 3) Document RTO/RPO

BACKUP NOT VERIFIED

Runbooks

Production runbooks

Build failed

Trigger: CI or worker build returns error

Contain
Classify
Generate recovery packet
Operator review
Retry only if safe

READINESS ONLY

Worker stuck

Trigger: Lease expired without heartbeat

Contain
Classify
Generate recovery packet
Operator review
Retry only if safe

READINESS ONLY

Queue backed up

Trigger: Pending count exceeds threshold

Contain
Classify
Generate recovery packet
Operator review
Retry only if safe

READINESS ONLY

Environment missing

Trigger: Required env var absent

Contain
Classify
Generate recovery packet
Operator review
Retry only if safe

READINESS ONLY

Unsafe claim published

Trigger: Public page exceeds verified capability

Contain
Classify
Generate recovery packet
Operator review
Retry only if safe

READINESS ONLY

Incidents

Incident response runtime

template template

We are investigating a runtime issue. No user data exposed.

Pause affected lane
Quarantine risky output
Alert operator

PLANNED

Runtime QA checklist

Check	Status	Finding	Risk
No fake worker status	pass	Workers labeled WORKER NOT RUNNING	LIVEShipped in production and reachable on this site.
No fake uptime	pass	No uptime claims displayed	LIVEShipped in production and reachable on this site.
No secrets exposed	pass	Env checker shows PRESENT/MISSING only	NO SECRET VALUES SHOWNOnly presence or missing status — secret values never displayed.
No auto-deploy	pass	NO AUTO DEPLOY labels applied	NO AUTO DEPLOYDeployment requires manual founder/operator approval.
Redis consumer	warn	Queue consumer not running	WORKER NOT RUNNINGWorker lane is represented in UI — no background process is executing this job.
Backup verified	fail	BACKUP NOT VERIFIED	BACKUP NOT VERIFIEDBackup status is not confirmed — do not claim backups are active.

Execution

Cursor / Aider runtime packets

Copy packets for controlled runtime implementation.

Add Production Runtime Route

route

Add Runtime Command Center

command

Add Worker Cloud Dashboard

worker

Add Job Scheduler

scheduler

Add Queue System Readiness

queues

Add Worker Lease System

leases

Add Timeout and Retry Rules

retry

Add Dead-Letter Queue

dlq

Add Recovery Automation Readiness

recovery

Add Quarantine System

quarantine

Add Logs and Observability

logs

Add Environment Variable Checker

env

Add Secret Safety Checker

secret

Add GitHub/Vercel Monitor Readiness

github

Add Supabase Runtime Readiness

supabase

Add Redis Queue Readiness

redis

Add Hetzner Worker Runtime Plan

hetzner

Add Telegram Alert Readiness

Add Cost and Runaway Controls

cost

Add Emergency Stop System

estop

Add Backup and Restore Readiness

backup

Add Production Runbooks

runbooks

Add Incident Response Runtime

incident

Add Runtime QA Checklist

Add Worker Cloud Final QA

final

Wire local worker lease consumer (Aider)

APPROVAL REQUIRED

Dashboard Trust center