12-Factor App Compliance Analysis

Reference: The Twelve-Factor App

Overview

The 12-Factor App methodology is a set of best practices for building Software-as-a-Service applications that are:

• Portable across execution environments

• Scalable without architectural changes

• Suitable for continuous deployment

• Maintainable with minimal friction

Input Parameters

Parameter Description Required

codebase_path

Root path of the codebase to analyze Required

Analysis Framework

Factor I: Codebase

Principle: One codebase tracked in revision control, many deploys.

Search Patterns:

Check for version control

ls -la .git 2>/dev/null || ls -la .hg 2>/dev/null

Check for multiple apps sharing codebase

find . -name "package.json" -o -name "pyproject.toml" -o -name "setup.py" | head -20

Check for environment-specific code branches

grep -r "if.production|if.development|if.staging" --include=".py" --include=".js" --include=".ts"

File Patterns: .git/ , package.json , pyproject.toml , deployment configs

Compliance Criteria:

Level Criteria

Strong Single Git repo, same codebase for all environments, no env-specific code branches

Partial Single repo but some environment-specific code paths

Weak Multiple repos for same app or significant code duplication across environments

Anti-patterns:

• Multiple Git repositories for the same application

• Environment-specific code branches (if production: ... )

• Different source files for dev vs prod

• Shared code not extracted to libraries

Factor II: Dependencies

Principle: Explicitly declare and isolate dependencies.

Search Patterns:

Python dependency files

find . -name "requirements.txt" -o -name "pyproject.toml" -o -name "setup.py" -o -name "Pipfile" -o -name "uv.lock"

JavaScript/TypeScript dependency files

find . -name "package.json" -o -name "package-lock.json" -o -name "yarn.lock" -o -name "pnpm-lock.yaml"

Check for system tool assumptions

grep -r "subprocess.*curl|subprocess.*wget|os.system.ffmpeg|shutil.which" --include=".py" grep -r "exec.curl|child_process.curl" --include=".js" --include=".ts"

Docker/container isolation

find . -name "Dockerfile" -o -name "docker-compose*.yml"

File Patterns: /requirements.txt , **/package.json , **/.lock , **/Dockerfile

Compliance Criteria:

Level Criteria

Strong Lock files present, dependency isolation (venv/Docker), no implicit system tools

Partial Dependencies declared but no lock files or isolation

Weak Dependencies in documentation only, relies on system-installed packages

Anti-patterns:

• Missing lock files (non-deterministic builds)

• Assuming system tools (curl, ImageMagick, ffmpeg) are available

• Different dependency managers in dev vs production

• No virtual environment or container isolation

Factor III: Config

Principle: Store config in the environment.

Search Patterns:

Environment variable usage

grep -r "os.environ|os.getenv|process.env|ENV[" --include=".py" --include=".js" --include=".ts" --include=".rb"

Hardcoded credentials (anti-pattern)

grep -r "password.=.['"]" --include=".py" --include=".js" --include=".ts" | grep -v "test|spec|example" grep -r "api_key.=.['"]" --include=".py" --include=".js" --include=".ts" | grep -v "test|spec|example" grep -r "secret.=.['"]" --include=".py" --include=".js" --include="*.ts" | grep -v "test|spec|example"

Environment-specific config files (anti-pattern)

find . -name "config.dev." -o -name "config.prod." -o -name "settings.development." -o -name "settings.production."

Database URLs in code

grep -r "postgresql://|mysql://|mongodb://|redis://" --include=".py" --include=".js" --include="*.ts" | grep -v ".env|test|example"

File Patterns: /.env , **/config/.py , **/settings.py , environment files

Compliance Criteria:

Level Criteria

Strong All config via environment variables, no hardcoded secrets, could open-source without leaks

Partial Most config externalized but some hardcoded defaults

Weak Hardcoded credentials, environment-specific config files

Anti-patterns:

• Hardcoded database URLs, API keys, passwords in source

• Config files like config/production.yml vs config/development.yml

• Environment grouping (if ENV == 'production': ... )

• Secrets committed to version control

Factor IV: Backing Services

Principle: Treat backing services as attached resources.

Search Patterns:

Database connection via config

grep -r "DATABASE_URL|DB_HOST|REDIS_URL|CACHE_URL" --include=".py" --include=".js" --include="*.ts"

Service initialization

grep -r "create_engine|MongoClient|Redis|Celery|boto3" --include=".py" grep -r "createPool|createClient|new Redis|S3Client" --include=".js" --include="*.ts"

Hardcoded service locations (anti-pattern)

grep -r "localhost:5432|localhost:6379|localhost:27017|127.0.0.1" --include=".py" --include=".js" --include="*.ts" | grep -v "test|spec|example|default"

File Patterns: **/database/.py , **/services/.py , **/db.py , connection configurations

Compliance Criteria:

Level Criteria

Strong All services via URL/connection string in config, swappable without code changes

Partial Most services configurable but some hardcoded defaults

Weak Hardcoded service locations, different code paths per environment

Anti-patterns:

• Hardcoded localhost for services in production code

• Conditional logic for local vs cloud services (if USE_S3: ... else: local_storage )

• Service-specific code paths based on environment

• Different drivers for dev vs prod

Factor V: Build, Release, Run

Principle: Strictly separate build and run stages.

Search Patterns:

Build/deploy configuration

find . -name "Dockerfile" -o -name "Makefile" -o -name "build.sh" -o -name "deploy.sh" find . -name ".github/workflows/*.yml" -o -name ".gitlab-ci.yml" -o -name "Jenkinsfile"

Build scripts in package.json

grep -A5 '"scripts"' package.json 2>/dev/null | grep -E "build|start|deploy"

Check for runtime compilation (anti-pattern)

grep -r "compile|transpile|webpack" --include="*.py" | grep -v "test|build"

File Patterns: **/Dockerfile , **/Makefile , /.github/workflows/ , CI/CD configs

Compliance Criteria:

Level Criteria

Strong Immutable releases, clear build/release/run stages, unique release IDs

Partial Build and run separated but release not immutable

Weak Runtime code modifications, asset compilation at startup

Anti-patterns:

• Runtime code modifications

• Asset compilation during application startup

• Configuration baked into build artifacts

• No release versioning

Factor VI: Processes

Principle: Execute the app as one or more stateless processes.

Search Patterns:

Session storage patterns

grep -r "session|Session" --include=".py" --include=".js" --include="*.ts" | head -20

In-process state (anti-pattern)

grep -r "global.cache|process_local|instance_cache" --include=".py" grep -r "global..=|module.exports.cache" --include=".js" --include="*.ts"

External session stores (good pattern)

grep -r "redis.session|memcached.session|session.redis" --include=".py" --include=".js" --include=".ts"

Sticky session configuration (anti-pattern)

grep -r "sticky.session|session.affinity" --include=".yml" --include=".yaml" --include="*.json"

File Patterns: **/middleware/.py , **/session/.py , server configurations

Compliance Criteria:

Level Criteria

Strong Stateless processes, all state in external datastores (Redis, DB)

Partial Mostly stateless but some in-process caching

Weak Sticky sessions, in-process session storage, shared memory state

Anti-patterns:

• In-process session storage (user_sessions = {} )

• Sticky sessions or session affinity

• File-based caching between requests

• Global mutable state shared across requests

Factor VII: Port Binding

Principle: Export services via port binding.

Search Patterns:

Self-contained port binding

grep -r "app.run|server.listen|serve|uvicorn" --include=".py" grep -r "app.listen|server.listen|createServer" --include=".js" --include="*.ts"

PORT environment variable

grep -r "PORT|port" --include=".py" --include=".js" --include="*.ts" | grep -i "environ|process.env"

Webserver as dependency

grep -r "uvicorn|gunicorn|flask|fastapi|express|koa|hapi" package.json pyproject.toml requirements.txt 2>/dev/null

File Patterns: **/main.py , **/server.py , **/app.py , **/index.js

Compliance Criteria:

Level Criteria

Strong Self-contained app binds to PORT, webserver is a dependency

Partial Port binding but not configurable via environment

Weak Relies on external webserver container (Apache, Nginx) to provide HTTP

Anti-patterns:

• Relying on Apache/Nginx/Tomcat to inject webserver functionality

• Hardcoded port numbers

• No PORT environment variable support

• CGI scripts or server modules

Factor VIII: Concurrency

Principle: Scale out via the process model.

Search Patterns:

Process definitions

find . -name "Procfile" -o -name "process.yml" -o -name ".foreman"

Multiple entry points

find . -name "worker.py" -o -name "scheduler.py" -o -name "web.py"

Background job systems

grep -r "celery|rq|sidekiq|bull|agenda" --include=".py" --include=".js" --include=".ts" grep -r "Celery|Worker|BackgroundJob" --include=".py" --include=".js" --include=".ts"

File Patterns: **/Procfile , **/worker.py , **/scheduler.py , queue configurations

Compliance Criteria:

Level Criteria

Strong Explicit process types (web, worker, scheduler), horizontal scaling

Partial Multiple process types but not easily scalable

Weak Single monolithic process, no separation of concerns

Anti-patterns:

• Single process handling all workloads

• Hard-coded worker counts in code

• No separation between web and background processes

• Vertical scaling only (bigger server, not more processes)

Factor IX: Disposability

Principle: Maximize robustness with fast startup and graceful shutdown.

Search Patterns:

Signal handlers

grep -r "signal.signal|SIGTERM|SIGINT|atexit" --include="*.py" grep -r "process.on.SIGTERM|process.on.SIGINT" --include=".js" --include=".ts"

Graceful shutdown

grep -r "graceful.shutdown|shutdown_handler|cleanup" --include=".py" --include=".js" --include=".ts"

Startup time

grep -r "startup|initialize|bootstrap" --include=".py" --include=".js" --include="*.ts" | head -20

File Patterns: **/main.py , **/server.py , lifecycle management code

Compliance Criteria:

Level Criteria

Strong Fast startup (<10s), SIGTERM handling, graceful shutdown, jobs returnable to queue

Partial Graceful shutdown but slow startup

Weak No signal handling, jobs lost on process death, slow startup

Anti-patterns:

• No SIGTERM/SIGINT handlers

• Slow startup (>30 seconds)

• Jobs lost if process crashes

• No cleanup on shutdown

Factor X: Dev/Prod Parity

Principle: Keep development, staging, and production as similar as possible.

Search Patterns:

Different services per environment (anti-pattern)

grep -r "if.development.sqlite|if.production.postgres" --include=".py" --include=".js" --include=".ts" grep -r "development.SQLite|production.PostgreSQL" --include=".py" --include=".js" --include=".ts"

Docker for parity

find . -name "docker-compose*.yml" -o -name "Dockerfile"

Environment-specific backends

grep -r "USE_LOCAL_|LOCAL_STORAGE|MOCK_" --include=".py" --include=".js" --include="*.ts"

File Patterns: */docker-compose.yml , environment configurations

Compliance Criteria:

Level Criteria

Strong Same services everywhere (PostgreSQL in dev and prod), containerized

Partial Mostly same but some lightweight dev alternatives

Weak SQLite in dev, PostgreSQL in prod; different backing services

Anti-patterns:

• SQLite for development, PostgreSQL for production

• In-memory cache in dev, Redis in prod

• Different service versions across environments

• "It works on my machine" issues

Factor XI: Logs

Principle: Treat logs as event streams.

Search Patterns:

Stdout logging

grep -r "print(|logging.info|logger.info|console.log" --include=".py" --include=".js" --include="*.ts" | head -20

File-based logging (anti-pattern)

grep -r "FileHandler|open..log|writeFile.log|fs.appendFile.log" --include=".py" --include=".js" --include=".ts" grep -r "/var/log|/tmp/..log|logs/" --include=".py" --include=".js" --include=".ts" | grep -v "test|example"

Structured logging

grep -r "structlog|json_logger|pino|winston" --include=".py" --include=".js" --include="*.ts"

File Patterns: **/logging.py , **/logger.py , logging configurations

Compliance Criteria:

Level Criteria

Strong Unbuffered stdout only, structured logging (JSON), no file management

Partial Stdout logging but with some file handlers

Weak Application writes to log files, manages rotation

Anti-patterns:

• Writing logs to files (FileHandler , open('/var/log/app.log') )

• Log rotation logic in application code

• Log archival managed by application

• Buffered logging

Factor XII: Admin Processes

Principle: Run admin/management tasks as one-off processes.

Search Patterns:

Management commands

find . -name "manage.py" -o -name "Rakefile" -o -name "artisan" grep -r "@cli.command|@click.command|typer.command" --include="*.py"

Migration scripts

find . -name "migrations" -type d find . -name "migration.py" -o -name "migrate.py"

Admin scripts with proper isolation

grep -r "bundle exec|source.venv|uv run" --include=".sh" --include="Makefile"

File Patterns: **/manage.py , **/cli.py , /migrations/ , admin scripts

Compliance Criteria:

Level Criteria

Strong Admin tasks use same dependencies/config, proper isolation, idempotent

Partial Admin tasks exist but different setup from app

Weak Manual database manipulation, scripts without isolation

Anti-patterns:

• Admin scripts not using app's dependency manager

• Direct SQL manipulation outside of migrations

• Admin scripts with hardcoded credentials

• Non-idempotent migrations

Output Format

Executive Summary Table

Overall: X Strong, Y Partial, Z Weak

Per-Factor Analysis

For each factor, provide:

Current Implementation

• Evidence with file:line references

• Code snippets showing patterns

Compliance Level

• Strong/Partial/Weak with justification

Gaps

• What's missing vs. 12-Factor ideal

Recommendations

• Actionable improvements with code examples

Analysis Workflow

Initial Scan

• Run search patterns for all factors

• Identify key files for each factor

• Note any existing compliance documentation

Deep Dive (per factor)

• Read identified files

• Evaluate against compliance criteria

• Document evidence with file paths

Gap Analysis

• Compare current vs. 12-Factor ideal

• Identify anti-patterns present

• Prioritize by impact

Recommendations

• Provide actionable improvements

• Include before/after code examples

• Reference best practices

Summary

• Compile executive summary table

• Highlight strengths and critical gaps

• Suggest priority order for improvements

Quick Reference: Compliance Scoring

Score Meaning Action

Strong Fully implements principle Maintain, minor optimizations

Partial Some implementation, significant gaps Planned improvements

Weak Minimal or no implementation High priority for roadmap

When to Use This Skill

• Evaluating new SaaS applications

• Reviewing cloud-native architecture decisions

• Auditing production applications for scalability

• Planning migration to cloud platforms

• Comparing application architectures

• Preparing for containerization/Kubernetes deployment