Platform Engineering: Building Internal Developer Platforms with Backstage and Crossplane

2025-02-15

How to build a self-service developer platform that reduces environment provisioning from weeks to minutes using Backstage, Crossplane, and ArgoCD

Platform Engineering has emerged as the answer to a critical question: how do you enable 200+ engineers to provision production-ready environments without creating a bottleneck on your infrastructure team? The answer is an Internal Developer Platform (IDP) that provides self-service infrastructure with built-in guardrails.

In this guide, I'll share how we built a platform that reduced environment provisioning from 2-3 weeks to under 10 minutes while maintaining SOC 2 compliance.

Internal Developer Platform Architecture

Why Platform Engineering?

The DevOps promise was "you build it, you run it." The reality? Most developers don't want to be infrastructure experts — they want to ship features. Platform Engineering bridges this gap by providing:

Golden paths: Opinionated, pre-configured templates that encode best practices
Self-service: Developers provision what they need without tickets
Guardrails: Automated policy enforcement prevents misconfigurations
Abstraction: Cloud complexity hidden behind simple, consistent APIs

The Architecture: Three Layers

Layer 1: Developer Experience (Backstage)

Backstage is the frontend of your platform. It provides a unified portal where developers can:

Discover existing services and their owners
Provision new environments from templates
View documentation, APIs, and dependencies
Monitor service health and SLOs

# backstage/templates/microservice/template.yaml
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: production-microservice
  title: Production Microservice
  description: Create a new microservice with CI/CD, observability, and security
spec:
  owner: platform-team
  type: service

  parameters:
    - title: Service Details
      required: [name, team, language]
      properties:
        name:
          title: Service Name
          type: string
          pattern: "^[a-z][a-z0-9-]*$"
        team:
          title: Owning Team
          type: string
          enum: [payments, identity, commerce, data]
        language:
          title: Language
          type: string
          enum: [go, typescript, python, java]
        tier:
          title: Service Tier
          type: string
          enum: [critical, standard, experimental]
          default: standard

    - title: Infrastructure
      properties:
        cloud:
          title: Cloud Provider
          type: string
          enum: [aws, azure, gcp]
          default: aws
        database:
          title: Database
          type: string
          enum: [postgres, mysql, dynamodb, none]
        cache:
          title: Cache Layer
          type: string
          enum: [redis, memcached, none]

  steps:
    - id: scaffold
      name: Scaffold Repository
      action: fetch:template
      input:
        url: ./skeleton
        values:
          name: ${{ parameters.name }}
          team: ${{ parameters.team }}

    - id: publish
      name: Create Repository
      action: publish:github
      input:
        repoUrl: github.com?owner=myorg&repo=${{ parameters.name }}

    - id: provision-infra
      name: Provision Infrastructure
      action: crossplane:claim
      input:
        apiVersion: platform.myorg.io/v1alpha1
        kind: Environment
        metadata:
          name: ${{ parameters.name }}
        spec:
          cloud: ${{ parameters.cloud }}
          tier: ${{ parameters.tier }}
          database: ${{ parameters.database }}
          cache: ${{ parameters.cache }}

    - id: register
      name: Register in Catalog
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: /catalog-info.yaml

Layer 2: GitOps Orchestration (ArgoCD)

ArgoCD ensures that the desired state in Git matches the actual state in the cluster. Every infrastructure change flows through Git:

# argocd/application-set.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: team-environments
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/myorg/environments.git
        revision: HEAD
        directories:
          - path: teams/*/environments/*

  template:
    metadata:
      name: "{{path.basename}}"
      labels:
        team: "{{path[1]}}"
        environment: "{{path[3]}}"
    spec:
      project: "{{path[1]}}"
      source:
        repoURL: https://github.com/myorg/environments.git
        targetRevision: HEAD
        path: "{{path}}"
      destination:
        server: https://kubernetes.default.svc
        namespace: "{{path[1]}}-{{path[3]}}"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true
          - ServerSideApply=true

Layer 3: Infrastructure Abstraction (Crossplane)

Crossplane is the secret weapon. It exposes cloud infrastructure as Kubernetes-native APIs, letting developers provision databases, caches, and queues with simple YAML:

# crossplane/compositions/environment.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: production-environment
spec:
  compositeTypeRef:
    apiVersion: platform.myorg.io/v1alpha1
    kind: Environment
  resources:
    # Kubernetes Namespace
    - name: namespace
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          forProvider:
            manifest:
              apiVersion: v1
              kind: Namespace
      patches:
        - fromFieldPath: metadata.name
          toFieldPath: spec.forProvider.manifest.metadata.name

    # PostgreSQL Database (if requested)
    - name: database
      base:
        apiVersion: rds.aws.upbound.io/v1beta1
        kind: Instance
        spec:
          forProvider:
            engine: postgres
            engineVersion: "15"
            instanceClass: db.t3.medium
            allocatedStorage: 20
            publiclyAccessible: false
            storageEncrypted: true
      patches:
        - fromFieldPath: spec.tier
          toFieldPath: spec.forProvider.instanceClass
          transforms:
            - type: map
              map:
                critical: db.r6g.xlarge
                standard: db.t3.medium
                experimental: db.t3.micro

    # Redis Cache (if requested)
    - name: cache
      base:
        apiVersion: elasticache.aws.upbound.io/v1beta1
        kind: ReplicationGroup
        spec:
          forProvider:
            engine: redis
            engineVersion: "7.0"
            nodeType: cache.t3.medium
            numCacheClusters: 2
            atRestEncryptionEnabled: true
            transitEncryptionEnabled: true

Developer Workflow: From Request to Running Service

Here's what the developer experience looks like:

Developer opens Backstage, selects "Create New Service"
Backstage scaffolds repository from golden path template
Git commit triggers ArgoCD sync
ArgoCD deploys Crossplane Claims to the cluster
Crossplane provisions cloud resources (RDS, ElastiCache, etc.)
Service is running with CI/CD, monitoring, and security — in under 10 minutes

# What developers see:
$ kubectl get environments -n payments-team
NAME              CLOUD   TIER       STATUS   AGE
checkout-service  aws     critical   Ready    8m
fraud-detector    aws     standard   Ready    6m

Policy Guardrails: Security Without Friction

The platform enforces organizational policies automatically:

# crossplane/compositions/policy-hooks.yaml
# Cost guardrails per tier
- fromFieldPath: spec.tier
  toFieldPath: metadata.annotations[cost-limit]
  transforms:
    - type: map
      map:
        critical: "5000" # $5000/month max
        standard: "1000" # $1000/month max
        experimental: "100" # $100/month max

Combined with OPA Gatekeeper:

# Prevent experimental workloads in production clusters
deny[msg] {
    input.review.object.metadata.labels.tier == "experimental"
    input.review.object.metadata.namespace == "production"
    msg := "Experimental tier workloads cannot be deployed to production"
}

Results: Measurable Impact

Metric	Before IDP	After IDP
Environment provisioning	2-3 weeks	< 10 minutes
Infrastructure tickets/month	200+	< 30
Teams fully self-serving	0	40+
Compliance violations	Frequent	Zero (automated)
Platform uptime	99.9%	99.99%
Annual cost savings	-	$420K (right-sizing)

Lessons from Production

Start with golden paths, not golden cages: Give teams escape hatches for genuinely unique needs, but make the golden path so good that 90% of teams choose it voluntarily.
Treat your platform as a product: Your engineers are your customers. Measure developer satisfaction, gather feedback, and iterate. If the platform is harder than doing things manually, teams will route around it.
Composability over completeness: Don't try to build everything at once. Start with the most common provisioning patterns and expand based on demand.
GitOps makes everything auditable: Every infrastructure change has a PR, a review, and a commit hash. This makes compliance evidence automatic.
Invest in documentation: A platform without documentation is a black box. Backstage's TechDocs integration makes documentation a first-class citizen.

Getting Started

The quickest path to an IDP:

Month 1: Deploy Backstage with a service catalog (just visibility)
Month 2: Add 2-3 golden path templates for common patterns
Month 3: Integrate Crossplane for infrastructure provisioning
Month 4: Add ArgoCD for GitOps-driven deployments
Month 5+: Iterate based on developer feedback

Remember: the best platform is one that engineers actually want to use. Start simple, prove value, and expand.