container/Sources/NativeBuilder/docs/ContainerBuildExecutor/Architecture.md

# ContainerBuildExecutor Architecture

## Overview

The ContainerBuildExecutor implements a clean, production-ready execution layer for ContainerBuildIR using the **Executor Pattern** rather than a complex scheduler.

## Architecture

### Core Components

1. **BuildExecutor** - The main orchestrator that executes complete build graphs
2. **OperationExecutor** - Executes individual operations (pluggable for different operation types)
3. **ExecutionContext** - Carries mutable state through the build process
4. **ExecutionDispatcher** - Routes operations to appropriate executors based on capabilities
5. **BuildCache** - Caches operation results to avoid redundant work
6. **Snapshotter** - Manages filesystem snapshots for layer creation

### Design Principles

- **Simplicity** - Clean interfaces with minimal complexity
- **Extensibility** - Easy to add new operation types and executors
- **Performance** - Parallel execution where possible, with efficient caching
- **Type Safety** - Leverages Swift's type system for correctness

## Execution Flow

```mermaid
graph TD
    A[BuildGraph] --> B[SimpleExecutor]
    B --> C[Stage Ordering]
    C --> D[For Each Stage]
    D --> E[Topological Sort]
    E --> F[For Each Node]
    F --> G{Check Cache}
    G -->|Hit| H[Return Cached Result]
    G -->|Miss| I[ExecutionDispatcher]
    I --> J[Find Suitable Executor]
    J --> K[Execute Operation]
    K --> L[Update Context]
    L --> M[Cache Result]
    M --> N[Next Node]
```

## Capability-Based Routing

The dispatcher matches operations to executors based on:

1. **Operation Type** - Does the executor support this operation kind?
2. **Platform** - Can the executor handle the target platform?
3. **Privileges** - Does the executor have required privileges?
4. **Resources** - Are resource requirements satisfied?

## Concurrency Model

### Stage-Level Execution

Stages are executed sequentially to respect dependencies:

```swift
for stage in graph.stagesInDependencyOrder() {
    let context = ExecutionContext(stage: stage, ...)
    let snapshot = try await executeStage(stage, context: context)
    stageSnapshots[stage.name] = snapshot
}
```

### Node-Level Parallelism

Within a stage, independent nodes execute concurrently:

```swift
let levels = try GraphTraversal.topologicalLevels(stage)
for level in levels {
    await withTaskGroup(of: ExecutionResult.self) { group in
        for node in level {
            group.addTask {
                try await executeNode(node, context)
            }
        }
    }
}
```

## State Management

### Snapshot Evolution

Each operation produces a new filesystem snapshot:

```
Initial Snapshot (S0)
    ↓
Operation 1 → Snapshot S1
    ↓
Operation 2 → Snapshot S2
    ↓
Operation 3 → Snapshot S3 (Final)
```

### Environment Propagation

Environment changes cascade through operations:

```swift
context.updateEnvironment(["FOO": .literal("bar")])
// This update is visible to all subsequent operations in the stage
```

## Caching Strategy

### Cache Key Generation

Cache keys include:
- Operation digest (content hash)
- Input digests (from dependencies)
- Platform identifier
- Additional context

### Cache Lookup Flow

1. Compute cache key for operation
2. Check cache for existing result
3. If hit: skip execution, use cached result
4. If miss: execute operation, store result

## Error Handling

Errors are categorized and handled appropriately:

- **Unsupported Operations** - No executor can handle the operation
- **Resource Constraints** - Requirements cannot be satisfied
- **Execution Failures** - Operation failed during execution
- **Cancellation** - Build was cancelled by user

## Extensibility Points

### Adding New Operation Types

1. Define the operation in ContainerBuildIR
2. Create a specific executor implementing `OperationExecutor`
3. Register the executor with the dispatcher

### Custom Caching

Implement the `BuildCache` protocol:

```swift
public protocol BuildCache: Sendable {
    func get(_ key: CacheKey, for operation: Operation) async -> CachedResult?
    func put(_ result: ExecutionResult, key: CacheKey, for operation: Operation) async
}
```

### Alternative Snapshotters

Implement the `Snapshotter` protocol for different backends:

```swift
public protocol Snapshotter: Sendable {
    func createSnapshot(from parent: Snapshot?, applying changes: FilesystemChanges) async throws -> Snapshot
    func prepare(_ snapshot: Snapshot) async throws -> SnapshotHandle
}
```

## Performance Considerations

1. **Lazy Snapshot Creation** - Only create snapshots when filesystem changes occur
2. **Parallel Execution** - Maximize concurrency within dependency constraints
3. **Efficient Caching** - Cache keys designed for fast lookup
4. **Resource Pooling** - Reuse expensive resources like container instances

## Future Enhancements

1. **Distributed Execution** - Execute operations across multiple machines
2. **Incremental Builds** - Skip unchanged portions of the graph
3. **Progress Reporting** - Real-time feedback during execution
4. **Resource Monitoring** - Track CPU, memory, and I/O usage