Bastion
Isolates 820+ enterprise B2B tenants on shared accelerator clusters and derives billing from an immutable append-only ledger customer teams can audit.
Engineering highlights
The core architectural breakthroughs.
High-scale engineering demand distilled into four verified production milestones.
Isolated on shared GPU clusters using NVIDIA MIG hardware slicing
Hardware memory partition stops cross-tenant out-of-memory cascades
eBPF kernel probes log exact token & GPU-second usage to WORM ledger
GPU fleet utilization jumped from 31.2% to 84.6%
Executive overview
The engineering challenge & solution.
The client hosted shared GPU infrastructure for 820 enterprise software companies. A single tenant running a heavy training script frequently starved adjacent customers of GPU memory (the 'noisy neighbor' problem). Furthermore, enterprise finance teams disputed monthly invoices because the provider could not show verifiable, fine-grained token and GPU-second usage logs.
We engineered a multi-tenant Kubernetes orchestration platform leveraging NVIDIA MIG (Multi-Instance GPU) hardware slicing. Workloads are partitioned into isolated hardware containers with dedicated VRAM. Every inference request generates a signed usage receipt committed to an append-only Kafka and PostgreSQL audit ledger.
Engineering governance
The three non-negotiables.
These are the architectural constraints that shaped every boundary — no trade-offs or compromises permitted.
Hardware-Enforced Tenant Isolation
Zero cross-tenant memory or compute bleed; tenant workloads isolated at the GPU hardware partition layer.
Cryptographic Usage Ledger
Every billable token, API call, and GPU second recorded into a tamper-evident, append-only Merkle ledger.
Replayable Billing Invoicing
Enterprise customers must be able to verify and replay their monthly invoice against cryptographic execution receipts.
Deterministic Usage Metering and Fair-Share GPU Kernel Scheduling Across 820 Concurrent Tenants
Measuring GPU execution time at the microsecond level across thousands of asynchronous model inference jobs introduced significant CPU profiling overhead, slowing overall cluster throughput by 14%.
We implemented an eBPF kernel tracing probe that intercepts CUDA driver kernel launch events directly in the Linux kernel. The eBPF module captures exact execution durations and tenant IDs with zero user-space context switching overhead, streaming binary telemetry to an append-only ledger.
Request lifecycle
How data moves through Bastion.
Production verification
Measured outcomes in production.
Every metric below was captured under real production traffic and audited against historical baseline data.
Operational resilience
Automated safety guardrails.
Systems fail in production. Here is how Bastion survives network partitions, upstream timeouts, and anomalous inputs without human intervention.
NVIDIA MIG Hardware Enclaves
Physical GPU memory partitioning prevents out-of-memory cascades between tenants.
Cryptographic Invoicing Receipts
SHA-256 Merkle proofs generated for every monthly customer billing statement.
Automated Fair-Share Throttling
Dynamic token buckets throttle tenants who exceed their contracted burst allowance.
Next step
Building a system with this level of demand?
Bring us the constraint that keeps your engineering leadership up at night — the latency ceiling, the compliance perimeter, or the unyielding reliability requirement.