Batching Efficiency at Inference: Where GPU Utilization Breaks Down
A detailed analysis of how batch size constraints imposed by latency requirements cause GPU SM underutilization in production inference workloads.
Engineering Notes
Thinking in silicon.
Technical writing from the Procunit team on inference hardware tradeoffs, model graph specialization, and the economics of production-scale ML. We write for engineers making infrastructure decisions — not for press coverage.
ASIC vs. GPU: The Data Center Economics Case for Custom Silicon
A framework for evaluating the total cost of ownership comparison between GPU clusters and dedicated inference ASICs at production inference scale.
When to Consider Custom Silicon for ML Inference
Not every ML team should be evaluating custom silicon. A decision framework based on model stability, workload volume, and infrastructure maturity.
How to Calculate Actual Throughput-per-Watt for Your Inference Workload
Most published throughput-per-watt numbers are measured at ideal conditions. Here's how to calculate the actual value for your production system configuration.
Model Graph Specialization: What a Compiler Actually Does to Your Frozen Model
A look inside the graph compiler pipeline — how ONNX and TensorFlow SavedModel files are analyzed, partitioned, and transformed into hardware-specific dataflow programs.
The GPU Wall: Why Production AI Inference Has a Structural Problem
GPU hardware was designed for training — a workload with fundamentally different compute patterns than frozen-model inference. This mismatch is getting worse, not better.