How do you benchmark and evaluate offline AI model performance?

Comprehensive offline AI benchmarking measures: latency (P50, P95, P99 inference times), throughput (inferences per second), memory footprint (peak and average RAM/VRAM usage), power consumption (watts per inference), model accuracy on domain-specific test sets, and startup time (model loading). Tools: use NVIDIA Nsight for GPU profiling, Intel VTune for CPU analysis, and custom harnesses measuring end-to-end pipeline latency. Methodology: warm up with 100 iterations before measurement, run 1000+ iterations for statistical significance, test across representative input distributions, and measure under realistic concurrent load. Key metrics for edge deployment: time-to-first-token for generative models, energy per inference for battery-powered devices, and memory watermark for multi-model scenarios. Establish regression testing in CI/CD: flag any deployment that degrades P95 latency by more than 10% or increases memory by more than 15%.