Browse the notebook

Performance

Articles in this topic

Jan 29, 20269 min read

AI & ML

Quantizing Llama 3 8B to W8A8: A Complete Guide with LLMCompressor and vLLM

Step-by-step tutorial for quantizing Meta's Llama 3 8B Instruct model to INT8 weights and activations (W8A8) using LLMCompressor, then serving it with vLLM for production inference with KV cache and continuous batching.

Jul 30, 20258 min read

Engineering

An Introduction to StarRocks: My Experience Using It (Plus My Docker Setup)

Deep dive into StarRocks OLAP database with real-world performance benchmarks, Docker setup guide, and production deployment insights for high-speed analytical workloads.

Jul 16, 202515 min read

Leadership

Understanding SIMD Performance: A Developer's Introduction with Real Benchmarks

Complete guide to SIMD performance optimization with AVX2, including real benchmarks comparing scalar vs vectorized code with GCC compiler analysis and practical implementation examples.