token-calculator.net
Token Calculator
LLM RAM Calculator
Token Generation Speed Simulator
Toggle theme
LLMs GPU RAM Calculator
Estimate the GPU RAM required for Large Language Models
LLM GPU RAM Calculator
Model Size (billion parameters):
Precision:
32-bit (FP32)
16-bit (FP16)
8-bit (INT8)
Calculate
FAQ
How much GPU memory is needed to serve a Large Language Model (LLM)?
What factors affect GPU memory usage for LLMs?
How can I optimize memory usage for LLM deployment?
What's the difference between 16-bit and 32-bit precision for LLMs?
How does model size affect GPU memory requirements?
Have more questions?
Contact us