Qwen

Qwen

Qwen/Qwen3.8-27B

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q4_k_s	Unsloth Unsloth	14.3 GB	N/A	N/A	N/A	Run
nvfp4	MLX Community MLX Community	15.0 GB	N/A	N/A	N/A	Run
4bit	MLX Community MLX Community	15.0 GB	19.9 tok/s	197.6 tok/s	Runs poorly	Run
q4_k_s	Unsloth Unsloth	15.0 GB	18.9 tok/s	190.9 tok/s	Runs ok	Run
8bit	MLX Community MLX Community	27.5 GB	N/A	N/A	N/A	Run

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.