Qwen

Qwen

Qwen/Qwen3.5-0.8B

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
ud-iq2_xxs	Unsloth Unsloth	322.6 MB	75.5 tok/s	2,137.9 tok/s	Runs great	Run
q3_k_m	Unsloth Unsloth	448.4 MB	72.1 tok/s	2,228.2 tok/s	Runs great	Run
q4_k_m	Unsloth Unsloth	507.8 MB	101.9 tok/s	2,985.9 tok/s	Runs great	Run
OptiQ-4bit	MLX Community MLX Community	570.4 MB	166.9 tok/s	2,512.2 tok/s	Runs great	Run
4bit	MLX Community MLX Community	596.3 MB	213.9 tok/s	2,913.2 tok/s	Runs great	Run
q8_0	Unsloth Unsloth	774.2 MB	108.8 tok/s	4,028.9 tok/s	Runs great	Run
8bit	MLX Community MLX Community	954.8 MB	168.6 tok/s	2,202.7 tok/s	Runs great	Run

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.