Lantier to LocalLLaMA@sh.itjust.worksEnglish · 4 days agoQwen/QwQ-32B · Hugging Facehuggingface.coexternal-linkmessage-square5fedilinkarrow-up115arrow-down10
arrow-up115arrow-down1external-linkQwen/QwQ-32B · Hugging Facehuggingface.coLantier to LocalLLaMA@sh.itjust.worksEnglish · 4 days agomessage-square5fedilink
minus-squareLantierOPlinkfedilinkEnglisharrow-up2·4 days agoGGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF
minus-squareJamonBear@sh.itjust.workslinkfedilinkEnglisharrow-up1·4 days agoYay! let’s try ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M /set parameter num_ctx 32768
GGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF
Yay! let’s try
ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M
/set parameter num_ctx 32768