GGUF quants are already up and llama.cpp was updated today to support it.