If I ask qwq32 anything that is even slightly complicated it will ramble until it exceeds the context window, then forget my question. Q4k which is all that fits (with context) on a 3090.
Gemma3 27B gives me a rapid 1shot response, and actually works really well for the type of rubber duck brainstorming partner I often need.
Try giving it a river crossing puzzle with substitutions. QwQ can take a lot of time but it will solve it. Gemma will just confidently give you a wrong answer, and will keep giving you wrong answers if you point out the mistakes.
Now, yes, QwQ will take a lot of tokens to get there (in one case it took it over 5 minutes running on Mac Studio M1 Ultra). Nevertheless, at least it can solve it.
Yeah, but how many river crossing puzzles and murder mystery games was it trained on, and how many times do I actually need to solve a river crossing puzzle?
Gemma3 27B gives me a rapid 1shot response, and actually works really well for the type of rubber duck brainstorming partner I often need.