It uses 261 W, probably more expensive than getting a 2nd Mistral subscription with the cost of power where I live (Belgium), but it will be free heat once I have solar panels and a battery.

I get about 45 t/s after some tweaking, getting the thermals under control helped a lot; mainly by reducing the power consumption to 261 W (and downclocking a little bit).

What vision model do you use?

I use Qwen 3.8 27b Q4 with 100k context (llama.cpp and the Mistral AI harness) on a 24 GB AMD Radeon 7900 XTX and it’s been great. It can handle pretty much anything I throw at it and I can let it work indepedently overnight / over days on long tasks.

3.8 MB, wtf did you do to this comic? This should have been less than 100 420 kB.