I Won't Miss The Cold...
This has nothing to do with technology, but just so you know- I'm a tropical beastie, and I absolutely will not miss the 22 degree weather this pass weekend. I am no longer built for this.
That is all.
Read more
Trying MiniMax M3 Without Sparse Attention in llama.cpp
tl;dr: When running MiniMax M3 Q8_0, dropping temp from 1.0 to 0.8 helped a lot with minor hallucinations and oddities, and disabling MSA also seems fairly promising so far, even though llama.cpp warns that its built-in dense fallback may degrade output. And it might;
Picking Models as a Mac User
After spending the past two weeks redoing all the models around the house, I realized it might make a good topic to chat about. I know that everyone and their brother has their own way to figure out what models they want to run on their hardware, but I figure
Quick Wilmer Update Check-in
Just a quick update- I know it's been quiet on the Wilmer front, but I have a huge update coming soon that I've been working on, and hope to release next weekend. A lot of quality-of-life stuff that I've been using myself
Using the System Prompt / Preferences Field
This is one of those "Obvious, but not everyone does it" things that I wanted to call out: the biggest quality of life change that you can make when using an LLM, regardless of whether it's a small locally hosted LLM or a big proprietary LLM,