Stanford Lectures...

Trying MiniMax M3 Without Sparse Attention in llama.cpp

tl;dr: When running MiniMax M3 Q8_0, dropping temp from 1.0 to 0.8 helped a lot with minor hallucinations and oddities, and disabling MSA also seems fairly promising so far, even though llama.cpp warns that its built-in dense fallback may degrade output. And it might;

Picking Models as a Mac User

After spending the past two weeks redoing all the models around the house, I realized it might make a good topic to chat about. I know that everyone and their brother has their own way to figure out what models they want to run on their hardware, but I figure

Quick Wilmer Update Check-in

Just a quick update- I know it's been quiet on the Wilmer front, but I have a huge update coming soon that I've been working on, and hope to release next weekend. A lot of quality-of-life stuff that I've been using myself

Using the System Prompt / Preferences Field

This is one of those "Obvious, but not everyone does it" things that I wanted to call out: the biggest quality of life change that you can make when using an LLM, regardless of whether it's a small locally hosted LLM or a big proprietary LLM,

Read more

Trying MiniMax M3 Without Sparse Attention in llama.cpp

Picking Models as a Mac User

Quick Wilmer Update Check-in

Using the System Prompt / Preferences Field