SomeOddCodeGuy's Ramblings

Using SSH Tunnels to Make Up for Lack of HTTPS on LAN

The reality is that a lot of folks who run open source apis/web front ends on their local LAN tend to run it as plain HTTP; whether its backend llm apis, the front end sites, or whatever other stuff you've tossed in: no TLS anywhere in sight.

Third Post's the Charm- Lack of Recent Updates

I haven't posted or made any updates on Wilmer in like a month, and then I suddenly dropped 3 blog posts all at once. If that doesn't tell you that I don't pay attention to things like trying to game SEO, not sure what

Llama.cpp's New MTP on MacOS

MTP So I decided to test out the new MTP in llama.cpp on Metal using my M2 Ultra, and figured I'd toss the results up here. This isn't meant to show the maximum tps you can get on Mac hardware; I'd have run

Building and Running Llama.cpp on an Air-Gapped Mac

If you ever tried to run Llama.cpp on a MacOS device that doesn't have internet on it, you've probably hit the annoying GateKeeper errors that it's downloaded from the internet and you should delete it. Generally I just build from source to avoid

A Quick-ish Rundown of LLM Basics

Over the past few days, I've realized that there are a lot of folks out there using LLMs that haven't had an opportunity to dig, even a little, into the basics of how LLMs really work. And I guess that makes sense; for the most part,

Qwen3.6, and WilmerAI OpenCode workflows

Just a random note, but Qwen3.6 35b a3b is putting a smile on my face. This little model feels like a big upgrade over 3.5's 27b or 35b a3b. Also- the Wilmer workflow for OpenCode is really going well. I need to test it more, because

Wilmer Tool Calling

So some year and a half after the request was made for me to put tool calling into Wilmer, I've finally got it in there. First off- it was a huge pain to implement; if I didn't have Wilmer itself and agentic coders to help, I&

A Quick Note on Gemma 4 Image Settings in Llama.cpp

In my last post, I mentioned using --image-min-tokens to increase the quality of image responses from Qwen3.5. I went to load Gemma 4 the same way, and hit an error: [58175] srv process_chun: processing image... [58175] encoding image slice... [58175] image slice encoded in 7490 ms [58175] decoding

A Few Tips for OCR With Qwen3.5 through Llama.cpp

Just a couple of quick tips. I am using the Unsloth Qwen3.5 27b gguf, and also tried the 122b gguf. First: The difference between the bf16 and fp32 mmproj is night and day. I was getting multiple hallucinations, errors, etc with the bf16. I swapped to the fp32 mmproj

Wrangling Qwen's Overthinking with Workflows

So I've been running Qwen3.5 122b a10b lately on the M2 Ultra (currently GLM 5 is sitting on the M3), and if you've used any of the Qwen3.5 family, you've probably seen or heard about the overthinking issue. The models are great

A New Toy...

The M5 Max Macbook Pro just arrived. First thing I did was fling llama.cpp, Wilmer and Open WebUI on it. Honestly, the speeds are really impressive, even considering that llama.cpp hasn't fully integrated the hardware changes yet (at least, that's my understanding). Here'

Slimming Down the Homelab Software Footprint

So my homelab setup post from a while back is already outdated. Not as much on the hardware part; rather the software side has consolidated dramatically. The original setup had somewhere around 20 to 30 separate WilmerAI instances running across my network. Each one was configured for a specific purpose: