SomeOddCodeGuy's Ramblings

Clawdbot...

Everyone and their brother is talking about Clawdbot, but as several others have pointed out- an agent with that many connections could be a security nightmare if it can be prompt injected. But since it supports OpenAI and Ollama endpoints... I wonder how well it would work if I stuck

M5 Max Macbook Pro Next Week?

Honestly been waiting for next week for a while. Even if the wait time is 2 months or longer on actually getting the order, having an M5 Max for the hardware matmul is going to be amazing and worth the wait. One nice thing about getting a new machine now

GLM 4.6 at UD_Q3_K_XL is surprisingly usable

So I currently run GLM 4.7 Q8 on my M3 Ultra, and after wrestling to find a solid model that would work well on the M2 Ultra 192GB, I finally decided to give the older GLM 4.6 UD_Q3_K_XL a try on it, seeing how much

Agents are Growing On Me

Go back in time a year and a half- it's mid-2024, LinkedIn has discovered AI and now the buzzword of the year is "agentic". Everyone and their brother was trying to convert every single task to be doable by an "agent" and, to be

The Resistance to Using Any AI...

When folks say "I can't find a use for AI", I think far too many of them are overthinking the use-cases, or expecting a much more grand difference in their lives. Without actually relying on LLMs to do the thinking for me, I can say without

Reddit: The Tale of Two Bans

So, my new account SomeOddCodeGuy_v2 just got permanently banned; not a shadowban this time, but a proper ban. So, a little backstory: My original reddit account, which had most of my benchmarks and whatnot on it, had gotten Shadowbanned after I posted a link to another reddit post, and

I haven't disappeared...

It's been a few weeks since I've posted or made any changes on Wilmer; I haven't stopped or lost interest, but rather I'm about to change jobs and I've been heads down on transition stuff before I leave my current

Understanding MoE Offloading

I was trying to answer someone's question about how Llama.cpp handles offloading with Mixture of Experts models on a regular gaming PC with a 24GB GPU, and ended up spending a few hours in a deep dive.

My Next Steps with Wilmer

When I first started Wilmer, it was for a very specific reason: I wanted a semantic router, and one didn't yet exist. The routers that were available were all specifically designed to take the last message, categorize that, and route you that way. I needed more, though; what

Microsoft's New User Role Model

So this looks like it could actually be a really fun model https://huggingface.co/microsoft/UserLM-8b I like this little specific purpose LLMs the most because it opens up some neat doors. They likely made this to act as the user-proxy in autogen, and they point out on their

Reddit woes; but a light at the end of the tunnel

After 3 months, /u/reddit finally messaged me to tell me the account was permanently banned. However, the section that should contain the reason for the ban is empty. It just says Your account has been permanently banned for breaking the rules. This account has been permanently closed. To continue

Agentic Coding Pt 2...

Every weekend for a while I've put out a release to Wilmer on Sunday; generally a few features I was able to knock out on Saturday and test on Sunday. Almost always using either some combination of local models with Wilmer via Open WebUI, or using Gemini 2.