I haven't disappeared...

It's been a few weeks since I've posted or made any changes on Wilmer; I haven't stopped or lost interest, but rather I'm about to change jobs and I've been heads down on transition stuff before I leave my current position.

Part of being a dev manager is doing what you can to help your team be fine when you're gone. So to that end, I've basically done nothing but work for the last 4 weeks, and will continue that for the next 2. Nights. Weekends. Even pulled an all-nighter the other week. I do my normal work during the day, and then write documentation after hours. So far, I've written close to 150 pages of detailed docs, amounting to close to 40,000 words.

Amusingly, despite my hobby, 0 words of that documentation are AI generated. All hand written. I would have loved to generate some, but I don't have the same level of AI tooling available to me at work that I do in my homelab, so it's really not an option.

So yea... here soon I'll be back to doing normal socg stuff, but for right now I basically just wake up, work, sleep, and start again, so there's just no time.

See y'all in December!

Understanding MoE Offloading

I was trying to answer someone's question about how Llama.cpp handles offloading with Mixture of Experts models on a regular gaming PC with a 24GB GPU, and ended up spending a few hours in a deep dive.

My Next Steps with Wilmer

When I first started Wilmer, it was for a very specific reason: I wanted a semantic router, and one didn't yet exist. The routers that were available were all specifically designed to take the last message, categorize that, and route you that way. I needed more, though; what

Microsoft's New User Role Model

So this looks like it could actually be a really fun model https://huggingface.co/microsoft/UserLM-8b I like this little specific purpose LLMs the most because it opens up some neat doors. They likely made this to act as the user-proxy in autogen, and they point out on their

Reddit woes; but a light at the end of the tunnel

After 3 months, /u/reddit finally messaged me to tell me the account was permanently banned. However, the section that should contain the reason for the ban is empty. It just says Your account has been permanently banned for breaking the rules. This account has been permanently closed. To continue

Read more

Understanding MoE Offloading

My Next Steps with Wilmer

Microsoft's New User Role Model

Reddit woes; but a light at the end of the tunnel