This post is a combination of some new results, old results, and reddit.com/u/invectorgator's results (with permission) to help give a clear picture of all testing so far. Links to the relevant posts can be found below. This was a lot of fun, and has lit
WilmerAI
Cross-Posting from Reddit IMPORTANT: This is an early development, barely even Alpha, release. Wilmer is a passion project for myself, but it felt stingy not to share it given how interested everyone was in it, so I released early. It's still months from what I'd consider
So, in the past I've mentioned that I use AI to assist in writing code for my personal projects, especially for things I use to automate stuff for myself, and I've gotten pretty mixed responses. Some folks say they do the same, others say AI can
WilmerAI
Ever since I first saw the group chat feature in SillyTavern, I've always wanted to have a team of AI to help me work on things. But I never liked the result of using 1 LLM to do it; it never really felt like it was doing me
So I've been digging into agents a bit. Specifically, been looking into AutoGen and CrewAI. These are super cool. I'm not sure what I'd actually use them for, but between the two I think I like CrewAI the best between the two. Next I&
For the past few months I've been working on a quiet little project on the weekends, whenever I can scrounge up time, and part of that project involves looking for the best models of each domain. Of course, there are some great coding, medical, math, etc finetunes, but
So after seeing a lot of folks recommending SillyTavern as a good front end for APIs, I finally decided to give it a better try. I mostly have been using Oobabooga, and while I had ST installed from many months ago, I never put a lot of time into understanding
Benchmarks
Previous Post Here are some real-world speeds for the Mac M2 Introduction In my previous post, I showed the raw real-world numbers of what non-cached response times would look like for a Mac Studio M2 Ultra. My goal was to demonstrate how well the machine really handles models at full
One thing I see a lot of when folks are talking about how fast a machine/gpu/whatever is that they will answer the question with this vague "I get 12 tokens per second!", which honestly doesn't feel like it gives a clear answer by itself
Just wanted to toss this out there since I don't see a lot of folks talking about it, but Deepseek 67b Chat has become one of my favorite general purpose models. This is also the first model I've run into that has clearly beaten out, for
IMPORTANT: This is not an exhaustive guide by far. Consider this a quick start tutorial to get you going. For a lot of models, what I have below is all you need. But for some models it looks like there's some wrestling involved with settings, vocab thingies, etc.
I realized I still had gpt4all v2.4.14 on my machine, so I’ve been using it to load up some of the old ggmls that I had laying around, and also see how it does with some of the llama2 models. I’ll miss the ones I can’
