A little library for my LLMs

I have been using LLMs for programming for the last year. Some things worked, others not, most are improving and in general I am happy with the results. I did things in the last year that I was not able to even start three years ago.

The tools work well out of the box. ChatGPT, Claude, Qwen, Gemini, they do the trick. Once I know what I want to do, and have an idea of how to do it, then it is a matter of time and hammering until my project is on the right shape, and I feel a lot more productive than some years ago, when I needed a full team around to do even the most trivial program.

However, my budget to mess around is limited, the API prices are all over the place, and the subscriptions change daily. That being the case, I was forced to be tool agnostic. I get results on any of them, and they are on the same level for the kind of things I do at the moment, then I use one tool until it run out of credit, then I move to the next one, and hopefully the limits on the first would be reset when I completed the cycle.

Maybe I could unify the subscriptions in a single provider, and get broader limits. I know a lot of people are doing that with good results. But there is something else. Each tool setup their own structure around. One create a memory.md. The other put an agent.md. The next do something else. Each companies change them all the time. Even if I investigate what are they storing and how, it will probably be different a week later.

Some time ago I saw the post from Karpathy about setting up a knowledge base for the LLM, and that made sense for me. I am using Ralph when I think it will work (with varying results, given that I am not able to do a 500 euros a day deployment for just messing around), and the controlled approach seemed easier to control. Trying to get some persistent memory around the things I do (and I do a couple of different things) looks like a good idea.

Brain-kit is what I implemented and use right now. In the next post I will go into how I am using it today.