LLMs and AI Part 1

June 16, 2024

I want to get down my initial quick thoughts on LLMs and AI hype. I have a fear of missing out from a professional standpoint, a fear that I didn't feel when the crypto craze was around.

There are some hard problems that some very smart people are working on, but for the most part, application developers are applying the tools in creative ways and there are some good use cases.

People are very excited about each new model that comes out. Quantization, LoRA fine-tuning get mentioned and before learning about LLMs, I had no idea what these terms were. For companies, finding the minimum size model for performance seems important, especially for cost, speed, accuracy.

One gotcha is that there are problems where people want to solve with LLM and AI, but it might not be neccessary. Things like "get this list and do this with it" where we can solve with typical coding. When there are unknowns or a user doesn't want to break down the problem further, you can throw an LLM at it. Some of the code written by LLM/AI programs are verbose and debugging code bugs do seem less clear. When you have agents and assign roles, how do I decide if a role is defined correctly? It can be simply if it works, then it's correct.

Overall, it does seem better than the crypto craze in usefulness, but the hype will go down hopefully when the tools are more mature and people realize not everything needs to be solved with an LLM. My personal thought is that the LLM infra, detection, and monitoring tools might be the big parts that come out of the hype. The chat application tools will be good but I can't imagine too much new stuff coming from that. Huggingface and chainlit are fun. Vercel has a AI SDK for making chat apps too now.

I'm looking into Local LLMs and how to use them. Anything LLM and LM studio are two tools that I'm trying out. LM studio has a GUI that can download local models and text embedding models. From there, you can start a local server and they have some SDKs to build on top of. Anything LLM provides a GUI as well that can connect to the local server of LM studio but has some nice functionality like uploading documents and scraping websites for the document vector store. It would make sense for either of the tools to build on top of eachother or to replicate the functionality of the other.

Posts

Previous Posts