The Best 7B LLM Models Tested with Oobabooga: Analysis and Insights
If you're passionate about local AI, you know how crucial it is to choose the right model for your setup. After testing several models available on Hugging Face using Oobabooga, I've compiled my impressions of some of the most popular 7-billion parameter (7B) Large Language Models. The goal is to evaluate how well these models can deliver immersive experiences, especially in RPG contexts and natural conversations.
What Does 7B Mean?
The term "7B" refers to the model's 7 billion parameters. More parameters generally mean a better ability to understand and respond to complex prompts. However, smaller models like 7B are optimized for hardware with limited resources, making them ideal for systems with GPUs offering just 8GB of VRAM or slightly more.
Tested Models and My Impressions
Below is a summary of the models I tested, highlighting their strengths and weaknesses. For my tests, I used a specific RPG character—a space pirate—to assess each model’s ability to remain consistent and engaging.
1. Dolphin-2.8-Mistral-7B
On paper, Dolphin-2.8-Mistral-7B seems promising, designed for natural conversations and programming tasks. However, in my RPG tests, it often came across as repetitive and failed to take the character's context into account. It frequently generated responses disconnected from the narrative or repeated similar phrases, compromising immersion. Despite these shortcomings, it can still be useful for simpler tasks like assisted writing.
2. Pygmalion-2-7B
Tailored for roleplay, Pygmalion-2-7B exhibited mixed performance. While it can produce engaging dialogues, it often derails, creating unwarranted storylines or out-of-context events. During my tests, the model sometimes initiated actions inconsistent with the character, making it difficult to maintain storyline coherence. This can be frustrating for users seeking a more structured experience.
3. Wizard-Vicuna-7B-Uncensored
This model is often praised by others, but I found it underwhelming. Its responses are often too brief, which makes RPG experiences frustrating, particularly in multiplayer AI chatbot scenarios. While it might have been a recommendation a few years ago, it no longer measures up in today’s context.
4. Yarn-Mistral-7b-128k
With a maximum context length of 128k tokens, Yarn-Mistral-7b-128k excels in extended dialogues and intricate storylines. In my tests, it delivered precise and consistent responses, showcasing a strong understanding of prompts. This versatile model adapts well to a variety of tasks.
5. OpenHermes-2.5-Mistral-7B
OpenHermes-2.5-Mistral-7B is a well-balanced model that provides high-quality responses in both narrative and technical contexts. In my RPG tests, it remained consistent with the character, making it a reliable choice for users seeking stability.
6. Starling-LM-7B-alpha
Starling-LM-7B-alpha is a solid choice for general conversations and roleplay. While it wasn’t the best in my RPG setup, it provided quality responses and can be effectively guided with the right prompts. Its reliability and simplicity make it a worthwhile option.
7. Toppy-M-7B
Toppy-M-7B leans more toward "spicy" roleplay scenarios and is not my go-to for immersive RPG experiences. However, it can deliver creative responses if used correctly.
8. Layris_9B-Q4_K_M-imat
Layris_9B-Q4_K_M-imat stood out as an excellent model for RPGs. Specifically designed for such usage, it provides engaging and consistent responses. Its filter-free nature makes it more suitable for unrestricted scenarios. Although a 9B model, the Q4_K_M version I tested runs perfectly on just 8GB of memory.
9. BuRP_7B-Q5_K_M-imat
BuRP_7B-Q5_K_M-imat is another model optimized for RPGs, capable of generating immersive, uncensored dialogues. This makes it a solid choice for those seeking unrestricted creative freedom.
10. InfinityRP-v1-7B-Q5_K_M-imat
InfinityRP-v1-7B-Q5_K_M-imat excels in RPG scenarios, delivering natural responses and a strong contextual understanding. Its lack of censorship makes it appealing for users seeking maximum creative flexibility.
11. una-thebeagle-7b-v1.Q5_K_M
This model stands out for creating realistic and coherent interactions in RPG contexts. Similar to others in this category, its lack of filters makes it less suitable for sensitive audiences.
12. Infinitely-Laydiculus-9b-Q4_K_M-imat
Infinitely-Laydiculus-9b-Q4_K_M-imat is one of the best models for RPGs. Despite being a 9B model, the Q4_K_M version used in my tests runs smoothly on just 8GB of memory. Designed specifically for this purpose, it delivers uncensored and immersive responses, making it an excellent choice for engaging interactions.
Important Note on Testing
It’s crucial to note that results may vary across different machines. These 7B models, while decent and respectable, are inherently limited compared to their larger counterparts. My tests were conducted with Q4 and Q5 quantized models, and results may differ depending on the chatbot platform and system configurations. Additionally, my tests focus on RPG scenarios, and these models may excel in other tasks. Therefore, consider these results indicative rather than definitive.
Final Thoughts
While some models, like Dolphin-2.8-Mistral-7B and Pygmalion-2-7B, struggled to meet expectations, others proved their worth, delivering an experience that felt like conversing with a real person. The key is experimenting with different models to find the one that best suits your interaction style.
My recommendations? Try OpenHermes-2.5-Mistral-7B, Yarn-Mistral-7b-128k, or one of the RPG-optimized models like Layris_9B-Q4_K_M-imat or InfinityRP-v1-7B-Q5_K_M-imat for a more immersive and engaging experience.
Feel free to share your thoughts or experiences in the comments!
Leave a Comment