My professor told me to ignore the hype around small language models for a year, but the new Llama 3.1 8B just proved him wrong.

I spent 6 months building a chatbot for our small business in Phoenix using a huge model, but after switching to the smaller one last week, our costs dropped by 60% with no drop in quality, so has anyone else found a specific small model that worked way better than expected?

4 comments

4 Comments

kevin33121d ago

The real shocker for me was seeing how much faster the small models can learn new things. We had to teach our system a bunch of local shipping rules, and the 8B model we tested picked it up in a few tries where the bigger one just kept getting confused. It's like they're more focused on the actual task you give them instead of trying to show off all their knowledge at once.

richard_young8021d ago

What model did you end up going with?

jessicap8221d ago

Honestly, the speed of change in this stuff is wild. My buddy was just complaining about his cloud bill for a simple app, then a smaller model he tried cut it in half overnight. Makes you wonder what else we're overpaying for without realizing it.

quinnm7721d ago

Tell me about it, feels like we've been throwing money at the big names just because they're the default choice. Bet half the cloud providers are sweating right now hoping nobody runs the numbers on these smaller options. It's the tech version of finding out the store brand cereal tastes exactly the same for half the price. What's the next overpriced thing we'll figure out we don't need?