You’d be surprised how small you can go. That’s IMO pretty much the future of AI - a shit ton of small specialized models. While the heavyweights have their use, they’re way too expensive and overkill for specialized tasks.
Some small models can comfortably run on the CPU as well, games can easily detect whether you have VRAM to spare and use GPU or CPU based on that.
It’s not there, yet, but what some of the small models can do is impressive. And if you train them extensively on fantasy scripts, I can see them generating NPC lines on the fly.
You’d be surprised how small you can go. That’s IMO pretty much the future of AI - a shit ton of small specialized models. While the heavyweights have their use, they’re way too expensive and overkill for specialized tasks.
Some small models can comfortably run on the CPU as well, games can easily detect whether you have VRAM to spare and use GPU or CPU based on that.
It’s not there, yet, but what some of the small models can do is impressive. And if you train them extensively on fantasy scripts, I can see them generating NPC lines on the fly.