Do you think it is likely that we will start to see Large Language Models integrated in to major video games? If so, are there some examples within gaming already?
There are a couple of mystery games that are available in early access/beta that already use LLM AI to generate dialogue.
I’ve messed with 1 and it’s alright. The problem with it is mostly in how it doesn’t necessarily regurgitate the info it knows into something coherent and logical to actually solve the mystery. That or they never ever tell you the truth. Since the only way to solve the mystery is by talking to NPCs to get enough info to piece the puzzle out, it often leads to unsolveable mysteries.
Now, if that problem with LLMs could be fixed, so it isn’t conflagrating multiple pieces of data into something new and novel, I think it would be awesome for making game worlds that are more alive and natural. Like instead of walking up to an NPC and clicking “rumors” you can actually just talk to them like a real person and they would respond in kind, as if the NPC was actually a person living in that world. It’s really the only thing I think would actually work well with generative AI chat bots, but the generative AI still has too many problems to make it truly viable.
It actually works better with trying to play D&D, because even IRL, players often misinterpret rules and argue. Which you genuinely have when trying to play with 3 AI characters.
We won’t see large language models. We will likely see a stripped down version like a small language model (or Domain Specific models if you want the fancy marketing wank term) because a NPC in a fantasy game doesn’t need to know about 13th century Europe or 19th century Asia.
Yes, LLMs are too costly for this and require a cloud service, smaller models could run on the client. The main difficulty is getting the training data and preparing it for machine learning.
I think using LLMs to provide the dialog for NPCs in a RPG is a use case that’s just begging to happen. Ie townsfolk that don’t just give the same few replies every time, and who react to things you’ve done in the past beyond just whatever prewritten options the developer thought of.
They did this in Free City, and one of the bank tellers became indistinguishable from a player.
For those that haven’t seen it, this produces hilarious results.
That is…actually far better than I thought it would be. It’s clearly not ready yet, but I could see the potential.
The AI model is too happy to serve the whims of the player, but if there was a better model that could actually be hooked in to me hanics like personality scores or reputation, I could see that as an interesting gameplay system. It also needs more checks on what they are and aren’t supposed to know (e.g. why would a Skyrim NPC associate the name Batman with heroism, or why would they know who Gandalf is?).
A (digital) setup like Westworld is probably in the cards someday. Hopefully with more checks in place to keep the AI from rising up though!
Even with the annoying YouTuber trying to make silly content out of it I’d say it actually kept into the roleplay pretty well
Wow, even with its flaws, that went way smoother than I expected
I feel like if an NPC doesn’t have something meaningful to say, then they may as well not exist. Otherwise they just serve to waste the players time.
“How dare there be an active world in my murder hobo wankfest?”
It is called “flavour”
A good fit would be random background NPCs. For example, pedestrians in a GTA like game. Can potentially increase the variety in the things they can say, and maybe even talk about things the player has just done.
Not anytime soon. Nvidia tried, and nobody liked it. LLMs still suck at creative writing and need a ton of RAM/VRAM just to work. They also often get confused or trail off in any discussion/roleplay.
The only game that sort of made it work was Suck Up!, where you’re a vampire that has to convince an AI to let you in their house so you can suck their blood. It’s a fun concept but even that game gets repetitive quick and the LLM is very stupid and random.
They don’t need a ton of ram if you use a tiny LLM customized for the game’s use cases, and that’s what games would be doing.
NVIDIA not just tried, but still doing it, and apparently soon you’ll play with these NVIDIA ACE NPCs in PUBG and a few other games.
I don’t know what sounds more robotic, the AI or the script read for the player.
deleted by creator
I think the PC vs. console divide is relevant here. I’m not sure how advanced text entry on consoles is these days, but I imagine PCs have the advantage with keyboards. Maybe if they use voice recognition on the consoles? But AAA games usually target both, and if interacting with the model is clunky for a big chunk of your market then the big developers might not use the technology.
Of course, indie devs that only target PC can go wild.Not sure about Switch, but PS5/Xbox support mouse and keyboard inputs if a game is designed for them.
Voice recognition seems more likely to me, though. PS5 already has an advantage there because every Dualsense controller has a mic in it.
Whisper from OpenAI is pretty solid for speech recognition (at least English), and it is small enough to deploy on mobile devices. If I recall correctly, both PS and Xbox controllers have mics built-in, so input device is covered.
