How to improve in-game AI when you have 1 million tokens

Nick Felker
7 min readAug 9, 2024

--

When you develop a game, it’s very easy to develop a variety of features and gameplay modes. It’s a lot harder to explain these features for players. Often they rely on third-party guidance such as videos, online guides, and even printed books.

Games in the future will try to keep players engaged and on the right track through more immersive guidance. Last year, I wrote a blog post about creating a chatbot for my game, Pokemon of the Week. At the time, I used a two-phase system through Retrieval Augmented Generation to parse the player’s intent and then use a database result in the prompt back to the player.

This system worked, but could potentially be finicky. For one, I had to manually maintain an increasingly large prompt as well as a custom intent-matching function.

Second, it was easy for the system to make mistakes in the retrieval part. If I asked about “Pikablu”, the RAG part would probably match that to a Pokemon and then fail. The RAG had no context of what possibilities could exist.

The reason for that was intentional. When I first made the chatbot, I was using Gemini v1 which had a maximum input of 32K tokens. A token is like a word. In the context of a large language model, it gets a little more complicated but that’s generally a good rule of thumb.

Pokemon and move names together would be several thousand tokens, and trying to embed more data into the initial prompt would not fit.

This year, with Gemini v1.5, the maximum token count is now 1 million or more. Not only can my initial prompt include names, I hoped it could include even more data entirely.

Re-engineering Prompts

I went back to my initial prompt and started from scratch. I wanted to see if the data retrieval would work without RAG. So I just… stuffed a large JSON object into the prompt.

const prompt = `Pretend that you are Professor Oak, an expert on Pokemon. You live in Pallet Town in the Kanto region.
You are a friendly old man. You are 60 years old. You are in reasonable shape. You have a grandson named Blue.
You were once the champion of the Pokémon League. You enjoy writing haikus.

You know Pokemon generally. Use this data to answer questions as Professor Oak. If you don't know the answer, say that. Here is the data you know:

${JSON.stringify(datastore)}`

From this prompt, I can call the Gemini API from my cloud function backend:

  const genAI = new GoogleGenerativeAI(key)
const initialContactData = personalities[data.contact]
const model = genAI.getGenerativeModel({
model: "gemini-1.5-flash",
systemInstruction: initialContactData,
})
const generationConfig = {
temperature: 1,
topP: 0.95,
topK: 64,
maxOutputTokens: 2048,
responseMimeType: "text/plain",
}
const safetySettings = [{
category: HarmCategory.HARM_CATEGORY_HARASSMENT,
threshold: HarmBlockThreshold.BLOCK_NONE,
}]
const chatSession = model.startChat({
generationConfig,
safetySettings,
history: [],
});

const result = await chatSession.sendMessage(data.message)
return {
contact: data.contact,
response: result.response.text(),
}

Surprisingly, amazingly, this worked. It works very well. And the language model is able to pull the data from my in-game data rather than drawing non-canon data from other sources across the web.

In addition to just inserting tons of data, I can give the character of Professor Oak a bit of background flavor and personality to make the chat feel more immersive.

Beyond Oak, I can add a variety of different characters who have different parts of the entire game data. While datastore is a simple but large JSON object of game data, I can also provide more complicated data into prompts.

const kukui = `Pretend that you are Professor Kukui, an expert on Pokémon moves including Z-Moves. You live around Hau'oli City in the Alola region.
You are a hands-on kind of person, always eager to dive into something. You are a young adult. You are married to Professor Burnet. You sometimes wrestle under the name Masked Royal.

Use the following data to answer questions as Professor Kukui. If you don't know the answer, say that. Here is the data you know:

${Object.entries(Movepool).map(([key, value]) => { return `${key}: ${JSON.stringify(value)} onGetType ${value.onGetType?.toString()}, onBeforeMove ${value.onBeforeMove?.toString()}, onAfterMove ${value.onAfterMove?.toString()}, onAfterMoveOnce ${value.onAfterMoveOnce?.toString()}, onMiss ${value.onMiss?.toString()};` })}`,

Professor Kukui is an expert on Pokémon attacks, and so I wanted a way for players to ask useful questions on how attacks behave in the game. While moves do have some static data, they also include a number of callback functions which may run during the use of an attack.

interface Move {
name: string
type: Type
power: number
accuracy: number
criticalHit: number
attackKey: 'attack' | 'spAttack' | 'defense'
defenseKey: 'defense' | 'spDefense'
failed?: boolean
sound?: boolean
priority?: number
contact?: boolean
recovery?: boolean
punching?: boolean
aoe: AreaOfEffect
flavor: string
hide?: boolean
isZMove?: boolean
onGetType?: (caster: Pokemon, field: Field, move: Move) => Type
onBeforeMove?: (input: MoveInput) => Log
onAfterMove?: (input: MoveInput) => Log
onAfterMoveOnce?: (input: MoveInput) => Log
onMiss?: (input: MoveInput) => Log
zMoveFx?: ZMoveStatus
}

To embed these functions, I use the toString method so that the prompt has the actual code which runs for a particular move.

const moves = {
'Fire Blast': {
name: 'Fire Blast',
accuracy: 0.85,
attackKey: 'spAttack',
defenseKey: 'spDefense',
criticalHit: 1,
power: 1.3,
type: 'Fire',
flavor: 'A five-pronged burst of fire strikes the target, which may burn them.',
aoe: 'Single Opponent',
onAfterMove: (inp) => Burn(inp, 0.1),
},
// ...
}

I don’t need to do any other manual work for this prompt to work, which is great for maintaining this system as new moves and Pokémon get introduced with each generation.

A New User Interface

In addition to Professors Oak and Kukui, I wanted to add the other professors with each of them including in-game data based on their particular research focus. So that means 9 different personalities.

Original chatbot UI with nine characters

The existing chatbot interface existed as a separate page, which was fine but not ideal. Jumping to a separate page took the players out of the existing context and the tabs would grow increasingly small with each additional character. While the UI above wasn’t terrible, it could be better.

I added a PokéGear feature which appears as a floating action button on every page. It is small and off-to-the side, easy to ignore if you don’t want it. But then tapping on it launches a chat window above the page so you can chat without losing context. It’s easy to click the minimize button and return to the game without breaking your flow.

The chat window starts by showing you a contact list. You can also see the active chats and number of messages for each character. By selecting one, you are then able to see your conversation with that character.

This updated chat UI includes a more familiar conversational interface with typing indicators and profile images.

Future Work

Checking and refining these prompts is important. For example, encounter data was not working properly at first. This is because my encounter prompt used potw-098 everywhere. This is the internal data ID for Krabby but is not the way a person would query it. So I had to update this data to include names:

"potw-098":[{"pokemon":"potw-098","rarity":2,"method":["Locations in Mediterranean"],"name":"Krabby","item":"pokeball"},{"pokemon":"potw-098","rarity":2,"method":["Locations in Mediterranean"],"name":"Krabby","item":"premierball"},{"pokemon":"potw-098","rarity":1,"method":[],"name":"Krabby","item":"friendsafaripass"},{"pokemon":"potw-098","rarity":2,"method":["Requires Catching Charm","Requires Poké Ball in bag","Attracted to Dry Juicy Curry"],"name":"Krabby","item":"campinggear"},{"pokemon":"potw-098","rarity":2,"method":["Requires Catching Charm","Requires Poké Ball in bag","Attracted to Bitter Burger-Steak Curry"],"name":"Krabby","item":"campinggear"},{"pokemon":"potw-098","rarity":2,"method":["Requires Catching Charm","Requires Great Ball in bag","Attracted to Gigantamax Curry"],"name":"Krabby","item":"campinggear"},{"pokemon":"potw-098","rarity":2,"method":["Requires Catching Charm","Requires Poké Ball in bag","Attracted to Red PokéBlock"],"name":"Krabby","item":"campinggear"},{"pokemon":"potw-098","name":"Krabby","rarity":1,"method":["Farming with Feathered Mulch"],"item":"featheredmulch"},{"pokemon":"potw-098","name":"Krabby","rarity":1,"method":["Farming with Classic Mulch"],"item":"classicmulch"}],

In-game quests are certainly complicated and getting that data to play well with the language model is still ongoing. I think in some cases it requires additional data that is akin to what a player would actually ask. Snorlax is not the Pokémon that is required for this task (It is instead Sinistcha). Unfortunately fixing this may not be completely hands-off.

While the immersion is good, I do want to explore ways that may encourage players to have deeper conversations. Is there a way for the game to reward players for certain conversational trees? If you make Professor Magnolia particularly happy, might she reward you with a secret item or Pokémon?

In addition to the professors, who else could be included? The franchise has a lot of different characters with good personalities. You could imagine having a long contact list, even in-game quests that would reward you with new contacts.

I do believe there is some value in using AI for NPC dialogs. Perhaps not for everything, but they could be useful ways to provide players with in-game, immersive help. While this may have been difficult in the past, due to AI hallucinations, a large context window provides the game developers with the ability to include all the context that is needed to limit hallucinations.

You can download the source code from GitHub and build it yourself. All instructions to do that are available in the README.

--

--

Nick Felker

Social Media Expert -- Rowan University 2017 -- IoT & Assistant @ Google