How do you train AI agents in language and understanding? Easy – drop them into a Zork-alike dungeon and let then find their own way out.
While the rest of the Microsoft world squeaked with excitement about new hardware and free collaboration tools, the boffins at Redmond’s Montreal Research Lab have looked to history as way to help Reinforcement Learning (RL) researchers skill up their AI agents.
The theory is quite simple – drop an AI agent into a turn-based, text-based game like Zork and see how it performs.
AI agents have been making strides in the world of text adventures in recent years, taking a break from the arduous task of beating crusty Atari 2600 games. There’s even a competition due to run at this year’s IEEE Conference on Computational Intelligence and Games (CIG), although you’d best get your skates on if you want to send your AI into battle because the deadline is a few short days away on 20 July.
Earlier this month, the team in Montreal revealed they created a tool called TextWorld to generate fresh game designs and an engine to allow players to interact with the thing. No visuals here, though – it's all through a text-based console familiar to anyone who remembers the excitement of Zork firing up for the first time on DEC PDP-10 or IBM PC.
TextWorld itself takes a knowledgebase of actions, themes and predicates, the number of rooms and objects, and a winning condition, and churns out a game using the Inform 7. The team acknowledges that it has limitations – quests have to be pretty simple, with each action dependent on the previous one, and no support for time-based events (such as doors automatically locking.) TextWorld also lacks Non-Player Characters and only one agent can play at a time, so Multi-User Dungeons (MUD) are right out.
Requiring Python 3 and a handy Linux distribution, TextWorld is relatively simple to get running. Your humble Reg hack, blessed only with Windows 10, coaxed it into life thanks to Microsoft’s Windows Subsystem for Linux, the version of Debian found in the Microsoft store, and a not inconsiderable amount of swearing.
Suffice to say that the instructions provided with TextWorld are somewhat… incomplete. However, with a few
apt-gets and the odd
sudo, and a near total disregard of the syntax suggested in the
readme, it is possible to generate a totally new text adventure and leap into it.
With luck, the AI agents will prove better at solving puzzles than this human.
Geoff Gordon, principal research manager at Microsoft Research Montreal, is excited about his new toy, combining as it does RL with natural language (unlike simpler challenges, such as beating Pac Man) and said: "These two technologies are both really important, but they don’t fit together that well yet. TextWorld will push researchers to make them work in combination."
Having spent more time than was strictly necessary playing with the output of TextWorld, The Register would humbly suggest that it seems unfair that AIs should have all the fun. In a world obsessed with frame rates and resolution, there is still a place for staring blankly at a screenful of densely written text.
The Reg, for one, looks forward to the moment when our future robotic overlords are able to crack Magnetic Scrolls' Jinxter. ®