What If AI Had to Learn a Game the Same Way You Do?

Most AI experiments in games give the model an API.

move()

attack()

find_target()

buy_item()

That works.

But I'm interested in a different question:

What happens if the AI doesn't get a special API at all?

I'm working on a future module for CHAOS, a persistent browser world I build as a non-commercial side project.

The idea is to let an AI model become an actual player.

Not an NPC.

Not a narrator.

Not an admin bot.

A player with its own account, money, files, applications, relationships, territory and history.

But there's one rule:

AI should never know or do anything that an equivalent human player couldn't.

So instead of giving the model access to the database or a list of every possible action, I'm thinking about giving it a second client for the same world.

The human sees pixels.

The AI sees semantic surfaces.

If a human opens the Map, the AI can open the Map.

If a human has to type help to discover a Terminal command, the AI has to discover it too.

If a file hasn't been opened, its content isn't magically part of the AI's knowledge.

If the player doesn't have access to an application, its capabilities shouldn't magically exist for the AI either.

The goal isn't to describe the entire game in one huge prompt.

It's closer to building a semantic operating system for an AI player: current window, focus, notifications, visible information, and only the interactions available at that exact moment.

And that's where I hit the question I find most interesting:

Where does a fair semantic interface end and a privileged AI API begin?

Would you make the model operate the real GUI through vision and mouse control?

Or would you build a semantic client — but force it to obey exactly the same information boundaries as a human?

This isn't implemented yet.

That's why I'm asking now.

If your first thought is "I'd design this differently", I'd genuinely like to hear how.

// Amn Tree

chaos.re

github.com/amnezja3/chaos

Build something.

Break something.

Question everything.

You are welcome.