The Value Function Is the Hard Part

Last month I won a silver medal in a Kaggle competition called Orbit Wars — small agents playing a game, ranked against each other. I'm glad I did it. But the thing I keep turning over afterward isn't the ranking. It's what Kaggle quietly takes out of the problem before you ever start: the question.

A Kaggle competition hands you the value function. There is one number, defined in advance, and your entire job is to make it go up. Be the best at this one scalar. Everything genuinely hard about the real world — deciding what is worth optimizing at all — has already been done for you. What's left is optimization, which is difficult, but difficult in a bounded, well-lit way.

And even inside that lit room, the questions did more work than the code.

The model would write agent code all day. Beam search, a hand-rolled world model, ML-hybrid scoring — I could ask for another variant and have it in a minute. What actually moved us wasn't any of those. It was the times I stopped asking for code. I spent more time asking the model to explain the strongest public agents than to write anything. I ran a local arena to rank them myself, and found that the "top" agents I'd been copying ideas from were only top in the open-sourced arena — the leaderboard I'd been trusting was measuring something narrower than I'd assumed. I deleted a pile of ELO numbers we'd been leaning on because they were stale, and leaning on a stale number is worse than having none.

The model could write agent code all day. It could not tell me which agent was worth writing.

The real turn in the whole competition wasn't a line of code either. It was a sentence I typed one evening: we've hit a dead end, this approach won't get us there, let's think out of the box — and then spun up a stronger model for no reason other than to reconsider the strategy. The answers had been cheap the entire time. Noticing that we were optimizing the wrong thing was the expensive part.

So Kaggle is the easy world. Not easy to win — easy in the sense that the question is settled before you arrive. Outside it, nobody hands you the scalar. You never quite know what you're optimizing against; the value function is implicit, contested, and moves while you look at it. Whoever can define it — say plainly what "better" means, here, for this — is already halfway to the answer. Often that half is the whole job.

Which is why the new agents make this more urgent, not less. When a competent answer costs almost nothing, all the remaining value collects on the side of the question. The scarce skill stops being "can you produce a solution" and becomes "can you state what you actually want." Ask a sharp question and the machine will race to a good answer. Ask a vague one and it will race just as fast to a useless one, only now it looks polished.

Deep Thought ran for seven and a half million years and returned 42, and only then did anyone notice that no one had understood the question. That used to read as a joke about slow computers. It reads differently next to fast ones.