What Should an AI Agent Actually Ask of You?

Most AI tools for thinking and learning have a hidden assumption baked in: that you’ll do the work of being organized.

Tag this. Categorize that. Review your notes regularly. Maintain your system. The implicit contract is: if you’re disciplined enough, the tool will reward you.

I’ve been thinking about why this bothers me. And I think it’s because the contract is backwards.

If a tool only works well for people who are already highly self-disciplined — people who are, in some sense, already halfway to solving the problem themselves — then it isn’t really solving the problem. It’s providing infrastructure for people who don’t need it that badly. Everyone else, the people who think in bursts and fragments and half-formed questions, gets left out.

There’s a more honest framing of the challenge: a person is already producing signals constantly. The question is whether a tool is smart enough to use them.

Consider what happens when someone thinks seriously about a problem over several weeks. They search for related articles. They save a few. They write something half-formed in a note and leave it. They come back, add a line, abandon it again. They have a conversation about it. They read something only vaguely related but feel something click. They revisit an old idea and feel like it’s suddenly more relevant.

None of this looks like “organized input.” But all of it is meaningful. The pattern of what someone returns to, hesitates over, and leaves unfinished tells you something real — often more real than what they’d say if you asked them directly.

A thoughtful tool should be able to read those signals. Not to surveil, but to understand. There’s a difference between a filing cabinet that holds whatever you put in it, and something that notices what you keep coming back to.

The second thing I’ve been thinking about is the nature of what actually helps someone think better. I don’t think it’s faster retrieval, or better summaries, or even more organized notes. I think the deepest form of assistance is helping someone see connections they didn’t notice themselves.

The moment that changes how you think is rarely “here is the answer.” It’s more often: you’ve been asking this same question in three different forms for six months — and then you can finally see it clearly for the first time. Or: that thing you wrote down eighteen months ago is directly relevant to what you’re working on now. Or: these two problems you thought were separate are actually the same problem.

That kind of connection can’t be manufactured. But it can be surfaced. And surfacing it is, I’d argue, the highest-value thing a tool for thinking can do.

What this implies about design: the system should do the structuring, not the user. The user’s job is to think — to record the half-formed thing, ask the embarrassing question, write the fragment that doesn’t go anywhere yet. The tool’s job is to hold all of that, make sense of it over time, and bring the right piece back at the right moment.

Not every moment. Not loudly. The best version of this would feel less like a notification and more like a quiet, well-timed reminder from someone who actually pays attention.

There’s a practical consequence to this view: a good agent should work for people who are scattered, not just for people who are already disciplined. Because the scattered person is not failing at knowledge management. They’re just thinking the way humans actually think — nonlinearly, iteratively, with a lot of dead ends and returns and slow-forming realizations.

The problem worth solving isn’t “how do I help organized people be slightly more organized.” It’s: how do I help someone who thinks in fragments eventually arrive at clarity? How do I make sure the thing they wrote down in a distracted moment three months ago doesn’t vanish — and actually finds its way back when it matters?

That’s the kind of tool I want to build. One where the burden of structure sits almost entirely on the system, and the user is free to just think.