Measuring Personality Through Conversation
Most personality tests ask you to describe yourself. “I see myself as someone who is curious about many different things.” Agree or disagree, on a scale of one to five.
The problem is that people are unreliable narrators of themselves. Not because they lie, but because self-report has two structural problems that no amount of careful question design can fully solve.
The problem with self-report
The first is social desirability bias. People answer questions not just about who they are, but about who they want to be — or who they think they should be. The aspiration bleeds into the assessment.
The second is context collapse. Personality isn’t stable across situations. How you engage with ideas at work differs from how you engage with ideas at a dinner table. Self-report averages across all contexts. The result is a generic portrait that misses the domain-specific patterns that actually matter for how you think.
What behavioral observation is
Behavioral observation is the alternative: instead of asking what you’re like, watch what you do.
Clinical psychologists have always known this. The richest personality data doesn’t come from questionnaires — it comes from structured observation. Watching how someone responds to challenge. What they reach for when they don’t know something. Whether they defend a position or revise it.
The challenge is that structured behavioral observation is expensive and hard to scale. You need a designed protocol and enough time to observe meaningful behavior. This is why self-report has dominated: it’s cheap, fast, and easy to score.
What personality looks like in conversation
When you observe someone in a structured debate, cognitive dimensions become visible in how they engage — not what they claim about themselves.
How someone responds to an opposing view is revealing. Do they treat it as a threat to neutralize, or an angle they hadn’t considered? Do they generate new framings mid-conversation, or stay within the frame they arrived with? Genuine revision — updating a position because a new angle was interesting, not because of pressure — is measurable.
How someone makes claims is equally telling. Do they cite evidence? Flag uncertainty? Track the logical consistency of what they’re saying across turns? Someone who holds themselves to a standard of rigor shows it — not by saying so, but by doing it.
Who initiates matters. Who introduces new threads? Who builds on what the other side said rather than waiting to respond? It’s less about talkativeness and more about directional ownership of the conversation.
How disagreement is handled is one of the clearest signals. Finding legitimate common ground while maintaining a distinct position, pushing back in a way that doesn’t foreclose the conversation — these behaviors are observable and meaningful.
Comfort with uncertainty is visible too. Can someone hold two contradictory possibilities without needing to collapse them? Or does discomfort push toward premature resolution?
Why domain-specific measurement matters
How you engage with climate policy differs from how you engage with finance. The same person may engage very differently across domains. Domain-specific measurement captures this — and that variance is informative, not noise.
It means you can configure an agent for a specific conversation, not just for a generic user profile. An agent calibrated for how you think about AI safety can be quite different from one calibrated for how you think about professional decisions.
What this makes possible for AI
The practical payoff is agents that actually know who they’re talking to — derived from behavioral evidence, not self-description. If you can reliably observe how someone thinks from conversation, you can configure agents to be genuinely complementary to how they think in that domain.
Not generic. Not average. Calibrated.
That’s the premise we’re testing.