Wise Crowds: The Merchant Advisory Panel That Actually Advises

The checkout customizer redesign had been stuck for three weeks not because we lacked opinions but because we had too many, all internal, all confident, and all slightly wrong in ways only a merchant running a real store could catch. So I did something different: I ran Wise Crowds with actual merchants in the room. Unlike most of the structures in this series, this one isn’t a mapping tool — there’s no board, no quadrant, no board full of sticky notes to photograph at the end. It’s structured peer consultation, and the entire artifact it produces is a changed decision in someone’s head.

What Wise Crowds actually is

Wise Crowds solves a narrower problem than the mapping structures: one person is stuck on a real, specific decision, and a room full of smart people is available but usually squanders that availability on either vague sympathy or unsolicited advice fired off before anyone understands the actual problem. The structure forces discipline onto both sides.

It runs in small groups of four to five — one client with a genuine, current challenge, the rest acting as consultants. Four steps, each tightly timed. Step one, one to two minutes: the client states the challenge and asks a specific question — not “what do you think of our checkout flow” but “should we let merchants disable the upsell drawer during checkout, or is that a decision we should make for them.” Step two, two to three minutes: consultants ask only clarifying questions. No advice, no “have you considered,” just questions that sharpen the group’s understanding of the actual problem. Step three, four to five minutes, is the move that makes the structure work: the client turns their chair around, back to the group, and says nothing. The consultants talk to each other about the challenge — generating advice, options, disagreements — as if the client had left the room. Step four, one to two minutes: the client turns back and says what landed, what they’ll actually act on.

Rounds run in roughly ten minutes each, so a room of twenty to thirty people can rotate through several clients in under an hour, often reshuffling which small group each client sits with between rounds. The turned chair does the real work here — it strips away the client’s instinct to defend, clarify, or subtly steer the group toward the answer they already wanted, and it frees the consultants to say the blunt thing they’d otherwise soften into uselessness.

Putting merchants in the consultant seats

The checkout customizer question — whether merchants should be able to fully disable the upsell drawer or only reposition it — had been argued internally for weeks with zero new information entering the room each time. Everyone on the team had a theory about merchant behavior. Nobody in the room actually ran a store on the theme. So for the next session we recruited five merchants from the theme catalog — a spread across store sizes and niches — and slotted them into consultant seats alongside two of our own designers, rotating across three small groups over ninety minutes.

Running Wise Crowds with external merchants instead of an internal team changes three things, and all three need deliberate handling before the session, not discovery during it.

The client role has to be genuinely ours, not a performance of open-mindedness. With an internal client, the group trusts that a real decision is on the table. With merchants in the room, there’s a real risk the “client” — usually a PM or designer — treats the session as a listening exercise they’ve already privately resolved, and merchants can smell that instantly. The fix is boring but necessary: the client states a question they don’t already know the answer to, and if the honest answer is “we’ve basically decided, we just want buy-in,” that’s not a Wise Crowds session — run something else, or be honest that you’re pressure-testing rather than deciding.

The silent-listening step needs active protection from becoming a pitch, in both directions. Merchants who’ve spent months wanting a feature will, the second the client’s back is turned, start selling rather than consulting — “you should just build X, here’s why it’ll work for everyone.” Facilitator’s job in that four-to-five-minute window is to nudge the group back toward the client’s actual question when it drifts into a wishlist session, without breaking the rule that the client stays silent. It happens fast and it’s easy to miss if you’re not actively listening for the drift.

Closing the loop afterward matters more with merchants than with an internal team. Internally, if the advice doesn’t get acted on, nobody outside the room notices. Merchants who spend ninety minutes giving structured advice and then see radio silence for two months conclude, correctly, that the “advisory” part of advisory panel was decorative. We sent a short follow-up to every participant a week later — what we heard, what we’re doing, what we’re not doing and why — even though nothing in Wise Crowds itself requires it. It’s the difference between a panel merchants want to sit on again and one they quietly stop responding to invitations for.

What came out of that session wasn’t consensus — one merchant group leaned toward full opt-out, another toward repositioning only — but a piece of information none of us had going in: merchants running higher-AOV stores wanted the drawer gone entirely because it read as pushy against their brand, while merchants running impulse-buy storefronts wanted it kept but wanted control over the copy. That’s not a finding a debate among five internal PMs was ever going to surface, because none of us had a brand to protect. We shipped a toggle with a copy-editable drawer rather than a binary on/off, which is a compromise, but it’s a compromise informed by two distinct merchant segments rather than two distinct internal opinions.

Failure modes

Using it when the question isn’t actually open. If the roadmap decision is already locked, Wise Crowds with merchants produces exactly the resentment you’d expect from anyone who gives real advice into a decision that was theater. Reserve it for questions you’ll genuinely let the room move.

Skipping the clarifying-question step because everyone’s in a hurry. It’s the shortest step and the one facilitators cut first under time pressure, but it’s what stops the consultants from solving the wrong problem confidently. A checkout drawer question that sounds simple often isn’t — merchants asking clarifying questions here routinely surface that “disable the drawer” actually means three different things to three different store types.

Running it with a group too homogeneous to generate friction. Five merchants who all run the same kind of store will converge fast and converge shallow. The value is in the disagreement between segments, so recruit for spread, not for whoever answered the email fastest.

Treating the chair-turn as optional or awkward and skipping it. It’s the single mechanism that makes this structure different from a regular feedback call, and cutting it collapses Wise Crowds back into an ordinary meeting where the client defends instead of listens — which is the exact dynamic the structure exists to break.

If you want the full set this structure sits inside, the Liberating Structures field guide has the rest.