25/10 Crowd Sourcing: Ranking a Room Full of Instructor Feature Requests in Fifteen Minutes

The instructor advisory call had become a predictable ritual: whoever spoke first and most persuasively about their feature request tended to walk away feeling heard, and everyone else’s requests got a polite “noted” that rarely turned into action. Not because the product team was playing favorites — it’s just what happens when prioritization gets decided by whoever’s argument the room heard most recently and most vividly. 25/10 Crowd Sourcing exists specifically to break that dynamic, and it did, in about fifteen minutes, with a room of twenty course instructors who’d never met a workshop structure that let all of them be heard equally before.

What it is

25/10 Crowd Sourcing (sometimes called 25/25, or 25/10) is a rapid, physically active structure for surfacing and ranking a large number of ideas or proposals using the crowd’s own judgment rather than a facilitator’s synthesis or a vocal minority’s persuasion. The name refers to the format’s two numeric anchors: each idea gets rated on a 1-to-5 scale, and after several rounds of pairwise comparison, the top ideas — often the top 10% of whatever was submitted — rise to the surface, ranked by the aggregate judgment of everyone in the room rather than by whoever pitched loudest.

How it runs

1. Silent individual writing (5 minutes). Pose a question — here, “what’s one feature or change you’d want in the platform for your courses?” Each participant writes their single best idea on an index card, one idea per card, no names attached.
2. Milling and pairing (1 minute per round, several rounds). Everyone stands and mills around the room. On a signal, each person pairs with the nearest other person. Each shares their card aloud, and each rates the other person’s idea on a 1-to-5 scale, writing the score on the card. Then they swap: each person now holds someone else’s idea, not their own.
3. Repeat for four or five rounds. Each round, mill, pair with someone new, share the idea you’re currently holding — which by round three or four is very likely not the idea you originally wrote — and get it rated again. The scores accumulate on the card as it travels, or participants keep a running tally.
4. Rank and share. After the final round, ideas are ranked by cumulative or average score. The top-scoring handful get read aloud to the whole room, and often a quick discussion or a further vote narrows further from there.

The mechanic that makes this work is subtle but crucial: because ideas circulate away from their original author, nobody is rating their own idea, and nobody knows whose idea they’re currently holding when they rate it. The whole room becomes an anonymous, distributed judging panel, and an idea’s final ranking reflects genuine, repeated, blind peer assessment rather than the charisma of whoever pitched it in a meeting.

Applying it: instructor feature requests, ranked without a popularity contest

Course instructors — the people authoring and running courses inside the LMS, distinct from the learners taking them — had been submitting feature requests through the advisory call for months, and the product team’s honest problem was that the requests from the two or three instructors who showed up to every call and spoke confidently had gotten disproportionate weight, not through any deliberate favoritism but simply because their voices were the ones the team heard most often and most vividly.

Twenty instructors joined a session run specifically to re-prioritize the request backlog, and the prompt for the silent-writing round was: “what’s one thing about authoring or running your course you wish worked differently?” Twenty cards, twenty different requests, submitted with no names and no chance for anyone to pitch or defend theirs before the rating rounds began.

What emerged after four rounds of milling and rating surprised the product team in a specific, useful way: the highest cumulative score didn’t go to any of the requests the vocal regulars had been pushing for months. It went to a request, submitted by an instructor who’d never spoken up on a call before, for the ability to duplicate an existing course as a starting template for a new one — a request that turned out to resonate broadly the moment it circulated anonymously, because nearly every instructor who rated it recognized the exact same pain (rebuilding similar course structures from scratch every time) even though almost none of them had ever mentioned it out loud in a group call, possibly because it felt too basic or too obvious to be worth a vocal advisory-call slot.

The two requests the regular vocal contingent had been pushing — more granular quiz analytics, and a richer certificate-design tool — did rank, but in the middle of the pack rather than at the top, once subjected to the same anonymous, repeated peer rating as every other card. That’s not a judgment that those requests were bad ideas; it’s a correction of a measurement bias that had nothing to do with idea quality and everything to do with who felt comfortable speaking up in a synchronous call.

Why anonymity and repetition both matter

Either half of this mechanism alone would be weaker. Anonymity alone — a single anonymous survey asking instructors to rate a pre-built list of feature ideas — still has the product team deciding which ideas made the list in the first place, reintroducing exactly the bias the exercise is meant to remove, just one step upstream. Repetition alone — without anonymity, if people rated ideas knowing whose they were — would just reproduce social dynamics through a different mechanism, since people rate a well-liked colleague’s idea more generously regardless of its actual merit. Combining both — the ideas themselves generated bottom-up by everyone in the room, and rated anonymously as they circulate — is what let the duplicate-course-template request rise on its own merit, discovered by the crowd rather than pre-selected by the facilitator or defended by its author.

The physical milling-and-pairing mechanic also does something a purely digital anonymous survey wouldn’t: it forces every idea through several independent judges in quick succession, so a single generous or harsh rater has limited influence on any one card’s final score — the equivalent of averaging across several independent samples rather than relying on one. A digital poll with everyone rating every idea once would get you anonymity but not this specific noise-cancellation property, and would also take considerably longer than fifteen minutes for twenty ideas from twenty people.

Failure modes and when to skip it

25/10 needs a genuinely large number of independently-generated ideas to be worth the physical choreography — with fewer than about fifteen participants, the milling rounds start to run out of new pairings quickly, and with fewer than five or six distinct ideas, a simpler show-of-hands vote gets you nearly the same ranking with far less setup. Save this structure for when you actually have a crowd and a real spread of proposals, not a small group with three obvious frontrunners.

It’s also weaker for ideas that require real technical or strategic judgment to evaluate, rather than lived experience to recognize — a room of instructors is well-positioned to judge “would a course-duplication feature help me,” because they live that pain directly; the same room is poorly positioned to judge “is this database migration approach sound,” because that’s not a domain where crowd intuition tracks quality. Use 25/10 for prioritizing ideas the room has direct standing to judge, and route ideas requiring specialized expertise through a different filter before crowd-ranking them, if at all.

Finally, don’t skip the discussion step after the ranking. The exercise produces a ranked list, but ranking isn’t the same as a decision — the top-ranked idea still needs someone with authority and context to sanity-check feasibility and cost before it becomes a commitment, and treating the crowd’s ranking as an automatic mandate skips a step that still needs a human product judgment applied on top of it.