Harvard Business School’s Iavor Bojinov and Edward McFowland III set out to test the workplace vibe we’ve all felt lately: “AI makes you feel like you can do anything. But can you do [a task] as well as people whose job it is?” In an experiment with 78 employees at IG Group, they looked at whether generative AI could help non–web analysts produce investing articles comparable to the specialists who write them for a living. The punchline: AI expands what people *can attempt*, but it doesn’t magically erase the value of domain experience.
The study split participants into three groups—12 web analysts (insiders), 26 marketing specialists (adjacent outsiders), and 40 technology specialists like software developers and data scientists (distant outsiders)—all using the same AI model. When it came to the final writing, the tech specialists hit the “GenAI wall”: their articles averaged 3.42/5 versus 3.96 for web analysts and 3.92 for marketing, a 0.5-point gap (about 13%). The researchers attribute the difference to “knowledge distance”—marketing’s closer overlap with content work made it easier to use AI output well, while those farther from the domain struggled to judge, shape, and execute to expert-level quality.
Where AI did level the field was upstream. In the conceptualization phase (outlines, baseline research), all three groups scored similarly: 4.05 (tech), 4.18 (marketing), 4.12 (web). And productivity gains were big across the board: conceptualization dropped to 23 minutes with AI from 63 without (nearly two-thirds less time), and writing fell to 22 minutes from 87 (nearly three-quarters less time). For advertising leaders, the takeaway is practical: use GenAI to speed briefs, outlines, and idea frameworks—and be realistic that “adjacent” talent can stretch further than “distant” talent without quality slipping, especially when the work demands context, taste, and lived experience rather than tidy, codifiable steps.
Read more at Harvard Business School.
Editor notes: Looks good. All stats verified against HBS Working Knowledge source: 78 participants (12/26/40 split), writing scores (3.42/3.96/3.92), conceptualization scores (4.05/4.18/4.12), time savings (23 vs 63 min, 22 vs 87 min), Bojinov quote. Voice is strong. Title cleanup: remove “| Working Knowledge” from WordPress post title.
Tags: generative AI, humans and machines, benchmarks, best practices, stats