Analysis of 713k employee prompts finds senior staff exhibit more sophisticated genAI use, with sophistication varying by role and increasing over time as domain expertise complements model capabilities.
Research on 713,564 employee prompts from nearly 4,000 back-office workers at a large firm (KPMG) reveals that senior employees exhibit significantly more sophisticated Generative AI use than junior staff. This sophistication, measured by prompt clarity, deliberate strategy use, and use-case diversity, increases with seniority because domain expertise complements AI capabilities, enabling complex knowledge-retrieval and strategic tasks rather than simple writing requests.
Contrary to the expectation that sophistication improves over time, the study found no sustained improvement in sophisticated use over an eight-month period, nor lasting gains from formal AI training. Instead, sophisticated use was consistently associated with longer initial prompts and greater iteration within conversations, suggesting that effective AI collaboration relies on delegation proficiency and accountability inherent in senior roles rather than just tool familiarity.
Key Findings: Seniority Bias: Above-manager employees showed the highest sophistication, utilizing AI for knowledge-intensive and strategic tasks, while staff employees focused on writing and personal requests. Functional Variation: Sophistication varied by department, with Strategy, Digital Innovation, and Project Management showing higher usage of advanced prompting techniques compared to Accounting & Finance. Training Impact: Formal AI training led to a temporary spike in sophistication during the month of completion, but no long-term behavioral change was observed in subsequent months. Observable Metrics: Managers can gauge sophistication through metadata like prompt length and iteration count (persistence/ambition) rather than just usage frequency, which correlated weakly with sophisticated behavior.
The paper presents large-scale field evidence on how employees actually use generative AI in a workplace setting, analyzing 713k prompts from a large firm. Rather than treating GenAI use as a binary adoption metric, it focuses on the quality and structure of employee prompts, examining how “sophistication” varies across seniority, role, and time. In doing so, the study offers a more granular view of AI use than typical usage analytics, connecting prompt behavior to organizational context and professional expertise.
Its central insight is that more senior employees tend to produce more sophisticated prompts, and that prompt sophistication is not uniform across the workforce. Instead, it varies by role and evolves over time, suggesting that effective GenAI use is shaped by domain knowledge, task familiarity, and experience with the model’s capabilities. This supports a complementarity view of AI adoption: human expertise does not merely coexist with model capability, but actively improves how the model is deployed.
The material matters because it provides rare real-world evidence on the micro-level practices underlying enterprise AI use. For organizations, it suggests that access to AI tools is insufficient without role-specific training, prompt practices, and feedback mechanisms. For researchers, it contributes a useful empirical lens for studying how foundation models are integrated into professional workflows, and how organizational learning may amplify or constrain the productivity gains from generative AI.