Recent studies have explored how performance varies across three approaches: human-only, AI-only, and human–AI collaboration.
The findings, however, are inconsistent. Some studies suggest that augmenting humans with AI yields better outcomes, while others show the opposite. This heterogeneity calls for a closer analysis to understand when is augmentation better than automation and when is it better to trust humans.
A recent research has made a systematic review and meta-analysis of 106 experimental studies. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations.
The study revealed the following key findings:
- Human–AI collaboration outperformed humans alone on average.
- Collaboration often underperformed compared to the best standalone performers (human or AI).
- Performance losses occurred in decision-making tasks, while gains were observed in creative tasks.
- Collaboration yielded gains when humans outperformed AI but resulted in losses when AI outperformed humans.
Implications for the augmentation/automation tradeoff
This study tempers the optimism about augmentation as the best AI or best humans still perform better than a combination of AI and human. However, the AI system did on average help humans perform better.
More precisely, the study reveals an important difference when it comes to the task type: combining AI and human performs better for creating content than making decisions.
This can be explained both by overreliance (when people rely too much on AI systems without seeking and processing more information) and underreliance (when people ignore AI’s suggestions because of adverse attitudes towards automation).
Strategic implications
The study underscores the relevance of both automation and augmentation, reinforcing the need for investments in data and AI capabilities.
The second implication of this study is that it confirms that augmentation or automation is no silver bullet and that understanding when to use AI, human or a combination of both will be a key driver for competitive advantage. To achieve this, companies need to engage in experiments and test by themselves the situations and settings where each of the three option performs better.
The third implication is that investing in HR leads to a higher productivity of technological investments. Training employees on AI systems and their limitations mitigates overreliance and underreliance, maximizing the systems’ impact. Moreover, cultivating in-house talent and processes enhances the uniqueness of internal resources, strengthening competitive advantage.
Photo de Johannes Plenio sur Unsplash
