As artificial intelligence becomes embedded in corporate hierarchies, a disconcerting gap has emerged between how quickly companies deploy AI systems and how carefully they manage them. Researchers at Boston University and Boston Consulting Group have documented a phenomenon that should concern business leaders across Southeast Asia and beyond: when organisations treat AI agents as legitimate employees—even listing them on organisational charts—managers systematically lower their guard, vetting work less rigorously and overlooking mistakes that would normally be caught. This paradoxical loss of vigilance represents a critical blind spot in the broader AI adoption rush that many Malaysian and regional firms are now pursuing.
The research began when Emma Wiles, a Boston University professor studying AI's workplace impact, observed human resources executives promoting the practice of anthropomorphising AI systems at a conference last October. What initially seemed a productivity-boosting innovation revealed a serious flaw upon investigation. In experiments spanning multiple companies, managers assigned to review documents from purported AI employees caught substantially fewer errors than those reviewing identical work attributed to humans. This wasn't a marginal difference; it reflected a fundamental shift in accountability psychology when humans interact with technology framed as a peer rather than a tool.
Wiles and her collaborators surveyed over one thousand corporate managers and discovered that approximately one-third of their organisations refer to AI systems as teammates or employees, with nearly a quarter maintaining AI agents on formal organisational structures. One manager even gave his AI system a name—Scout—and described it as an equivalent peer on his team. This anthropomorphisation matters profoundly because it reshapes how human supervisors approach their responsibilities. The research revealed that when managers believed they were reviewing work from an AI employee rather than an AI tool, they significantly reduced their quality control efforts. Those overseeing companies with formalised AI employees proved least diligent, suggesting that institutional legitimacy amplifies the accountability paradox.
The psychological mechanism underlying this behaviour appears rooted in established professional norms. Managers accustomed to overseeing human subordinates operate from an ingrained assumption that mistakes reflect their own supervisory failure. This internalised responsibility drives meticulous review. Similarly, when assessing outputs from clearly labelled tools, managers maintain defensive caution. But AI employees occupy a conceptual middle ground that confuses accountability. Managers at organisations formally recognising AI agents seem to assume responsibility lies elsewhere—with the technology team, with executives who mandated AI adoption, or with the machines themselves. This diffusion of accountability creates a dangerous void where critical scrutiny should exist.
This problem extends beyond simple oversight failures. Researchers have discovered that AI language models themselves demonstrate systematic bias toward content produced by other AI systems. A 2025 study found that resume-evaluation algorithms consistently favoured applications written with AI assistance over those composed entirely by humans. When Ohio State University researchers presented these findings to recruiting firms, they uncovered widespread ignorance of this bias among companies already deploying such systems. Jane Yi Jiang, an operations professor at Ohio State, noted that firms were eager to learn how to improve their processes only after researchers spelled out the problem explicitly. This reactive rather than proactive approach suggests most companies are implementing AI without anticipating consequences.
For Malaysian enterprises considering similar deployments, the implications warrant serious reflection. Regional companies moving rapidly into AI adoption often lack the institutional frameworks to identify such subtle but consequential problems. The pace of implementation frequently outpaces risk assessment capacity. Jiannan Xu, a University of Maryland researcher collaborating on hiring algorithm studies, emphasised that most large language models fundamentally misunderstand human behaviour and decision-making. These systems tend to apply cold game-theoretic rationality when evaluating business scenarios, potentially steering companies toward aggressive competitive tactics that trigger destructive outcomes no human strategist would recommend.
One illustrative case involves AI systems making pricing and location decisions. When left to optimise purely rational outcomes, AI models may recommend undercutting competitors so aggressively that industry-wide price wars develop, destroying profitability across the sector. Humans, even competitive ones, naturally cooperate and seek mutual benefit more frequently than theoretical rationality suggests they should. AI lacks this cooperative instinct, creating a fundamental mismatch between algorithmic recommendations and the human business ecosystems where those recommendations must operate. Companies deploying AI for strategic decisions without human safeguards risk catastrophic miscalculations.
The scope of unknown problems remains vast and largely unmapped. Wiles herself acknowledged that even researchers studying AI extensively understand only a fraction of the defects these systems introduce into organisations. As researchers and companies identify one problem—whether resume bias, oversight lapses, or game-theoretic dysfunction—they simultaneously uncover evidence of additional undetected flaws. Each solution often requires deliberate intervention, such as making managers explicitly accountable for AI employee mistakes or rebuilding trust in AI outputs through transparent validation processes. Yet few companies have implemented such safeguards systematically.
The broader context of AI adoption reveals a consistent pattern: companies became aware of certain AI shortcomings—bias against minorities, confidentiality breaches, inaccurate confident assertions—after public incidents or pressure from advocates. Yet even as this awareness grew, firms continued adopting AI for increasingly consequential decisions without learning from previous failures. The psychological patterns Wiles documented suggest that formal integration into organisational structures may actually worsen rather than improve oversight quality. Companies that maintain AI strictly as tool rather than team member may inadvertently exercise stronger quality control.
For Southeast Asian businesses, this research carries particular relevance as the region races to build AI capabilities and capitalise on the technology's purported efficiencies. Malaysian, Singaporean, and Indonesian companies often adopt innovations proven successful elsewhere without necessarily adapting implementation practices to local contexts. The findings suggest that importing AI-as-employee models from North American firms without simultaneously importing stronger oversight mechanisms could prove costly. Regional companies might benefit from deliberately rejecting anthropomorphisation of AI systems, maintaining psychological and organisational distance that preserves critical scrutiny.
The fundamental challenge, according to Wiles, stems from the mismatch between ancient human management practices and the novel challenge of overseeing non-human intelligence. Over centuries, societies developed reliable frameworks for managing human employees, entire bodies of law and custom governing accountability. But when organisations treat fundamentally different entities—systems without intentions, agency, or responsibility—as members of human teams, that accumulated wisdom becomes irrelevant. Managers operate intuitively rather than from established best practices, making errors no seasoned supervisor would contemplate with human subordinates.
Moving forward requires deliberate institutional choices rather than natural evolution. Companies must resist the seductive framing of AI as employees or teammates. Instead, organisations should maintain clear categorical distinction: AI systems remain tools requiring layers of human validation, with explicit accountability assigned to supervisory humans. Implementation should move at speeds that permit identification and correction of problems before they cascade. Most critically, organisations deploying AI for consequential decisions must invest in understanding not just how the technology works, but how organisational psychology responds to it. The current blind rush to integrate AI without understanding these dynamics represents a significant and largely unquantified business risk.
