AI coding assistants — GitHub Copilot, ChatGPT, and their kin — have moved from novelty to everyday tool in the space of just a few years. Surveys consistently report that a majority of developers now use at least one AI tool in their workflow. The productivity claims are compelling: faster boilerplate, quicker documentation, reduced context-switching between documentation and editor. And yet, as someone who studies how people actually work with technology, I find myself asking a different question: are developers truly teaming with these tools, or are they simply being led by them?

What Does "Teaming" Actually Mean?

The concept of human-machine teaming comes from human factors research, and it carries specific implications. A team is not just two entities working in proximity — it involves shared goals, complementary roles, mutual awareness of each other's capabilities and limitations, and the ability to recalibrate when things go wrong. By that standard, most current human-AI coding interactions fall considerably short.

Today's AI coding assistants are extraordinarily capable at pattern completion. They excel at producing syntactically plausible code based on context. But they have no model of your goals, no understanding of the broader system you are building, and no way to flag when their confident-sounding suggestion is subtly wrong for your specific situation. The burden of calibration sits entirely with the developer.

The Automation Bias Risk

One of the most well-documented risks in human-automation interaction is automation bias — the tendency to over-rely on automated suggestions and reduce independent judgement. We see this in aviation, in medical decision-support systems, and in radiology. There is every reason to believe it applies equally to AI code generation.

When a developer is tired, time-pressured, or working outside their area of expertise, an AI suggestion that looks right is highly likely to be accepted without adequate scrutiny. Security vulnerabilities introduced via AI-generated code are already appearing in empirical studies. The code compiles, the tests pass — and the flaw lives on in production.

This is not a failure of the tool. It is a predictable consequence of deploying a powerful automation in conditions where human oversight is cognitively demanding and where the cost of close inspection feels higher than the perceived risk of accepting the suggestion.

Towards Effective Human-AI Collaboration

So what would genuinely effective teaming look like? A few principles seem important:

• Transparency over fluency. Tools that communicate uncertainty — flagging when a suggestion is drawn from limited or ambiguous context — support better human judgement than tools that always present a polished answer. Fluency can be a form of deception.
• Friction in the right places. Not all friction is bad. Asking developers to briefly articulate why they are accepting a suggestion in a high-stakes context (security-sensitive code, complex logic) could act as a forcing function for deliberate reasoning.
• Skill maintenance, not skill atrophy. If developers consistently offload the thinking parts of coding to AI tools, the expertise needed to catch AI errors will erode. Effective teaming preserves and exercises human skill rather than substituting for it.
• Shared context. Future tools that maintain a richer model of the codebase, the developer's intent, and the team's design decisions could begin to approximate genuine collaboration. We are some distance from that today.

A Research Perspective

These questions are not merely theoretical for me. My current research examines how developers engage with AI tools in software engineering practice — what strategies they use, when they trust AI outputs, and how team dynamics shift when AI becomes a participant in the development process. I presented early findings at CHASE 2025 in Ottawa, and I expect this line of inquiry will keep us busy for years to come.

The stakes are real. Software shapes infrastructure, healthcare, finance, and social interaction. The people who build it need tools that amplify their capability and judgement, not tools that quietly erode both. Getting human-machine teaming right in software engineering is, I would argue, one of the more consequential design challenges of the coming decade.

← Back to Blog