Your developers are shipping code faster than they did a year ago, and some of it nobody wrote by hand. That isn't a problem. What's changed is which part of the job is scarce.
The first draft used to be the expensive part. Now it's close to free, and the expensive part is knowing whether the draft is right. That one shift moves the goalposts on how you hire, what you test for and who's genuinely worth paying for.
The draft is the cheap part now
Ask an assistant for a caching layer, a migration script or a REST endpoint and you'll get something that runs. It will look confident. It will follow the conventions of whatever it was trained on, which may or may not be the conventions of your codebase.
What it won't tell you is that the cache invalidates on the wrong event, or that the migration locks a table you can't afford to lock at 9am. Those are judgment calls, and they need someone who has been on the wrong end of both.
Used well, these tools are a real multiplier. Boilerplate, test scaffolding, unfamiliar syntax, the first pass at a refactor: all faster, and your senior people spend less of the week on work that never needed them.
Used badly, it produces plausible code nobody understands, at a volume nobody can review properly. The bug rate doesn't climb because the tool is bad. It climbs because the reading didn't keep pace with the writing.
So the interview has to change, and most haven't. Asking someone to write a function from scratch tests the skill that just got cheap.
What you want to know is whether they can find a problem in code they didn't write. Hand them a working implementation with a subtle fault in it, tell them it's already in production, and watch what they do. Do they read the query plan or guess? Do they ask what the traffic pattern looks like before optimizing anything? Do they notice the retry that will duplicate a charge, or the timeout nobody set? The engineers who catch those things are the ones who've operated something real, and they behave the same way whether the code came from a colleague, a contractor or a model. That's the signal, and it's the same signal that always mattered, just worth more than it used to be.
Then ask them to explain the fix to someone non technical. Anyone who can say "orders aren't being lost, they're slow because the payment provider is slow" is worth more to your team than someone who can only say it in code.
Review is the bottleneck now, not writing
Judgment first, tools second
It's the standard we set out in how to identify a strong backend developer, and it's how we screen for every role: PHP, SQL and Shopify alike. Tools change every year. What we test for doesn't.
None of this is an argument against the tools. Your team should use them, and the teams that use them well will out ship the ones that don't. It's an argument about what you screen for once the writing stops being the hard part.
If you're hiring into that reality, the skills worth testing are diagnosis, operational instinct and the ability to explain a tradeoff out loud. That's what we look for across our stack, and it shapes how we choose the tools we build on.