AI Wrote the Code. Who Is Responsible When It Breaks?

A few years ago, implementing a new feature usually meant opening the editor, understanding the existing code, writing the implementation, running tests, fixing errors, and repeating the process until everything worked. Today, a developer can describe the feature in a prompt and have an AI coding agent generate a large part of the implementation in minutes.#

At first, this feels like a huge productivity improvement. And it can be.

The interesting problem starts after the code is generated. What happens when the code looks correct, the tests pass, the pull request is merged, and the application still breaks in production?

• Who is responsible for that code?
• The AI that generated it?
• The developer who accepted it?
• The reviewer who approved the pull request?
• The answer is much simpler than the question: the developer and the engineering team are still responsible for the software they ship.

AI can write the code, but it does not own the system. That distinction becomes increasingly important as AI coding agents become capable of making changes across an entire codebase instead of simply completing a few lines inside an editor.

The Code Can Be Correct and Still Be Wrong#

One of the easiest mistakes to make with AI-generated code is to judge it by whether it looks reasonable.

Consider a simple Django API. Suppose we have an endpoint that returns a list of orders:

  class OrderListView(ListAPIView):
      queryset = Order.objects.all()
      serializer_class = OrderSerializer

The serializer returns information about the customer:

  class OrderSerializer(serializers.ModelSerializer):
      customer_name = serializers.CharField(
          source="customer.name",
          read_only=True,
      )
  
      class Meta:
          model = Order
          fields = ["id", "customer_name", "total"]

An AI assistant might look at this code and tell us that everything is fine. It is valid Django code. It is easy to understand. It may even pass the existing tests. But imagine the endpoint returns 2,000 orders. If Django loads each customer separately, the endpoint can suddenly execute thousands of database queries. Nothing is syntactically wrong. Nothing necessarily crashes. The code simply does not behave well at scale. The correct implementation may require:

  Order.objects.select_related("customer")

This is the kind of problem that makes AI-assisted development interesting, AI can know how select_related() works. It can even suggest it when asked. But it may not know that your particular endpoint receives thousands of records in production, or that another part of the system depends on the current query behavior.

The missing information is not syntax.

It is

context

.

AI Does Not Understand Your Entire System#

This is where experienced developers still have an important role, A real application is much larger than the file currently being edited, A small change to a Django model can affect serializers, views, permissions, Celery tasks, database queries, reports, background jobs, and integrations with other services.

For example, imagine an AI agent is asked:

Add a status field to the Order model and update the API.

The task sounds simple, But the developer knows something the prompt may not mention:

• The order status is also used by a Celery task that creates shipments.
• There is a database constraint that depends on the old status values.
• A reporting query assumes that completed is the final state.
• Another service consumes the API and expects the previous response format.

Suddenly, adding one field is no longer just a model change.

The difficult part of software development has never been typing:

  status = models.CharField(...)

The difficult part is understanding what that change means to the rest of the system.

The Dangerous Part Is Code That Looks Good#

Bad code is usually easy to notice, A function that is 500 lines long, contains duplicated logic, and has obvious bugs will probably attract attention during code review. AI-generated code can be more dangerous when it is clean, consistent, and convincing. It might use the correct framework APIs. It might follow the project's naming conventions. It might even look better than the code written by a developer six months ago. But it can still contain a problem that is invisible without understanding the system.

Consider database transactions, An AI agent might implement:

  order.status = "confirmed"
  order.save()
  
  create_payment(order)
  create_shipment(order)

The code looks perfectly reasonable. But what happens if create_payment() succeeds and create_shipment() fails?. Now the order says it is confirmed, the payment exists, but the shipment does not. The problem is not that the AI doesn't know Python. The problem is that the implementation does not correctly represent the business transaction. A developer needs to ask questions such as:

• Should these operations happen inside one transaction?
• Can the payment operation be retried?
• What happens if the task runs twice?
• Should shipment creation happen synchronously?
• What happens when the external payment provider times out after receiving the request?

These are engineering questions. They are not solved simply by generating more code.

Tests Are Important, But Passing Tests Are Not Proof#

This is another trap in AI-assisted development.

You ask the agent to implement a feature.

• It writes the code.
• It writes tests.
• The tests pass.

Everything looks good. But what exactly did those tests prove? Tests can prove that specific scenarios behave as expected. They cannot prove that the software satisfies every business requirement or that the implementation is safe under every production condition.

For example, a test might verify:

  def test_create_order():
      response = client.post("/api/orders/", data)
  
      assert response.status_code == 201

That tells us the endpoint can create an order. It does not necessarily tell us:

• whether duplicate requests create duplicate orders,
• whether unauthorized users can access another customer's order,
• whether the endpoint performs efficiently with 10,000 records,
• whether concurrent requests cause inconsistent data,
• whether a failed payment leaves the order in the correct state,
• whether the database transaction is correct.

The test passed but system can still be wrong, This is why AI-generated tests should be reviewed with the same care as AI-generated application code.

So What Should Developers Actually Review?#

When an AI agent creates a feature, I don't think the most useful question is:

"Does this code look good?"

A better question is:

"What could this code do wrong?"

That changes the entire review process. For a backend application, I usually want to think through several areas.

Database#

Ask:

• Does this create unnecessary queries?
• Can this produce an N+1 problem?
• Are indexes required?
• Can the migration lock a large table?
• Are constraints correct?
• Can duplicate records be created?

Transactions#

Ask:

• Which operations must succeed together?
• What happens when the middle operation fails?
• Can the operation be safely retried?
• Can two requests modify the same record at the same time?

Security#

Ask:

• Can a user access another user's data?
• Are permissions checked at the correct level?
• Can input be trusted?
• Are sensitive fields exposed in the serializer?
• Can an endpoint be abused by repeated requests?

API contracts#

Ask:

• Does this change the response structure?
• Will existing clients continue working?
• Are error responses consistent?
• Are optional and required fields handled correctly?

Background jobs#

Ask:

• Is the Celery task idempotent?
• What happens when it fails?
• Will retries create duplicate data?
• What happens if the worker crashes halfway through the operation?

Production behavior#

And finally:

• What happens when the database is slow?
• What happens when Redis is unavailable?
• What happens when an external API times out?
• What happens when two users perform the operation simultaneously?
• What happens when the data is 100 times larger than the developer's test data?

These questions are often more important than the code itself.

The Developer's Role Is Changing#

AI does change software development. There is no point pretending otherwise. If an AI agent can implement a feature in ten minutes that previously took a developer two hours, developers should take advantage of that capability. But the valuable skill is moving upward. Instead of spending most of the time typing code, developers increasingly need to spend time understanding the system, describing the requirements clearly, reviewing generated changes, designing tests, investigating failures, and making architectural decisions.

The developer becomes less of a typist and more of an engineer responsible for the outcome. That is an important distinction.

If an AI agent writes 500 lines of code and the developer does not understand those 500 lines, the developer has not really saved 500 lines of work. They have moved the work somewhere else. The work becomes reviewing, validating, testing, and understanding what was generated.

How I Would Use an AI Coding Agent#

For a real project, I would avoid giving an AI agent a large feature and simply saying:

"Build this."

Instead, I would gradually give it enough context to make good decisions.

First, ask it to understand the existing architecture.

Then ask it to identify the files and components involved.

Then ask it to explain the current flow.

Only after that would I ask it to implement the change.

For example:

  First inspect the existing order workflow.
  
  Do not modify any files.
  
  Explain:
  
  1. How an order is created.
  2. Which services are involved.
  3. Which models are updated.
  4. Which background tasks are triggered.
  5. Which database transactions are used.
  6. Which existing tests cover this flow.
  
  Then identify the files that would need to change
  to implement the new requirement.

This approach gives the developer an opportunity to catch misunderstandings before code is generated.

Then the implementation request can be much more specific:

  Now implement the requirement.
  
  Reuse the existing architecture.
  
  Do not introduce a new abstraction unless necessary.
  
  After implementation:
  
  1. Run the relevant tests.
  2. Check for N+1 queries.
  3. Check transaction boundaries.
  4. Check permission behavior.
  5. Check backward compatibility.
  6. Summarize the files changed and why.

The AI becomes a development tool inside the engineering process rather than becoming the engineering process itself.

The Most Important Skill Is Still Understanding#

There is a temptation to think that AI means developers no longer need to understand the code they write.

I think the opposite is becoming true.

When developers write every line manually, they naturally spend time reading and thinking about the implementation.

When an AI agent can generate hundreds of lines in seconds, it becomes easier to accept code without fully understanding it.

That makes understanding more valuable, not less.

You need to understand enough to recognize when something is wrong.

You need to understand databases to recognize a bad query.

You need to understand distributed systems to recognize a race condition.

You need to understand security to recognize a broken permission check.

You need to understand your business domain to recognize an implementation that technically works but violates the actual requirements.

AI can help you write the code.

It cannot take responsibility for the software you put into production.

Who Is Responsible When It Breaks?#

So, who is responsible when AI-generated code breaks production?

The answer isn't the AI.

The AI is a tool.

The responsibility belongs to the people and organization that decide to ship the software.

That doesn't mean developers should avoid AI. Quite the opposite. AI coding agents can be incredibly useful when they are used as part of a disciplined engineering process.

The important thing is to keep the responsibility in the right place.

Let AI generate code.

Let it investigate problems.

Let it write tests.

Let it suggest refactoring.

But make the final engineering decisions yourself.

Because production doesn't care who wrote the code.

It only cares whether the system works.