Becoming AI-Native Without Losing Our Minds to Architectural Amnesia — Lessons from QCon AI NY 2025

I sat in the QCon AI NY 2025 room. The lights were bright. The ideas were faster. Andrew Hoblitzell gave the talk. He warned us about architectural amnesia. He...

Introduction

I sat in the QCon AI NY 2025 room. The lights were bright. The ideas were faster. Andrew Hoblitzell gave the talk. He warned us about architectural amnesia. He warned us about losing our system memory when AI moves in.

I have built systems at Google, Microsoft, Amazon, and Zulily. I have seen the same mistake many times. You can be AI-native and sane. You must be deliberate.

What does AI-native mean?

AI-native means you design systems with AI at the center. It means data, models, and pipelines are first-class parts of your architecture. It is not an add-on. It is the foundation.

Why architectural amnesia kills AI projects

Architectural amnesia is when teams forget why systems were built the way they were. They add models. They skip contracts. They break interfaces. Over time, the system loses meaning. It becomes fragile. Costs rise. Bugs hide in the wiring.

Stories from the field

Google: A team rolled out an ML model without stable data contracts. The model changed a feature name. Upstream jobs failed. The pipeline threw silent errors for days. The fix was simple. The damage was not.

Amazon: A team tightly coupled model logic into core services. When the model changed, the whole service needed changes. Deployments stalled. We introduced model serving layers. We gained speed.

Zulily: A small team used clear data contracts and strong observability. They caught drift fast. The app stayed healthy.

Core principles to avoid architectural amnesia

Define data contracts early. Treat them as APIs. Version them.
Separate model training from model serving. Keep boundaries clean.
Automate tests for data quality, model outputs, and backward compatibility.
Add observability. Log features, predictions, and feedback.
Use feature stores or shared descriptors to avoid duplicate logic.
Govern models. Keep an audit trail of who changed what and why.

Practical steps for being AI-native

Start with a small, clear contract. Use JSON schema or Protobuf. Version it. Publish it. Enforce it.
Build a model gateway. The gateway hides model changes from callers. It shapes requests and responses.
Deploy shadow testing. Let the new model run in parallel. Compare results. Do not cut the cord too fast.
Monitor drift and data quality. Alert early. Prefer small fixes to late rewrites.
Run chaos tests on your ML pipeline. Fail fast in staging. Fix before prod.
Treat models like people: document them. Add lineage, metrics, and grade.

Tools and patterns that work

Feature stores to centralize features. Model registries to track versions. CI/CD for models (MLOps) with data tests. Observability tools for predictions and inputs. Data contracts and schema registries.

Quick checklist

Do we have a data contract? Yes/No.
Is training separate from serving? Yes/No.
Do we log inputs and outputs? Yes/No.
Do we version models and features? Yes/No.
Can we run a model in shadow mode? Yes/No.

A faith of engineering

I am Ermenildo Valdez Castro, Jr. I also go by Ernie Castro. I have been humbled by systems that failed and grateful for systems that held. A little like a parish on Sunday. We prepare and then we watch.

Conclusion

Becoming AI-native is necessary. It is also risky. Architectural amnesia is real. Fight it with contracts, boundaries, tests, and observability. Start small. Build clear layers. Ship often. Learn fast.

— Ermenildo V Castro (Ernie) 😊