What Building with AI Actually Means for Software Engineers

When engineers ask me what comes after AI-assisted coding, I do not think the answer is simply more AI-assisted coding.

Writing code is becoming easier. Connecting a model to an inbox, adding an MCP server or asking an agent to update a ticket can be useful, but those examples left me with a more difficult question: what are software teams actually building with AI?

I found my answer while helping to shape a trend-detection proof of concept for an influencer-marketing business. It was not the project I first thought it was. That change in understanding showed me where much of the next layer of software engineering complexity sits.

I describe that layer as Building with AI: using existing models and AI capabilities as components inside a product or operational system. It sits between using an AI product and building the underlying models themselves.

Connecting an AI Tool Is Only the Starting Point

If I connect an assistant to my email and ask it to summarise unread messages, I am using AI. If I expose email through an MCP server, I have created a reusable connection. MCP gives applications a standard way to provide models with resources, prompts and tools, including executable actions. That is valuable infrastructure, but the connection is not yet the whole system.

The engineering begins when I have to decide which messages the model may read, which actions it may take, what approval is required, how duplicate actions are prevented, what happens after a partial failure, and how I can reconstruct a bad decision later. The difficult part is not giving a model access. It is designing the boundary around that access.

This distinction helped me separate a demonstration from a product. A demonstration proves that a model can call a function. A product must remain useful when the input is incomplete, the model is uncertain, an external service is unavailable or a user wants to know why something happened.

The Trend-Detection Project That Changed My Definition

The business problem sounded simple: spot an emerging content trend early enough for a campaign team to act on it. My initial mental model was an ingestion problem. We would collect signals from external platforms, enrich the data and ask AI to identify what was gaining momentum.

Further discovery changed that architecture. The client already had a working multimodal search capability. Videos had been converted into frame grids, an LLM described the visual content, audio had been transcribed, and embeddings made the resulting records searchable. A campaign manager could type a phrase and find semantically related content.

That system solved search for something somebody had already named. The actual gap was different: surface a pattern before anybody knew what phrase to type.

We did not need a better chatbot. We needed a detection system.

The proposed proof of concept shifted towards clustering the existing embedding space, comparing activity across time windows and maintaining a persistent registry of candidate trends. Persistence mattered because a trend could appear, go quiet and return. A stateless weekly run might treat the same behaviour as three unrelated discoveries.

Confidence could not come from a model declaring, “this is a trend.” It had to be composed from observable signals: the size and growth of a cluster, engagement, duration, recurrence and the completeness of the underlying records. Missing music identifiers or imprecise timestamps could not bring the pipeline down, so the system needed to degrade deliberately and expose where its confidence had weakened.

There was also no historical dataset containing every real trend and the exact moment it began. We could not honestly promise a conventional accuracy score. We framed success as a feasibility result: run the pipeline, inspect whether its candidates were coherent, replay a few known cases and let a campaign manager decide whether the shortlist was actionable.

That was the moment the phrase “building with AI” became concrete for me. The models and embeddings mattered, but they were components. The work was in turning uncertain signals into a bounded, inspectable decision-support system.

What Software Teams Are Actually Building with AI

I now look for systems in which AI changes how an application interprets information, chooses a path or produces an outcome. That includes more than chat interfaces:

• Retrieval systems that find relevant text, images, audio or video by meaning rather than exact keywords. Vector search works by comparing embeddings, numerical representations that preserve useful semantic relationships.
• Decision-support systems that rank, cluster or explain weak signals while leaving an accountable person in control of the final decision.
• Workflow systems in which a model selects tools, prepares structured inputs and hands work between people and services.
• Adaptive interfaces that generate structured answers or user-interface elements based on the user’s context, while still respecting a schema.
• Evaluation and monitoring systems that measure whether model-assisted behaviour remains useful after prompts, data, tools or models change.

Some of these systems contain an agent; many do not need one. In the trend project, clustering and temporal tracking were more important than an autonomous loop. Adding an agent would not have solved the missing ground truth or the need to preserve the identity of a trend over time.

This is why I no longer start with “Where can we add an agent?” I start with the decision or workflow, then ask which part benefits from semantic interpretation, generation or tool selection.

The New AI Toolchain Follows the New Failure Modes

A growing collection of tools now exists around this work. I find them easier to understand when I connect each category to the problem it is trying to manage.

• AI application SDKs, such as the Vercel AI SDK, standardise model calls, streaming, structured output and tool calling. Their schemas and approval controls exist because a model-generated tool request needs validation before ordinary code executes it.
• Model gateways and provider abstractions centralise model selection, fallbacks, budgets and usage controls. They help when a system cannot assume that one provider, model, price or latency profile will remain suitable.
• MCP clients and servers standardise how models discover context and tools. They reduce bespoke integration work, while leaving authentication, authorisation and safe execution to the application design.
• Embedding models and vector search make semantic retrieval and similarity-based discovery practical across text and other media. In our project, that made it possible to move from named searches towards groups of related content.
• Agent and workflow runtimes manage multi-step execution, state, retries and checkpoints. They become useful when a process genuinely needs branching or repeated tool use rather than one model response.
• Evaluation and observability platforms record traces and test behaviour against examples. LangSmith, for example, separates offline evaluation on curated datasets from online evaluation of production interactions and turns failed traces into future test cases.
• Guardrails and permission layers constrain inputs, outputs and actions. Their job is not to make a model infallible; it is to limit what a failure can affect and make important actions reviewable.

These tools are not evidence that every application needs an AI stack. They are evidence that teams have encountered recurring problems that a model API alone does not solve.

The Building-with-AI Jobs I Can See Today

The titles are still unsettled, but the work is already visible. At the time I checked, Anthropic grouped roles such as Applied AI Engineer, Applied AI Architect, Applied AI Security Architect and Forward Deployed Engineer under Applied AI. OpenAI’s description of an Applied AI Engineer spans use-case selection, architecture, prototypes, evaluation, integrations, production launch and decisions about retrieval, tools, reliability, latency, cost, safety and governance.

I read those listings as examples of responsibility clusters, not a final map of the labour market:

• Applied AI or AI product engineers turn model capabilities into reliable product behaviour.
• Forward-deployed engineers and AI architects combine discovery, domain constraints and hands-on implementation in a customer environment.
• AI platform engineers provide shared model access, telemetry, security, cost controls and deployment patterns.
• AI evaluation and reliability engineers build datasets, graders, traces and feedback loops for behaviour that ordinary unit tests cannot fully cover.
• AI security engineers concentrate on prompt injection, data exposure, permissions and the consequences of giving models tools.

None of this makes full-stack experience obsolete. I used the same instincts I would use in any serious software project: clarify the user’s decision, inspect the data, define interfaces, isolate failure, preserve state and agree what success means. The difference was that part of the system behaved probabilistically and the evidence was incomplete.

Where the Engineering Complexity Is Moving

AI-assisted coding reduces the effort required to produce some code. I have felt that change in my own day-to-day work. But code production was never the whole job, and making it cheaper increases the relative value of the decisions around it.

The harder questions move upwards and outwards:

• What outcome is valuable enough to build around?
• Which decisions may be probabilistic, and which must remain deterministic?
• What context may the system access, and under whose authority?
• How will we evaluate a useful answer when no perfect answer exists?
• How do we detect behavioural regression after changing a prompt, model or data source?
• How do we control latency and cost across several model and retrieval steps?
• Where must a person review, override or explain the result?

In the trend-detection work, writing the pipeline was only one concern. We also had to avoid pretending that early detection was prediction, avoid inventing a benchmark we did not possess, distinguish a recurring trend from a new one and keep incomplete fields from becoming invisible assumptions.

The next complexity is not syntax. It is accountability for systems whose behaviour cannot be specified line by line.

My Practical Test for Building with AI

I use a simple test now. If removing the model leaves the same product with a less convenient interface, I may only be using AI. If removing it changes the system’s ability to interpret, retrieve, rank, decide or act, I am probably building with AI.

Then I ask a second question: what new failure has that capability introduced? The answer points towards the real engineering work and towards the tools that may deserve a place in the architecture.

For a full-stack developer taking a first step, I would not begin by collecting frameworks. I would choose one bounded workflow with a real user and an observable decision. Add one AI capability. Define what good and harmful outputs look like. Record the intermediate steps. Put a human checkpoint before any consequential action. Then test the awkward cases.

That exercise is small enough to finish, but it contains the work I now associate with this field. Building with AI begins when the model stops being the demo and becomes one governed part of a system you are prepared to own.