The Hidden Cost of Session State in Scale-Out Runtimes

Why modern systems treat compute as disposable and state as infrastructure.

In the previous article, we explored why analytical systems eventually become temporal systems.

Organizations stop asking:

What exists?

And begin asking:

What changed?

Answering that question requires preserving history.

Versioned records.

Append-only timelines.

Historical state transitions.

The platform accumulates increasingly large volumes of information.

Naturally, this creates a new architectural temptation.

If data is valuable, why not keep it close to the application?

The answer turns out to be one of the defining lessons of modern infrastructure.

Because state and scale rarely coexist peacefully.

The Comfortable Illusion

Most applications begin life as a single process.

One server.

One runtime.

One memory space.

Everything feels simple.

A user logs in.

Session information is stored in memory.

A few objects are cached.

User preferences remain resident inside the application.

The architecture works perfectly.

Because there is only one place for requests to go.

The Day a Second Server Appears

Success introduces complexity.

Traffic grows.

A second application instance is deployed.

Then a third.

Then ten.

Suddenly the architecture changes:

           User
             ↓
      Load Balancer
         ↙     ↘
   Server A   Server B

At first glance nothing appears different.

The application code remains identical.

The database remains identical.

Users continue interacting with the same platform.

But a subtle problem has emerged.

Server A and Server B do not share memory.

The Session Problem

Imagine a user logs in.

Their session is stored inside Server A.

The next request arrives.

The load balancer sends it to Server B.

Server B knows nothing about the user.

The session has vanished.

Not because it was deleted.

Because it never existed on that machine.

The system now faces a decision.

Either:

• Force requests back to the original server

or

• Move session state somewhere shared

This decision shapes the architecture of the entire platform.

Sticky Sessions: The Shortcut

The easiest solution is usually sticky sessions.

The load balancer remembers where a user originally landed.

Future requests are routed to the same server.

The problem appears solved.

And for a while, it is.

Many systems operate successfully this way.

The difficulty emerges later.

Because sticky sessions create a hidden dependency between users and infrastructure.

Server A is no longer interchangeable with Server B.

Requests become tied to specific machines.

The architecture begins losing one of the most valuable properties in distributed systems:

Replaceability.

The Fallacy of Stateful Compute

Cloud infrastructure introduced a radically different idea.

Application instances should be disposable.

A server should be able to disappear at any moment.

A container should be replaceable.

A runtime should be restartable.

None of these actions should affect users.

This philosophy becomes impossible when application state lives inside the runtime itself.

The moment a server owns state, the server becomes special.

Special infrastructure is difficult infrastructure.

And difficult infrastructure rarely scales elegantly.

Horizontal Scaling Changes Everything

When architects talk about horizontal scaling, they often focus on capacity.

More servers.

More requests.

More throughput.

But horizontal scaling is really about independence.

Every application instance should be capable of serving every request.

That requires a simple rule:

No application instance should own information that other instances require.

The runtime executes.

The runtime processes.

The runtime responds.

The runtime forgets.

State lives elsewhere.

State Becomes Infrastructure

Modern systems eventually arrive at the same architecture.

Instead of:

Application
     ↓
Session State

they evolve toward:

Application
     ↓
Shared State Layer

This layer may be:

• Redis
• PostgreSQL
• DynamoDB
• Distributed caches
• Object storage
• Session stores

The technology matters less than the principle.

State becomes an infrastructure concern rather than an application concern.

Applications consume state.

They do not own it.

Why Historical Systems Make This Worse

The challenge becomes even more significant when we connect it back to the previous article.

Week 7 introduced temporal systems.

Historical records.

Versioned entities.

Append-only timelines.

These systems naturally accumulate far more data than current-state systems.

Imagine attempting to keep portions of that information inside application memory.

The result is predictable.

Memory consumption grows.

Synchronization complexity grows.

Recovery becomes difficult.

Horizontal scaling becomes fragile.

The larger the historical dataset becomes, the more dangerous stateful runtimes become.

The architecture begins fighting itself.

Failure Recovery Reveals the Truth

One of the simplest ways to evaluate an architecture is through a thought experiment.

Ask:

What happens if this server disappears right now?

In a stateless architecture, the answer is usually boring.

Another instance takes over.

Traffic continues.

Users remain unaware.

In a stateful architecture, the answer becomes complicated.

Sessions disappear.

Caches vanish.

User context is lost.

Recovery becomes a restoration exercise.

This distinction reveals an uncomfortable truth.

Many systems appear scalable until the first failure occurs.

The Cloud's Most Important Lesson

Cloud computing is often described through virtualization, containers, orchestration, and managed services.

Those are implementation details.

The deeper lesson is architectural.

Compute became temporary.

State remained permanent.

Modern infrastructure succeeds because these concerns are separated.

Applications can be replaced.

State persists.

Applications can scale.

State remains consistent.

Applications can fail.

State survives.

This separation unlocked the elasticity that defines contemporary platforms.

A More Important Lesson

This article is not really about sessions.

Nor is it about cookies.

It is about ownership.

Earlier in this series we argued that dashboards should not own business logic.

Operational systems should not own analytical definitions.

Dependencies should not own system stability.

Now we arrive at another separation.

Application runtimes should not own state.

Every time a component becomes responsible for something beyond its intended role, complexity accumulates.

State inside runtimes appears convenient.

Until scale arrives.

Then convenience becomes a constraint.

Looking Ahead

So far, this series has explored how systems accumulate complexity through data, infrastructure, dependencies, memory, and state.

Next we return to the data layer once again.

Because even when architectures become properly distributed, another invisible cost remains.

Moving data.

Serialization.

Network transport.

Execution boundaries.

The industry spent decades optimizing storage and computation.

Increasingly, the bottleneck is neither.

The bottleneck is the cost of moving information between them.

And that is where we go next.

Part 8 of the Architecture of Information Systems series.