A repeated HTTP request does not always mean that the user intended to perform the same action twice.
Sometimes an order has already been created, but the client never receives the response because of a network failure. The client sees a timeout, assumes the operation failed, and sends the same request again. For a normal GET, this usually isn’t a problem. For POST /orders, the consequences are very different: two identical requests may create two separate orders.
That is why API design should consider not only how an operation is executed, but also what happens if the client repeats it.
For operations with side effects—such as creating an order, reserving inventory, or starting another critical process—this becomes especially important.
**The Problem Starts After a Successful Operation
**Imagine a normal request:
POST /orders Content-Type: application/json { "product_id": "SKU-1842", "quantity": 1 } The server receives it, creates the order, and tries to return the response.
Client API │ │ │──── POST /orders ─────>│ │ │ │ Create #18452 │ │ │<──── response ──── X │ │ connection lost │ From the server’s perspective, everything succeeded.
From the client’s perspective, it didn’t.
So the client retries the request:
POST /orders → Order #18452 timeout POST /orders → Order #18453 Now the system contains two orders, even though the user initiated only one business operation.
The problem isn’t the retry itself. Retrying after a timeout is perfectly reasonable client behavior.
The real problem is that the API cannot distinguish a retry of the previous operation from a genuinely new command.
**Adding an Idempotency-Key
**One way to solve this is to assign a unique identifier to the operation.
POST /orders Idempotency-Key: 8c9f21a7-42d1 Content-Type: application/json { "product_id": "SKU-1842", "quantity": 1 } Before executing the operation, the API checks whether the key has already been used:
key = request.headers["Idempotency-Key"] existing = idempotencyStore.find(key) if existing: return existing.response result = createOrder(request.body) idempotencyStore.save( key, result ) return result If the client repeats the request with the same key, the server does not create another order. Instead, it returns the result of the original operation.
At this point, the solution looks almost complete.
This is actually where the interesting part begins.
**A Simple
find()
Is Not Enough
**Imagine two identical requests arrive almost simultaneously:
Request A → find(key) → NOT FOUND Request B → find(key) → NOT FOUND Request A → createOrder() Request B → createOrder() Both requests check the key before either one has saved anything.
The result is still two orders.
So an if key exists check alone does not make an API idempotent.
The registration of the key must be atomic.
One way to enforce this is at the database level with a unique constraint:
CREATE TABLE idempotency_keys ( key VARCHAR(128) PRIMARY KEY, request_hash VARCHAR(64) NOT NULL, status VARCHAR(20) NOT NULL, response_body TEXT, created_at TIMESTAMP NOT NULL ); Now two processes cannot register the same key at the same time.
The logic becomes closer to this:
try: reserveIdempotencyKey(key) except DuplicateKey: return waitForExistingOperation(key) result = createOrder() completeIdempotencyKey( key, result ) return result This closes the race condition that a simple pre-check cannot solve.
**One Key Shouldn’t Represent Two Different Requests
**Now another scenario appears.
First, the client sends:
Idempotency-Key: abc-123 { "product_id": "SKU-10", "quantity": 1 } Then, by mistake, the same key is reused for a different request:
Idempotency-Key: abc-123 { "product_id": "SKU-77", "quantity": 4 } Simply returning the result of the first request would be incorrect.
The key is the same, but the business command is not.
That is why it makes sense to store a hash of the meaningful request parameters together with the key:
requestHash = sha256(normalize(request.body)) record = idempotencyStore.find(key) if record exists: if record.requestHash != requestHash: return Conflict return record.response Now the same key with the same payload is treated as a retry.
The same key with different content is treated as a conflict.
It is a small detail, but details like this determine whether an API is genuinely idempotent or merely appears to be.
**What If the Operation Fails Halfway Through?
**There is a more difficult scenario.
The key has already been registered, but the process crashes before the order is created—or before the result is stored.
In that case, the fact that a record exists is not enough.
The idempotency record needs its own lifecycle:
PROCESSING COMPLETED FAILED For COMPLETED, the behavior is straightforward: a repeated request receives the stored response.
PROCESSING means the operation is already underway. What happens next depends on the architecture: the client might wait, receive a special status, or be asked to retry later.
FAILED is more complicated. Sometimes the operation can be safely retried. In other cases, the system first needs to determine whether part of the business process completed before the failure occurred.
This is where it becomes clear that idempotency cannot be implemented using an HTTP header alone.
It has to understand the state of the business operation itself.
**How Long Should Idempotency Keys Be Stored?
**Keeping idempotency keys forever usually doesn’t make sense.
But setting the TTL too short creates a different problem.
Imagine that the record has already expired and been deleted when the client retries an old request. The API no longer knows that the operation was previously executed and treats it as a new one.
This is why retention should not simply be set to something arbitrary like “24 hours because it’s convenient.”
The TTL depends on the lifecycle of the specific operation and the expected behavior of the client.
For a short-lived technical command, the window may be small. For orders, payments, or other critical state-changing operations, the requirements may be very different.
That is why we treat TTL not as a universal infrastructure setting, but as part of the contract of the specific operation.
**Where Is Idempotency Actually Needed?
**Adding an Idempotency-Key to every API endpoint is not a good idea either.
This mechanism is especially useful where repeated execution can create an unwanted side effect:
creating another order;
reserving inventory twice;
creating the same resource again;
starting the same process twice;
repeating a business-state transition.
For read operations, the issue is usually addressed by HTTP semantics themselves.
For state-changing operations, the correct behavior depends on the specific business scenario.
That is why we stopped treating idempotency as middleware that can simply be attached to every endpoint.
It is a property of a particular business operation and of how that operation should behave when repeated.
**What Changed in Our Approach
**The most important lesson was not about the Idempotency-Key itself.
We simply stopped treating repeated requests as something unusual.
A timeout does not tell the client whether the operation completed.
The connection may be lost after the transaction has already committed.
A client application may automatically retry the request.
Two identical requests may arrive almost simultaneously.
If an API performs a critical business operation, all of these scenarios need to be considered before production—not after the first incident.
We apply the same principle when designing other internal platform components: the system should be designed not only for the ideal path, but also for situations where the network, client, or external service behaves differently from what we expected.
Further reading:
Why Internal Tools Deserve the Same Thoughtful Architecture as Customer-Facing Products https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia
The first article in the engineering series:
How We Realized Our Initial Product Catalog Model Was No Longer Enough https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed
**Conclusion
**An idempotent API is not simply an endpoint that accepts an Idempotency-Key.
A reliable implementation needs to account for concurrent requests, atomic key registration, reuse of the same key with a different payload, incomplete operations, and result retention.
But the core lesson is even simpler:
A repeated request is a normal part of a distributed system, not an exceptional scenario.
If one user action is supposed to create exactly one order, the API must preserve that rule even when the network, client, or infrastructure causes the HTTP request to be sent more than once.

Most features are initially designed around the ideal scenario: the user submits a request, the data is validated, saved to the database, and the system returns a successful response. While an application is still small, this approach is often sufficient.
As the platform grows, however, it becomes clear that the most difficult problems do not occur when everything works as expected. They arise when one step in the process unexpectedly fails.
During the development of Droplox, we gradually came to a simple conclusion: a system’s reliability is defined not only by how well it performs under normal conditions, but also by how predictably it behaves when something goes wrong.
An external API may become temporarily unavailable. A message may be delayed in the queue. The database may return a transient error. A user may submit the same request multiple times. In distributed systems, situations like these are not exceptional—they are expected.
Initially, our logic looked something like this:
createOrder() reserveInventory() sendNotification() return Success As long as every step completed successfully, everything seemed straightforward.
But the moment one operation failed, new questions appeared.
What happens if the order has already been created but inventory reservation fails?
What if the external service never receives the notification?
Can the operation be safely retried?
What will the user see if part of the workflow succeeds while another part fails?
After encountering several scenarios like these, we realized that the problem wasn’t individual failures.
The problem was the approach itself: we had designed the system as if failures were rare and had little impact on the overall workflow.
Over time, the question changed.
Instead of asking:
“How should this feature work?”
we began asking:
“What happens if any step of this feature fails?”
That shift transformed not only our exception handling but our architecture as well.
Every component now needs to know whether an operation can be safely retried, which failures are temporary, and which require the process to stop immediately.
In simplified form, the workflow evolved into something like this:
try: processOrder() except TemporaryError: retryLater() except ValidationError: rejectRequest() except ExternalServiceError: scheduleRetry() The most important change wasn’t the introduction of retries.
It was that we stopped treating failures as exceptional events.
Failures became part of the normal lifecycle of the system—something that must be considered before writing the primary success path.
This also changed how we test new features.
It is no longer enough to verify that the happy path works correctly and performs well.
We also need to understand what happens if:
an external service becomes unavailable;
a message is delayed;
an operation is executed multiple times;
the user submits the same request repeatedly.
If the system cannot handle these situations predictably, then the feature is not yet ready for production.
At the same time, our approach to observability evolved.
For every important process, we now need to know:
where the failure occurred;
how long the operation took;
whether a retry was performed;
whether the system recovered automatically.
Without this information, incident investigations quickly become little more than blind guesswork.
We apply the same philosophy when designing the platform’s internal components.
Good architecture is defined not only by how quickly the happy path executes, but also by how gracefully the system handles failures.
Learn more about this approach here:
https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia
This article also continues our first engineering note on how the platform’s architecture evolved as its requirements grew:
https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed
**Conclusion
**Today, we no longer treat failures as rare exceptions.
In distributed systems, failures are inevitable.
The question is not whether they will happen, but how predictably the system will respond when they do.
A good system is not one that never fails.
A good system is one that already knows what to do when something inevitably doesn’t go according to plan.

In the early stages of development, Soft Delete often seems like the perfect solution. Instead of physically removing a record from the database, you simply add a deleted_at or is_deleted field and exclude those records from normal queries. Users can restore deleted data, historical information is preserved, and the risk of accidental deletion appears minimal.

That is why many teams begin using Soft Delete almost everywhere.

We followed the same path.

While Droplox was still a relatively small platform, this approach worked without any noticeable issues. However, as the system grew, it became clear that what initially seemed like a universal solution was gradually becoming a source of unnecessary complexity.

The first problem was surprisingly simple. Every query now had to remember that deleted records existed. Almost every data retrieval operation required an additional condition such as deleted_at IS NULL. At first, this looked insignificant. Over time, however, these checks appeared in nearly every repository, service, and newly developed feature. Forgetting a single condition could expose data that users had considered deleted long ago.

Other consequences gradually emerged as well. The database continued storing large numbers of logically deleted records, making indexes less efficient. Queries became more complex, analytics required additional filtering, and relationships between tables became less intuitive than before. This was particularly noticeable when one entity had already been deleted while related records continued appearing in query results.

Eventually, we realized that Soft Delete itself was not the problem.

The real issue was that we had adopted it as the default strategy without asking whether it actually made sense for each type of data.

After that, our approach changed.

Instead of searching for one universal solution, we began evaluating the lifecycle of each entity individually.

If data is important for auditing, legal compliance, historical tracking, or recovery, Soft Delete is completely justified.

If a record is temporary and has no value after deletion, a physical delete is usually the better choice.

If information may be needed in the future but should no longer remain in operational tables, archiving is often the most appropriate solution.

This change significantly simplified our architecture. Operational tables stopped accumulating large volumes of unused records, queries became easier to understand, and the risk of accidentally exposing deleted data was greatly reduced.

The experience reminded us of another important lesson.

In software architecture, there are very few universal solutions.

What works perfectly for one entity may be entirely unnecessary for another.

That is why, when designing new components today, we begin with a simple question:

“What should happen to this data after it is deleted?”

The answer is not always the same.

But it usually leads us to the right strategy.

We try to apply the same principle throughout the rest of the platform as well. Instead of relying on universal patterns, we choose architectural approaches that genuinely fit the specific problem. We discussed this philosophy in more detail in our article explaining why internal tools deserve the same level of architectural attention as customer-facing features:

https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

This article also continues our first engineering note about how the product catalog architecture evolved as the platform grew:

https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm\_source=hashnode&utm\_medium=feed

Conclusion

Soft Delete remains a valuable tool—when used intentionally.

It works extremely well where preserving history or enabling data recovery is genuinely important.
However, when applied automatically to every entity, it gradually makes queries more complex, introduces hidden dependencies, and reduces the overall transparency of the system.

Over time, we reached a simple conclusion:

There is no universal strategy for deleting data.

Good architecture does not begin with choosing between Soft Delete and Hard Delete.

It begins with understanding what role that data should play after the user clicks “Delete.”

During Droplox’s first internal releases, we evaluated API performance primarily by looking at the average response time. As long as the average remained low, the system appeared to be performing well. However, after some time, users began reporting that certain operations occasionally felt much slower than usual—even though our primary performance graph barely changed. The problem turned out to be the metric itself. Average response time provides a good overall picture, but it can easily hide rare yet extremely slow requests. If 99 operations complete in 80–100 ms while one takes five seconds, the average may still look perfectly acceptable. For the user who experienced that five-second delay, however, the statistics are irrelevant—that delay defines their perception of the product. Because of this, we stopped treating average response time as our primary performance metric and began focusing on latency distribution instead:

Average: 110 ms

P50: 82 ms

P95: 430 ms

P99: 2100 ms

These numbers provide a much more honest view of system performance. P50 shows that half of all requests complete in about 82 ms. P95 means that 95% of requests finish within 430 ms, while roughly one out of every twenty requests is slower. P99 reveals the heavy tail: approximately one request out of every hundred takes more than two seconds. That heavy tail is almost completely hidden when looking only at the average. After that realization, the way we investigated performance issues changed as well. Previously, we asked: “Why did the average response time increase?” Today, we usually begin with a different question: “What happened to p95, and which operations ended up in the slow tail?” The answer is not always related to the API code itself. Sometimes the delay comes from an external integration. Sometimes a background job holds a lock longer than expected. In other cases, only one specific request type becomes slow while everything else continues to perform normally. The average response time blends all of these scenarios into a single number. Percentiles reveal that the problem affects a particular workflow rather than the entire system. However, p95 alone isn’t enough. To understand the root cause of latency, we began attaching technical context to every slow operation:

{ "operation": "CreateOrder", "duration_ms": 2187, "database_ms": 180, "external_api_ms": 1760, "retry_count": 1, "queue_wait_ms": 140 }

This log shows much more than the total duration of 2,187 ms. It reveals exactly where the time was spent. In this example, the database performed relatively quickly, queue waiting time was minimal, and most of the delay came from an external API. Without breaking the timing down by component, we would only know that CreateOrder was slow and might waste time optimizing the wrong part of the system. Over time, it became clear that performance cannot be evaluated using a single metric. Today, we typically monitor several indicators together:

P50 — the typical response time.
P95 — latency experienced regularly by a meaningful percentage of users. P99 — rare but most expensive performance outliers.

Error Rate — the percentage of operations that fail.
Retry Rate — the percentage of operations that require retries.
For example, a stable P50 combined with a sharp increase in P95 often means that most requests remain healthy while one dependent component experiences intermittent slowdowns. An increasing Retry Rate with unchanged database latency may indicate problems with an external service or the network. A high P99 while P95 remains stable can reveal rare edge cases that are almost impossible to discover during conventional testing. This perspective also changed how we design new features. Today, it is no longer enough to verify that an operation performs well on average. We also try to understand: which external dependencies it relies on; where retries might occur; how long requests could spend waiting in queues;
what diagnostic context will be needed if a production issue occurs.
Good observability does not make a system faster by itself.
But it allows engineering teams to understand why the system became slower much more quickly.
We apply the same philosophy when designing internal platform components.
A well-designed architecture is not only about implementing correct business logic—it is also about making it easy to identify exactly where a problem occurred within a workflow.
Earlier, we explained why internal tools deserve the same architectural attention as customer-facing products:
https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia
This article also continues our first engineering note about how our product catalog architecture evolved as the platform’s requirements grew:
https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

**Conclusion
**Average response time remains a useful metric, but it rarely reflects what users actually experience.
A system can appear fast on average while consistently producing noticeable delays for a subset of requests.
If the goal is to build a platform that is not only fast but also consistently reliable, it is essential to analyze latency distribution and understand the causes behind the slow tail—not just the “average request.”
That is why, when evaluating the performance of a new feature today, we first examine P95 and P99, review Error Rate and Retry Rate, and only then look at the average response time.

Many of the most important business breakthroughs rarely emerge during strategic planning sessions or after hours of meetings.

More often, they happen unexpectedly.

An unusual customer question.

A surprising pattern in customer behavior.

A mistake that leads to a better way of solving a problem.

A conversation between people from different departments.

Or an observation that initially seemed too insignificant to matter.

If you look at the history of many successful companies, one pattern becomes clear: their best ideas often were not planned. They emerged from unexpected intersections of people, experience, and data.

As companies grow, however, these moments become increasingly rare.

It is not because employees become less creative.

The real issue is different—the organization itself gradually becomes too predictable.

Each department focuses only on its own area of responsibility.

Every meeting follows a predefined agenda.

Every process becomes carefully standardized.

Every metric belongs to a specific team.

On one hand, this improves efficiency and helps the business scale.

On the other, it reduces the opportunities for people from different disciplines to encounter each other’s ideas by chance.

Yet it is precisely at the intersection of different perspectives that some of the most valuable innovations emerge.

When departments operate in isolation, companies become increasingly effective at solving familiar problems.

But they become less likely to recognize new opportunities.

That is why many technology companies intentionally create environments where people from different teams interact on a regular basis. Not because it is a fashionable management trend, but because innovation rarely emerges within a single function. More often, it appears where different knowledge, experiences, and perspectives intersect.

This is especially evident in eCommerce.

Sometimes an important improvement begins not with an analytical report, but with a question from a customer support representative.

Not with an executive meeting, but with an observation made by a warehouse employee.

Not with a new technology, but with an unexpected pattern in the behavior of a handful of customers.

Companies that recognize these signals early—and connect them before everyone else does—often gain a competitive advantage long before the market realizes that a new trend is emerging.

As Droplox has evolved, we have increasingly come to believe that great decisions are driven by more than simply having large volumes of data. Equally important is creating an environment where information, processes, and teams do not exist in isolation, but instead help reveal the broader picture of how the business operates.

That is why modern digital infrastructure is about more than automating operations.

It is also about connecting data, processes, and people in ways that create new relationships—and new ideas.

We explored this topic in more detail here:

https://droplox0.wordpress.com/2026/07/26/what-is-digital-commerce-infrastructure-and-why-is-the-future-of-e-commerce-built-on-connected-platforms/

This idea also aligns closely with our engineering article on why internal tools deserve the same architectural attention as customer-facing products:

https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

Ultimately, a company’s ability to grow depends on more than how well it executes established processes.

It also depends on preserving its ability to notice what nobody was intentionally looking for.

Perhaps these serendipitous discoveries remain one of the most overlooked sources of long-term competitive advantage.

At the beginning of almost every application, API controllers are remarkably simple.

They receive an HTTP request, validate the input, invoke a few methods, persist changes to the database, and return a response to the client.

At this stage, everything feels perfectly reasonable. The controller seems like the natural place for this kind of logic.

But as the product evolves, this approach gradually begins creating problems.

If you haven’t read the previous article in our engineering series, it provides useful context for understanding how our architectural thinking at Droplox has evolved over time:

https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

How a Controller Gradually Becomes the Center of the Entire System

As the platform grows, every new business workflow introduces additional requirements.

We need to:

Verify user permissions.
Ensure an order is in a valid state.
Update inventory levels.
Record an audit trail.
Publish events to other services.
Perform additional validation.
Handle failures gracefully.

Individually, each responsibility seems small.

And almost every time, the thought is the same:

“It’s easier to add just a few more lines to the controller.”

Then it happens again.

And again.

Eventually, a controller that once contained only a few dozen lines becomes one of the most complicated files in the entire project.

But size is not the real problem.

The real issue begins when a single component becomes responsible for far too much.

When One Class Knows Too Much

Over time, the controller stops being just an entry point.

Instead, it begins to:

Receive HTTP requests.
Validate incoming data.
Execute business rules.
Interact with the database.
Communicate with external services.
Build responses for clients.

Every new feature touches multiple layers of the application.

As a result, code reviews become more difficult.

Testing requires more effort.

Reusing existing business logic becomes increasingly challenging.

Worst of all, business processes begin depending on the behavior of a specific HTTP endpoint.

That was the moment we realized the problem wasn’t controllers themselves.

The problem was the amount of responsibility we had gradually assigned to them.

The Question That Changed Our Approach

At one point, we asked ourselves a very simple question.

If REST APIs disappeared tomorrow and were replaced by another interface, should our business logic have to change?

The answer was obvious.

No.

Business processes should never depend on how a command reaches the system.

REST.

GraphQL.

A message queue.

A CLI.

A background job.

These are simply different ways of delivering commands to the application.

The domain rules should remain exactly the same regardless of the transport mechanism.

That realization led us to gradually move business logic out of controllers and into dedicated components responsible for executing specific use cases.

The Controller Returned to Doing Only Its Job

After the refactoring, the controller’s responsibility became dramatically simpler.

Today, its job consists of only three steps:

Receive the request.
Delegate it to the appropriate use case.
Return the result to the client.

Almost everything else happens outside the HTTP layer.

The controller no longer makes business decisions.

It simply connects the external interface to the application’s internal logic.

At first glance, the change appears relatively small.

In reality, its impact reached much further.

More Than the Code Changed

As the architecture evolved, so did the way our engineering team discussed software.

Previously, we often asked:

“What else needs to be added to this controller?”

Today, our conversations are completely different.

Instead, we ask:

Which business process does this use case represent?
Where should responsibility for this business rule belong?
Can this use case be reused elsewhere in the platform?

At first glance, the difference may seem purely semantic.

In practice, it fundamentally changes how architecture is designed.

The team stops thinking in terms of HTTP requests.

Instead, we begin thinking in terms of business processes.

What Changed in Practice

Once responsibilities were separated, many aspects of development became much simpler.

The same use cases can now be reused throughout different parts of the application.

Business logic is no longer tied to REST APIs.

Testing became easier because validating business rules no longer requires spinning up HTTP infrastructure.

When a new communication channel is introduced, we don’t duplicate existing logic—we simply reuse it.

The result is a far more flexible architecture that adapts much more easily to change.

This Principle Turned Out to Be Valuable Beyond APIs

Later, we applied the same thinking to other parts of the platform.

For example, while designing our internal tools.

Over time, it became increasingly clear that architectural quality is not determined by the number of modern technologies or fashionable design patterns.

What matters far more is understanding which component owns which responsibility—and where the boundaries between architectural layers should exist.

We explored this topic in greater depth in another engineering article:

https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

A Controller Is an Entry Point—Not the Center of Business Logic

Today, we view controllers very differently.

They should not make business decisions.

They should not contain domain rules.

They should not understand the application’s internal implementation.

Their responsibility is much simpler:

Receive a request.

Delegate it to the appropriate use case.

Return the response.

The less a controller knows about business processes, the easier the system becomes to maintain, test, and evolve over time.

The Question We Ask Ourselves Today

Whenever we design a new capability, we ask ourselves the same question.

If REST APIs disappeared tomorrow, how much business logic would we need to rewrite?

If the answer is:

“Almost all of it,”

then our architecture is still too tightly coupled to the transport layer.

If the answer is:

“Almost none,”

then responsibilities have been separated correctly.

That is exactly what we strive for today.

Conclusion

After years of building software, we arrived at a somewhat unexpected conclusion.

The best API controllers are usually quite boring.

They contain very little business logic.

They make almost no business decisions.

They know almost nothing about the application’s internal processes.

And that is precisely what makes them so valuable.

When controllers focus only on their intended responsibility, applications become easier to test, simpler to maintain, and safer to evolve as new requirements emerge.

Ultimately, great software architecture is built around business processes, not HTTP requests.

Everything else is simply a way of interacting with the system.

⸻

Further Reading

How We Realized Our Initial Product Catalog Model Was No Longer Enough
https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

Why Internal Tools Deserve the Same Thoughtful Architecture as Customer-Facing Products
https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

Over the past few years, businesses have become remarkably good at collecting data. Online stores monitor sales in real time, marketing platforms track dozens of performance metrics, analytics tools generate increasingly sophisticated reports, and new dashboards appear faster than teams can review them.

It seems logical that the more information a business has, the easier it should be to make the right decisions.

In reality, however, the opposite is happening more often than ever.

The problem is rarely a lack of data.

More often, businesses are overwhelmed by having too much of it.

More Information Doesn’t Automatically Mean More Understanding

Most modern platforms can measure almost everything.

Website traffic.
Conversion rates.
Customer acquisition costs.
Average order value.
Order processing time.
Return rates.
And dozens of other metrics.

Yet not every number actually helps run a business.

Many metrics exist simply because they are easy to collect. They look impressive on dashboards but rarely answer the most important question:

What should we do next?

As a result, executives spend hours reviewing charts and reports when only a handful of truly meaningful signals are needed to make an informed decision.

When Every Department Has Its Own Version of the Truth

As organizations grow, another challenge emerges.

Different teams begin looking at the business through different metrics.

Marketing focuses on customer acquisition costs.

Sales tracks the number of orders.

Operations measures fulfillment speed.

Finance monitors expenses and profitability.

Each of these metrics is important.

However, if they are not connected by a shared understanding of the business, the company gradually loses its overall perspective.

That is why more organizations are focusing not only on collecting information but also on organizing it effectively.

We explored this topic in greater detail in our article about Master Data Management (MDM)—an approach that helps maintain consistent, accurate, and reliable data across an entire organization.

Large Volumes of Data Cannot Replace a Single Source of Truth

Another common problem appears when the same information differs across multiple systems.

For example, the sales report shows one number.

The CRM shows another.

The financial system reports something entirely different.

Instead of discussing what action to take, the team ends up debating which dataset can actually be trusted.

Hours—or even days—can be lost this way.

That is why the concept of a Single Source of Truth (SSOT) has become increasingly important for modern organizations.

When everyone works from the same trusted data, discussions become shorter, alignment improves, and decisions are made much faster.

Measuring Everything Rarely Leads to Better Decisions

At first glance, it seems that the more metrics a company tracks, the better it manages its business.

In reality, an excessive number of metrics often produces the opposite result.

Teams become preoccupied with minor fluctuations in individual numbers while gradually losing sight of broader trends.

Instead of focusing on the changes that truly matter, attention becomes scattered across dozens of secondary indicators.

There is more monitoring.

But not necessarily more understanding.

The Best Companies Are Not Defined by the Number of Reports They Generate

Companies that consistently make decisions faster than their competitors rarely succeed because they have more analytics tools.

Their advantage usually comes from something much simpler.

They deliberately identify a limited number of key performance indicators that genuinely influence business outcomes.

The remaining information does not disappear.

It remains available whenever deeper analysis is needed, but it does not distract the team from the signals that matter most.

That is why the speed of decision-making is increasingly becoming a competitive advantage in its own right.

How This Relates to Droplox
This philosophy shapes the way we build Droplox.

Our goal is not to overwhelm users with as many dashboards and reports as possible.

Instead, we focus on helping them quickly recognize the changes that genuinely require attention and providing clear, actionable information that supports better decisions.

That is why AI Advisor within Droplox functions as a Decision Support System (DSS). It analyzes business data, identifies meaningful patterns, and highlights potentially important changes while leaving every final decision entirely under the user’s control.

We believe great analytics should reduce information overload—not create it.

If understanding the current situation requires opening dozens of browser tabs and comparing multiple reports, the problem is probably not a lack of information.

It is how that information has been organized.

Conclusion

In the years ahead, the companies that succeed will not necessarily be the ones collecting the most data.

The real advantage will belong to those that learn to identify which data truly matters—and understand it quickly.

When information stops being an end in itself and becomes a tool for decision-making, businesses respond to change faster, allocate resources more effectively, and move forward with greater confidence.

Data alone does not create value.

Value is created when the right information reaches the right person at the right time.

⸻

Further Reading

What Is Master Data Management (MDM), and Why Is Data Becoming E-Commerce’s Most Valuable Asset?
https://medium.com/@patrik_kramer_vp/what-is-master-data-management-mdm-and-why-is-data-becoming-e-commerces-most-valuable-asset-4555bb4e8940

What Is a Single Source of Truth (SSOT), and Why Does It Matter in E-Commerce?
https://medium.com/@droplox/what-is-a-single-source-of-truth-ssot-and-why-does-it-matter-in-e-commerce-4512ccb6a381

Droplox: Decision-Making Speed Is Becoming E-Commerce’s Biggest Competitive Advantage
https://app.qwoted.com/press_releases/droplox-decision-making-speed-is-becoming-e-commerce-s-biggest-competitive-advantage

Why the Same Data Stored Across Multiple Systems Almost Always Falls Out of Sync
https://dev.to/droplox/why-the-same-data-stored-across-multiple-systems-almost-always-falls-out-of-sync-j87

Official Droplox Website
https://droplox.com

In our previous engineering notes, we shared how our thinking about the product catalog architecture evolved over time. First, it became clear that the original model could no longer support the growing requirements of the platform. Then we completely redesigned how product variants were represented. But as we continued designing the system, another question kept coming up—one that initially seemed surprisingly simple:

What is an order, really?

If you haven’t read the first article in this series yet, we recommend starting there. It explains how our approach to domain modeling gradually evolved as Droplox grew:

https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

At First, Everything Seemed Straightforward

During the early stages of development, the order model felt perfectly natural.

It contained a list of products, customer information, shipping details, payment status, timestamps, and a handful of other required fields. For the first version of the platform, that was more than enough.

As the product evolved, however, we encountered scenarios that no longer fit comfortably into such a simple model.

A payment could succeed—or fail.

A single order could be fulfilled in multiple shipments.

A return might involve only some of the purchased items.

Inventory reservation happened at one point in time, picking and packing happened later, while delivery updates arrived independently from external logistics providers.

Each of these scenarios was completely normal on its own.

The problem was something else.

We were still trying to treat everything as one object.

When One Model Starts Owning Everything

Over time, the Order entity gradually became the place where numerous unrelated business processes intersected.

Every change began affecting multiple workflows simultaneously. Even the smallest update had to be verified against payment processing, shipping, returns, inventory reservation, and many other parts of the system—even when those processes had very little to do with one another.

Eventually, we noticed something interesting.

The conversations within the engineering team had started changing.

Instead of asking:

“Where should we add another field?”

We increasingly found ourselves asking:

“Which business process actually owns this information?”

That question fundamentally changed how we viewed the order module.

An Order Turned Out to Be a Collection of Processes

Gradually, we realized that an order should not be viewed as a single business event.

Instead, it represents a collection of independent processes, each following its own lifecycle.

Payments have their own workflow.

Fulfillment follows different business rules.

Shipping evolves independently.

Returns form an entirely separate business process.

They are connected by the same customer order, but that does not make them the same responsibility.

When all of this is forced into one object, that object slowly becomes responsible for far too much.

Instead of Expanding the Order Model, We Started Separating Responsibilities

From that point forward, we stopped continuously expanding the Order model and began moving responsibilities into dedicated components.

Each service became responsible only for the information it truly owned and only for the business logic that belonged to its specific process.

This did not necessarily make the architecture simpler.

It made it significantly clearer.

Changes no longer produced unexpected side effects in unrelated parts of the system. Code reviews became easier. Testing became more focused. Discussing new features became more productive. Predicting the impact of changes became much more manageable.

Instead of debating which new field belonged inside the Order entity, our discussions gradually shifted toward defining clear boundaries between services and designing how they should interact.
That was the moment we felt the architecture had finally started reflecting the actual business processes.

We Applied the Same Thinking Throughout the Platform

This wasn’t an idea limited to the order module.

As Droplox evolved, we became increasingly convinced that the quality of internal services influences the long-term success of a product just as much as the features customers actually see.

That is why we invest significant effort into the architecture of our internal tooling.

We explored this philosophy in more detail here:

Why Internal Tools Deserve the Same Thoughtful Architecture as Customer-Facing Products

https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

The Biggest Lesson Went Far Beyond Orders

Looking back, we realize that the most important lesson had very little to do with the Order module itself.

It was really about domain modeling as a whole.

It is remarkably easy to think of any business process as a static object with a collection of fields.

Real business processes rarely stay that simple.

They evolve at different speeds.

They develop their own business rules.

They integrate with different external systems.

Gradually, they become independent parts of the overall platform.

If the architecture continues hiding all of that complexity inside one large object, that object eventually becomes the bottleneck that slows down future development.

That is why, whenever we design a new module today, we almost always ask ourselves the same question:

If this process starts evolving independently six months from now, should it still belong to the same model?

There is no universal answer.

But asking that question helps us identify architectural problems long before they turn into technical debt.

Conclusion

At first glance, an order really does look like nothing more than a database record.

As a product evolves, however, it becomes clear that this single object actually represents numerous independent business processes, each following its own rules and lifecycle.

Recognizing that reality changed not only the architecture of our order module, but also the way we design the entire Droplox platform.

Today, we spend far less time asking which fields belong inside an existing model.

Instead, we focus on understanding where one business process ends and another begins.

Users rarely notice architectural decisions like these.

Yet they are often what allow a platform to evolve for years without allowing a single model to become the centre of the entire business logic.

⸻

Further Reading

How We Realized Our Initial Product Catalog Model Was No Longer Enough
https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

Why Internal Tools Deserve the Same Thoughtful Architecture as Customer-Facing Products
https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

In the first engineering note, we explained why the original Droplox product catalog model eventually stopped matching the needs of a growing platform. That redesign solved several immediate problems, but it also exposed another architectural issue that had been easy to overlook during the early stages of development. We had been treating product variants as if they were simply additional properties of the product itself.

If you haven’t read the first article, it provides useful background for this one:

How We Realized Our Initial Product Catalog Model Was No Longer Enough
https://droplox.hashnode.dev/how-we-realized-our-initial-product-catalog-model-was-no-longer-enough?utm_source=hashnode&utm_medium=feed

The original model worked… until it didn’t

At first, the decision felt completely natural. A product could have different colours, sizes or configurations, so storing everything inside one Product model kept the implementation simple. Queries were easy to write, the catalog remained understandable, and there were very few moving parts. For an early version of the platform, that approach worked exactly as intended.

The problems appeared gradually rather than all at once. One variant needed its own images. Another required different pricing rules. Some variants came from different suppliers, while others had independent inventory levels or operational states. None of these requirements looked significant on their own, so the obvious solution was always the same: add another property, introduce another condition or extend the existing model one more time.

Nothing seemed fundamentally wrong. But every small change made the Product model responsible for something new, and over time it became responsible for almost everything.

The real issue wasn’t complexity—it was ownership

Eventually we realised the problem wasn’t the amount of data stored inside the model. The real problem was that we were mixing responsibilities that evolved independently.

Instead of asking “Where should this new property live?”, we started asking a different question:

Who actually owns this information?

That question completely changed how we looked at the catalog.

A product represents something relatively stable: its identity, shared description and general information. A variant represents one specific sellable configuration with its own SKU, inventory, supplier, pricing and availability. Although they are closely connected, they don’t necessarily change for the same reasons or at the same speed.

Treating them as one object blurred those boundaries. Eventually the Product model became a place where unrelated concerns met simply because they happened to appear on the same page in the user interface.

Similar on the screen doesn’t mean identical in the system

From a customer’s perspective, variants feel like part of the product. Someone opens a product page, chooses a colour or size and completes the purchase without thinking about the underlying architecture.

The software, however, has different responsibilities.

A black T-shirt being out of stock shouldn’t affect the white one. One variant may have a different supplier, another shipping method or a temporary promotional price. Those aren’t product-level changes—they belong to a specific configuration.

Once we accepted that distinction, the solution became much clearer. Instead of continuing to grow one large Product model, we separated responsibilities into independent components that could evolve without constantly affecting one another.

The objective wasn’t to introduce more abstraction. It was to reduce unnecessary coupling.

Clearer boundaries changed the way we developed features

After responsibilities became more explicit, development became noticeably more predictable. Features that previously touched several unrelated parts of the catalog were now usually limited to a single component.

Code reviews became easier because ownership was obvious, testing became more focused, and developers spent less time worrying about unintended side effects.

Interestingly, almost none of these improvements were visible to users. The interface looked almost identical, and there wasn’t a major feature announcement associated with this work. What changed was the engineering experience behind the scenes.

The catalog became significantly easier to extend without constantly revisiting old assumptions.

We kept seeing the same pattern elsewhere

While redesigning other internal parts of Droplox, we noticed that this wasn’t really a story about product variants.

It was a story about boundaries.

Internal tools, administrative systems and domain models benefit from thoughtful architecture just as much as customer-facing features because both determine how quickly future changes can be delivered. We’ve written about that idea in more detail here:

Why Internal Tools Deserve the Same Thoughtful Architecture as Customer-Facing Products
https://dev.to/droplox/why-internal-tools-deserve-the-same-thoughtful-architecture-as-customer-facing-products-5cia

Software rarely becomes difficult because one class or one service is too complicated. More often, complexity appears because responsibilities slowly begin overlapping. One exception leads to another, another shortcut seems harmless, another property gets added, and eventually developers spend more time understanding the existing model than implementing new functionality.

One question changed our design discussions

Today, whenever we evaluate a new feature, one question appears surprisingly early:

If this requirement changes six months from now, which component should change with it?

If the answer involves several unrelated parts of the system, it’s usually a signal that the current design deserves another look before more code is written.

That question doesn’t automatically trigger refactoring, and sometimes the existing implementation is perfectly reasonable. But asking it consistently helps us identify architectural risks before they turn into long-term maintenance problems.

The redesign of our catalog influenced much more than one part of the platform. It changed how we think about ownership, responsibilities and long-term evolution across the entire system.

Final thoughts

Looking back, product variants turned out to be much more than a small detail inside an e-commerce catalog.

They taught us that the most valuable architectural improvements aren’t always new frameworks, new databases or more sophisticated abstractions. Sometimes the biggest improvement comes from recognising that two concepts which look almost identical from a user’s perspective actually evolve independently and therefore deserve different responsibilities inside the system.

Good architecture rarely attracts attention on its own. Its value becomes obvious only when the product continues growing without every new feature turning into another large engineering project.

Every product starts with simple decisions.

When we began building the product catalog for Droplox, a traditional data model seemed more than sufficient. A product had a name, description, price, and category—what more did we need for the first release?

At that stage, the approach worked exactly as expected.

It allowed us to move quickly, validate ideas, and avoid investing in a complex architecture before it was actually needed.

But as the platform evolved, we started noticing that the original model was gradually becoming less capable of supporting new requirements.

In hindsight, that was completely expected.

A Good Model Works… Until the Product Starts Growing

The first changes seemed perfectly ordinary.

We needed to support multiple variants of the same product.

Then came additional attributes.

After that, images tied to specific variants.

Later, supplier information, different product statuses, and other types of data that no longer fit comfortably into the original structure.

Each time, the easiest solution was to add another field.

That’s exactly how many projects evolve in their early stages.

Eventually, however, we noticed something important.

Every new requirement started affecting more parts of the system.

A seemingly small feature suddenly required changes across multiple modules.

That was our first real warning sign.

The Problem Wasn’t the Number of Fields

At first glance, a larger model doesn’t seem like a problem.

Add a few new properties, update the database schema, and continue building.

In practice, things turned out to be much more complicated.

When a single entity becomes responsible for everything, it gradually becomes too heavy.

Every modification begins to ripple through neighboring parts of the system.

Developers have to account for an increasing number of dependencies.

Testing takes longer.

The risk of unintentionally breaking stable functionality grows.

Most importantly, introducing new features becomes increasingly difficult.

At that point, we realized the issue was no longer individual fields.

The issue was the model itself.

We Stopped Treating a Product as a Single Object

Instead of continuing to expand one large entity, we decided to rethink how we viewed the catalog.

In reality, a product isn’t a single object.

It’s a collection of independent components connected by relationships.

There is the core product information.

There are product variants.

Images.

Attributes.

Supplier information.

And many other pieces of data that follow different rules and evolve at different rates.

Rather than placing everything inside one model, we began separating responsibilities into dedicated components.

Each component became responsible for only one area of the system.

As a result, the architecture became easier to understand and less sensitive to changes in neighboring modules.

It’s important to note that this isn’t a universal blueprint for every project.

Architecture always depends on the product’s specific requirements.

We’re simply sharing an approach that proved valuable during our own development process.

Product Development Became Easier in Unexpected Ways

The most interesting changes appeared after the refactoring.

We didn’t suddenly gain one major new feature.

Instead, we noticed dozens of small improvements.

Changes became localized.

When new requirements appeared, they usually affected a single component rather than the entire catalog.

The code became easier to read.

Testing became easier to organize.

Technical discussions inside the team became shorter because responsibilities were much clearer.

Sometimes those are the improvements that matter most.

Users rarely notice them directly.

But they make future development significantly easier.

We Stopped Trying to Predict Everything

Perhaps that became the biggest lesson of the entire process.

When designing software, it’s easy to fall into one of two extremes.
Either you try to anticipate every possible scenario years in advance.

Or you optimize exclusively for today’s requirements.

Both approaches have obvious drawbacks.

In the first case, the architecture becomes unnecessarily complex long before it’s needed.

In the second, the product quickly runs into its own limitations.

Over time, we adopted a more balanced mindset.

Instead of trying to predict the future, we focus on building systems that can evolve alongside the product.

This Lesson Extended Beyond the Catalog

After redesigning the catalog, we started asking ourselves the same question whenever we worked on a new feature.

“If the requirements change six months from now, how difficult will it be to adapt this solution?”

The answer wasn’t always encouraging.

But asking that question early helped us identify architectural limitations before they became development bottlenecks.

Eventually, this principle spread far beyond the catalog itself.

Today, we try to design every part of the platform with change in mind, treating evolution as an expected part of software development rather than an architectural failure.

What This Means for Users

Most users will never see the internal structure of a product catalog.

And that’s perfectly fine.

They don’t need to understand how the data is organized behind the scenes.

What they do experience is the result of those architectural decisions.

Good architecture makes it easier to introduce new features, resolve issues more quickly, and continue improving the product as it grows.

Great architecture is rarely visible on its own.

Its absence, however, eventually becomes noticeable to everyone.

Final Thoughts

The product catalog may seem like one of the most ordinary components of an e-commerce platform.

Behind that familiar interface, however, lies a series of architectural decisions that shape the future of the entire product.

While building Droplox, we learned a simple lesson.

Sometimes the best way to improve a system isn’t to keep adding new pieces.

Sometimes it’s to step back, rethink the underlying model, and build a foundation that’s ready to evolve.

Those decisions rarely appear in release notes.

But they’re often the reason a product can continue growing without turning every future update into a major architectural compromise.