A second datastore has to earn its operating cost

Search and analytics can run over the same records and still be different workloads. Interactive search wants fast discovery over indexed fields. Analytics wants to aggregate, rank and scan many rows cheaply.

When those differ enough, giving each its own store makes sense:

search engine → interactive search / filtering
column store  → aggregation / ranking / reporting

The cost side

Every extra datastore brings:

• a pipeline to project data into its shape;
• schema changes that may need coordinating across projections;
• freshness that has to be stated, since the stores can lag each other;
• more monitoring, deployment and recovery work.

So the test is whether this workload boundary is worth its operating cost, not whether the second store is faster at something. On a solo project that question is very concrete, because I’m the one on call.

Things I try to avoid

• Replacing a component that already works, to solve a problem it doesn’t have.
• Telling the “X couldn’t cope, so we added Y” story when the real reason was fit and cost.
• Treating “fewer technologies” as the rule when two workloads clearly differ. It’s a good default, not a law.

Where I’ve applied it

• Elasticsearch for search, ClickHouse for analytics: kept Elasticsearch for search and added ClickHouse only for analytics.
• Reputation scoring over 64 million wallets: ClickHouse for aggregation and ranking over approximately 64 million scanned wallets.

Still unsure

When does the boundary stop being worth it? Small datasets, light reporting or a short-lived product all push towards one store. I don’t have a clean rule for the threshold yet.