Part II of The Art of Data War — Knowing Your Position, PRAISE Framework. Previously:
“The consummate leader cultivates the moral law, and strictly adheres to method and discipline; thus it is in his power to control success.” ~ The Art of War, by Sun Tzu.
Most firms trust their data because they trust the engineers who built it. When a pipeline breaks, the person who wrote it fixes it, and for a while that works. You’ve made data a leadership priority, built the teams to carry it, accounted for what you have and who uses it, and put it to work in business decisions. But none of that matters if your systems break under load, quality issues go undetected until they corrupt decisions, or your best engineers burn out responding to the same failures repeatedly. This is where many organizations discover their Achilles’ heel: data platforms held together by heroic individuals rather than systematic processes. A few people cannot grow as fast as the business, and growth is where that shows. What holds is a system: pipelines built to one standard so that anyone can run them, quality checked before a decision depends on it, and an operations team that keeps the lights on 24 X 7.
Operations & Reliability, the Stabilize dimension of PRAISE, addresses whether your data infrastructure can sustain growth without constant firefighting: whether it operates reliably at scale, catches problems before they cascade, and frees your talented teams to innovate rather than firefight.
Uber — From Wild West to Systematic Scale
By 2025 Uber’s data platform served about 189 million active customers and more than 7 million active drivers across more than 15,000 cities in over 70 countries, and held some 350 petabytes of data. During its early years of rapid growth that data infrastructure was built team by team: silos grew out of necessity, quality came with no guarantee, and the company hit a reliability wall.
During Uber’s early growth, the company onboarded overlapping data workflow systems: Apache Oozie, Airflow, Jenkins, custom Python and Clojure scripts. Users chose from several tools for each use case. While this allowed agile growth, engineers had to learn several systems that did the same job, and every one of them had to be maintained, debugged and taught to its users. This is how custom solutions become tribal knowledge concentrated in individuals. In 2017 it led to Piper, Uber’s centralized data orchestration system built on Apache Airflow but re-architected for Uber’s scale. Piper replaced the wild west with unified standards managed by a dedicated team. By December 2023 it supported 1,000 teams and 200,000 active pipelines.
Uber built integrated systems to catch quality issues proactively, before the data is consumed downstream. Databook, which began as a set of static catalogue pages, was rebuilt around a metadata graph. The Unified Data Quality platform (UDQ) auto-generates standard quality tests from each dataset’s metadata and shows the results in Databook. By 2021 it supported over 2,000 critical datasets and had detected around 90% of data quality incidents. Schema changes are reviewed and signed off before they are deployed. The checks did not catch everything: a fares data incident affecting ML models went undetected for 45 days. Uber answered with D3, a framework that automates anomaly detection across more than 300 of its most critical datasets, and reports that average time-to-detection fell to 2 days, more than a twentyfold improvement.
Uber started Site Reliability Engineering in 2014, when the load on its systems was outgrowing an engineering organization that doubled every six months. For its big-data jobs the company built DataCentral, which gives granular performance insights and automatically surfaces error messages with root cause suggestions, so that the owner of a failing job is told what went wrong and what to try. Tools like this rest on standardized telemetry that describes the state of the plant, the platform as it runs day to day, across thousands of datasets.
The Signals to Look For
Read the case back and it shows three things, in order. Piper put Uber’s pipelines on one standard. UDQ and D3 were built to check quality before a decision depends on it. DataCentral reads the error when a job fails and tells its operator what went wrong and what to try. The scale is Uber’s. The three things are what the signals below look for in your own organization.
1. Are pipelines built to one standard, so that anyone can run and repair them?
What to look for: Do your data flows run through a central platform, automated and standardized, not through ad hoc processes each team keeps to itself? Is manual work the exception? Does a new dataset follow a proven pattern, stored, accessed, versioned and monitored like every other? When a pipeline fails, can someone who did not build it find the fault and fix it?
A central data platform is a set of capabilities managed through common governance, automation, and standards while remaining open for usage and access. It does not have to be a single technology, or a single team controlling everything. The platform carries each dataset through its full lifecycle, from acquisition to distribution, and automation speeds mundane tasks like onboarding new datasets or setting up distribution channels. Standardization, in how data is stored, accessed, versioned, and monitored, makes management simpler without requiring institutional knowledge per dataset. It is what brings a new dataset in within hours, where a custom pipeline takes weeks. Imagine creating custom pipelines for hundreds or thousands of datasets versus following proven patterns. Standardized telemetry is also what lets AI take part in operations: a model can watch and act on thousands of pipelines only when they all report their state the same way.
Without a central platform, teams build custom solutions for every need. This problem commonly emerges in systematic hedge funds with engineering powerhouses. Each engineer tends to own the pipelines they brought in, so every pipeline has one owner. Solutions become tribal knowledge concentrated in individuals who become indispensable heroes. When these people leave, their systems become black boxes. Teams spend the lion’s share of their time on plumbing: acquiring, cleaning, and storing data, with little left for analysis. The organization accumulates technical debt faster than it can pay it down.
These custom solutions create cascading problems. Onboarding new datasets takes weeks instead of hours. Maintaining hundreds of custom pipelines becomes impossible: systems drift, break silently, or run indefinitely without anyone knowing if they’re still needed.
2. Are datasets qualified before a decision depends on them?
What to look for: Do automated quality checks catch issues before data reaches consumers, not in a post-mortem? Do datasets arrive complete per SLA before critical business moments? Does lineage tracking document each dataset’s sources, transformations and versions, so that a root cause can be found and a result reproduced? Are historical versions preserved for audit and post-decision analysis, so that an audit is routine and not a panic? Is quality logic transparent and crowd-sourced from users who understand their use cases?
Central platforms aren’t just about consolidation—they’re built to govern quality and distribution at scale with minimum effort. Datasets delivered without qualification pose risks to downstream consumers who discover issues too late. Quality check logic must be transparent to users, who know best what works for their use cases. The best rules come from two sources. Where datasets are recorded the same way, AI can draft a new one’s automated checks from similar datasets and from how the data is used. The people who use the data add what they know of their own downstream requirements. Understanding the original data product matters too: what inherent issues or characteristics does the source data carry? This is where the catalog from Account becomes critical: profiling and data representation reveal source data patterns, anomalies, and limitations that inform what quality checks and enrichments make sense downstream.
Two numbers show whether this is working: the share of quality incidents your own checks catch, and how long a bad dataset goes unnoticed. Quality isn’t just accuracy—it’s timeliness. Latency monitors track dataset arrival and completeness against SLAs. When critical feeds run late, alerts escalate before business impact: traders need overnight positions before the market opens, retailers need inventory before launching promotions.
Lineage isn’t a fancy term—it’s documenting the complete lifecycle of every data point from original sources through transformations to final consumers. Without lineage that tracks both data sources and generation logic, each under version control, tracing issues or evaluating existing datasets becomes impossible. When metrics suddenly shift, lineage reveals whether source data changed or transformation logic evolved. Auditors and regulators ask a firm to reproduce a result on the data it had at that moment. Datasets get restated as normal business: vendors correct errors, companies revise earnings. Maintaining original versions in a bi-temporal setting matters for regulatory audits and understanding decision quality: what did we know when we acted, versus what we learned later?
3. Is there a dedicated DataOps function maintaining trust 24 X 7?
What to look for: Is there a dedicated DataOps function with clear escalation protocols, where first responders have the tools to diagnose and intervene before escalating to data owners or infrastructure teams? Does standardized telemetry across all datasets enable normalized monitoring regardless of type, frequency, or size? Is incident response systematic, with root cause analysis and prevention, not heroic firefighting and reactive scrambling when issues surface?
Imagine software development without DevOps or application support: engineers writing code and personally responding to every production incident at 3 AM, no systematic deployment processes, no monitoring tools, no escalation protocols. Talented developers would flee to saner environments. Their expertise would be wasted on repetitive troubleshooting instead of building new capabilities. This is exactly what happens in data organizations without dedicated DataOps functions. Brilliant data scientists and engineers leave jobs because fire drills and 24 X 7 support consume their careers. The cost reaches the business too. One owner per pipeline cannot respond in time, and portfolio managers end up running models on late or broken data, in the worst cases at a loss. They chase a data team that does not answer, and the team’s standing falls with every incident. Engineers under that pressure onboard new datasets with the minimum of checks, which produces the next failure. DataOps provide systematic operational support far cheaper than expensive specialized talent wasted on firefighting, ensuring the plant runs healthy while both data owners and DataOps teams innovate in their respective domains.
Effective DataOps run on three or more tiers of support. In a common setup DataOps teams are the first responders, data owners are the second-level specialists, and behind them sit the platform and infrastructure specialists. DataOps teams need power and tools: health dashboards, drill-down interfaces, automated remediation for common issues, clear runbooks for intervention before reaching specialized knowledge. Where the pipelines look alike and the platform gives operators those tools, the first tier can be entrusted to existing support groups that are already watching trading systems around the clock. When DataOps can resolve 70-80% of issues without waking data owners, engineers stay focused and the organization scales without scaling headcount proportionally
The control room, and the three tiers of support behind it.
The central platform and automated quality checks from the earlier signals are prerequisites for scalable DataOps. Without them, operational support becomes impossible at scale. Innovation in DataOps emerges from systematic protocols built on this foundation. Automated SLA monitoring generates alerts when datasets run late or fail quality checks, triggering immediate investigation. Standardized telemetry and operational metadata describe plant state across thousands of datasets regardless of type, frequency, or size. This normalization lets monitoring and intervention tools work uniformly: the same dashboard shows batch equity data, real-time market feeds, and alternative web-scraped datasets. Without standardization, DataOps teams drown in bespoke monitoring systems, each requiring specialized knowledge. The same standardization lets AI take the first look. It can find the likely cause of a failure, propose the known fix, and enable the incident communication to vendors, specialists and users. There is art in making the whole ecosystem work together and for each other.
Now read your own organization. Are your pipelines built to a standard anyone can pick up? Is your data qualified before the decision is made? When a pipeline fails at 3 AM, is answering it someone’s job? Three yeses mean your data holds because of a system. Three noes mean it holds because of a few people, and a few people cannot grow as fast as the business. Start with the standard, then the checks, then the team: each depends on the one before.