Written by: Ben DeBow | Founder and CEO of Fortified | Author of End of Abundance in Tech
Blog Series: The Evolution of Technology’s Role in Business.
In part four of his blog series, The Evolution of Technology’s Role in Business, Ben DeBow exposes a costly oversight in how enterprises manage their IT infrastructure. For years, organizations have focused intensely on preventing downtime, investing heavily in monitoring and observability tools. However, Ben argues that this narrow focus on avoiding rare system failures has created an even bigger problem: a bloated ecosystem of redundant tools, underutilized data, and missed opportunities for optimization.
Through examples like GlobalTech and FinServ Corp, Ben reveals how businesses are spending millions on overlapping tools and unnecessary data retention, all while ignoring the potential to improve performance during the 99.995% of the time when everything is working fine. He calls for a major shift in how enterprises approach monitoring—advocating for an efficiency mindset that prioritizes the collection of relevant data and continuous system optimization. This, Ben suggests, is the key to not just cutting costs but driving real value through smarter, leaner IT operations.
Introduction
In the high-stakes world of enterprise IT, uptime is king. For decades, companies have poured billions into keeping their systems and applications running smoothly. But what if this laser focus on preventing downtime is actually causing us to miss the forest for the trees? What if our approach to monitoring and observability is not just inefficient, but fundamentally flawed?
Picture this: GlobalTech, a Fortune 500 company, proudly boasts a 99.995% uptime for their critical systems. It’s an impressive figure that translates to just 26 minutes of downtime per year. To achieve this, GlobalTech has invested millions into monitoring and observability tools – over 100 different solutions, capturing terabytes of data every day. The problem? Most of this data sits idle, only accessed during those rare moments of crisis or when the system is not performing well. It’s like buying an expensive insurance policy that you hope never to use, except you’re paying premiums every second of every day.
GlobalTech isn’t alone. Across industries, enterprises are caught in a cycle of over-monitoring and underutilizing. Take the case of FinServ Corp, a major financial services provider. In a recent audit, they discovered they were using five different tools to monitor the same set of database servers. Each tool was capturing identical metrics, storing them separately, and even alerting different teams about the same issues. The result? A bloated IT budget, confused response teams, thousands of non-actionable messages, and licensing and storage costs that made the CFO’s eyes water. FinServ Corp was effectively paying five times over for the same information, all in the name of “comprehensive” monitoring.
The Hidden Costs of Overlapping Monitoring Tools
But it’s not just about duplicate data collection. Many enterprises fall into what we might call the “redundancy trap.” They accumulate layers of overlapping tools, each added during different projects or by different teams, without a holistic view of their monitoring ecosystem. Consider Legacy Corp, a multinational retail corporation. Over the years, they had accumulated a legacy system monitoring tool from the 1990s, a “next-gen” application performance management (APM) suite added in the mid-2000s, a cloud-native observability platform adopted during their recent cloud migration, and various open-source tools championed by different development teams. Each of these systems was capturing data, some of it unique, but much of it redundant. The kicker? When a major outage hits, teams spent more time reconciling conflicting data from different tools than actually solving the problem.
The financial impact of this approach is staggering. A recent study by Datamation estimates that 84% of enterprises struggle with costs, complexity, and tools. This translates to millions, sometimes tens of millions, of dollars annually. But the cost isn’t just financial. There’s also increased system overhead from constant data collection, higher cognitive load on IT staff managing multiple tools, slower incident response times due to tool sprawl, and missed opportunities for optimization during normal operations.
Shifting the Focus: From Crisis Management to Continuous Optimization in Monitoring
So, what’s the solution? It’s time for a fundamental shift in how we approach monitoring and observability. Instead of obsessing over the 0.005% of the time when things go wrong, we need to focus on the 99.995% when everything is running smoothly. Imagine if enterprises, instead of just collecting data for crisis moments, used their monitoring tools to continuously optimize their systems during normal operations. They could identify and eliminate performance bottlenecks before they become critical, optimize resource allocation in real-time, potentially reducing infrastructure costs, and predict and prevent issues using AI and machine learning, rather than just reacting to them.
But to achieve this shift, we need to address a crucial oversight: the hidden costs of data retention. Everything in technology has a cost, both immediate and long-term. Is 1-year-old or even 3-month-old monitoring data truly valuable to your company? If so, what’s the relative value of this data for a non-production server versus a Tier 2 server versus a mission-critical server?
Too often, we approach data collection for monitoring with a one-size-fits-all mentality, storing data for years, capturing the same metrics across the enterprise and only differentiating at the alerting level. This approach ignores the vastly different importance and cost implications of various systems within our infrastructure. The reality is that around 90% of the systems, non-production and non-mission critical systems, in most companies can survive on just capturing 10 metrics or less, with retention periods as short as 48 hours or at most a week.
How GlobalTech Cut Costs and Streamlined Monitoring with a Tiered Approach
Let’s revisit GlobalTech. After an audit of their monitoring practices, they realized they were retaining a year’s worth of data for all their systems, regardless of importance. By implementing a tiered approach, they reduced data collection on 80% of their systems to just 8 key metrics with a 72-hour retention period. For their Tier 2 systems (about 15% of their infrastructure), they collected 25 metrics with a 1-month retention. Only for their most critical 5% of systems did they maintain comprehensive monitoring with extended retention. The result? A 70% reduction in data storage costs, a 50% decrease in the processing power required for data collection, and a significantly streamlined monitoring system that actually provided more actionable insights to their teams. These benefits also yield resource savings on the systems being monitored since we are polling less frequently and reducing the tools overlap on the data collection.
As technology adoption continues to accelerate and costs keep rising, we need to design our monitoring and observability systems with an efficiency mindset. This means critically evaluating each metric we collect, tailoring retention periods to the actual value of the data, regularly reviewing and optimizing our monitoring strategies, and considering the total cost of ownership – not just storage costs, but also the processing power required to collect and analyze the data, and the cognitive load on our teams.
The future of monitoring and observability isn’t about collecting more data—it’s about collecting the right data, in the right amounts, for the right duration. By adopting an efficiency mindset and a tiered approach to data collection and retention, companies can not only reduce costs but also improve the effectiveness of their monitoring systems.
Unlocking Real ROI: Focusing on Continuous Improvement Over Crisis Management in IT Monitoring
The billion-dollar blind spot in enterprise IT is not just the risk of downtime or the wasteful overspending on redundant monitoring – it’s the missed opportunity for continuous improvement during the 99.995% of the time when systems are running smoothly. By shifting our focus from crisis management to ongoing optimization, we unlock tremendous value. This approach not only reduces technology spending by eliminating unnecessary tools and data storage, but more importantly, it makes applications faster and more efficient.
Imagine the cumulative impact of small, continuous improvements made throughout the year, rather than just firefighting during those 26 minutes of downtime. This is where the real ROI lies – in the relentless pursuit of efficiency and performance during normal operations. As we navigate the ever-expanding technological landscape, remember not all systems are created equal, and neither should our monitoring strategies be. It’s time to stop the one-size-fits-all approach to monitoring and start building smarter, more efficient observability systems that drive continuous improvement. Your bottom line, your IT teams, and your end-users will all reap the benefits of this paradigm shift.