Today, Priyendra Deshwal, Satyam Shekhar, Abhishek Rai, and I are thrilled to introduce you to NetSpring, the company we founded just over 18 months ago. Our mission is to help businesses everywhere become more agile through operational intelligence from data. We’ve been hard at work building the NetSpring Operational Intelligence Platform, and now we’re excited to show you what it can do for your business.
What is operational intelligence, and why do we believe it’s one of the biggest opportunities in data analytics? We’ll explain — starting with some background on how we got to where we are today.
In 2012, we started a company called ThoughtSpot. Our goal was to make business intelligence (BI) as easy as using Google Search. Today, ThoughtSpot is a leader in the BI space, used by hundreds of large businesses in every industry.
Over the years, as we worked with hundreds of enterprise customers, we became aware of an increasingly urgent need for a new kind of analytics: operational intelligence.
Operational intelligence (OI) is an emerging term that describes how raw event data created in the past seconds, minutes, or hours can be used to drive insights and make decisions. Operational intelligence is also about analyzing and influencing event patterns that lead to the final states of business entities. While business intelligence is strategic, historical reporting on the final states, OI is tactical, showing you what’s happening now and why — so you can respond to issues or seize opportunities before it’s too late. OI and BI are complementary — like the spinal cord and brain, together making up the central nervous system.
Operational intelligence is a new kind of analytics
When you look at its key differentiating features, it’s clear that operational intelligence represents a dramatically new kind of analytics:
- Operational: Rather than running as an offline process, it lives online where the data is created, and is integrated into the everyday operations of the business.
- Event-Oriented: Rather than working with summarized subsets of data, it treats every single business event as a first-class citizen, so users can dig deep into complex event flow patterns.
- Real-Time: Rather than looking at batch reports on stale data, it lets users interactively run low-latency queries on the freshest data as it’s streaming in.
- Continuous: Rather than pull-based, it’s push-based, always monitoring and sending alerts relevant to a user’s context and job function.
What’s driving the need for operational intelligence?
No matter what industry you look at, the pace of change is accelerating. The landscape is more competitive than ever, and those that can’t adapt their tactics on the fly are being pushed out by newcomers. Furthermore, with COVID-19 driving adoption of digital solutions and services, businesses suddenly have a massive amount of raw event data streaming in. From eCommerce marketplaces, product instrumentation, financial trading, IoT sensors, supply chain systems, and more sources, the volume and velocity of this data is only increasing.
As a result, every product and business team is looking for timely, high-fidelity, actionable insights from their event data. We call this actionable business observability. Companies want to avoid being blindsided by issues that should have been caught and fixed sooner, like customer experience degradations, API outages, risk exposure, or supply chain disruptions. Teams want to detect issues in the moment and dig into detailed event flows to quickly find the root cause. Then, they want to take immediate action or automate their response with AI.
This is not just about finding problems, but also leveraging new opportunities. Businesses want to be able to deliver dynamic pricing, personalized product experiences, and continuous customer engagement. They want a fine-grained understanding of usage patterns so they can proactively ensure that every customer is experiencing the best quality of service — and spot things their competitors miss.
From working with hundreds of business operations and product teams, we’ve observed that there is an unsolved need for operational intelligence at enterprises, which is becoming key to staying competitive — and even surviving.
Why is this an unsolved problem?
We’ve deeply studied the landscape of tools and looked at what businesses have done to get a pulse on their mission-critical, time-sensitive operations. Here’s what we found:
1. The tools that are out there don’t work for OI
In the majority of companies, we’ve seen at best a hodgepodge of use-case specific systems with a very high total cost of ownership, siloed analytics, and poor scalability. These “solutions” break down into the following categories:
- Machine-data oriented infrastructure monitoring, log analysis, and time-series specific tools, repurposed to monitor business metrics. They can’t bring in rich context from multiple business systems, or do the advanced event flow and dimensional slice-and-dice analytics needed for business metrics.
- BI tools on top of plain-SQL databases. Since they’re fundamentally designed for offline reporting on static data, BI tools have performance and modeling issues when incorporating event data.
- Product analytics tools that provide basic reporting templates. These canned templates aren’t flexible enough to incorporate business context, do advanced ad-hoc analytics, or scale to a high data volume.
- OLAP cubing systems that provide query acceleration with pre-computations. They have a high cost of building and managing hundreds of cubes and lack full analytical capabilities such as complex schema modeling, continuous queries, full joins, and event flow analysis.
- Stream processing platforms that are good for coding programs that read small batches of data from a stream and write to persistent stores. They do not scale as generic, ad-hoc, converged streaming and batch query processing systems.
2. To build adequate tooling, there are immense technical challenges
Occasionally, we encountered a company that had cobbled together dozens of open source tools to build a colossally complex home-grown system for OI. Even with a large engineering team, these projects rarely lasted more than a year or two in production, highlighting the many technical challenges of building and maintaining OI at scale.
First, there’s the challenge of processing event data as it streams in rather than with a batch, offline approach. You need to be able to handle streaming constructs such as exactly-once semantics, watermarks, transactional guarantees, out-of-order arrival, late arrival, updates, etc. — and do it all at high volume and high speed.
To incorporate business context, you then need to be able to combine streaming event data with static historical data. You must connect to a variety of data sources and query across them continuously with high performance. You need to support joins fully, with guarantees of data consistency across sources. On top of this, you have to be able to easily model the data in a way that expresses temporal metrics, event flows, and advanced business rules.
Finally, to make the data actionable with monitoring and alerting, you need to build and operationalize ML models, which can take months or quarters. You need to be able to deliver sub-second alerting with low noise. And you need to seamlessly integrate with collaboration tools and transactional business applications.
The solution: The NetSpring Operational Intelligence Platform
Putting together a world-class team of seasoned data analytics experts, we set out to solve the problem of operational intelligence. Today we’re excited to unveil the NetSpring Operational Intelligence Platform.
The NetSpring Operational Intelligence Platform is a unified operational intelligence “command center” for product managers, business/operations analysts, customer success/support managers, and data and application engineers. The platform gives users the tools to deeply understand every product interaction, customer experience, process flow, or data pipeline. With ML-driven monitoring and alerting on arbitrarily complex business metrics, you can instantly detect, analyze, and act on every anomaly or opportunity.
We’ve built an integrated data and application platform providing an end-to-end solution that abstracts complexity away from the user. It provides seamless data connectivity and caching for any static or streaming data source, without the need for job or cluster management. We’ve developed a new analytics language called NetScript that provides a higher level of abstraction, composability, and reusability than SQL to provide rich expressibility of complex analytic constructs. We provide business users a no-code development framework for self-service application composition using a library of generic and use-case specific templates.
The platform is purpose-built from scratch with a modern cloud-native architecture optimized for scale, performance, and cost. Our Converged Analytical Processing (CAP) technology brings about a true convergence of streaming and batch query processing, with integrated, optimized storage. It’s designed for streaming ingestion, efficient incremental computations, event flow computations, time travel, and continuous SQL queries — at extreme scale. Our Relational Event Streams technology enables complex event flow analytics on top of the familiar and powerful relational model, combining the best of a relational database and specialized event analytics tools.
Try it for yourself
While we’re early in our journey as a company, our platform is already in production and adding value at Fortune 500 companies. We’re ready to show you what it can do for your business. If you have mission-critical workflows that demand time-critical insights, lack the right level of visibility and actionability around your business, and are looking to gain a competitive edge with a modern cloud data intelligence stack, we are confident you will find value in NetSpring.
Adopting the NetSpring Operational Intelligence Platform is very simple. You can be up and running in an hour, and you can build out a production-worthy operational intelligence application in a day. Contact us at email@example.com or register here to learn more.
NetSpring and the modern data intelligence stack
We’ve been pushing the boundaries of what an analytics platform can do, and we have a lot of exciting work ahead of us. Our mission is to empower every enterprise to reach peak operational efficiency and profitability. Our goal is to serve as the go-to platform in enterprises for all operational business insights from any data.
We have a bold vision that some years from now, if you look at the data analytics infrastructure at any major enterprise, you will see three key things powering a large class of core analytics workloads: a streaming pipe like Apache Kafka, a cloud data lake or warehouse, and the NetSpring Operational Intelligence Platform. We call this the modern data intelligence stack. We strongly believe that this will result in massive simplification, reduced cost, and increased analytic sophistication at enterprises, making a huge impact on business agility.
Every decade or so, a new generation of analytics is born. We are at the forefront of ushering in the next one. We invite you and your business to work with us. Let’s make history together!