200,000 downloads per month don’t happen everyday in the enterprise software universe, especially when you’re talking about a piece of infrastructure software enabled search in Big Data. ElasticSearch – based in Amsterdam – was putting up those numbers and growing even faster. Clearly, they were doing something special.
Big data is all the buzz, even to the point where Gartner is suggesting that we are headed for the “trough of disillusionment.” But, when you talk to customers about what they really want from their Big Data – they will tell you about the insights that can be unearthed from those volumes of data. Insight that helps sell products; that improves consumers’ experiences; that drives business strategies. For years we ignored the vast amount of data that was at our fingertips. With Hadoop, we now have a data architecture that allows us to collect, store, and organize that data. But, true value is created from those insights - those are the true gems of big data.
In the past few months, we’ve heard fascinating stories from many big data analytics companies. It’s probably one of the “hottest” segments in enterprise venture capital today. We loved many of the ideas, but didn’t invest any of them. Analytics are a tricky business because ultimately, the products end in two scenarios: (a) a skilled user to get the desired result from them; or (b) high degrees of structure that don’t answer the really big questions. We began to look elsewhere – what if you just wanted to “talk” to your data – sort of like how we have become accustomed to finding what we want as consumers from Google.
“Enterprise Search” is a sector that has been around for a while, but historically, companies have not gotten it quite right. It has been complex to implement and scale and generally pointed at finite data sets. What if talking to your data meant you could ask it complex questions? What if you could ask it questions in free text form? What if you could ask it a lot of question, very quickly, in succession? What if you could engage with a search engine with powerful and scalable APIs?
Shay Banon answered those questions when he wrote much of the original versions of ElasticSearch. The foundation of ElasticSearch is the Apache project – Lucene. It’s an amazing foundation and the most broadly adopted search software (aside from Google) in the world. But, Lucene is raw code. Shay took Lucene, and, developed server software around that made scalable and robust. He made it well-suited for the cloud. He aligned with today’s use-cases with Mongo, Hadoop, and much more. And, he instrumented it so that it would be much more than search. It would be the foundation of the way analytics would be implemented. After all, a graph is just a visual result of the questions you ask you data.
We were pretty excited when we saw what Shay and his team had built. But, we got even more excited when we saw the team around Shay. The Open Source world is a tight knit one. Many have a hard time believing that a sustainable business can be built on a foundation of software that can be freely read and used by anyone. Yet, there are a few teams who have successfully created great businesses from that Open Source foundation. At Index, we had the fortune of supporting MySQL in its journey to becoming both an enormously successful Open Source database and a great business. Another such team created SpringSource, which was acquired a few years back by VMware. ElasticSearch’s business team was assembled from SpringSource executives that had been to the show before. Steve Schuurman, Nick White, and others knew how to build a world-class business founded on Open Source technology.
Great technology, big market, terrific team – its no surprise that we are excited today to announce our investment in ElasticSearch. We haven’t seen momentum like this for an Open Source project in quite some time – which is quite exciting. But, perhaps even more so is the vision of ElasticSearch. Through its peaks and valleys, Big Data is here to stay. Business and consumer experiences will never be the same because we will have vast amounts of information to make these experiences better. But, that vision will only come to be when we can talk to our data the way we talk to our own minds. ElasticSearch is a huge leap ahead in that direction.