The Fast and the Furious: How ClickHouse, the World’s Fastest Open-Source Database, is Creating the First Real-Time Data Warehouse
Developing a database is a lot like building a company. At its core, a database is only as good as the way it’s built, the data people put into it, and the use cases built on top of it. Similarly, a company’s success boils down to its foundational principles, the people in charge, and the strategies put in place to meet market needs and adapt to changing environments.
ClickHouse creator Alexey Milovidov nailed the first part when he committed the first lines of code back in 2009. With his team of engineers, Milovidov had a vision to create a new kind of analytical database — one that was faster, more resource-efficient, and optimized to generate analytical reports from massive data scales updating in real-time. Launched in 2012 to power the world’s second-largest web analytics platform, ClickHouse was open-sourced in 2016 and quickly became one of the fastest-growing and most widely celebrated databases of all time.
Today, things look a little different for ClickHouse. While technical excellence remains a core part of the vision, the second part of the equation — turning a wildly popular open-source project into a successful company — has come into sharper focus. What began as an experiment is now its own fully independent company, with headquarters in the Bay Area and a large engineering hub in Amsterdam. Milovidov is joined by two co-founders, Aaron Katz and Yury Izrailevsky, who bring decades of go-to-market and product experience from companies like Salesforce, Netflix, and Google. They lead a globally diverse team of nearly 200 employees, united behind a common goal of providing industry-leading, real-time analytics for companies like Microsoft, Walmart, Cisco, Sony, Lyft, and many others.
“The foundational technological advantages of having gone cloud-first, combined with this highly performant, lightning-fast database, gives us a lot of excitement about where this company can go.”
—Mike Volpi, Index Ventures
In their first year as a company, the ClickHouse team built a managed cloud service in record time and released it to wide acclaim. Now with more than 1,000 paying customers and counting, their sights are set on capturing a market that Katz, the company’s CEO, describes as “limitless.” Given the array of use cases that ClickHouse powers — observability, real-time analytics, fraud detection, sentiment analysis, marketing, and sales analysis — not to mention the criticality of its infrastructure, the scalability of its database, and the unmatched speed and efficiency of the product — it’s hard not to agree with that assessment.
“The end-market opportunity is massive,” says Mike Volpi, partner at Index Ventures and an early investor in ClickHouse. “The foundational technological advantages of having gone cloud-first, combined with this highly performant, lightning-fast database, gives us a lot of excitement about where this company can go.”
And yet, turning a highly successful open-source project into a commercial entity, and doing so at record speed, has not been without its challenges. The story of ClickHouse is one of ambitious yet disciplined leadership, clear and direct communication, and a team-wide commitment to putting customers first at every step of the way.
A Company is Born
In the spring of 2021, Aaron Katz was thinking about what to do next. He had just spent six years as CRO at Elastic, helping the open-source startup go public in 2018. Before that, he spent 12 years at Salesforce, where again he saw the company grow from startup to IPO. On the heels of two transformative experiences, he found himself a free agent for the first time in decades. He did some growth advising and angel investing. He thought about starting a new company, weighing the time and risk involved with embarking on that process in his mid-forties.
Then one day, a simple introductory call with Yandex started him down a path nobody could have predicted. Yandex was looking for advice on what to do with a database management system called ClickHouse, which had been developed inside the company more than 10 years earlier. Katz was aware of it from his time at Elastic, and he knew it had been growing in popularity since being released as an open-source project. He suggested to Yandex that if they were willing to contribute ClickHouse to a company to be formed by Katz, he would arrange financing and run the company, following a playbook used by companies like Confluent and Hortonworks.
Around the same time, Katz reached out to his friend Mike Volpi at Index Ventures. The two had worked together at Elastic — Volpi was instrumental in recruiting Katz out of Salesforce — and kept in touch during COVID, including discussing different opportunities in Volpi’s portfolio should Katz decide to step back into an operating role. As Katz began negotiations with Yandex, he asked Volpi for his thoughts on the potential deal — including whether Index might be interested in joining as an investor.
“I don’t think I had to complete the sentence,” Katz says with a laugh.
Volpi had known of ClickHouse for several years, and had seen it gaining in popularity with everyone from the Fortune 20 to Index’s Series A portfolio companies. In fact, seeing the product’s mass adoption, he had briefly explored his own deal with Yandex. When Katz told him he was working on a deal to extract ClickHouse from Yandex, he could see right away that Katz — someone who as CRO at Elastic had deep experience commercializing an open-source product; who had been through the exact revenue journey, taking it from $15 million to more than $500 million in sales; and who knew the entire playbook, from who to recruit to how to position and sell the product, to how to turn free users into paid users — was the perfect person to lead the new business.
“I kept thinking, if I could get a third co-founder to run product and engineering, and if I could find someone with the same level of experience building distributed systems on open-source that I have on go-to-market, it could be a really compelling combination.”
—Aaron Katz, ClickHouse
Joined by Peter Fenton from Benchmark, Katz and Volpi got to work negotiating an ownership structure with Yandex. Volpi recalls the negotiations as “interestingly complex.” While Yandex wanted to maintain control over ClickHouse as a subsidiary, Katz was clear from the outset that the founders and their investors would collectively require majority control over the company, and it would need to be operated as an independent, venture-backed Delaware corporation. Katz also stipulated that ClickHouse creator Alexey Milovidov and his “brilliant” team of a dozen engineers leave Yandex to join the new company, trade in their liquid RSUs for early equity in the newly formed company, and relocate to Amsterdam.
While negotiations were still ongoing, Katz decided they needed a third co-founder. “I kept thinking, if I could get a third co-founder to run product and engineering, and if I could find someone with the same level of experience building distributed systems on open-source that I have on go-to-market, it could be a really compelling combination,” Katz says.
The first person he thought of was Yury Izrailevsky. The two had been introduced by Volpi several years earlier at an open-source event hosted by Index Ventures, when Izrailevsky was leading engineering at Netflix. Izrailevsky had since moved on to Google, where as VP of Engineering he was responsible for a large portfolio of cloud products and developer tools, with a team of more than a thousand engineers. When Katz told Volpi about his plans to pry Izrailevsky away from Google, Volpi replied, “Brilliant idea, but good luck.”
Katz went ahead and made his pitch. He describes Izrailevsky’s initial response as a polite “Strong no.” And yet, Katz saw a glimmer in Izrailevsky’s eye. It was enough to give him the confidence to keep selling him on the idea — and to recruit Volpi and Fenton to do the same.
Izrailevsky knew of ClickHouse from his younger brother, who worked at a leading technology company that had been trying to evolve its analytics platform and big data architecture, but the systems it had weren’t scaling. They held a bake-off to explore other technologies, with ClickHouse coming out as the “runaway winner.” After Katz reached out with his proposal of turning ClickHouse into a managed cloud service, Izrailevsky downloaded the open-source version to his laptop and played around with it himself.
“I was blown away,” Izrailevsky says. “It was so fast and so slick and so easy to work with.”
“The more I thought about this idea of doing ClickHouse with Aaron and Alexey, the more I was like, 'This is it. This is the opportunity.'”
—Yury Izrailevsky, ClickHouse
While Izrailevsky wasn’t eager to leave what he calls an “amazing” job at Google, he saw a unique opportunity to be part of “something special,” and to fulfill a lifelong dream. After beginning his career working in startups, he had spent 20 years at large companies like Yahoo!, Netflix, and Google. He missed the scrappy, dynamic startup environment. Deep in his heart, he had always known that one day he wanted to build a company on his own.
“The more I thought about this idea of doing ClickHouse with Aaron and Alexey, the more I was like, ‘This is it. This is the opportunity,’” he says.
In the meantime, negotiations continued. While the parties took turns walking away from the table, Volpi says one thing that kept bringing him back was talking to ClickHouse users as part of his due diligence. He describes “glowing review after glowing review” with users raving about ClickHouse’s speed and ease of use. Even as talks dragged on for months, his confidence in the deal kept growing. Katz felt similarly, and says that despite the group’s individual differences, they shared a persistence and a desire to get a deal done.
Finally, they found a structure that worked. In August of 2021, they announced the incorporation of ClickHouse, Inc., along with $50 million in Series A funding led by Index Ventures and Benchmark. Barely two months later, they raised another $250 million at a $2 billion valuation. Evidently, the world was eager to see what would happen when you took a hugely popular open-source project like ClickHouse and made it available in the cloud.
Ascending to the Cloud
When Tanya Bragin joined ClickHouse as VP Product in January of 2022, she knew what she was getting into. As a senior product leader at Elastic, where she worked closely with Katz, Bragin had seen many workloads leave for ClickHouse, which was quickly becoming known as the world’s fastest analytical database. As she learned more about the team’s vision of turning that best-in-class database into a managed cloud service, she saw ClickHouse as an ideal next step in her development, and a unique opportunity to make an impact with a team and company she believed could shape the future of real-time analytics.
Specifically, Bragin saw a need in the market for a cloud service to support the rise of interactive data-driven applications. By offering a solution that combined the vast data handling capability of traditional data warehouses with the immediate response of real-time databases — what she has since coined the “real-time data warehouse” — Bragin envisioned ClickHouse powering a wide swath of use cases across many industries. But first, they had to build it.
“A cloud service is ultimately where customers will derive the most value. It’s the most cost-efficient, it’s the most scalable, it’s the most secure, it’s the most reliable.”
— Aaron Katz, ClickHouse
In most playbooks, the way you commercialize an open-source product is to start by adding a set of proprietary features, along with advanced technical support, and selling that “enterprise” edition to self-hosted free users. It’s a shorter path to revenue than the second step and eventual model, which involves building a managed service in the cloud. Other open-source companies like Elastic, Confluent, and MongoDB had developed their cloud offerings slowly, over the course of many years. ClickHouse took an ambitious path from day one, choosing not only to skip the first step and go straight to building a cloud product, but to do so in a single calendar year.
According to Volpi, “The easy thing would have been to say, ‘We’ll just sell the enterprise edition and make some money early on.’ But by doing the hard part first, we could come to market with the end-state product and not tiptoe around some ‘halfway’ business model.”
“A cloud service is ultimately where customers will derive the most value,” Katz adds. “It’s the most cost-efficient, it’s the most scalable, it’s the most secure, it’s the most reliable.”
Bragin and her team began by devising a plan that would help them achieve their goal of building ClickHouse Cloud in less than a year. They set clear milestones — Private Preview in May, Public Beta in October, General Availability in December — determining what was feasible by each date, and what resources were required to get there. The aggressive timeline forced them to be judicious about prioritizing tasks and allocating resources. They continued to set ambitious targets, but regularly reassessed and adjusted as needed.
From the outset, the ClickHouse team wanted to build a cloud service that any developer could use without deep knowledge of analytical databases. They designed the system to manage data efficiently, ensuring it was secure, reliable, and could scale to meet the needs of any size of business. During the initial testing phase, they worked closely with private users to gather feedback and make improvements. This hands-on approach helped the team refine the platform to ensure it met real-world needs, making it as intuitive and user-friendly as possible.
In October of 2022, ClickHouse launched its cloud offering in beta. The team had taken bets on how many signups they might see on the first day, with most estimates in the “several hundreds,” according to Bragin. But by the end of the day, the launch held the top spot on Hacker News, and signups were in the “high thousands.” Izrailevsky, who remembers feeling nervous throughout the day, says, “The response was overwhelmingly positive. It really reinforced our confidence and let us know that we were on the right track.”
What Bragin remembers most about that day isn’t the number of signups; it’s the feedback they received from customers, specifically around pricing. After some internal debate, they had developed a pricing model based on infrastructure usage that included clear dimensions like storage and compute, along with more opaque “read” and “write” units. Early private preview users had understood the model; but when ClickHouse Cloud opened to the public, customers were confused. Their feedback helped Bragin and her team simplify the pricing model, removing the opaque units to make it more transparent and easy to understand.
I see ClickHouse as a technology every developer will need to adopt... and the reason for that is the rise of interactive data-driven applications. If you want to build anything in today’s world, you need an analytical database that’s optimized for real-time performance.Tanya Bragin,
ClickHouse VP Product
“You never know what’s going to happen during a launch,” Bragin says. “You can have amazing success in terms of signups, but feedback in an area you didn’t anticipate. As a product leader, that’s exactly what I need in order to know what’s working and what’s not. It means I can adjust on the way to General Availability, and everyone’s going to be happy.”
ClickHouse Cloud officially launched in December of 2022, less than a year after the team set what everyone could agree was an aggressive timeline. And while pent-up demand for the cloud product seemed to boost signups early on, there was no slowdown in interest after the product launched. In fact, signups continued to accelerate.
“So much work goes into that point, and you just put it out there and the world gets to vote on whether or not you’ve built something that’s valuable and of high quality,” Katz says. “In the end, we surpassed even our most aggressive expectations by almost every measure.”
Bragin, who describes that first year as a “whirlwind,” says the experience was exactly what she hoped for when she joined the company. Now, more than a year after the launch of ClickHouse Cloud, she and the team have their sights set on expanding the platform’s capabilities while helping the world understand the power of real-time data analytics.
“I see ClickHouse as a technology every developer will need to adopt,” she says. “And the reason for that is the rise of interactive data-driven applications. If you want to build anything in today’s world, you need an analytical database that’s optimized for real-time performance.”
Endless Possibilities
One of the challenges of starting any company is finding alignment with your co-founders. Since ClickHouse, Inc. was born during COVID, and because Milovidov and his team lived overseas, he and Katz had never met in person. And while Katz and Izrailevsky had known each other for years, they never worked together. As the co-founders discussed the type of company they wanted to build, Katz was surprised to learn how aligned they were as leaders.
“The ClickHouse I first built and the ClickHouse of today resemble each other. It’s a pleasure to work with Aaron and Yury to build the team that is executing on my vision.”
—Alexey Milovidov, ClickHouse
“It feels like I'm playing chess with two queens,” Katz says. “Yury and Alexey are both so incredibly talented and strong at what they do. They’re very different, but they’re so aligned in terms of the product we want to build, how we take it to market, the quality of the product, the pace of innovation. It’s the most productive engineering culture I’ve been a part of, delivering the highest-quality product in the shortest period of time I’ve ever seen.”
“When I open-sourced ClickHouse, it was because I believed that current databases didn't meet the requirements of analytics at scale,” Milovidov says. “To see my passion for ‘fast’ and my belief in the importance of analytics recognized by my co-founders was important. But learning that they believed in developer-first adoption the way I do was an important moment in our partnership. The ClickHouse I first built and the ClickHouse of today resemble each other. It’s a pleasure to work with Aaron and Yury to build the team that is executing on my vision.”
“It’s a real trifecta,” Volpi says of the dynamic between the three founders. “With Yury, there’s the large-scale cloud-building experience. With Aaron, you’ve got go-to-market leadership. And then with Alexey, you have the guy who invented the most resource-efficient and high-performing database in the world.”
In most successful companies, culture starts at the top, and ClickHouse is no different. Bragin highlights bias to action and direct communication as two parts of ClickHouse’s culture that go hand in hand, and that stem directly from its founders. In a fast-growing startup, she says, you have to make fast progress at every stage. This requires team members to be explicit about what they’re working on, including what has and hasn’t worked. Empathy and trust are key in this type of environment, especially among a globally distributed team.
“One thing we’ve learned is to not jump to conclusions,” Bragin says. “Sometimes things get misunderstood or lost in translation. The important thing, just like with bias to action at the task level, is to iterate on communication and quickly find out what’s not being understood.”
“You’ll be surprised at what customers like and don’t like, what they want and don’t want. You need to iterate quickly and pivot. Over time, that’s how you build the best products.”
—Yury Izrailevsky, ClickHouse
Izrailevsky describes ClickHouse’s culture as an amalgamation of things its leaders learned from their previous companies and partners. He attributes the focus on engineering excellence, talent density, and data-driven decision-making to his time at Google. He says the company’s customer obsession — talking to users constantly, deeply understanding their needs — comes from Katz’s experience at Salesforce. And he says that Netflix is where he learned the value of a culture of rapid experimentation, which requires hiring well, trusting team members, and giving them the freedom and responsibility to move quickly and try new things.
“You should never assume that what you think is a good idea will be the right thing,” he says. “You’ll be surprised at what customers like and don’t like, what they want and don’t want. You need to iterate quickly and pivot. Over time, that’s how you build the best products.”
Since launching Cloud over a year ago, one thing the team continues to explore in its conversations with customers is the diversity of use cases ClickHouse can support. Increasingly, today’s businesses need to process and use information on a real-time basis to gain a competitive advantage. This is true of virtually any industry dealing with vast data sets that require quick processing and immediate insights, from ecommerce to telecommunications to financial services to healthcare. ClickHouse’s blazing speed, combined with its scalability, reliability, and affordability, make it an ideal choice as a real-time data warehouse, powering data-intensive applications that run on real-time and historical data.
“There are a lot of point solutions in enterprise software that solve a very specific problem, but the addressable market is limited,” Katz says. “At ClickHouse, we see the opportunity as unlimited, evidenced by all the ways people are using this analytical database.”
He also points to the business criticality of a technology like ClickHouse, particularly in a hard economic environment. At a time when any “nice-to-have” product is tough to secure budget for, ClickHouse, as the place where companies store their most valuable asset — their data — stands in stark contrast. When you combine this diversity of use cases and mission-critical infrastructure with a product that’s faster and cheaper than alternatives, it’s easy to see why Katz is so excited about the future of ClickHouse, and why its customer base continues to grow.
Looking back over the past few years, reflecting on the journey so far, Katz pauses for a full minute, trying to think of his most memorable day. Finally, he settles on October 11, 2022 — the day he met Milovidov in person for the first time. Arriving at the team’s Amsterdam office, he wasn’t sure what to expect — a handshake, maybe, or a pat on the back. Instead, Milovidov grabbed him in both arms and gave him a big hug.
“ClickHouse is Alexey’s life's work,” Katz says. “He created the software. He named it. It's short for Clickstream Data Warehouse. He open-sourced it. He's the primary committer. And I simply helped him unleash the potential of it to the world by creating this unbelievable company around it.”
Published — June 13, 2024
- This link opens the post, "The King of Candy: How Riccardo Zacconi and King Conquered Mobile Gaming" The King of Candy: How Riccardo Zacconi and King Conquered Mobile Gaming
- This link opens the post, "Cloud Captains: How Assaf Rappaport and His Extraordinary Co-Founders Built the World’s Fastest-Growing Company" Cloud Captains: How Assaf Rappaport and His Extraordinary Co-Founders Built the World’s Fastest-Growing Company