What this covers

Yanev walks through Databricks as a company-building story. It starts with Apache Spark, the open source engine that Databricks turned into a proprietary, faster and cheaper version, and which displaced Cloudera's batch-oriented Hadoop platform as the dominant big-data stack. Yanev frames the growth path as Technology to Product to Solution Progression: Spark was the technology, the proprietary engine the product, and vertical offerings for health care, industry and the public sector the solution. Commercial mechanics get plenty of room. Billing is by usage in increments as small as a second, revenue comes from the analytics engine, the lakehouse and an AI product added with the Mosaic acquisition, and sales reach was extended through partners such as Deloitte and IBM. The move onto Azure, pushed by SineWave for its security standard, is told as a case of buyers choosing one new thing at a time.

The second half looks forward. Yanev argues that manufacturing lines, fraud detection and ad targeting all need decisions made while data is being produced, and that modern AI depends on analyzing unstructured data in real time. He also claims enterprises want AI grounded in their own governed data, and that OpenAI's commercial restrictions make Databricks the better partner. More contested calls include the forecast that most data infrastructure will become marketplace-driven, with Apple's forced opening in Europe as the example, and the claim that efficiency gains enlarge cloud spending rather than shrink it. He also predicts niche data startups will struggle against broad platforms.

Sharpest takeaway

Yanev argues Databricks wins by pairing a free open-source Spark community with an enterprise layer that is open to any tool or model, which makes it a value-added platform on top of the hyperscale clouds that will pull ahead of Snowflake in real-time and AI workloads rather than a head-to-head rival.

  • Real-time processing of unstructured streaming data is the capability AI requires, and it is Databricks' core strength while Snowflake grew from structured data
  • Openness (any tool, any model, any cloud) builds stickiness and a marketplace that closed competitors cannot match
  • Efficiency gains expand total usage, so Databricks and the hyperscalers are complements more than competitors

The argument · threads10 threads · 62 claims
0.82

Real-time data demand across industries favors Databricks over Snowflake in the AI era.

11 pointscentrality 5/5
  • Modern AI applications such as facial recognition and ChatGPT would not be possible without the ability to analyze unstructured data in real time, and this dependency is a main reason for Databricks' recent acceleration.

    “ChatGPT, these all are not possible if you can't analyze unstructured data in real time.”

  • Structured data is organized in a predictable, column-based format stored in databases and spreadsheets and is easy to search and analyze, whereas unstructured data (text, images, video, social media) lacks a standard format and is hard to extract meaning from without specialized algorithms or expertise.

    “structured data is data that's organized and formatted in a predictable way”

  • Although the two companies converged functionally, AI will separate them because AI requires unstructured-data analytics, Databricks' strongest capability, while Snowflake does not come from that world, so Snowflake will become more of an underlying infrastructure layer like AWS or Microsoft.

    “AI fundamentally requires unstructured data analytics, which is, by far, Databricks' strongest muscle.”

  • Databricks began in the unstructured, real-time streaming world while Snowflake began in structured historical data, and both have since expanded to do each other's core work, with Databricks moving directly into Snowflake's territory via the lakehouse.

    “Databricks started in the unstructured data world... and Snowflake started in the structured world”

  • Manufacturers, banks and recommendation engines need to act on data while it is being produced, because historical analysis is too late: manufacturing lines need real-time sensor and video data, banks need to catch fraud as it happens, and ad platforms need to target users during the session when they are most likely to act.

    “it's not useful for me to know that this fraud was committed after it's already happened”

  • The shift to real-time decision-making was already underway but was accelerated by COVID and supply chain disruptions, which made efficiency and performance gains requiring real-time decisions a top priority across industries.

    “COVID helped us a lot, but it was already a trajectory that was going somewhere.”

  • Before streaming analytics, companies organized data into files or libraries and analyzed only historical records, so every insight was retrospective and manual.

    “So historically, there wasn't this ability to analyze streaming data.”

  • Databricks lets enterprises analyze user behavior in real time to decide which ad to serve or what to recommend at the moment a consumer is most likely to act, and enterprises can charge brands more because of higher hit rates.

    “they're going to pay a lot higher price for something like that because it should have a higher hit rate.”

  • Customers such as Honeywell and large pharmaceutical companies use Databricks to process streaming sensor, machine, and video data in real time on manufacturing lines, and banks use it to detect fraud as it happens rather than after the fact.

    “it's not useful for me to know that this fraud was committed after it's already happened.”

  • Before streaming analytics, organizations organized data into files or libraries and analyzed only historical records, whereas today the ability to make decisions in real time is becoming central across automotive, manufacturing, logistics, and energy companies.

    “So historically, there wasn't this ability to analyze streaming data.”

  • Data is now produced from more and more locations, including everyday devices and sensors, which requires processing data in a quick way to make decisions across many industries.

    “coming from your toaster oven and coming from the field of the rails for a train going down the tracks.”

0.71

Open, non-lock-in marketplace platforms beat closed proprietary rivals that frustrate customers.

12 pointscentrality 4/5
  • Open source provides no competitive advantage because it is available to everyone, so any commercial advantage must come from proprietary additions built on top of it.

    “there's no competitive advantage in open source because it's for everybody.”

  • OpenAI's models carry restrictions that prevent many commercial uses, whereas Databricks allows many models to be used commercially and to integrate with proprietary data, making Databricks the better partner for enterprise AI.

    “interestingly, OpenAI is not so open. You're not allowed to use their models for anything commercial.”

  • Very few companies have successfully turned an open-source platform into their own enterprise version, with Red Hat cited as an earlier attempt that did not achieve the same result as Databricks.

    “very few have been able to create their own version of this open source platform like Databricks did.”

  • Databricks released its own LLM called Dolly, but its AI strategy is to be a marketplace hosting hundreds of datasets and models, most not built by Databricks, so customers can pick the model that wins.

    “what Databricks has released is their AI platform or solution model... called Dolly. But you don't just need to use Dolly.”

  • Hyperscalers would like to compete with data platforms but have chosen a marketplace approach, and the reason they cannot effectively build everything is that buyers want choice across many tools.

    “I think they'd love to be competitive, but they have not taken that approach, and I don't think they could achieve it effectively”

  • Over time most data and infrastructure companies will become marketplace-driven because closed platforms are hard to sustain, as customers demand multi-cloud and hybrid flexibility and regulators force openness, citing Apple's required opening in Europe.

    “most of these companies will ultimately be marketplace-driven companies if they're smart about it”

  • The main risk to Databricks' continued growth is failing to maintain its democratizing, open ethos and to integrate new AI solutions and add security, provenance and cost-efficiency layers, since customers do not want to choose between Databricks and others.

    “making sure that they maintain this democratizing ethos and integrate effectively those new solutions”

  • Customers adopt new platforms more easily when the platform works with tools they already know, because they change the insights they get from the business rather than how the business operates.

    “You give them something familiar. So they're not necessarily changing the way they do their business.”

  • Being Hadooped describes getting locked into the Hadoop-based batch-processing ecosystem, which was great for structured data but poorly suited to real-time streaming, and Cloudera had been the dominant player built on that architecture.

    “Cloudera had been built on Hadoop, great for structured data”

  • Cloudera was the biggest data and infrastructure company at the time, built on Hadoop and poorly suited to real-time streaming, and Databricks overtook it early on, moving on to provide a value-added layer atop Microsoft and Amazon clouds.

    “So I think Cloudera was the biggest company at the time for data and for infrastructure in the cloud, and Databricks overtook that.”

  • Because Databricks works with nearly every familiar tool, adopting it is a leap to the broader analytics marketplace rather than a lock-in to one vendor, which helps customers move without changing how they run their business.

    “If you're familiar with something, it's almost 100% certain that it works with Databricks.”

  • Apple and Oracle run more closed platforms that restrict customers, which frustrates many of them and has pushed some away, whereas Microsoft's more open approach lets customers use many tools.

    “If you're on Oracle, you have to use their software and platform, which really frustrates many, many customers”

0.66

Rapid revenue growth and seven-figure customer spending signal Databricks' value to customers and investors.

10 pointscentrality 4/5
  • Snowflake is the closest public comparable to Databricks and trades at roughly a 19x multiple, which roughly matches the valuation implied by Databricks' recent funding round.

    “Snowflake is the closest. It's at a 19x multiple. If you look at Databricks' recent round, it about pegs them there.”

  • Databricks has tens of thousands of customers, more than 1,200 partners, and over 300 customers spending more than $1 million, which Yanev cites as evidence of deep entrenchment.

    “Databricks actually has, I think, it's over 300 customers now that spend over $1 million.”

  • Databricks' lakehouse product, which combines structured and unstructured data and directly competes with Snowflake, grew from zero to over $200 million in revenue in 18 months.

    “In just 18 months, it went from 0 to over $200 million in revenue.”

  • Delta Sharing, a newly added data-sharing feature, will become another revenue stream for Databricks.

    “which again is a new element called Delta Sharing that they've added that, I think, will become another revenue stream for them”

  • Fortune 500 customers that adopt Databricks typically spend in the seven figures annually, many in the double-digit millions, and spend grows within each account; no customer is yet at the three-digit million level.

    “they are definitely using in the seven figures, so in your millions and many of them, especially in that size company in the double-digit millions.”

  • Databricks charges customers by usage in increments as small as one second on a pay-as-you-go basis, with revenue coming from three product lines: the Spark-based analytics engine, the lakehouse, and an AI product added with the Mosaic acquisition.

    “they charge by usage down to increments that are as small as a second, and it's a pay-as-you-go model”

  • When SineWave invested, Databricks was growing revenue at 149 percent and about 40 percent of its customers had exceeded their memory quota, which signaled heavy usage and value to customers.

    “this was at 149% growth rate at the time. They had incredibly strong usage. I think it was 40% of their customers exceeded their memory quota”

  • Someone attempted to acquire Databricks for $1 billion in its early days, which Yanev cites as a sign of value and says gave SineWave comfort that it would have a good outcome either way.

    “there was also an attempt to acquire them back then for $1 billion. So it was a pretty good bet either way”

  • Making decisions from data in motion, in real time, is the holy grail that previous platforms could not achieve, and Databricks created the industry around it so that it is now the standard for data-driven decisions.

    “It's the holy grail. We haven't been able to do that before. Databricks created that industry, and now it is the standard”

  • Cloudera, built on Hadoop's batch processing, was the dominant big-data platform until Databricks overtook it by accelerating in-memory processing, which makes analysis much faster and more versatile than organizing data one book at a time.

    “What Databricks did to displace Hadoop is they invented the ability or accelerated the ability to do in-memory processing.”

0.56

Spark-based efficiency lowers customer costs and creates a sticky one-stop platform.

6 pointscentrality 4/5
  • Customers keep paying because Databricks lowers their cost and raises performance, and the decisive test customers apply is whether the efficiency in operating decisions, not just cheaper compute from pay-as-you-go pricing, saves them money.

    “If Databricks can continue to always increase that efficiency and value, then customers will always pay for it”

  • Databricks reports gross margins of about 85 percent, compared with roughly 56 percent for Snowflake, and Databricks avoids paying a hardware cost layer to AWS that Snowflake pays because Snowflake's infrastructure runs on AWS hardware.

    “their margins are 85%. So think about Snowflake's margins, which are around 56%”

  • Spark originated in the Berkeley lab of Ion Stoica, and Databricks was founded around 2009 by roughly seven or eight people, including Matei Zaharia, all of whom are still executives at the company.

    “it was around seven or eight or so that are considered the founders of Databricks and of Spark, and all seven of those are still executives at the company.”

  • Databricks took open-source Spark, built a proprietary, more efficient version with faster performance and lower cost, and by reducing infrastructure spend so much, the software effectively felt free, creating value that exceeded what open source alone could capture.

    “they were able to reduce the cost of that so significantly that essentially the software didn't only become free, but it's almost like you were making money”

  • Opening Databricks to other analytical tools and then to the full analytics stack, including the warehouse and lakehouse, turned it into a one-stop platform that customers would not leave for competitors, the same way travelers use KAYAK to compare everything in one place.

    “If Databricks had stayed Spark only, I think it would not have had the potential or been as big as it is now”

  • SineWave's thesis was that streaming analytics must be applied in ways that drive customer return on investment by reducing cost or increasing revenue, and it used Databricks' open-source traction as evidence that the team could build an enterprise layer on top.

    “At the end of the day, a customer cares about whether it can do something that reduces its cost or increases its performance”

0.38

Scale and broad tool integration beat niche players and closed in-house builders.

4 pointscentrality 3/5
  • When AI becomes ready for real enterprise deployment, companies will be forced to adopt the tools needed to stay competitive, and Databricks is positioned to be the tool they must use even if they run Snowflake.

    “AI fundamentally requires unstructured data analytics, which is, by far, Databricks' strongest muscle.”

  • Databricks' advantage for enterprise AI lies in security, data governance, and provenance, which let companies control data access and avoid leakage of proprietary data, capabilities most AI application startups lack.

    “Do I know what I'm getting? Is there a leakage of data? ... Those are all pieces that most AI applications don't have”

  • Companies that insist on building everything in-house and keep closed ecosystems fall behind the innovation curve and get displaced, as Cloudera was displaced and as Microsoft, Amazon, Snowflake and OpenAI may face.

    “those ultimately struggle. It by default keeps you behind the innovation curve and you ultimately get displaced.”

  • Data-specific or application-specific niche players will struggle against Databricks because scale and broad reach are decisive in the data market, so startups should integrate broadly with other tools rather than focus narrowly.

    “that scale component is massive. You need that broad reach and being data-specific or application-specific just is not appealing and doesn't work.”

0.38

Early vendor lock-in through partners and cloud access makes platform adoption sticky and hard to displace.

4 pointscentrality 3/5
  • Databricks used channel partnerships with firms such as Deloitte and IBM, training consultants so that the partner's client base could deploy Databricks, which expanded its sales capacity far beyond its size as a small company.

    “All of Deloitte's customers by training various folks at Deloitte, all of Deloitte's customers can now deploy Databricks”

  • Early on, Databricks ran only on AWS, and SineWave pushed it to move to Microsoft Azure because Microsoft had a security standard that unlocked regulated public-sector and enterprise buyers, and buyers prefer one new choice at a time.

    “Databricks was on Amazon only. And we push them very hard to move as quickly as possible to Microsoft, because Microsoft had a security level that Amazon did not have.”

  • Choosing a data platform is a multigenerational decision because once developers and data pipelines are built on it, switching is hard, so the first vendor to onboard customers gains a sticky advantage that competitors struggle to break.

    “when customers are making a decision between Databricks and Snowflake, they're making a multigenerational decision to some extent.”

  • Databricks went on a massive hiring spree after investor pressure, which alarmed Snowflake's people because Databricks was recognizing the race to lock in customers early.

    “we pushed them and they went on a massive hiring spree. It scared the hell out of Snowflake people”

0.35

Vertical and public-sector focus built a sticky business worth hundreds of millions.

3 pointscentrality 3/5
  • The public sector is the slowest technology adopter of all, which makes it a place to look for large, entrenched revenue streams once a vendor gets in.

    “the government is the last adopter of everything. So if you're thinking about the slowest moving beast, in many respects, that's the place to look.”

  • Databricks started targeting specific verticals about six years ago, beginning with health care, industrial and public sector, and tailoring the platform to each vertical made adoption quicker even though the product applied to all customers.

    “they started to focus on specific verticals... they started with health care, industrial and public sector.”

  • Building the public-sector vertical from scratch with Databricks produced a business worth hundreds of millions of dollars that is very sticky.

    “building their public sector vertical, which is hundreds of millions of dollars for them and very sticky”

0.29

Patient build-out from technology to product to solution preceded revenue ramp and valuation.

6 pointscentrality 2/5
  • Spark's open-source platform came out of UC Berkeley, where Ion Stoica's lab produced the technology that became Databricks.

    “it was around seven or eight or so that are considered the founders of Databricks and of Spark, and all seven of those are still executives at the company.”

  • Companies must move from technology to product to solution: the open-source Spark was the technology, Databricks' proprietary version was the product, and vertical-specific solutions for health care, industrial and public sector were the step that accelerated growth.

    “every company needs to move from technology to product to solution”

  • Revenue matters, but a company does not need revenue on day one or even day ten; Databricks built a sticky developer community and enterprise usage first, which allowed a hockey-stick revenue ramp once monetization opened.

    “But that doesn't have to be where you are on day 1. It doesn't even have to be where you're on day 10 as long as you can get there by day 20.”

  • Investors expected Databricks to become a multibillion-dollar company, but nobody predicted a $50-plus billion valuation, which was driven by the AI-era acceleration that came later.

    “So we always thought it would be a multibillion-dollar company. Did we think it would become a $50-something billion company? I don't think anyone”

  • Databricks' trajectory positions it well for an exit for early-stage investors, and public markets will reward it once investors are hungry for growth again, which Yanev expects to happen.

    “the person overtaking all of those is Databricks itself, which positions them very well for their ultimate exit for its early-stage investors.”

  • Enterprise use of AI is still very early, and organizations still need to figure out how to use it effectively and train decision-makers to deploy it, so Databricks' bet is on being ready for whichever AI solution wins.

    “Now the use of AI in enterprises is still very early.”

0.24

Revenue grows with per-customer usage, and cheaper data processing expands total spend.

4 pointscentrality 2/5
  • Enterprises need AI grounded in their own proprietary data to avoid outputs based on other people's hallucinations or distortions, and governance over that data is what makes AI usable in day-to-day business.

    “insights that aren't based on other people's hallucinations or distortions and are actually meaningful for their business”

  • The key to the business model is that revenue continues to grow within each customer over time, and the ultimate size of each account depends on how broadly a company deploys the analytics, with AI potentially pushing large accounts to multiples or double digits of revenue.

    “these companies could be multiple to double digits of revenue for Databricks each”

  • Making data processing more efficient does not reduce total spend on infrastructure, because cheaper and easier data use increases the overall amount of data processed, expanding the market size faster than efficiency shrinks per-unit consumption.

    “maybe the processing will be more efficient, but there will be a lot more of it.”

  • The third product, which came with the Mosaic acquisition, lets enterprises run AI models on their own proprietary data, and revenue scales with usage.

    “this new product that came with the acquisition of Mosaic”

0.19

Efficiency-driven cloud usage makes AWS and Microsoft beneficiaries rather than losers.

2 pointscentrality 2/5
  • AWS and Microsoft act as partners to both Snowflake and Databricks as well as underlying infrastructure, and they have chosen a marketplace approach instead of trying to build every analytics tool themselves.

    “they're both partners with both Snowflake and Databricks”

  • Databricks and Snowflake are more of a challenge to Microsoft and Amazon than competitors with each other, because the efficiency they bring increases total cloud usage: efficiency is offset by a much larger market, so the cloud providers ultimately benefit from more compute.

    “maybe the processing will be more efficient, but there will be a lot more of it”