Why does a technology win? Does it have to be technologically superior? Does it need it come at exactly the right time to fill a (latent) need?

There are fabled examples of inferior technology winning - remember the video format wars between Video2000, VHS and Betamax (my age is showing). And examples of products so far ahead of the curve they didn’t catch on (early tablets vs. iPad).

Fundamentally, what I am trying to grasp is how it’s possible there are so many tells there is a huge market for a product, and yet people aren’t buying and instead settle for something inferior. This question has kept me busy for some time now, as I have tried to wrap my head around the developer experience (DevX or just DX) tooling market, and more specifically products and services in the application platform as a service (aPaaS or just PaaS) world.

Indications there is a need for PaaS

Recently I selected a representative sample started a poll on Twitter to get my bearings.

The results indicate PaaS adoption will grow (by a lot). Several blogs underline the developer experience is the next major competitive front in enterprise tech.

We’ve needed it for 10+ years

But you don’t have to trust my ‘research’, there has been demand for PaaS for 10+ years. Anybody remember Heroku? It was so far ahead of the curve it was an example for many but ran out of steam and lost its magic.

It did however serve as an example for many how to do PaaS right. One in particular - Cloud Foundry - is super interesting as it is open-source and quickly amassed a huge ecosystem of vendors, partners and especially users with an enormous total market cap. Other PaaS like products and services are the proprietary big cloud offerings, and infrastructure products that get enriched with flavoring to be more PaaS like (OpenShift).

It is my firm facts based believe we can only get to real multi-cloud and non-zero-sum game mechanics with an open-source core. Examples include Linux in the OS space, and Kubernetes in the container orchestration space. In the PaaS space Cloud Foundry is still the only serious open-source player.

So why isn’t everyone using Cloud Foundry?

It cannot be due to lack of success stories. Cloud Foundry has been instrumental as supporting technology for countless digital transformations of huge corporates.

If anything, Cloud Foundry has perhaps been too successful: teams of 3-4 operating a platform supporting 1000+ applications, while upgrading with zero downtime during office hours. And the operators in question were not Google tech gods but regular people working in (semi)-government, finance, insurance, manufacturing, retail.

So why isn’t everyone using Cloud Foundry? I believe the answer is not trivial as there are multiple factors involved.

Reason I - Distractions

The possible choices in technology for building a PaaS are without exaggeration enormous. How you obtain a PaaS is in the end a buy or make discussion: do you trust a governing body carefully assessing and integrating technologies into one curated stack supported by multiple vendors? Or do you invest yourself in taking the time away from your core business to select the best of breed technology in every category and do the same work without support on the integrations?

If framed like this, it’s a no-brainer. Somehow though, it seems most organizations end up with the latter.

Role of the users - taking the easy way out

One of the reasons - in the words of doctor Julz - is “that technologies that let you rename what you were already doing (‘unopinionated systems’) and think you’re done are almost inevitably more popular, especially with big companies, than opinionated systems, because they’re way way easier to adopt. However, the cost of their ease of adoption is the ease with which you can adopt them for very little payback.”

So to reap the rewards of unopinionated systems - or rather deliberately pick an opinionated system - organizations are required to have a clear strategy and vision, and combine them with efficient execution. Now think about the fraction of organizations you know that have this requirement covered.

Role of the DevOps architects - perverse feedback loop

Another distraction is provided by the ‘DevOps architects’ (mileage on role name will vary), who make their careers and resumes integrating complex technologies.

This pattern has a (perverse) positive feedback loop: because building it yourself is more inefficient, it results in more practitioners (see attendance numbers KubeCon), drawing more attention on the technologies. To an external observer it may seem the technology and practice are hugely successful, but you may question the value they provide to the organizations they get implemented in as opposed to the value they provide to the practitioners.

Once established, this is the market reality hyping on shiny new toys and sub-optimal (local) optimizations. The real irony here is that it’s inescapable: my team at ITQ currently spends ~85% of their time on building, designing, and integrating these kinds of unopinionated and unstructured platforms, whereas the remainder is spend on opinionated (PaaS) platforms. The ratio between the values they provide to the organizations they run in, is probably the inverse if not worse.

For some perspective, see this overview of make vs. buy in various tech stacks: There really is a lot to do - and get wrong - if you choose make (image credits Daniel Jones)

The big cloud players all cater to these DevOps architects with their go to market, by putting most marketing behind the make movement, as customers tie themselves up this way making it harder to ever move away from their specific platform.

It begs the question who are the real buyers - and so the market? Is it the CIO? Is there even a conscious decision involved, or is an unstructured PaaS situation the inevitable outcome of hiring DevOps architects?

Reason II - Barriers to mass adoption - enterprise product

Cloud Foundry was always positioned as an enterprise version of Heroku: it had built in support for multi-tenancy, high availability and resiliency from the get-go. Combined with the BOSH life cycle management tooling for the platform itself, it is insanely robust.

Because of all of this, it comes with a large footprint - in terms of required compute, storage, number of Virtual Machines. Not too large for the enterprises it is targeted at, but large enough to not easily fit onto a developer laptop/workstation (although some crazy Dutch people made it run on Raspberry Pis). In time more developer friendly solutions were launched, but they failed to get much developer traction. The barrier to get something running on CF as a developer if you first had to set up the platform itself remained high enough not to get ecstatic about it.

The lack of indie mindshare and grassroots adoption means that every introduction of Cloud Foundry in an organization has to start at the top. Therefore, the (enterprise) sales cycle is almost exclusively consultative (vs. transactional): long cycles, huge deals, and necessarily high price points.

This is why Cloud Foundry is a massive success in a limited number of large organisations, with little exposure outside it.

Reason III - Role of the vendors - delay & confusion

Fundamentally Cloud Foundry offers a promise to developers and operators in the form of an API. It provides functionality but doesn’t promise anything about how it’s implemented. This is great because it offers the ones implementing the technology underneath the freedom to replace components as they see fit when newer, better, and more standardized solutions arrive on the scene. A few examples:

• the routing plane which went from custom Ruby to custom Go, to finally a combination of Istio and Envoy.
• the container runtime went from custom ‘droplet’ execution to containerd (originating in Docker).

When it became clear Kubernetes would become the winner in the container orchestration wars, a logical next step would be to replace the custom ‘Diego’ container orchestration engine with Kubernetes under the covers and all would be peachy, right? Well, that’s the story for custom (user) applications running on the platform, but what about the lifecycle management of the platform itself, shouldn’t we divorce ourselves from BOSH and run everything on Kubernetes?

It was this question in early 2018 that incited mass confusion as it would break the operational model/promise. It did not help that Pivotal (now VMware) - once the guiding light among the vendors - chose to sit this one out in an attempt to challenge the vendor community.

Several initiatives started on how to transform the Cloud Foundry ‘application’ itself. Inception. The initiatives were roughly:

• replatform - KubeCF packaged existing BOSH artifacts as Kubernetes Helm charts. It works but is the band aid solution. Breaking the operational model with little value, just to quickly get on the Kubernetes bandwagon platform
• refactor - cf-4-k8s. Most Cloud Foundry components are changed or replaced with new components. This would obviously take much more time. Fast forward to today, and it is not clear yet if the developer promise can be kept. cf-4-k8s is 1.0 (GA) but is not feature complete. VMware/Pivotal released a tech preview 2 years ago, but after all this time, it seems the core project is not in active development anymore:

It was only in early 2020 (after Heptio won the internal battle) VMware did announce they would also divorce themselves from BOSH:

The problem with all this?

Cloud Foundry has existing users. A lot has happened but ultimately everyone using CF is still on BOSH for production. CF is as awesome as ever, and BOSH is as stable as it always was, but it’s now a second tech stack to support beside Kubernetes. None of these users have a clear migration path. Currently it’s not even clear what they will be migrating to. This may not be an immediate problem: when they get a lot of value from the developer experience, and still so much value gained every day, this is something to worry about later. But for users where application modernization value is less visible, CF may seem like yet another hosting platform with a separate tech stack and a much higher price point.

Cloud Foundry could have had many new users. Because it’s an awesome platform. It provides enormous value. But time stood still: users are not committing to technology the vendors themselves have a hard time committing to.

Reason IV - Role of the community

One of the triggers for me writing this post were some other much more famous community members being vocal about the future of Cloud Foundry. Where Engineer Better’s Daniel Jones paints a bleak outlook and embraces one of the hyperscalers’s proprietary offerings, Wayne Sequin from Stark & Wayne is more positive.

Some of what they said resonates with me, but this won’t give us a forward looking promise. We need to get back to the value proposition narrative. Early 2015 Cloud Foundry was breathing Developer Experience and value line. From 2018 to 2021 all we could talk about was implementation details and changing the operational model. We - as a community - allowed ourselves to be sidetracked.

What now?

We need to take a step back.

What is it we think the market ultimately wants? A great developer experience.

No developer is happy continuing writing infrastructure and yaml boilerplate (unless they suffer from Stockholm syndrome).

The original great developer experience is still the promise of CF push, as written by Onsi Fakhouri:

• here is my source code
• run it on the cloud for me
• I do not care how

Yes, for large organizations migration paths and (API) compatibility is important. But you know what’s more important? Having a great story. Commitment. To the CF push promise.

In the words of James Watters - CF push is sacred.

Let’s start there.

Let’s start lightweight.

Let’s commit.

Let’s make 2021 the year of PaaS.