As someone who has been working on platform engineering for two decades, and more specifically, building platforms as products, I’ve built platforms for some of the world’s most recognisable brands and founded companies that have gone on to be acquired by much larger organisations. Along the way, I’ve made many (all the??) mistakes and had several successes.
I aimed to share my most important learning in a recent PlatformCon talk, “20 Years' Platform Engineering, 10 Years' Platform as a Product, 5 Lessons Learned”, and wanted to expand upon several topics in this article.
Let’s dive into the five lessons that I regard as hard-won takeaways (because things can and do go wrong!) from my platform-as-product experience:
Employ a “product mindset”.
Find the right abstractions for your organisation.
Democratise your platform.
Everywhere is brownfield; everything is complex.
Day one is easy; day two thousand is hard.
Employ a product mindset for building a platform
What does “product mindset” mean when building a platform? In a white paper on platforms as products, which I wrote with Joe Fitzgerald, we provide a definition for thinking about the “productisation” of the platform that can reshape how we think of building platforms:
“Your platform is not an off-the-shelf piece of software; it is an evolving set of reusable services, integrated with your existing systems, that creates valuable outcomes for your business. The platform’s capabilities should change in response to the needs of its users — your app developers — among whom it is a recognizable internal brand. In other words, your platform should be treated as a product.”
If we don’t build from a product mindset, our platform ends up being project-orientated. It will likely be restricted to a specific start and end date with a release and certain features, operating much like most short-term projects. Like most off-the-shelf software releases, it goes out into the world, and we hope for the best. And then we repeat this project-focused process again. This incremental approach to platform building isn’t really aligned with the definition given above and leaves important opportunities on the table because good platform engineering is more than just engineering.
The user is central to building great platforms. What do our users want and what will they use? The platform evolves based on this use and the feedback we actively seek from our user base. The platform, like most things, should respond to changing development or business needs and use patterns to deliver value.
A mix of feasibility from an engineering perspective, viability from a product management perspective and desirability from a design and usability perspective underpin what we mean by “successful product”. This means that while platform engineering is necessary, it isn’t sufficient on its own because the platform product requires product management and design skills to become a successful platform product.
Key takeaways
Bringing real and ongoing value to the platform as a product demands understanding its users, first and foremost, and keeping the following in mind:
Remember, platform engineering is just one part - product and design are also essential.
Collaborate with users and get firsthand experience of what they are doing with the platform; use X-as-a-service to ensure scalability.
Capture user metrics and respond to feedback to continuously adapt to user needs and inject ongoing innovation.
Find the right abstractions for your organisation
Talk to any CTO, and they will tell you that their application teams spend way too much time on non-development tasks. A 2019 survey published in The New Stack reported that developers spend less than a third of their time writing or improving code. And my own informal chats with CTOs reveal that somewhere upwards of an estimated 80% of application teams’ time is spent on non-value-delivering toil. That is, developers are building their own internal platforms, focusing on operational workloads and any number of other non-creative tasks.
CTOs and developers alike want to reduce this burden. Platform engineering can help – if an organisation invests in finding the right abstractions for their business. That’s a big if because there’s no one-size-fits-all platform, which is one potential roadblock.
One common approach has been a classic bad DevOps “you write it, you run it” mentality in which individual teams decide to build their own platform only to discover that multiple other internal teams have also invested in building their own platforms. Leaving aside the organisational communication silos that let this happen, this is wasteful, inefficient and fails to take advantage of the power of platform engineering.
How to determine the right abstractions
What are the “right abstractions”? A good place to start is to ask what key attributes we need to get from a platform to make it worthwhile and useful. To start, how will it provide maximum efficiency, security and productivity to the organisation? Achieving this demands looking at what is common to the teams in the organisation but unique to the business. If something is not unique to the business, it probably makes more sense to buy something off the shelf or get an existing cloud service to serve the need rather than trying to build it yourself.
One way to decide on the right abstractions is to understand the wrong ones. Forcing everyone down a prescriptive “golden path” that aims to show exactly how things should be done creates friction and resistance. The “dictatorial approach” comes up as a common conflict because differing opinions and experiences clash with the “here is how everything is going to work” approach. Whether it’s CI/CD, the exact framework in use, staging and production, or other tools, it’s important to create (de-)composable abstractions to ensure people are more productive and have some flexibility. Having options in the form of high-level abstractions helps provide some guideposts that won’t turn into an electric fence.
Key takeaways
The right abstractions will be unique to an organisation and its needs, meaning:
Finding what is common to your teams but unique to your business.
A promised “golden path” without flexibility reduces productivity and creates friction.
Don’t dictate to application teams; they will have their own opinions about how things work. You should try to make their lives easier, but cooperatively, without overruling their experience.
Democratise your platform
To deliver meaningful productive valuable software in an enterprise often involves a lot of stakeholders. Give these people an early “in” or face struggle(s) down the line in getting value into production and delivering value for the organisation’s customers.
Many people throughout the process are empowered to say “no” to anything you are trying to do - for a variety of reasons. Democratising the platform from the get-go helps include these stakeholders and brings their value to the platform early. Time-to-value is a major consideration for enterprises when making technology investments, and in some organisations, complexity, compliance, and governance (among other things) slow value delivery way down.
An example of the power of democratising the platform: an insurance company that had six-month to two-year deployment cycles could reduce the time to value by folding all ship-code-to-production activity into a single platform and automating it, seeing their deployment cycles slashed and being able to deploy multiple times a day.
Key takeaways
Getting the right stakeholders involved and bringing them and the value they possess to the table makes a big difference in terms of the platform you end up with. Some keys to this democratisation process:
Create a self-service API for your platform with different interfaces to access it
Help other teams (identity, CI/CD, database) contribute their value to that platform
Enable teams like security, networking and other enablement bring what they offer, e.g., subsystem teams gain value from enablement teams, etc.
Everywhere is brownfield, everything is complex
The world is not a pristine, green space, and this holds true in organisations where a lot of legacy stuff exists. Yes, everything is brownfield – and complex, and even the most useful solutions aren’t going to be able to replace everything or be used everywhere.
As a real-world example, I talked to a CTO at one of the world’s top banks, who explained that he loved what Cloud Foundry could do but wondered what would work for the other 99% of workloads he had responsibility for. Essentially, it’s important – and sometimes even critical – to think about what actually needs to get done and what can be done with what you already have. If you’re small or have very basic apps, using something like Heroku, Netlify, or another platform as a service will make perfect sense. However, complexity is inevitable for other organisations facing layers of scalability, compliance, and governance issues, as well as myriad long-living heritage applications that don’t conform to the new platforms and services being introduced.
Most enterprises are brownfield, almost unimaginably complex and can be challenging to work with as a result. This reality informs the honest way we need to think about our platforms and how and whether they can adapt to that kind of landscape rather than trying to dictate it.
Underneath that, at most organisations, there is no single infrastructure layer. It’s impossible to say “everything’s on Kubernetes”. Instead, it’s likelier to see Terraform, Pulumi, cloud APIs, and infrastructure as code tools like Chef, Puppet, lots of scripting, some declarative and some imperative stuff, and so on. And the platform is going to have to work with all of this if it is going to provide real value to the organisation. When building a platform, it’s important to enable people to bring their value in an X-as-a-service mentality, meaning you need a clear, consistent API in front of the value people have been bringing over time. This API needs to be there for both old and new resources if the platform is going to deliver true value inside the organisation.
Key takeaways
Be prepared for complexity and legacy. Some key considerations for tackling this reality include:
Adapt, don’t dictate.
Understand and plan to build a platform that will work with a lot of brownfield stuff.
Deliver a clear, consistent API for both old and new resources.
Day one is easy; day two thousand is hard
On day one, it’s easy to create something from scratch. On day 2,000, it’s a lot harder to maintain what was built on day one. Even with careful planning, a lot happens in an organisation over time – people leave and take a lot of institutional knowledge with them; layers are built on layers, and what exists on day 2,000 is a different beast from day one.
How do you work against this? Some challenges, as noted, are impossible to foresee, but organising and standardising into a manageable structure is step one. Taking control of the things we need to manage into a structured setup to understand where they are at, which versions they are, and what software is really installed there, and this can be done with an X-as-a-service mentality, thinking about standardising what you offer as-a-service. Standardising and caring for an as-a-service offering rather than sharing a code base (which is a ticking time bomb) helps ensure that users are coming along on the journey rather than taking on their own maintenance nightmares.
Key takeaways
Day 2,000 does not have to be as hard as it seems. Consider the ways to make sure your platform serves your users from day one to day 2,000:
Organise and standardise
Consider that it’s easy to build applications but much harder to maintain
Sharing code is a maintenance nightmare; deliver things as a service (manage the fleet, take everyone on the journey, keep them updated and secure)
Keep the five lessons alive
Whatever part of the platform engineering journey an organisation finds itself in, these five lessons can come in handy in evaluating and designing a platform for the future:
Product mindset: Make your platform a product.
Find the right abstractions: Make your platform a product that actually works for you.
Democratise your platform: Make your platform more robust by taking on as much value from stakeholders early on and gaining easier buy-in along the way.
Everywhere is brownfield, everything is complex: Make your platform adaptable to the existing complexities of the legacy enterprise and its infrastructure and systems.
Day one is easy; day 2000 is hard: Organise and standardise aspects of your platform as much as possible to ensure ease of use and maintenance from day one to day 2,000 and beyond.
Comments