Spotify getting into enterprise IT tools (opens in new tab)

Backstage is great at slowly migrating dysfunctional organizations into slightly less dysfunctional organizations. I deal with 30 years of random tech debt in a company with immense shadow IT (domains randomly registered, vendors randomly hired to do X, often nobody knows things are still needed, our zonefiles are a mess, unclear responsibilities with our team of 4 being the default fallback), and Backstage at least puts some semblance of insight into that entire mess and provides a way forward to formalize such processes. Also, putting documentation where developers commit to every day instead of a Wiki where it's out of sight / out of mind is very underrated.

Companies grow - today you have 50 engineers and a few microservices, in 2 years you might have 200 engineers and tons of APIs and web applications. I think if old orgs could have bought off the shelf developer portals years ago, they would have. Now companies that grow have this option.

It might not be a massive market, but there are a few interesting companies building this kind of tools and they sell it

We were that big company (50,000 employees, probably about 10,000 or so in technical roles across the enterprise) that had a bunch of custom portals and internal sites for all our dev stuff, and also Stoplight (Spotlight?) for APIs etc..

We were early adopters of Backstage, back in 2020. We've migrated almost all of our portals and "meta-infrastructure" to Backstage, and it's certainly not perfect, and not everyone has moved over, but the majority of our teams are in one service catalogue and people can much more easily find service owners, dependencies and more.

It's been a great tool and a pretty big internal success story for us.

But this move from Spotify tells me that they are no longer stewards of an open source project, they now use open source as a loss-leader for their paid offerings.

Which is totally fine, but it will change how we look at that relationship.

The big company use case is the ideal backstage use. Backstage is just a framework. you can use it to interface with the old system behind the scenes and provide a nicer layer on top for your application devs.

Glad I'm not the only one confused. Even the product purpose seems vague...

> Libraries, websites, ML models — you name it, Backstage knows all about it

Yes, and it’s not just poking. You have to actually code lots of stuff from scratch and maintain it. Feels like you almost need a full time person maintaining it.

The customers for Backstage aren't startups but mostly big corporations with hundreds of developer teams. Those corporations want to get rid of these hacky internal solutions and many don't even have one.

> One is a mature, public B2C, and the other is a dev tooling startup.

Amazon/AWS?

I think you are describing spin-in's. Made famous by Cisco. Didn't catch on much.

Many companies, like uber, use their other more profitable business units to subsidize other business units. It doesn’t always make sense to spin out a company / product

Backstage is an open source CNCF project. It needs a corporate sponsor to keep it alive.

We piloted two jobs ago and aborted. Wouldn't recommend at your size because it seemed to do effectively, you need a "Backstage Team" which is not Ops Team who occasionally does Backstage work. Probably why Spotify is considering it because sweet consulting money.

While I don't know your exact situation, I'm currently at company of your size engineers and stop torturing your Developers with Helm is off the shelf advice I would give. See if Kustomize would get the job done and be honest. I've found Devs can handle Kustomize a ton easier since kubectl kustomize > testoutput.yaml is alot easier for them to parse.

Helm is great if you have a bunch of different things to launch with a ton of different options but most companies don't do that. They need HPA/Deployment/Service/Ingress which is easily handled by Kustomize.

My company uses it. Our team is about the same size. I have mixed feelings. It has been useful for things like making sure a new repository is created with all of the CI goodies every project should have. You can also use it to automate things like opening PRs against your gitops repo to get apps deployed from a template.

Creating software templates can be a large investment. It would be great if there was a software template marketplace. The backstage contributors took a stab at this by creating a template repo that has not been updated in about a year. https://github.com/backstage/software-templates

If you do invest in creating a template, be sure to get plenty of stakeholder feedback so the thing you make is something that actually gets used.

I’ve introduced it at a company with ~3,000 engineers and it’s been quite successful, though we do not yet leverage all of its power and we do have a dedicated team to run it and build on top of it

We have a lot of in-house tooling and Backstage allows us to move big chunk of UIs into one place - it helps us keep UIs consistent, and makes it easier for everyone to find the tools we have. However with a lot of older, more mature in-house tooling it’s been a struggle to migrate to their data model, introduce their software templates etc.

For an org with ~50 engineers I’d recommend something more off-the-shelf like Cortex, Port, or OpsLevel

We use it, and apparently it’s great. It didn’t help with the silos though because I don’t have access.

We are trialing it. Setting up VPN connections, firewall rules, automated secret rotation, API key creation for centralized external services, DNS entries for private endpoints, new team onboarding, new developer onboarding, anything time consuming repeatable task that usually takes our ops or sec teams used to deliver manually in support of the various tech teams to deliver their apps into our clusters. The various “plugins” create PR’s for the ops/sec team to double check and approve to keep the auditors happy. Servicing ~500 developers. It’s definitely a platform that needs a dedicated team though.

Ive watched a couple of botched attempts at implementing it and just like with other spotify fads you need to be all in or not at all. Also im highly sceptical it is actually going to “break your silos” or whatever they are promising because silos usually come from the management and the culture not the type of idp management happened to cargo cult on you

Look into projen. It’s not as heavy and doesn’t require any services. But makes it easy to template and spin up projects. But the key is that templates are living and updatable. If you want an intro and a free consultation reach out via email in my profile.

It depends how you define successful. I work in a large org, and it work well for governance, and as a console, but it has its limitation. Pushing too many plugins on it makes management of the console hard

We're 30-40 developers, and we have just stopped using backstage. We started using it for APIM behind Okta SSO, but all the custom frontend and functionality tweaking needed to make it look nice was daunting.

If we start looking at it again, I'd try a simpler use-case, but definitely interested in trying it again.

we use it through roadie.so but it doesn't feel very polished compared to other developer portals...

I did not expect that. So Backstage asks you to manually insert plugin components into the app source code? Yeah... that really doesn't feel like a plugin architecture to me. More like patch the source code so it does what you need it to do. Nothing wrong with that, but not a plugin architecture.

and the account that submitted here has zero comments.

The scaffolder is the part Datadog won't get you, and in my experience it is the most useful feature.

To me it looks like a SharePoint for engineering teams.

With plugins focused around projects/deployments/metrics.

Which sounds actually interesting.

By default it offers you a few tools to better understand and manage your software landscape:

- you have all your micro services catalogues in one place, with a fairly decent data model (services are grouped into systems, they are owned by teams, they have declared dependencies on each other)

- you have documentation for each service that you can easily find

- you have a template catalogue that allows you to easily create new services. You need a full-stack app? Here, recommend option is Java + React, use this template. You need a new ML system? Here’s what you should use, click “create” and we’ll bootstrap it for you

- plus a few more, like customizable home page

If you are looking for a way to start cataloguing and standardizing services across your org, the developer portal (whether Backstage or other solution) is a good thing to have

In a past job of a fairly decent sized org (around 80 engineers, maybe 10 different active projects, a few dozen microservices, component library, template generators, and some custom CLTs) it was very useful as a way of documentation for your engineers.

I'm sure you used stuff like confluence but it was always a terrible way to search across code bases and projects to understand how they worked.

Backstage was interactive documentation. I can see what projects we have, I can click on one, see who is maintaining it, what their communication is, what their various perf/benchmark scores were. It was nice.

Apparently it's even grown beyond documentation needs from what I can tell and it provides platform tooling now.

What’s wrong with PostHog? Do you mean Postman?

I think you have the wrong product; PostHog is for startups, not enterprises.

What's bad about PostHog? I haven't tried but I thought it looks good

Spotify's shuffle algorithm can't be changed easily because too many people rely on its existing quirks. Which means that Spotify's ability to keep it at a reliably stable level of unreliability for so long, through so many other platform changes, makes it a prime example of why Spotify is a perfect choice for enterprise! ;)