Ask HN: How did web development become so bizarrely complex?

My half cynical take is "resume driven development." The more different tools one uses, the better they can pad their resume, and resumes don't say 90% of these tools weren't needed.

A more objective take would be simply that "they don't know better." Hire a bunch of junior developers right out of college, and throw them in a pool of layers, and they'll think this is how "a real job" is done - they've never seen anything better. It doesn't help that their go-to example of best, most polished websites (imagine google docs) are built by hundreds of engineers, with as many layers.

Once it becomes an ingrained culture, it would be extremely hard to turn around.

I hear you, and we are actually in the beginning stages of replacing our create-react-app with Phoenix server side rendered templates.

We're also going to use esbuild to compile our very minimal alpinejs code. Most interactivity will be built using phoenix liveviews.

We're hitting the eject button on this whole ecosystem. There is a rot in node/npm that I think will never really go away as it's baked into the culture.

That seems very strange indeed.

Makes me curious about the context. I guess it’s a company that maybe has managed to either raise too much money, or prioritized quite poorly where to spend the engineering power…?

I can relate so much to this. I've been doing this since 2010, so over a decade.

Just yesterday, I needed a function that consumed some data from an endpoint, and did one of two things:

- Open a window with a PDF in so the user can print.

- Focus a god damn input field in a form.

That was it.

My manager, who isn't even a "front-end" guy, took my PR. Then changed it to some overly complex mess with an entire separate file. Huge comments. Overly built for a hundred different use cases and did it "the react" way.

We're literately opening up a new window or focusing an input.

I tried to push back but it went nowhere. He's right and I'm wrong. Never mind my 10+ years in the industry. He's the manager.

As production becomes more and more specialized and more and more automated workers lose connection to the product they are producing. A shoemaker cares about the shoes they make because it is their person creation. A worker in a shoe factory who perhaps is just pulling a lever over and over again doesn't care at all about the end result. This is called alienation. Specialization causes workers to become alienated from their work.

We can assume that alienation is rife in the software industry. Someone writing a module for some extension interface for some in house editing tool for the accounting department may not care much. So they prioritize other things. An engineer wants to learn new programming languages because it looks good on their cv and a manager wants to lead large groups because it also looks good on their cv. Thus overcomplicated projects written in newfangled programming languages is in both their best interest. Especially if they are contractors. Then the simplest solution is almost always the wrong one.

> In addition we seem to be grabbing the PDFs from the server, converting to Base64 and then rendering as an object in the react app (rather than just linking to the pdf file directly).

> What benefit do we get from this complexity compared to just having static HTML pages that someone would update manually?

In the good old days, HTML was simple because it was just text + <h1>, <p>, <b>, etc markup and HTML forms, and web authors were ok / expected that the web browser would do the heavy lifting and render it appropriately for the client OS using native toolkits / widgets.

These days, the publisher wants 100% control over everything, so all that rendering / toolkit that was previously left to the browser is re-invented in JS.

Ask your superiors why you need to embed a PDF viewer in your page instead of just linking the to PDFs and letting the client open it in their local browser / Acrobat / PDF-viewer of choice. They'll probably mumble something about a consistent experience, what if the user doesn't have acrobat installed, etc.

Without a better description of the requirements of the site it is hard to give an answer about your specific situation with regards to what is taking so long.

Technologies have improved, and requirements have gotten more complex. It sounds like the team you are working on is either doing things in an awful way or the requirements are such that there is a lot that needs to be done.

20 years ago it was very difficult to build anything remotely complex. 2-3 people could build a very simple, static site over a few weeks. If you showed web developers 20 years ago what a modern web app looks like today they would be impressed.

It is certainly possible to use a bunch of technologies that complicate things unnecessarily but even then it shouldn't take 9 months for four engineers to build a simple site.

I agree with most comments here. Just wanted to highlight this is not just a "frontend" thing. I'm seeing the same thing happening in the backend (when overcomplicating everything with Elixir or Go, etc) and infrastructure (Kubernetes, etc...) for what would be a single landing page or a very minimal "CMS" kind of site that with Rails or Django you'd be up and running in less than a week. Instead we spend months with "modern" stacks. Worst part is these stacks have a HUGE amount of custom glue code and what happens is that people eventually leave the company, leaving new joiners with a system which is impossible to understand in a reasonable amount of time. So these new joiners call this old system "legacy" and back to the start. Where if you had used a simpler, well documented, well tested, opinionated, popular solution (rails/django/laravel/etc) you'd at least have a good baseline despite any mess developers could create on top of it.

At my last job I was horribly looked down at when suggesting doing simpler things. People want to master a single gold hammer and use it for everything, no matter what the business problem is. React is THE WAY, Go is THE WAY, kubernetes is THE WAY and now it doesn't matter (we don't even think about it!! what's the actual f**ng business problem we're trying to solve). We just use THE WAY tools because otherwise we're doing "legacy".

It's sad, really sad.

I know a bunch of people that can build web apps in react and they didn't know how to write html in a text editor. They all looked at me like I was some kind of wizard.

They all worked in marketing an their company had them do a react course.

I think it comes down to one of two things; the first is ignorance. If the only tool you have is a hammer, everything starts to look like a nail. If you only know about React and Sass, any layer beneath that is a complete mystery and not something you'd even consider as a solution because you're not aware of it.

The second, and my preferred theory, is that the modern web stack has not adequately followed increasing user requirements, forcing us to take much longer to develop what seems like trivial new features. It's not that i18n, or larger images or maintaining state in the browser is some huge, insurmountable problem that can't be solved. It's just that our tools are so woefully immature that we spend more time wrestling with them than we do actually working on the features we're being paid to work on.

And for what? For some vague illusion of choice that for the vast majority of people doesn't even matter? I don't care if I use Redux or one of the bazillion different variations on the idea. Ditto for Sass, Tailwind or what have you. The web layer needs to solidify already, give us a decent bedrock to build our apps and websites on and just get out of the way.

A whole generation has grown up floating effervescently a layer too high above the foundational stack. They have never written a website in "raw" HTML.[a] They have never configured Nginx, let alone Apache.[b] They have never read a complete request response, nor are they aware that one of the Ts in HTTP stands for text![c] They have been removed from the Internet.

[a] React, Angular, Vue [b] EC2, Droplets, VPS [c] Request, Axios

Developing in React is using HTML, CSS and JavaScript. It's a declarative approach so the programmer states describes the end result and the React library does the work (code) to update to he DOM. This works really well and is not as hard as the comments make it out to be here. You could write all th JS out to achieve the same thing but it would be silly.

It forces the developer to break things into components which is essential for any maintainable UI.

Since React is the the most popular JS library it is easier to hire for.

SPAs that are done well are a better user experience because the page 'loads' once and then feels faster. That is what users expect.

If a company has multiple sites, it makes sense to be consistent with tech across those sites.

It's ready for future requirements. If you didn't use React and you kept getting requirements for the next few years you would be more likely to end up with a ball of mud if you were doing plain HTML and JavaScript.

One part that I agree seems complicated is the build and dev environment.

Why people develop "static" web sites with React is beyond me.

> In addition we seem to be grabbing the PDFs from the server, converting to Base64 and then rendering as an object in the react app (rather than just linking to the pdf file directly).

You know if you just click the link your browser will render it right?

> There are no concerns from our project sponsors around how long this is taking or the costs involved.

Who and where are these people? And where's your development team?

Think for a moment of the amount of people with Internet access, globally, and you will get to your answer.

Scalability. Scalability is the answer.

People have dynamic conceptions of what it means to be a good developer. It started with making composable components which led to an inflation of components in a codebase. Connecting all these little things led to global state management, which led to stuff like Redux, which led to a second abstraction layer on top of composition. Doing this meant you are a good developer.

Now, types in Typescript for every little thing means you are a good developer. Not necessarily judicious use of types and small type footprints, but insane obsessive, compulsive, exhaustive list of types for even the most basic function signature. This is what it means to be a good developer.

What we lack as a group is the ability to have a sit down in the current and say ‘some of this is not needed, we tried it, it’s always worth a trying, but perhaps we can cut down on some of this granular component composition, perhaps we can only use a little bit of typescript, perhaps we can reduce state management’.

We don’t have the mechanism to self-reflect, so our only form of healing is to keeping barreling forward. In other words, lie to ourselves, and run from the problem.

It all started with XMLHttpRequest (slightly serious).

Ajax spawned "Web 2.0", which in turn spawned the batteries-included-JS-frameworks (which overtook simpler libraries & plugins). Resume driven development then made them the standard, and we now have a generation of programmers who think a GET request only returns JSON, and were never taught it can also return HTML ;)

I'm not a web-dev and I have nothing to add, except how crazy it is that something as seemingly simple as a header, three columns, and a footer is so difficult. That, and centering things on a page.

https://en.wikipedia.org/wiki/Holy_grail_(web_design)

I'm kinda glad I'm back-end only.

I think Business decisions are less and less in touch with the technical realities.

I just quit a full time job in a Huge-Non Profit where my main project was to make a simple CRM, based on about 10 simple but intertwined tables. This CRM would have saved the hours of hundreds of overworked/burned out people.

The frontend was made entirely with Bootstrap. A functional, life-saving MVP without any dynamic content could have been made in a month with a team of 3 developers.

However, all of the feedback we constantly got was only to make it look more modern and dynamic (and debatably worse from a UX standpoint).

Mid-development we were forced to add Vue.js to our tech stack. I quit because my conflicts with the business side were becoming hopeless, the project has still not seen an MVP, 1.5 years and two avoidable burnouts in the making.

My thinking is that people feel the need to deliver something as modern and sexy as possible to get more credit. Completely losing track of the functional aspects of things.

Long ago, I worked with a man who had spent his life writing copy for marketing presentations. He was a perfectionist, and had spent much time getting things such as hyphens in the right place at the end of lines, etc.

It took quite a bit of convincing to get him to just let text wrap in a web page. Eventually he got used to this new normal.

I suspect much the same is happening here... instead of just letting people see the PDF link, they have to emulate the PDF viewer, and likely other things are being done in a similar vein, trying to keep control of something that should be left up to the browser.

This is a 2-3 person over a few weeks project indeed.

The more people you add to a project, the more the scope balloons in complexity, not faster.

I have seen this play out so many times.

It's because some people think the way Google, Microsoft, Facebook, etc. build web pages is considered "best practice" and try to emulate them.

It's actually worst practice for all but the largest companies and teams. We as an industry need to start doing a much, much better job communicating to all relevant parties that the best way to build most web sites/apps most of the time will look radically different from how Big Tech does it, and that's actually expected and reasonable.

Kafkaesque is the word that describes it pretty well. Every few months i glance at that imposing black mountain of web development and it still makes an ominous impression. I 've stuck with PHP/Jquery and can still build whatever stupid idea comes in my head but always wonder what's behind the door.

> The gateway to the law is open as it always is, and the doorkeeper has stepped to one side, so the man bends over to try and see in. When the doorkeeper notices this he laughs and says, "If you're tempted give it a try, try and go in even though I say you can't. Careful though: I'm powerful. And I'm only the lowliest of all the doormen. But there’s a doorkeeper for each of the rooms and each of them is more powerful than the last. It's more than I can stand just to look at the third one.”

What I've seen is a loss in the ability to architect solutions that are simple. There seems to be a feeling that if doesn't include every technology available that it's missing something.

Good architectures (think both software and construction, both) only use what they need. They are complete not when there is nothing left to add, but nothing left to take away. Good architectures make changes and extensions effortless.

Instead, as Alan Kay says, "Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves." He said that a while ago and it's getting worse.

The problem domain has become quite complex. Many people are complaining that developers are "complicating" front-end development but it's the other way around.

HTML has become complex to account for new features that users want. So did CSS (which makes you rely on SASS) and so did JavaScript (and now you complex apps require a strong-typed language like TypeScript).

A few years ago, HTTPS was used by prominent websites only. Now, it's required for even the simplest website. Implementing HTTPS can be simple (if you use Cloudflare) or a bit complex. (if you are running Nginx). But the complexity level did definitively increase.

The state of web development (and software development) is what it is because of the nature of many engineers. Ever piece of the code base is now a beautiful abstraction, that is scalable, extensible, configurable and... over-engineered.

There are probably a handful of sites that needs this level of engineering. But, for everyone else, that isn't running at a huge scale, it's hugely taxing.

Do you know how hard it is for a non-elite developer to make sense of this massive range of technologies? Now imagine a non-elite business dude who can barely figure out how to use email. Those dudes control 99.99%+ of world capital. No one cares about the underlying tech.

People want to list everything on there resume, and that is the main problem. Most of the startup mvps and SMB softwares can easily be developed without any fancy tech, but developers will do it any way.

I remember a mid sized saas app of one client, built completely on plain vanilla php,css, javascript(jquery was used) and mysql database.

It was handling millions of request from thousands of paying clients without a hitch. Only fancy thing he did was an automatic backup system using custom script using replicated database. It just works.

Without a list of requirements answering this question is impossible. There might be 1 required (e.g. authN/Z) that for _reasons_ upgrades the level of complexity considerably.

That said, the web is complex today because user expectations grew significantly. A website must be compliant with all browsers, mobile ready, save state without users clicking a 'save' button, etc. A CMS might do all that out of the box with boilerplate generic code.

I built a 52 page web app from scratch in 3 months learning Typescript while doing it. It wasn’t hard. The key is to use an absolute minimum of dependencies. Only add a dependency if you can truly prove a large benefit/cost advantage compared with writing what you need yourself. Also, use meta programming (code generation/declarative programming) to remove all repetitive work.

If you have learned to program by copy/pasting other people’s code, and fiddling around with it until it works, then you don’t know better. If you have learned to program by starting from nothing, and build what you need yourself bottom up, then you are in a much better position when it comes to making the right architectural decisions.

Once in my career I was a frontend architect, but since then I have become a reactionary when it comes to view rendering, I always opt for server side rendering and when needed add some dynamic rendering with JavaScript for that page only. Fast to implement, fast to run.

Complexity may come from people aligning with a clan for help and support e.g. react.

Personally I like to understand what I am writing. My last project was php, bootstrap and plain javascript.

I appreciate that others want to piggy back on the work and support of others. We all do.

It's always a trade-off on what technology you use. You can easily use HTML files and edit them directly, but you'll soon find pain-points that plain old HTML can't alleviate.

That's when the layers of complexity begin to creep in...

From an organizational perspective, I would like to add my own experience of how things go unnecessarily complex with FE and BE.

A while ago (maybe 7-10 years), I consulted for a company with over 3000 technical people (devs, sys, support, PMs, POs ...). My work was primarily with management regarding what people now call DX (Developer Experience), another new term that I will not address now.

When I was there, they were starting the transformation from Waterfall to Agile. One of the outcomes of this process was the realization that the teams cannot work agile because their technical architecture does not fit this model. So let's first take a moment to underline this: they did not decide to change the technology because the clients had some needs that their current architectures and technologies could not fulfill, but because their current tech stack did not allow an agile way of working.

They went full-on and hired an external company to teach them the newest technologies and work with them. So here is what happened: these cool new guys who were speaking at dev conferences and writing articles about the future of technology came. After one year, everything was transformed into "Javascript," "Microservices," and Reac. I was first amazed on how much energy both management and employees have put into this transformation, how active they were, how much learning happened. Kudos to them for having that much focus and power to achieve this transformation. Then I participated to an internal hackathon, 2-3 full days, where each team was working on their ideas or projects. What was the result? In the end, many Git repositories with microservices and react apps but almost no real solution for a problem solved. Mostly only designers were the only ones trying to create an interface that could solve the problem they choose. The devs spent most of their days setting up big automated infrastructure, focusing on build pipelines and setting up local and production environments, and losing sight of the main purpose: solving a real problem.

I realized that the main issue with tech companies or any other company is transforming toward an organization or architecture without having the actual need that the new structure solves. Because this way of doing things usually does not leave a lot of room for critical thinking about why something is better or worse as a technical solution. It defaults to using what you already know, but what you already know might not be fit as it was not the result of a kind of organic learning based on encountered problems but was imposed on you by a stronger outside context.

So now, what kind of developers do you think the company hires: mostly young developers that know microservices and react. So every newly started project is a SPA on Microservices.

Its all boils down to leadership.

I worked on a recent project where after spending 1 year they still don't have a basic auth working.

I think if you push DRY and SOLID as far as you can, it ends up in today's version of web.