People will, at one point in your career, talk to you (or with you) about the broken window phenomenon. That is an observation that, if windows are left visible broken in part of a town, the surroundings will usually start to deteriorate at a faster rate than if someone took the time to fix the original windows at some point. It's usually brought up to make sure that some flaw is fixed before it leads to the recognition that mediocrity might, in the end, be acceptable. Which would, in turn, lead to more mediocre things.
I wholeheartedly agree with that sentiment. From a end product perspective. But there's also an angle here to consider when actually building the product itself. You want to be absurdly fast, at least theoretically, when building something. And now I want you to have an honest look at yourself - how long does your CI pipeline run? Is that (whatever that number is) the best you can really do?
Everything on this blog is naturally an opinion piece, but I guess this is more of an opinion piece than the others. And here it comes: Spend some time to make your CI builds really crazy fast. Like at most 2-3 minutes. Longer than 5 minutes is dubious, longer than 10 minutes is weird and longer than 20 minutes is just outright abuse of infrastructure. Let me remind you here that it's at least the year 2024 when you're reading this, and whatever you're compiling and building is probably not more complex than the Linux kernel - and that thing takes less than a minute to build on modern hardware. Adjust your goals accordingly.
If you've read more than 2 other posts on here you'll also realize that I'm mostly focusing on value, so let's get to the point on why it's so imperative to have fast builds: You don't want to have people waiting for machines. It's bad enough that we have people waiting on people - and that's harder to avoid at times. But there's no point to having people wait for machines. If you want to merge something, you should be able to do so in a few minutes, and if you want to deploy something, you should be able to do so in a few minutes as well. Make things fast. It removes friction, it removes idle times, it removes context switches. All of that stuff doesn't add value, is annoying as fuck and can easily be optimized away.
How?
Well, I don't know too much about every stack in the world, but from a common sense point of view, start with only doing the bare minimum in every CI run. For a backend project that might be compiling, building an image and pushing that somewhere. Do you need Sonarqube, Linting and super slow tests for every PR? Probably not. I usually try to find a subset of tests that make sure that the most critical flows are covered, while deferring longer running tests to nightly cadences. Again, no one in their right mind is challenging the importance of automated tests, but your task is to weigh two things against each other: Is it more important to be able to regularly work fast with as little friction as possible, risking a broken build or some broken functionality every once in a while - or do you want to always play it safe?
Make it easy to build forward, make it easy to rollback. Don't make people wait on machines.