Read how funda streamlines the Azure DevOps Experience

Funda's Platform Team simplifies developers' lives. In this blog post, Software Engineer Marek Broda reveals how a product mindset allowed for a streamlined Azure DevOps experience. Discover our workflow optimization through UI enhancements and faster builds.

This is the fourth and final post of our CI/CD engineering series. In the first blog post, Tom introduced composability for pipeline management, while Pascal highlighted the Backstage developer portal's templates for accelerated integration in the second post. In the third in the series, Kateryna explained about efficient testing practices in our Azure DevOps pipeline, and in this final post I will reveal how we streamlined our Azure DevOps experience.

DevEx and the Platform Team

As the Platform Team at funda, we want to make the lives of the engineers working here easier, empowering them to build features, solve problems and improve the experience for our end users.

In accordance with the team roles outlined in Team Topologies, the Platform Team works to provide an environment that minimises the cognitive load placed on stream-aligned teams, reduces friction in their working processes and leaves developers feeling empowered to make changes. We are one of the key teams responsible for providing a positive Developer Experience (DevEx) at funda.

DevEx can be described as the overall satisfaction and effectiveness of developers when interacting with the tools, platforms and workflows that they use to fulfil their roles at the company.

A positive DevEx is hugely important within any company. It empowers engineers, making them feel capable of making an impact without being restricted by the eco-system within which they are operating. At most tech companies, the speed of software development is one of the main limiting factors on innovation, and a good DevEx, incorporating useful tooling, consistent environments and low friction, will only increase productivity, allowing more to be achieved.

Improvements

We have written in the past about providing Golden Paths, which are standard practices that guide engineers through the process of developing features. We have also detailed our adoption of Backstage, a developer portal where engineers can see a catalogue of services and make use of the afore-mentioned Golden Paths. These are examples of larger projects that we have undertaken which have had the stated goal of improving DevEx.

However, DevEx is not only improved by larger undertakings such as those described in previous blog posts. As a Platform Team, we have taken steps to incorporate DevEx improvements into our regular stream of work. As part of our schedule, we analyse where engineers need to reach outside of their own team for assistance and discuss with them the pain points that they experience whilst using our tools. Reflecting upon these topics allows us to make regular improvements to DevEx at funda.

In this blog post, I will describe some of the steps we have taken to streamline the Azure DevOps experience for our developers with precisely this approach.

Linking our build and deploy pipelines

As detailed in our blog post on Modular Pipelines, we recently transitioned our CI/CD processes to Azure DevOps (AZDO). The way that AZDO has its pipelines set up means that often you see builds and deploys all taking place as distinct stages within the same pipeline. In that post, we also detailed the reasons why this solution would not work for us.

We produced a solution in which the build pipelines would publish Docker images, which the deploy pipelines would then deploy to the developer’s chosen environment.

However, this meant that to deploy a service to an environment, you would need to go to the build pipeline, find the name of the resultant Docker image, and then use that as a parameter when triggering the deploy pipeline. This would result in a rather cumbersome workflow.

Moreover, it meant that developers needed to know that they needed the Docker image, where to find it and how to use it. This was just another unnecessary layer of complexity that we wanted to avoid.

In creating our AZDO deploy UI, we made it very simple for engineers to deploy a particular build to their chosen environment, reducing friction in developer workflows. You can find documentation on writing your own AZDO extensions here.

In addition, the cognitive load on developers is lessened, as they do not need to think about the implementation details of the pipelines themselves. They can focus on their own work and see their results quickly and easily.

Cleverly handling pipeline cancellations and failures

During one of our regular sessions analysing developer pain points, we realised that developers were reaching out to SRE (Site Reliability Engineering) because a pipeline of theirs was blocked. They were finding that pipeline runs would result in an error that the terraform state was locked, or that there was already a helm release in progress.

As the developers may not have been aware of the internal workings of the pipeline modules, they may not have known what caused this to happen and would just reach out to SRE to get it unblocked before carrying on with their work.

Upon looking into these requests further, we realised that we should improve the handling of errors or cancellations during the execution of our pipeline modules. These events were causing the deployment to become locked, and we needed to cleverly deal with these scenarios to prevent this happening.

After adding the safeguards that allowed us to cleverly handle an early exit from the pipeline execution, we saw a reduction in requests to SRE to be unblocked. This means that developers wasted less time trying to figure out why they were unable to deploy their services and did not find their flow state interrupted by the tooling that they were using.

Decreasing build times by leveraging Docker layers

From the feedback that we received from our developers after we released our service pipeline modules, we also learned that whilst most builds were only taking a few minutes, some larger or older services comprising more complicated Docker images could take significantly longer. Discussion with the developers of these services alerted us to the fact that builds could take 10+ minutes to complete and that using the pipelines could be a frustrating experience.

Upon analysing our own build module, and the Docker files of these services, we realised that we could speed up the process significantly.

Using this excellent blog post by Andrew Lock as inspiration, we adjusted our build module to extract, build and cache intermediate stages for future use, so that subsequent runs of a pipeline did not always have to build the entire image from scratch. With cache hits, we saw significant performance increases to our build module with execution times less than half of what they had been beforehand.

Listening to feedback from our developers and acting meant that we were able to increase developer satisfaction with the tooling that we provided and reduced the friction that developers felt when working on older or more complicated services.

The rewards of adopting a product mindset

We have seen how effective UI changes have reduced the cognitive load on developers, allowing them to focus their energies on their own work. We have also seen how improved build times and error handling reduce the friction that developers feel whilst trying to iterate and deploy their work.

In isolation, the examples that I have listed above would have had a limited effect on the experience of our developers when using our new Azure DevOps tooling. However, the cumulative effect of our regular DevEx focus is a system that works efficiently and improves the day-to-day experience of being an engineer at funda.

Key to making regular DevEx improvements is having a Platform-as-a-Product mindset. In part, this means thinking about the developers at funda as key stakeholders, because they are the end users of the platform. In addition to this, a product must be both long-lived and evolvable.

The logical conclusion of this is to iterate what you offer, constantly trying to make changes to the benefit of the company's developers. This will create a platform that is compelling to use, because developers will feel as though it is the easiest option to achieve their goals.

We have seen the benefits of doing this, not only with our modular pipelines, but also with the rest of the platform that we offer. If you adopt a product mindset, putting developer experience at the heart of your decision-making and workflows, you give your platform the best chance of success.