The benefits of having a lighthouse architecture

At funda, we love exploring new ways of working that contribute to our development process. Aleix Domènech, Staff Engineer, explains what our lighthouse architecture is, why we designed it, and how it helps funda become a better company.

Funda has been helping its users find their dream homes for over 20 years. During that time, the world has undergone incredibly fast technological advances. Can you imagine receiving search results on CD-ROM via postal mail? That’s what happened before funda. Today, we are a microservices company running natively in the cloud.

At funda, we have always adopted state-of-the-art technologies and practices, and to great success. At the same time, this has meant we've accumulated a whole bunch of very old stuff that, in one way or another, we have had to keep maintaining. At some point, the capacity to build new features for our customers was being greatly hindered by the amount of time we needed to spend just keeping what had been previously made running.

What is lighthouse architecture?

After the redesign of the website and the migration to the cloud, we decided that it was about time to do some spring cleaning and come up with a more robust and future-proof way of working. We called this new approach lighthouse architecture.

A lighthouse is a tower, building, or another type of structure designed to emit light from a system of lamps and lenses and serve as a beacon for navigational aid for maritime pilots at sea or inland waterways. -Wikipedia

By definition, if you were to reach the lighthouse on your boat, you'd end up sinking. That was the recurring joke when we originally came up with the terminology – but it was intentional. The lighthouse architecture is all about the journey, not the destination.

The goals of our lighthouse

The goals of the lighthouse architecture are derived from funda’s strategy for the upcoming years: to strengthen our position as the leading real estate platform in the Netherlands, and add more value to our customers' journeys. In essence, this means doing what we are doing a little bit better and adding more features faster than ever before. Specifically, the goals are as follows:

Team independence: Teams have to work without waiting for other teams. They also need to provision any infrastructure they require and should not be blocked by processes outside their control.
Decoupling: Our applications and services must remain decoupled from each other, preventing catastrophic cascading error events and other sorts of unwanted situations. Decoupled services are easier to maintain and make our developers happier.
Cloud native: Having a cloud-native architecture is the only way to go if your want to maximize your throughput while minimizing the cloud invoice at the end of every month.
Quality: We need clear quality metrics. We release features that provide value, work well, and deliver a nice and cohesive user experience.

We understand that in the world of web development, we will never be able to keep all of our services up to date, and that is ok. We build our platform to add new components without compromising the reliability of the whole, and that lets us play around, innovate and focus on creating new stuff, while getting rid of old features that no one likes to maintain.

The lighthouse architecture revolves around our teams, which are aided by their team leads and, of course, by me and my colleagues at the Staff Engineering department. The teams own their services; they have complete freedom to take any decisions when it comes to development and design, as long as it makes sense and fits the rest of the platform. This freedom allows them to move extra-fast and remain decoupled from the rest of their peers.

Case in point: new search experience

An excellent example of this freedom is one new feature that we will be releasing soon, and are very excited about: our new and revamped search experience. Previously, our Search Team spent a lot of time patching and keeping the old search function updated and working fast. They could not make drastic changes in the architecture or contracts because other teams depended on it. Thanks to the lighthouse architecture, the team was empowered to implement a revamped search experience with specialized and diverse software stacks, took the search component out of the old monolith, and even adopted a new search engine. In fact, what they did is so cool that other teams are starting to follow the same design, which is becoming a new standard at funda.

How it works

Figure 1: Team space

As we can see in figure 1, a team operates in their own space (1). In that space, they have complete freedom and end-to-end ownership of one or more bounded contexts (2). Bounded contexts are characterized by containing highly cohesive groups of services. We divide our bounded contexts into features (3), each of which can be composed of several services (4).

One example of a bounded context at funda is the listings. There, we can find everything that has to do with how listings enter our platform, how they are presented and how customers can interact and find them. Some examples of services inside the listings-bounded context include the search and detail pages services, ingestor services that make data available, and the saved search and saved listings services.

Benefits of this way of working

Making sure teams remain independent is key to leveraging their freedom, decreasing development time and improving developer happiness. At the same time, having fully independent teams comes with its own set of challenges. For example, how do other components outside the listings-bounded contexts have access to listings data without depending on services from the listings context? Our model is fully event-driven, which means that all of our microservices use a message broker pattern to receive updates from other features. This decouples them and makes them more resilient to change.

Figure 2: Team boundaries

Figure 2 illustrates how several independent bounded contexts distributed across two teams publish both message contract schemas (1) and event notifications for other bounded contexts to subscribe to (2). When it comes to messaging contracts, we've recently adopted the AsyncAPI specification, which integrates seamlessly with our developer portal, effectively shortening our lines of communication even more.

A feature in the lighthouse architecture is defined as one or more components that work together to achieve a common goal. It is serviced and maintained by a single team, meaning a feature will contain end-to-end logic to reach its purpose. A significant advantage of dividing our system into smaller features is that it allows teams to iterate quickly and innovate more. Teams can use this freedom to their advantage to reduce the time they spend working on older applications whilst having fun creating new stuff using the latest technologies. Additionally, the fact that a feature contains end-to-end logic makes it very easy for teams to fully own it because they only depend on themselves.

Figure 3: Features

In figure 3, we can see different possible feature sizes found within our system. They can be a simple UI component or a massive set of separate services and data sources working in harmony to provide a unique experience to our users. Again, teams have the freedom to decide how they want to design their features. Developers have examples available and shared packages of patterns and tools that have already been tested by other teams. Therefore we know they integrate well into our platform. At the same time, developers can decide to ignore all these recommendations and make their own choices; ultimately, it's up to them.

Conclusions

We don’t see the lighthouse architecture as an arbitrary set of rules defined by distant figures but as a framework that allows developers to have the freedom to make their own choices. Even though it’s all relatively new and we just started adopting this way of working, we’ve already found several improvements that tell us that we are definitely on the right track. What we like the most about our new way of working is that it allows our developers to naturally fall into the pit of success*, which prevents us from wasting time and resources and allows us to focus on the nicest part of our job: delivering value to our customers.

* The pit of success term was initially coined by Rico Mariani when talking about software language design. Make sure you read the original article and possibly the sequel, the pit of success for organizations.