Airbnb's massive deployment technique: 125,000+ times a year

Airbnb is one of the hottest startups today. They have reformed the hotel industry and have become a go-to for travellers looking for unique, affordable accommodations. Apart from the business success, to a Developer, the most impressive point is their deployment schedule.

According to recent data, Airbnb deploys its code over 125,000 times yearly. That is an average of 345 deployments per day. The question is: How does Airbnb keep up with such a demanding schedule? This involves state of the art scaling and deployment techniques for a ever changing codebase of Airbnb.

The short answer is through Multicluster Kubernetes.

• Initially, Airbnb started with Ruby on Rails monolith which they called Monorail, but as the company started growing, they realized that this architecture could not scale.

• To solve this issue, they shifted to Service Oriented architecture (SOA) on Amazon Elastic Compute Cloud. They divided Monorail into two major services:

  • Hyperloop
  • Treehouse
    While service oriented architecture solved tight coupling issues, the scaling issues were still there.

• To solve scaling issue, Airbnb migrated to Kubernetes. Airbnb's engineering team started to use YAML templates to minimize complexity and abstract configurations.

• They started using Git to store configuration files, which helped them streamline their review, updates, and commits process.

• Later in the same year, they migrated to Kubernetes, and their deployment started failing due to the etcd (key value store) cluster getting out of memory. Fortunately, Kubernetes stopped any deployment and scaling in the process whenever etcd failed.

• This issue was finally resolved by upgrading to the V3 data format of etcd. When their cluster node doubled in size again, they started facing more issues. By using SmartStack- Airbnb's service Mesh they transitioned to a multi-clustered environment.

• AirBnb then created the kube-system (in-house method for deploying clusters), which allowed them to deploy to clusters and ensured equal cluster performance.

With the migration to Kubernetes, Airbnb has reached no less than 125,000 production deployments per year.

Airbnb is one of the biggest success stories of Kubernetes. It has not only benefited from it but has also contributed to its development.

Airbnb's microservices migration story is one of those rare tales that start with "it was a total mess, but we learned from our mistakes, and now everything is awesome." It is a great example of how a company can use technology to overcome organizational challenges and emerge stronger on the other side.