Etsy is an online marketplace that connects people across the globe to buy and sell handmade and vintage goods.
Founded in 2005 in Brooklyn, Etsy has grown rapidly.
As a result, Etsy has need to scale its infrastructure to serve its growing user-base as well as its developers.
In this talk, Bethany Macri, Core Platform Engineer at Etsy, will discuss three software projects that demonstrate how Etsy changed its infrastructure to scale and maintain high availability:
- a four-year data migration project
- writing a new API that allowed clients to fetch data concurrently, and
- scaling datasets infrastructure.
This talk will propose a model for scaling as well as discussing possible ways Etsy will continue to scale as it grows.
- The speaker is on the Core Platform team which is an Infrastructure team.
- 36 Million Items for sale
- 22 Million active buyers
- Transacts nearly every country in the world
- 9 offices in 7 countries
- 800 employees. 51% women
- Certified B corporation
Etsy Engineering culture: Code as craft:
- More senior engineers take junior engneers under heir wing - apprenticeship model
- Practical ways practicing the craft:
- Continous Integration – push to the site 35 times daily. (a push is not a release).
-
- Measure everything - metrics driven organization.
- Blameless most mortems.
Scalability
Scalability: It's the ability of a program or a system to continue to function as its input grows in size and/or quantity.
Scalability on Wikipedia
Growth in terms of
- Number of users (increased traffic and data to store, more shops, more images)
- Size of code base
- Number of engineers (over 200 now)
- Number and complexity of features.
- Sharding
- API v3
- Big Data Access pattern
Technical stack
- PostgreSQL
- PHP
- LAMP stack