Newsletters
Tech
Megan Rose Dickey
2013-04-23T11:32:00Z
Pinterest engineers Yashwanth Nelapati and Marty Weiner recently shared some insights and lessons learned while developing and scaling the company.
Advertisem*nt
Pinterest is a site for collecting and sharing photos of interesting objects around the web.
In a little over three years, Pinterest has grown from zero page views a day to billions per month. The most recent tally: 3.4 billion monthly page views from its 25 million members worldwide.
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
So they started dropping off technologies and did a massive restructuring of the architecture.
Advertisem*nt
Advertisem*nt
Pinterest uses Amazon EC2/S3 for a few reasons. The main one: you can have new instances ready in a matter of seconds.
Advertisem*nt
Advertisem*nt
Open source database MySQL has proven to be solid choice for Pinterest. It's incredibly mature and you can hire for it, as lots of engineers know MySQL.
Advertisem*nt
Pinterest realized that during its rapid growth, it needed to spread the data evenly to handle the load. So they defined a spectrum of options between clustering and sharding.
Advertisem*nt
But sharding is a completely manual data placement process. It's used to separate databases into smaller, faster, and more manageable data pieces called shards.
Advertisem*nt
Advertisem*nt
If there's a massive bug, it will impact every single node. A SPOF brought down Pinterest four times.
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
Advertisem*nt
With an increased load on a database, Pinterest replicated a server to handle some of the data nodes.
Advertisem*nt
Since Pinterest is on AWS and MySQL queries took about 3 milliseconds, they decided to build the location into the ID.
If Pinterest has 50 IDs, for example, they split them up and run them in parallel. This is what Pinterest's lookup/rendering structure looks like.
Advertisem*nt
Advertisem*nt
How Pinterest brings up a user profile. Most of the calls are served from the cache (Memcache or Redis)
Pinterest built a huge scripting farm to move 500 million Pins and 1.6 billion follower rows. Scripting is what happens when you need to move from the old, unsharded system to the sharded one.
Advertisem*nt
Advertisem*nt
Advertisem*nt
Need to give your brain a rest?
Read next
NEW LOOK
Sign up to get the inside scoop on today’s biggest stories in markets, tech, and business — delivered daily. Read preview
Thanks for signing up!
Access your favorite topics in a personalized feed while you're on the go.
Advertisem*nt