Connect with us


A logical migration of Coinbase’s internal ledger

In 2019, Coinbase come into being to strengthen the infrastructure upon that its merchandise are engineered and to form a solid foundation to support current and future product lines. As a part of that effort, we tend to set to adopt AWS RDS PostgreSQL as our information of selection for relative workloads and AWS DynamoDB as our key-value store.

One of the primary areas we tend to set to concentrate on was a way to keep track of balances and move funds. Our merchandise had every devised their own solutions and a few heritage systems were conjointly littered with school debt and performance problems. the power to quickly and accurately method transactions is central to Coinbase’s mission to create an open financial set-up for the globe.

We designed and engineered a brand-new ledger service to be quick, accurate, and serve all current and future desires across merchandise and have undertaken our biggest migration as of nonetheless, moving over one billion rows of company and client dealings and balance info from the previous information storage to our new PostgreSQL-based answer, while not care and no perceptible impact to users.

Our key learnings:

  • Make it repeatable — you’ll not get wise right the primary time.
  • Make it quick — thus you’ll quickly ingeminate.
  • Make it quiet — By planning the method so it runs while not disrupting traditional business operations.

Here’s however we tend to do it:

Migration necessities

Accuracy and Correctness: Since we’re handling funds, we tend to know this is able to be a really sensitive migration and wished to require each precaution, confirm that each last Santoshi is accounted for.

Repeatable: to boot, the form of our information was fully totally different within the new system vs the heritage system. Further, we tend to had to handle technical debt and cruft accumulated over time in our monolithic application. we tend to knew we wanted to account for the chance of not obtaining everything right during a single go, thus we tend to wished to plan a repeatable method that we tend to might ingeminate on till obtaining it right.

No Maintenance Downtime: we tend to wish each dealing on Coinbase to execute whereas performing on this. we tend to didn’t wish to try and do any care or take any period for the transition.

The Setup

We can interpret the migration into a pair of separate problems: switch live writes and reads over the new service, and migrating all of the historical information into the new service.


The repeatable Backfill method

We thought-about multiple approaches to backfilling the billion-plus rows that represent all the transactions administrated on Coinbase from its origin, all with execs and cons.

The simplest answer would care to try and do it all at the applying level, investing the ledger consumer implementation we tend to have in situ for the twin writes. it’s the advantage of travail equivalent code ways we’ve in situ for live writes — there would be one mapping from recent to unaccustomed maintain. However, we might have had to change the service interface to permit for the backfill and that we would have had to line up long-running processes in conjunction with a checkpointing mechanism just in case of failure. we tend to conjointly benchmark this answer, and located that it’d be too slow to fulfill our necessities for quick iteration.

Continue Reading


Name Price
Bitcoin (BTC)
Ethereum (ETH)
Tether (USDT)
Litecoin (LTC)
Bitcoin Cash (BCH)



Copyright © 2020, powered by - 𝐈𝐧𝐯𝐞𝐬𝐭𝐦𝐞𝐧𝐭 𝐚𝐝𝐯𝐢𝐜𝐞 𝐝𝐢𝐬𝐜𝐥𝐚𝐢𝐦𝐞𝐫: The information contained on this website is provided for educational purposes, and does not constitute investment advice. - 𝗱𝗼𝗹𝗹𝘆𝗯𝘂𝘇𝘇.𝗰𝗼𝗺 is not a registered broker, analyst, or investment advisor.