Netflix’s Story of Migration to AWS
History: Netflix Before AWS
Netflix was founded on 29 August, 1997 by Marc Randolph and Reed Hastings in Scotts Valley, California. At the start, it offered a per-rental model for each DVD. The per-rental model was dropped by early 2000, allowing the company to focus on the business model of flat-fee unlimited rentals without due dates, late fees, shipping and handling fees, or per-title rental fees.
In January 2007, the company launched a streaming media service, introducing video on demand via the Internet. Netflix built two datacenters, located right next to each other but the company could not build data centres fast enough to meet the often-spiky demand of its users And as soon they got everything working they would run out of capacity, and the whole process had to start over again.The experience of building data centres taught Netflix an important lesson — they weren’t good at building data centres. What Netflix was good at was delivering video to their members. Netflix would rather concentrate on getting better at delivering video rather than getting better at building data centres.
In August 2008, Netflix experienced major database corruption for three days. This is when Netflix decided to go to AWS, even though Amazon is its biggest competitor. At this time, AWS was just getting established.
Why did Netflix choose AWS ?
In order to transition from vertically scaled single points of failure, such as relational databases on our datacenter, to highly reliable, horizontally scalable, distributed systems in the cloud, Netflix picked AWS as their cloud provider.
They switched to AWS because it offered them the most scope and the widest range of services and capabilities. Netflix wanted cloud computing to spread globally without having to construct its own data centres, but its outdated data centres couldn’t do so. How close Netflix was to going all-in on the AWS cloud was disclosed in an August 2015 Wall Street Journal piece.
Netflix relies on AWS to help it innovate with speed and consistently deliver best-in-class entertainment. AWS provides Netflix with compute, storage, and infrastructure that allow the company to scale quickly, operate securely, and meet capacity needs anywhere in the world.
Migration to AWS
It took 7 years for Netflix to migrate to AWS. The majority of their systems, including all customer facing services, had been migrated to the cloud prior to 2015. After that, the time taken was to figure out the perfect secure and durable path for their billing infrastructure and all the aspects of our customer and employee data management.
Finally in January 2016, Netflix finally completed its cloud migration and shut down its last remaining data centre bits by their data service. Though most of its systems are handled by AWS, due to an increase in the number of customers, Netflix decided to build its own Content Delivery Network (CDN) named “Open Connect” for storing and transmitting data. CDN basically copies the website and its media content to its servers spread around the world.
The migration improved Netflix’s scalability and service availability and the velocity by which the company could release new content, features, interfaces and interactions. It also freed up the capacity of engineers, cut the costs of streaming, drastically improved availability and added the experience and expertise of AWS.
How AWS Benefits Netflix
Moving to the cloud has brought Netflix a number of benefits. It got 8x more streaming members than they did in 2008, and they are much more engaged, with overall viewing growing by 3 times in 8 years. Netflix started its service in around 130 countries with millions of new customers after its migration.
After this, Netflix continued to evolve rapidly, incorporating many new resource-hungry features and relying on ever-growing volumes of data. Supporting such growth would have been difficult on Netflix’s own data centres. AWS’s Elasticity of the cloud allowed them to add thousands of virtual servers and petabytes of storage within minutes, making such an expansion possible.
On January 6, 2016, Netflix expanded its service to over 130 new countries, becoming a truly global Internet TV network. Leveraging multiple AWS cloud regions, spread all over the world, enabled Netflix to dynamically shift around and expand its global infrastructure capacity, creating a better and more enjoyable streaming experience for Netflix members wherever they are.
Netflix relies on the cloud for all of our scalable computing and storage needs — business logic, distributed databases and big data processing/analytics, recommendations, transcoding, and hundreds of other functions that make up the Netflix application. Video is delivered through Netflix Open Connect, our content delivery network that is distributed globally to efficiently deliver our bits to members’ devices.
The cloud also allowed Netflix to significantly increase their service availability. Cost reduction was not the main reason they moved to the cloud. However, their cloud costs per streaming start ended up being a fraction of those in the data centre. This is possible due to the elasticity of the cloud, enabling them to continuously optimise instance type mix and to grow and shrink their footprint near instantaneously without the need to maintain large capacity buffers.
AWS Services Netflix Uses
Netflix has a very complex architecture on AWS and they use all of them together as a hybrid system to stream and power their services like Amazon KDS, AWS Lambda, Amazon Route 53, EC2, ELB, TTL, ML, etc. Some of the key services used are are:-
Amazon Kinesis Data Streams (KDS)
Netflix is able to continually gather terabytes of data per second from tens of thousands of sources with the aid of KDS, and the data is made available in milliseconds for use in real-time analytics and stored in Amazon S3. This may be utilised to enhance both the customer experience and the referral programme.
AWS Lambda
With the help of AWS Lambda, Netflix created a self-managing architecture based on rules and replaced ineffective procedures to cut down on mistakes and gain crucial time.
With lambda, we can utilise the rules that are triggered by data modifications to determine what needs to be backed up, what needs to be duplicated to off-site storage, and to verify that it arrived securely. We can also restart the copies and recheck and revalidate if it failed to raise any alarms.
Additionally, it is utilised for automating security. With lambda, they can verify that every new instance is built and configured in compliance with the laws and circumstances, and they can also start a shutdown process or send out notifications when unauthorised instances enter their infrastructure.
Amazon Route 53
More than 190 different nations have Netflix access. A complicated DNS infrastructure is needed to make it robust for customers, which is where AWS Route 53 helps Netflix. All utilise distinct AWS Regions.
Divert 53 enables Netflix to swiftly and effectively route traffic to alternate regions in the event of a server or region failing or becoming overloaded.
Present Status & Conclusion
In its 20-year history, Netflix has grown from a DVD rental website with 30 employees to a global streaming service with over 5,000 titles, 130 million subscribers and $11 billion annual revenue that has drastically transformed the entertainment industry. Its service is now available in more than 190 countries.
In a nutshell, it uses its CDN for media storage and transmitting it, but for rest all the services like Recommander system, user Information, traffic Routing, Log, DNS Routing, etc., Netflix relies on AWS.
