What is YouTube?

YouTube is a popular video streaming service where users upload, stream, search, comment, share, and like or dislike videos. Large businesses as well as individuals maintain channels where they host their videos. YouTube allows free hosting of video content to be shared with users globally. YouTube is considered a primary source of entertainment, especially among young people, and as of 2022, it is listed as the second-most viewed website after Google by Wikipedia.

YouTube’s popularity

Some of the reasons why YouTube has gained popularity over the years include the following reasons:

• Simplicity: YouTube has a simple yet powerful interface.
• Rich content: Its simple interface and free hosting has attracted a large number of content creators.
• Continuous improvement: The development team of YouTube has worked relentlessly to meet scalability requirements over time. Additionally, Google acquired YouTube in 2007, which has added to its credibility.
• Source of income: YouTube coined a partnership program where it allowed content creators to earn money with their viral content.

Apart from the above reasons, YouTube also partnered up with well-established market giants like CNN and NBC to set a stronger foothold.

YouTube’s growth

Let’s look at some interesting statistics about YouTube and its popularity.

Since its inception in February 2005, the number of YouTube users has multiplied. As of now, there are more than 2.5 billion monthly active users of YouTube. Let’s take a look at the chart below to see how YouTube users have increased in the last decade.

Increase in monthly active users of YouTube per year

YouTube is the second most-streamed website after Netflix. A total of 694,000 hours of video content is streamed on YouTube per minute!

Let’s evaluate your understanding of the building blocks required in the design of YouTube. Based on the functional and nonfunctional requirements given below, identify three key building blocks needed to curate the design of YouTube.

• Functional requirements:
  • Stream videos
  • Upload videos
  • Search videos according to titles
  • Like and dislike videos
  • Add comments to videos
  • View thumbnails
• Nonfunctional requirements:
  • High availability
  • Scalability
  • Good performance

You should focus on the building blocks required for back-end storage and efficient data transmission to end users. Please also specify the reason(s) behind the selection of a building block.Want to know the correct answer?

List three key building blocks necessary to design a system like YouTube.

Provide your answer here!

Let’s understand how to design YouTube in the next lessons.

How will we design YouTube?

We’ve divided the design of YouTube into five lessons:

1. Requirements: This is where we identify the functional and non-functional requirements. We also estimate the resources required to serve millions of users each day. This lesson answers questions like how much storage space YouTube will need to store 500 hours of video content uploaded to YouTube per minute.
2. Design: In this lesson, we explain how we’ll design the YouTube service. We also briefly explain the API design and database schemas. Lastly, we will also briefly go over how YouTube’s search works.
3. Evaluation: This lesson explains how YouTube is able to fulfill all the requirements through the proposed design. It also looks at how scaling in the future can affect the system and what solutions are required to deal with scaling problems.
4. Reality is more complicated: During this lesson, we’ll explore different techniques that YouTube employs to deliver content effectively to the client and avoid network congestions.
5. Quiz: We reinforce major concepts we learned designing YouTube by considering how we could design Netflix’s system.Our discussion on the usage of various building blocks in the design will be limited since we’ve already explored them in detail in the building blocks chapter.

Requirements of YouTube’s Design

Let’s start with the requirements for designing a system like YouTube.

Functional requirements

We require that our system is able to perform the following functions:

1. Stream videos
2. Upload videos
3. Search videos according to titles
4. Like and dislike videos
5. Add comments to videos
6. View thumbnails

Non-functional requirements

It’s important that our system also meets the following requirements:

• High availability: The system should be highly available. High availability requires a good percentage of uptime. Generally, an uptime of 99% and above is considered good.
• Scalability: As the number of users grows, these issues should not become bottlenecks: storage for uploading content, the bandwidth required for simultaneous viewing, and the number of concurrent user requests should not overwhelm our application/web server.
• Good performance: A smooth streaming experience leads to better performance overall.
• Reliability: Content uploaded to the system should not be lost or damaged.

We don’t require strong consistency for YouTube’s design. Consider an example where a creator uploads a video. Not all users subscribed to the creator’s channel should immediately get the notification for uploaded content.

To summarize, the functional requirements are the features and functionalities that the user will get, whereas the non-functional requirements are the expectations in terms of performance from the system.

Based on the requirements, we’ll estimate the required resources and design of our system.

Resource estimation

Estimation requires the identification of important resources that we’ll need in the system.

Hundreds of minutes of video content get uploaded to YouTube every minute. Also, a large number of users will be streaming content at the same time, which means that the following resources will be required:

• Storage resources will be needed to store uploaded and processed content.
• A large number of requests can be handled by doing concurrent processing. This means web/application servers should be in place to serve these users.
• Both upload and download bandwidth will be required to serve millions of users.

To convert the above resources into actual numbers, we assume the following:

• Total number of YouTube users: 1.5 billion.
• Active daily users (who watch or upload videos): 500 million.
• Average length of a video: 5 minutes.
• Size of an average (5 minute-long) video before processing/encoding (compression, format changes, and so on): 600 MB.
• Size of an average video after encoding (using different algorithms for different resolutions like MPEG-4 and VP9): 30 MB.

Storage estimation

To find the storage needs of YouTube, we have to estimate the total number of videos and the length of each video uploaded to YouTube per minute. Let’s consider that 500 hours worth of content is uploaded to YouTube in one minute. Since each video of 30 MB is 5 minutes long, we require 305530​ = 6 MB to store 1 minute of video.

Let’s put this in a formula by assuming the following:

𝑇𝑜𝑡𝑎𝑙 𝑠𝑡𝑜𝑟𝑎𝑔𝑒​: Total storage requirement.

𝑇𝑜𝑡𝑎𝑙 𝑢𝑝𝑙𝑜𝑎𝑑/𝑚𝑖𝑛​ : Total content uploaded (in minutes) per minute.

• Example: 500 hours worth of video is uploaded in one minute.

𝑆𝑡𝑜𝑟𝑎𝑔𝑒 𝑚𝑖𝑛​ : Storage required for each minute of content

Then, the following formula is used to compute the storage:

𝑇𝑜𝑡𝑎𝑙 𝑠𝑡𝑜𝑟𝑎𝑔𝑒 = 𝑇𝑜𝑡𝑎𝑙 𝑢𝑝𝑙𝑜𝑎𝑑/𝑚𝑖𝑛 × 𝑆𝑡𝑜𝑟𝑎𝑔𝑒 𝑚𝑖𝑛​

The numbers mentioned above correspond to the compressed version of videos. However, we need to transcode videos into various formats for reasons that we will see in the coming lessons. Therefore, we’ll require more storage space than the one estimated above.

Note: In a real-world scenario, YouTube’s design requires storage for thumbnails, users’ data, video metadata, users’ channel information, and so on. Since the storage requirement for these data sets will not be significant compared to video files, we ignore it for simplicity’s sake.

Bandwidth estimation

A lot of data transfer will be performed for streaming and uploading videos to YouTube. This is why we need to calculate our bandwidth estimation too. Assume the upload:view ratio is 1:300—that is, for each uploaded video, we have 300 video views per second. We’ll also have to keep in mind that when a video is uploaded, it is not in compressed format, while viewed videos can be of different qualities. Let’s estimate the bandwidth required for uploading the videos.

We assume:

𝑇𝑜𝑡𝑎𝑙 𝑏𝑎𝑛𝑑𝑤𝑖𝑑𝑡ℎ: Total bandwidth required.

𝑇𝑜𝑡𝑎𝑙 𝑐𝑜𝑛𝑡𝑒𝑛𝑡: Total content (in minutes) uploaded per minute.

𝑆𝑖𝑧𝑒-𝑚𝑖𝑛𝑢𝑡𝑒: Transmission required (in MBs) for each minute of content.

Then, the following formula is used to do the computation below:

𝑇𝑜𝑡𝑎𝑙 𝑏𝑎𝑛𝑑𝑤𝑖𝑑𝑡ℎ=𝑇𝑜𝑡𝑎𝑙 𝑐𝑜𝑛𝑡𝑒𝑛𝑡 𝑡𝑟𝑎𝑛𝑠𝑓𝑒𝑟𝑟𝑒𝑑 × 𝑆𝑖𝑧𝑒-𝑚𝑖𝑛𝑢𝑡𝑒​

We calculate bandwidth in bits per second (bps), as shown above. Therefore, the detailed calculation of the above number (480 Gbps) is given below.

Let’s convert the numbers to appropriate units first:

• No. of video hours per minute = 500 hours
• No. of video minutes per minute = 500 * 60 = 30,000 minutes
• MB per minute = 120 MB/minute
• MB per second = 120/60 = 2 MB/second

The 8 bits are used to convert bytes to bits. By convention metric of bandwidth is in bits per second.

Number of servers estimation

Let’s assume that we have 500 million daily active users of YouTube. Considering our assumption of using daily active users as a proxy for the number of requests per second to find the number of servers for peak load times, we get 500 million requests per second. Then, we use the following formula to calculate the number of servers:

Note: Concurrent requests significantly impact the number of required servers compared to requests spread over time. For a more in-depth exploration of refining peak load assumptions, refer to the “Back-of-the-envelope Calculations” chapter.

Building blocks we will use

Now that we have completed the resource estimations, let’s identify the building blocks that will be an integral part of our design for the YouTube system. The key building blocks are given below:

Building blocks in a high-level design:

• Databases are required to store the metadata of videos, thumbnails, comments, and user-related information.
• Blob storage is important for storing all videos on the platform.
• A CDN is used to effectively deliver content to end users, reducing delay and burden on end-servers.
• Load balancers are a necessity to distribute millions of incoming clients requests among the pool of available servers.

Other than our building blocks, we anticipate the use of the following components in our high-level design:

• Servers are a basic requirement to run application logic and entertain user requests.
• Encoders and transcoders compress videos and transform them into different formats and qualities to support varying numbers of devices according to their screen resolution and bandwidth.