As an organization wide architecture, each organization recommends to use tools and components from some cloud provider. So let's assume that we are recommended to use AWS components for our design choices. In real world architecture at any organization, architecture is usually a mix of in-house components and components provided by different cloud providers.
In AWS cloud, the implementation of the micro service instances can be achieved by utilising Elastic Compute Cloud (EC2) instances. EC2 is a web service that provides scalable compute capacity in the cloud.
We can create multiple EC2 instances to deploy our micro services and configure them behind an Elastic Load Balancer (ELB) to distribute the incoming traffic among the instances. This setup allows us to scale our microservices horizontally by adding more instances based on demand. Additionally, we can use auto-scaling groups to automatically adjust the number of instances based on metrics such as CPU utilization or network traffic.
We can also use AWS Elastic Container Service (ECS) or Kubernetes to deploy and manage containerised micro services. These services abstract away the underlying infrastructure and allow us to focus on managing and scaling our microservices. We can define our microservices as Docker containers and deploy them to an ECS cluster or a Kubernetes cluster. Both ECS and Kubernetes provide features such as automatic load balancing, auto-scaling, and rolling updates that help in managing micro services at scale.
API Gateway can be implemented using a service like AWS API Gateway, which provides features such as API management, routing, and security.
API Gateway can be used to create and manage RESTful APIs that allow clients to interact with the backend services in a secure and scalable way.
It also supports various authentication and authorisation mechanisms, including AWS IAM, OAuth 2.0, and custom authorisers.
Additionally, API Gateway can be configured with caching and throttling options to improve performance and prevent overloading of backend services. It can also integrate with other AWS services like AWS Lambda, which can be used to create serverless functions to process API requests.
The API Gateway can translate incoming requests from one protocol to another. For example, it can translate RESTful API requests to gRPC requests or vice versa.
The load balancer can be implemented using Amazon Elastic Load Balancing (ELB) depending on factors such as the expected traffic, scalability requirements, cost, and deployment environment.
Amazon ELB is a fully-managed load balancing service provided by Amazon Web Services (AWS) that automatically distributes incoming traffic across multiple targets, such as Amazon EC2 instances or containers, and can scale up or down based on demand. If the system is deployed on AWS, using ELB might provide better integration and performance.
Considering the requirement of high read QPS and low latency, a NoSQL database would be a more suitable choice for storing media metadata. Some popular NoSQL databases that can be considered for this use case are MongoDB, Cassandra, and Amazon DynamoDB. These databases are known for their ability to handle large volumes of data with high throughput and low latency.
We will use Amazon DynamoDB to store media metadata. Amazon DynamoDB is capable of handling large amounts of unstructured data, which would be suitable for storing media metadata such as title, description, type, metadata as key-value pairs, genre id, language id, asset user id, and other relevant information. They also provide high availability and scalability, which is important for a system that needs to handle a large volume of media content. Additionally, Amazon DynamoDB has good integration with the AWS Cloud, which would be useful if we were to deploy our system on AWS.
In AWS, there are several storage services that can be used for media storage, depending on the specific requirements of the application. However, the most suitable storage service for the "media storage" component in our design would be Amazon S3 (Simple Storage Service).
Amazon S3 provides highly scalable and durable object storage for any type of data, including media files. It offers low latency access to data and can handle high read and write QPS (queries per second) required for media storage. It also offers various storage classes, such as Standard, Infrequent Access (IA), and Glacier, which can be used based on the access frequency and cost requirements of the application.
In addition, Amazon S3 integrates well with other AWS services, such as Amazon CloudFront for content delivery and Amazon Elastic Transcoder for media transcoding. It also offers advanced security features, such as server-side encryption, access control, and auditing.
Overall, Amazon S3 provides a reliable and cost-effective solution for media storage in the AWS cloud.