WikiGalaxy
Principles of Scalable System Design
Load Balancing:
Distribute incoming network traffic across multiple servers to ensure no single server bears too much load, improving the system's availability and reliability.
Caching:
Store copies of frequently accessed data in a 'cache' to reduce latency and improve data retrieval speed.
Database Sharding:
Partition a database into smaller, more manageable pieces called shards to improve performance and scalability.
Asynchronous Processing:
Decouple the request and response process to handle tasks asynchronously, improving responsiveness and throughput.
Microservices Architecture:
Design the application as a collection of loosely coupled services to enhance scalability, flexibility, and fault isolation.
Load Balancing
Round Robin:
Requests are distributed equally across all servers in a cyclic order. This method is simple but may not consider server load variations.
Least Connections:
Directs traffic to the server with the fewest active connections, optimizing resource utilization.
IP Hashing:
Uses the client's IP address to determine which server receives the request, maintaining session persistence.
Weighted Round Robin:
Assigns weights to servers based on their capabilities, allowing more powerful servers to handle more requests.
Geographic Load Balancing:
Routes user requests to the nearest data center, reducing latency and improving user experience.
import java.util.List;
import java.util.Random;
class LoadBalancer {
private List servers;
private Random random = new Random();
public LoadBalancer(List servers) {
this.servers = servers;
}
public String getServer() {
int index = random.nextInt(servers.size());
return servers.get(index);
}
}
public class Main {
public static void main(String[] args) {
List servers = List.of("Server1", "Server2", "Server3");
LoadBalancer lb = new LoadBalancer(servers);
System.out.println("Redirecting to: " + lb.getServer());
}
}
Round Robin:
This Java example demonstrates a simple random load balancer that selects a server randomly from a list. This approach is similar to round robin but does not guarantee equal distribution.
Least Connections:
Implementing a least connections strategy would involve tracking active connections to each server and selecting the one with the least load.
IP Hashing:
IP hashing can be implemented by converting the client's IP address into a hash and using it to select a server, ensuring consistent routing for the same client.
Weighted Round Robin:
Weights can be assigned to servers based on their capacity, and the load balancer can adjust the probability of selection accordingly.
Geographic Load Balancing:
Geographic load balancing requires a more complex setup involving multiple data centers and DNS-based routing to direct users to the closest server.
Console Output:
Redirecting to: Server2
Caching
In-Memory Caching:
Stores data in RAM for fast access, suitable for frequently accessed data.
Distributed Caching:
Spreads cache data across multiple nodes, enhancing scalability and fault tolerance.
Content Delivery Network (CDN):
Caches static content at edge locations closer to users, reducing latency and load on origin servers.
Write-Through Caching:
Updates cache and database simultaneously to maintain consistency.
Read-Through Caching:
Fetches data from the cache first and updates it from the database if necessary.
import java.util.HashMap;
import java.util.Map;
class Cache {
private Map cache = new HashMap<>();
public void put(String key, String value) {
cache.put(key, value);
}
public String get(String key) {
return cache.getOrDefault(key, "Not Found");
}
}
public class Main {
public static void main(String[] args) {
Cache cache = new Cache();
cache.put("user1", "John Doe");
System.out.println("User1: " + cache.get("user1"));
System.out.println("User2: " + cache.get("user2"));
}
}
In-Memory Caching:
This Java example illustrates a simple in-memory cache using a hashmap to store key-value pairs for quick access.
Distributed Caching:
Distributed caching can be implemented using tools like Redis or Memcached, which distribute data across multiple nodes.
Content Delivery Network (CDN):
CDNs cache static assets like images and scripts at various edge locations globally to serve content faster to users.
Write-Through Caching:
Write-through caching ensures data consistency by updating both the cache and database during writes.
Read-Through Caching:
Read-through caching reduces latency by checking the cache first and only querying the database if necessary.
Console Output:
User1: John Doe
User2: Not Found
Database Sharding
Horizontal Sharding:
Splits data across multiple tables with the same schema, distributing rows across shards.
Vertical Sharding:
Segments data by columns, storing different columns in separate databases.
Range-Based Sharding:
Divides data into ranges and assigns each range to a shard, useful for ordered data.
Hash-Based Sharding:
Uses a hash function to assign data to shards, providing even distribution.
Directory-Based Sharding:
Maintains a lookup table to map data to shards, offering flexibility in data placement.
import java.util.HashMap;
import java.util.Map;
class ShardManager {
private Map shards = new HashMap<>();
public ShardManager() {
shards.put(0, "Shard0");
shards.put(1, "Shard1");
}
public String getShard(int userId) {
int shardKey = userId % shards.size();
return shards.get(shardKey);
}
}
public class Main {
public static void main(String[] args) {
ShardManager sm = new ShardManager();
int userId = 12345;
System.out.println("User ID " + userId + " is in " + sm.getShard(userId));
}
}
Horizontal Sharding:
This example uses a hash-based approach to determine the shard for a given user ID, distributing data evenly.
Vertical Sharding:
Vertical sharding involves separating columns into different databases, which is not demonstrated here.
Range-Based Sharding:
Range-based sharding would require defining ranges and assigning them to shards, ideal for ordered data.
Hash-Based Sharding:
Hash-based sharding is implemented here by using the modulus operator to assign user IDs to shards.
Directory-Based Sharding:
Directory-based sharding uses a lookup table to map data to shards, offering more control over data placement.
Console Output:
User ID 12345 is in Shard0
Asynchronous Processing
Message Queues:
Decouple processes by using message queues to handle tasks asynchronously, improving scalability and fault tolerance.
Event-Driven Architecture:
Triggers actions in response to events, enabling real-time processing and responsiveness.
Task Scheduling:
Schedules tasks to run at specific times or intervals, managing background processes efficiently.
Callback Functions:
Use callback functions to execute code after an asynchronous operation completes, maintaining flow control.
Non-Blocking I/O:
Handles input/output operations without blocking the execution thread, improving performance.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
class Task implements Runnable {
private String name;
public Task(String name) {
this.name = name;
}
@Override
public void run() {
System.out.println("Executing task: " + name);
}
}
public class Main {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(2);
executor.submit(new Task("Task1"));
executor.submit(new Task("Task2"));
executor.shutdown();
}
}
Message Queues:
This example demonstrates asynchronous task execution using a thread pool, similar to message queue processing.
Event-Driven Architecture:
Event-driven architecture can be implemented using event listeners and handlers to respond to system events.
Task Scheduling:
Task scheduling can be managed using libraries like Quartz Scheduler to run tasks at specific intervals.
Callback Functions:
Callbacks can be used in asynchronous operations to execute code upon completion, maintaining control flow.
Non-Blocking I/O:
Non-blocking I/O allows the system to continue executing other tasks while waiting for I/O operations to complete.
Console Output:
Executing task: Task1
Executing task: Task2
Microservices Architecture
Autonomous Services:
Each service operates independently, allowing for easier scaling and deployment.
Decentralized Data Management:
Each microservice manages its own data, reducing dependencies and improving resilience.
API Gateway:
Acts as a single entry point for client requests, routing them to appropriate services.
Service Discovery:
Allows services to find each other dynamically, facilitating communication and scaling.
Fault Isolation:
Isolates failures to individual services, preventing system-wide outages.
class UserService {
public void getUser() {
System.out.println("Fetching user data...");
}
}
class OrderService {
public void getOrder() {
System.out.println("Fetching order data...");
}
}
public class Main {
public static void main(String[] args) {
UserService userService = new UserService();
OrderService orderService = new OrderService();
userService.getUser();
orderService.getOrder();
}
}
Autonomous Services:
This example illustrates two independent services, UserService and OrderService, operating autonomously.
Decentralized Data Management:
Each service could manage its own data, reducing cross-service dependencies and improving resilience.
API Gateway:
An API gateway can route requests to these services, acting as a single entry point for clients.
Service Discovery:
Service discovery mechanisms can help these services find and communicate with each other dynamically.
Fault Isolation:
If one service fails, it doesn't impact the others, demonstrating fault isolation in microservices.
Console Output:
Fetching user data...
Fetching order data...
