Interview preparation · Technical guide

System Design Terminology: Questions and Answers

System design concepts, trade-offs, and interview prompts. Numbers and named technologies are illustrative; capacity, consistency, and security decisions require explicit requirements.

Examples are independent teaching snippets and may require application types, imports, packages, schema, and configuration. Framework behavior is version-dependent. Corrections address identified issues; the complete source code collection has not been compiled or integration-tested.

1. What is System Design?

System design is the process of defining the architecture, components, modules, interfaces, and data for a system to satisfy specified requirements. It involves creating a blueprint for how different parts of a system will work together to achieve the desired functionality, performance, scalability, and reliability.

Key aspects: - Architecture patterns and decisions - Component interactions - Data flow and storage - Performance optimization - Scalability considerations - Security and reliability

2. What are the Key Components of System Design?

Core Components:

1. Frontend/Client Layer - User interfaces and client applications
2. API Gateway - Entry point for client requests
3. Load Balancer - Distributes traffic across servers
4. Application Servers - Business logic and processing
5. Database Layer - Data storage and retrieval
6. Cache Layer - Fast data access
7. Message Queue - Asynchronous communication
8. CDN - Content delivery optimization
9. Monitoring & Logging - System observability

3. How do you Approach a System Design Interview Question?

Step-by-Step Framework:

1. Clarify Requirements - Functional requirements - Non-functional requirements (scale, performance, availability) - Constraints and assumptions

2. Estimate Scale - Traffic volume - Storage requirements - Data processing needs

3. High-Level Design - System components - Data flow - Technology choices

4. Detailed Design - Component interactions - Data models - API specifications

5. Identify Bottlenecks - Performance issues - Scalability concerns - Single points of failure

6. Optimize - Caching strategies - Load balancing - Database optimization

4. What is Scalability?

Scalability is the ability to handle increased load while meeting defined service objectives, often by adding resources or improving efficiency. It is not synonymous with availability, reliability, or low latency. State the load dimension: requests, concurrent users, data volume, throughput, or geographic regions.

5. What is Availability?

Availability is the fraction of time a service is able to serve its intended requests under an agreed definition. It depends on the measurement window, exclusions, dependency behavior, and whether partial functionality counts. Four nines is a target, not a design property by itself.

6. What is Reliability?

Reliability is the ability of a system to perform its required functions under stated conditions for a specified period of time. It includes fault tolerance and error handling.

// Circuit Breaker Pattern Implementation
public class CircuitBreaker
{
    private CircuitState _state = CircuitState.Closed;
    private int _failureCount = 0;
    private readonly int _threshold;
    private readonly TimeSpan _timeout;

    public async Task<T> ExecuteAsync<T>(Func<Task<T>> operation)
    {
        if (_state == CircuitState.Open)
        {
            throw new CircuitBreakerOpenException();
        }

        try
        {
            var result = await operation();
            OnSuccess();
            return result;
        }
        catch (Exception)
        {
            OnFailure();
            throw;
        }
    }

    private void OnSuccess()
    {
        _failureCount = 0;
        _state = CircuitState.Closed;
    }

    private void OnFailure()
    {
        _failureCount++;
        if (_failureCount >= _threshold)
        {
            _state = CircuitState.Open;
        }
    }
}

7. What is Consistency?

Consistency ensures that all nodes in a distributed system see the same data at the same time. It's about maintaining data integrity across the system.

Types: - Strong Consistency - All nodes see the same data simultaneously - Eventual Consistency - All nodes will eventually see the same data - Weak Consistency - No guarantee of consistency

// Strong Consistency Example
public class ConsistentDataStore
{
    private readonly IDistributedCache _cache;
    private readonly IDatabase _database;

    public async Task<bool> WriteWithConsistencyAsync(string key, string value)
    {
        // Use distributed lock for strong consistency
        using var lockHandle = await _cache.CreateLockAsync(key);

        if (lockHandle.Acquired)
        {
            try
            {
                await _database.WriteAsync(key, value);
                await _cache.SetAsync(key, value);
                return true;
            }
            finally
            {
                await lockHandle.ReleaseAsync();
            }
        }

        return false;
    }
}

8. What is Partition Tolerance?

Partition tolerance is the ability of a distributed system to continue operating even when network partitions occur (some nodes become unreachable).

// Partition Tolerant Service
public class PartitionTolerantService
{
    private readonly List<INode> _nodes;
    private readonly IConsensusAlgorithm _consensus;

    public async Task<OperationResult> ExecuteOperationAsync(Operation operation)
    {
        var quorum = _nodes.Count / 2 + 1;
        var responses = new List<NodeResponse>();

        foreach (var node in _nodes)
        {
            try
            {
                var response = await node.ExecuteAsync(operation);
                responses.Add(response);
            }
            catch (NetworkException)
            {
                // Node is partitioned, continue with others
                continue;
            }
        }

        // Continue if we have quorum
        if (responses.Count >= quorum)
        {
            return await _consensus.ReachConsensusAsync(responses);
        }

        throw new PartitionException("Insufficient nodes available");
    }
}

9. What is CAP Theorem?

CAP concerns behavior during a network partition in a distributed data system: a design cannot simultaneously guarantee both strong consistency and availability for every request when partitions occur. It does not say a system chooses only two properties forever, and partition tolerance is not optional for systems that communicate over networks.

10. What is Load Balancing?

Load balancing distributes incoming network traffic across multiple servers to ensure no single server bears too much load.

// Load Balancer Implementation
public class LoadBalancer
{
    private readonly List<Server> _servers;
    private readonly ILoadBalancingAlgorithm _algorithm;

    public async Task<Server> GetNextServerAsync()
    {
        var healthyServers = _servers.Where(s => s.IsHealthy).ToList();

        if (!healthyServers.Any())
        {
            throw new NoAvailableServersException();
        }

        return await _algorithm.SelectServerAsync(healthyServers);
    }
}

// Round Robin Algorithm
public class RoundRobinAlgorithm : ILoadBalancingAlgorithm
{
    private int _currentIndex = 0;
    private readonly object _lock = new object();

    public Task<Server> SelectServerAsync(List<Server> servers)
    {
        lock (_lock)
        {
            var server = servers[_currentIndex % servers.Count];
            _currentIndex++;
            return Task.FromResult(server);
        }
    }
}

11. What is Caching?

Caching stores frequently accessed data in fast-access storage to improve performance and reduce load on primary data sources.

// Multi-Level Cache Implementation
public class MultiLevelCache
{
    private readonly IMemoryCache _l1Cache; // Fastest
    private readonly IDistributedCache _l2Cache; // Slower but shared
    private readonly IDataStore _dataStore; // Slowest

    public async Task<T> GetAsync<T>(string key)
    {
        // L1 Cache Check
        if (_l1Cache.TryGetValue(key, out T l1Value))
        {
            return l1Value;
        }

        // L2 Cache Check
        var l2Value = await _l2Cache.GetAsync<T>(key);
        if (l2Value != null)
        {
            // Populate L1 cache
            _l1Cache.Set(key, l2Value, TimeSpan.FromMinutes(5));
            return l2Value;
        }

        // Data Store Check
        var dataValue = await _dataStore.GetAsync<T>(key);
        if (dataValue != null)
        {
            // Populate both caches
            await _l2Cache.SetAsync(key, dataValue, TimeSpan.FromMinutes(30));
            _l1Cache.Set(key, dataValue, TimeSpan.FromMinutes(5));
        }

        return dataValue;
    }
}

12. What is Sharding?

Sharding is a database architecture pattern where data is horizontally partitioned across multiple databases or tables based on a shard key.

// Database Sharding Implementation
public class ShardedDatabase
{
    private readonly Dictionary<string, IDatabase> _shards;
    private readonly IShardingStrategy _shardingStrategy;

    public async Task<T> GetAsync<T>(string key)
    {
        var shardKey = _shardingStrategy.GetShardKey(key);
        var shard = _shards[shardKey];
        return await shard.GetAsync<T>(key);
    }

    public async Task SetAsync<T>(string key, T value)
    {
        var shardKey = _shardingStrategy.GetShardKey(key);
        var shard = _shards[shardKey];
        await shard.SetAsync(key, value);
    }
}

// Hash-based Sharding
public class HashShardingStrategy : IShardingStrategy
{
    private readonly int _shardCount;

    public string GetShardKey(string key)
    {
        var hash = key.GetHashCode();
        var shardIndex = Math.Abs(hash) % _shardCount;
        return $"shard_{shardIndex}";
    }
}

13. What is Replication?

Replication is the process of copying data from one database server to another to ensure data availability and fault tolerance.

// Database Replication Implementation
public class ReplicatedDatabase
{
    private readonly IDatabase _primary;
    private readonly List<IDatabase> _replicas;

    public async Task<bool> WriteAsync(string key, object value)
    {
        // Write to primary
        var primaryResult = await _primary.WriteAsync(key, value);

        if (primaryResult)
        {
            // Replicate to all replicas asynchronously
            var replicationTasks = _replicas.Select(replica => 
                replica.WriteAsync(key, value));

            await Task.WhenAll(replicationTasks);
        }

        return primaryResult;
    }

    public async Task<object> ReadAsync(string key)
    {
        // Read from any available replica for better performance
        foreach (var replica in _replicas)
        {
            try
            {
                return await replica.ReadAsync(key);
            }
            catch (Exception)
            {
                continue;
            }
        }

        // Fallback to primary
        return await _primary.ReadAsync(key);
    }
}

14. What is a CDN?

A Content Delivery Network (CDN) is a distributed network of servers that deliver web content to users based on their geographic location.

// CDN Service Implementation
public class CDNService
{
    private readonly Dictionary<string, List<string>> _cdnNodes;
    private readonly IGeolocationService _geolocationService;

    public async Task<string> GetOptimalNodeAsync(string userLocation)
    {
        var nearestNode = await _geolocationService.FindNearestNodeAsync(userLocation);
        return nearestNode;
    }

    public async Task<bool> CacheContentAsync(string contentId, byte[] content)
    {
        var cacheTasks = _cdnNodes.Values.SelectMany(nodes => 
            nodes.Select(node => CacheOnNodeAsync(node, contentId, content)));

        await Task.WhenAll(cacheTasks);
        return true;
    }

    private async Task<bool> CacheOnNodeAsync(string node, string contentId, byte[] content)
    {
        // Implementation for caching content on specific CDN node
        return true;
    }
}

15. What is a Reverse Proxy?

A reverse proxy is a server that sits between client devices and backend servers, forwarding client requests to appropriate backend servers.

// Reverse Proxy Implementation
public class ReverseProxy
{
    private readonly Dictionary<string, string> _routingRules;
    private readonly ILoadBalancer _loadBalancer;

    public async Task<HttpResponseMessage> ForwardRequestAsync(HttpRequestMessage request)
    {
        var targetService = DetermineTargetService(request);
        var targetServer = await _loadBalancer.GetNextServerAsync(targetService);

        return await ForwardToServerAsync(request, targetServer);
    }

    private string DetermineTargetService(HttpRequestMessage request)
    {
        var path = request.RequestUri.AbsolutePath;

        if (path.StartsWith("/api/users"))
            return "user-service";
        else if (path.StartsWith("/api/products"))
            return "product-service";
        else
            return "default-service";
    }
}

16. What is a Message Queue?

A message queue is a form of asynchronous communication between services, where messages are stored in a queue until they are processed.

// Message Queue Implementation
public class MessageQueue<T>
{
    private readonly Queue<Message<T>> _queue = new Queue<Message<T>>();
    private readonly object _lock = new object();
    private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(0);

    public async Task EnqueueAsync(T message)
    {
        lock (_lock)
        {
            _queue.Enqueue(new Message<T> { Data = message, Timestamp = DateTime.UtcNow });
        }
        _semaphore.Release();
    }

    public async Task<Message<T>> DequeueAsync(CancellationToken cancellationToken = default)
    {
        await _semaphore.WaitAsync(cancellationToken);

        lock (_lock)
        {
            return _queue.Dequeue();
        }
    }
}

// Message Consumer
public class MessageConsumer<T>
{
    private readonly MessageQueue<T> _queue;
    private readonly Func<T, Task> _processor;

    public async Task StartConsumingAsync()
    {
        while (true)
        {
            var message = await _queue.DequeueAsync();
            await _processor(message.Data);
        }
    }
}

17. What is Eventual Consistency?

Eventual consistency means replicas may temporarily differ but are expected to converge if writes stop and replication succeeds. It does not specify a convergence deadline, conflict-resolution rule, read guarantee, or absence of lost updates; the protocol must define these.

18. What is Strong Consistency?

Strong consistency ensures that all nodes in a distributed system see the same data at the same time, with no stale data.

// Strong Consistency Implementation
public class StronglyConsistentStore
{
    private readonly IDatabase _primary;
    private readonly List<IDatabase> _replicas;
    private readonly IDistributedLock _lock;

    public async Task<bool> WriteAsync(string key, object value)
    {
        using var lockHandle = await _lock.AcquireAsync(key);

        // Write to primary first
        await _primary.WriteAsync(key, value);

        // Synchronously replicate to all replicas
        foreach (var replica in _replicas)
        {
            await replica.WriteAsync(key, value);
        }

        return true;
    }

    public async Task<object> ReadAsync(string key)
    {
        // Always read from primary for strong consistency
        return await _primary.ReadAsync(key);
    }
}

19. What is Horizontal Scaling?

Horizontal scaling (scaling out) involves adding more machines or nodes to a system to handle increased load.

// Horizontal Scaling Implementation
public class HorizontallyScalableService
{
    private readonly List<IServiceInstance> _instances;
    private readonly IServiceDiscovery _serviceDiscovery;
    private readonly ILoadBalancer _loadBalancer;

    public async Task<bool> ScaleOutAsync()
    {
        // Create new service instance
        var newInstance = await CreateNewInstanceAsync();

        // Register with service discovery
        await _serviceDiscovery.RegisterAsync(newInstance);

        // Add to load balancer
        _loadBalancer.AddInstance(newInstance);

        return true;
    }

    public async Task<bool> ScaleInAsync()
    {
        // Remove least loaded instance
        var instanceToRemove = await _loadBalancer.GetLeastLoadedInstanceAsync();

        // Deregister from service discovery
        await _serviceDiscovery.DeregisterAsync(instanceToRemove);

        // Remove from load balancer
        _loadBalancer.RemoveInstance(instanceToRemove);

        // Terminate instance
        await TerminateInstanceAsync(instanceToRemove);

        return true;
    }
}

20. What is Vertical Scaling?

Vertical scaling (scaling up) involves adding more power (CPU, RAM, storage) to existing machines.

// Vertical Scaling Implementation
public class VerticallyScalableService
{
    private readonly IResourceMonitor _resourceMonitor;
    private readonly IScalingPolicy _scalingPolicy;

    public async Task<bool> ScaleUpAsync()
    {
        var currentResources = await _resourceMonitor.GetCurrentResourcesAsync();
        var scalingDecision = await _scalingPolicy.EvaluateScalingAsync(currentResources);

        if (scalingDecision.ShouldScaleUp)
        {
            // Request more resources from infrastructure
            await RequestMoreResourcesAsync(scalingDecision.RequiredResources);

            // Restart service with new resources
            await RestartWithNewResourcesAsync();

            return true;
        }

        return false;
    }

    private async Task<ResourceSpecification> RequestMoreResourcesAsync(ResourceSpecification required)
    {
        // Implementation for requesting more CPU, RAM, etc.
        return new ResourceSpecification();
    }
}

21. What is a Microservices Architecture?

Microservices architecture is an architectural style where an application is built as a collection of small, independent services that communicate over well-defined APIs.

// Microservice Example
public class UserService
{
    private readonly IUserRepository _userRepository;
    private readonly IEventBus _eventBus;

    public async Task<User> CreateUserAsync(CreateUserRequest request)
    {
        var user = new User
        {
            Id = Guid.NewGuid(),
            Name = request.Name,
            Email = request.Email
        };

        await _userRepository.CreateAsync(user);

        // Publish event for other services
        await _eventBus.PublishAsync(new UserCreatedEvent(user));

        return user;
    }

    public async Task<User> GetUserAsync(Guid userId)
    {
        return await _userRepository.GetByIdAsync(userId);
    }
}

// Service Communication
public class ServiceCommunication
{
    private readonly IHttpClientFactory _httpClientFactory;

    public async Task<T> CallServiceAsync<T>(string serviceName, string endpoint, object data)
    {
        var client = _httpClientFactory.CreateClient(serviceName);
        var response = await client.PostAsJsonAsync(endpoint, data);
        return await response.Content.ReadFromJsonAsync<T>();
    }
}

22. What is a Monolithic Architecture?

A monolithic architecture is where an entire application is built as a single, unified unit where all components are tightly coupled.

// Monolithic Application Example
public class MonolithicApplication
{
    private readonly IUserService _userService;
    private readonly IProductService _productService;
    private readonly IOrderService _orderService;
    private readonly IInventoryService _inventoryService;

    public async Task<OrderResult> ProcessOrderAsync(OrderRequest request)
    {
        // All services are part of the same application
        var user = await _userService.GetUserAsync(request.UserId);
        var product = await _productService.GetProductAsync(request.ProductId);
        var inventory = await _inventoryService.CheckInventoryAsync(request.ProductId);

        if (inventory.IsAvailable)
        {
            var order = await _orderService.CreateOrderAsync(request);
            await _inventoryService.UpdateInventoryAsync(request.ProductId, -1);
            return new OrderResult { Success = true, OrderId = order.Id };
        }

        return new OrderResult { Success = false, Message = "Out of stock" };
    }
}

23. What is Service Discovery?

Service discovery is a mechanism that allows services to find and communicate with each other in a distributed system.

// Service Discovery Implementation
public class ServiceDiscovery
{
    private readonly Dictionary<string, List<ServiceInstance>> _services;
    private readonly IHealthChecker _healthChecker;

    public async Task<List<ServiceInstance>> DiscoverServicesAsync(string serviceName)
    {
        if (_services.TryGetValue(serviceName, out var instances))
        {
            // Filter healthy instances
            var healthyInstances = new List<ServiceInstance>();

            foreach (var instance in instances)
            {
                if (await _healthChecker.IsHealthyAsync(instance))
                {
                    healthyInstances.Add(instance);
                }
            }

            return healthyInstances;
        }

        return new List<ServiceInstance>();
    }

    public async Task RegisterServiceAsync(string serviceName, ServiceInstance instance)
    {
        if (!_services.ContainsKey(serviceName))
        {
            _services[serviceName] = new List<ServiceInstance>();
        }

        _services[serviceName].Add(instance);
    }
}

24. What is API Gateway?

An API Gateway is a server that acts as an entry point for all client requests, routing them to appropriate microservices.

// API Gateway Implementation
public class ApiGateway
{
    private readonly IRoutingService _routingService;
    private readonly IAuthenticationService _authService;
    private readonly IRateLimiter _rateLimiter;

    public async Task<HttpResponseMessage> ProcessRequestAsync(HttpRequestMessage request)
    {
        // Authentication
        var authResult = await _authService.AuthenticateAsync(request);
        if (!authResult.IsAuthenticated)
        {
            return new HttpResponseMessage(HttpStatusCode.Unauthorized);
        }

        // Rate limiting
        if (!await _rateLimiter.AllowRequestAsync(request))
        {
            return new HttpResponseMessage(HttpStatusCode.TooManyRequests);
        }

        // Route to appropriate service
        var targetService = await _routingService.GetTargetServiceAsync(request);
        return await ForwardRequestAsync(request, targetService);
    }

    private async Task<HttpResponseMessage> ForwardRequestAsync(HttpRequestMessage request, string targetService)
    {
        // Implementation for forwarding request to target service
        return new HttpResponseMessage();
    }
}

25. What is Rate Limiting?

Rate limiting is a technique used to control the rate of requests a client can make to an API within a specified time window.

// Rate Limiting Implementation
public class RateLimiter
{
    private readonly Dictionary<string, Queue<DateTime>> _requestHistory;
    private readonly int _maxRequests;
    private readonly TimeSpan _window;

    public async Task<bool> AllowRequestAsync(string clientId)
    {
        var now = DateTime.UtcNow;

        if (!_requestHistory.ContainsKey(clientId))
        {
            _requestHistory[clientId] = new Queue<DateTime>();
        }

        var queue = _requestHistory[clientId];

        // Remove old requests outside the window
        while (queue.Count > 0 && now - queue.Peek() > _window)
        {
            queue.Dequeue();
        }

        // Check if request is allowed
        if (queue.Count < _maxRequests)
        {
            queue.Enqueue(now);
            return true;
        }

        return false;
    }
}

// Token Bucket Algorithm
public class TokenBucketRateLimiter
{
    private readonly Dictionary<string, TokenBucket> _buckets;
    private readonly int _capacity;
    private readonly double _refillRate;

    public async Task<bool> AllowRequestAsync(string clientId)
    {
        if (!_buckets.ContainsKey(clientId))
        {
            _buckets[clientId] = new TokenBucket(_capacity, _refillRate);
        }

        return await _buckets[clientId].ConsumeTokenAsync();
    }
}

26. What is Throttling?

Throttling is a technique to limit the rate of processing requests, often used to protect system resources.

// Throttling Implementation
public class Throttler
{
    private readonly SemaphoreSlim _semaphore;
    private readonly int _maxConcurrentRequests;

    public Throttler(int maxConcurrentRequests)
    {
        _maxConcurrentRequests = maxConcurrentRequests;
        _semaphore = new SemaphoreSlim(maxConcurrentRequests);
    }

    public async Task<T> ExecuteAsync<T>(Func<Task<T>> operation)
    {
        await _semaphore.WaitAsync();

        try
        {
            return await operation();
        }
        finally
        {
            _semaphore.Release();
        }
    }
}

// Adaptive Throttling
public class AdaptiveThrottler
{
    private readonly Queue<double> _responseTimes;
    private readonly int _windowSize;
    private double _threshold;

    public async Task<bool> ShouldThrottleAsync()
    {
        var avgResponseTime = _responseTimes.Average();

        if (avgResponseTime > _threshold)
        {
            return true; // Throttle requests
        }

        return false;
    }
}

27. What is Circuit Breaker Pattern?

The circuit breaker pattern prevents a system from making calls to a failing service, allowing it to fail fast and recover gracefully.

// Circuit Breaker Implementation
public class CircuitBreaker
{
    private CircuitState _state = CircuitState.Closed;
    private int _failureCount = 0;
    private readonly int _threshold;
    private readonly TimeSpan _timeout;
    private DateTime _lastFailureTime;

    public async Task<T> ExecuteAsync<T>(Func<Task<T>> operation)
    {
        if (_state == CircuitState.Open)
        {
            if (DateTime.UtcNow - _lastFailureTime > _timeout)
            {
                _state = CircuitState.HalfOpen;
            }
            else
            {
                throw new CircuitBreakerOpenException();
            }
        }

        try
        {
            var result = await operation();
            OnSuccess();
            return result;
        }
        catch (Exception)
        {
            OnFailure();
            throw;
        }
    }

    private void OnSuccess()
    {
        _failureCount = 0;
        _state = CircuitState.Closed;
    }

    private void OnFailure()
    {
        _failureCount++;
        _lastFailureTime = DateTime.UtcNow;

        if (_failureCount >= _threshold)
        {
            _state = CircuitState.Open;
        }
    }
}

public enum CircuitState
{
    Closed,
    Open,
    HalfOpen
}

28. What is Bulkhead Pattern?

The bulkhead pattern isolates different parts of a system so that a failure in one part doesn't bring down the entire system.

// Bulkhead Pattern Implementation
public class BulkheadIsolation
{
    private readonly Dictionary<string, SemaphoreSlim> _bulkheads;
    private readonly Dictionary<string, int> _limits;

    public async Task<T> ExecuteInBulkheadAsync<T>(string bulkheadName, Func<Task<T>> operation)
    {
        if (!_bulkheads.ContainsKey(bulkheadName))
        {
            var limit = _limits.GetValueOrDefault(bulkheadName, 10);
            _bulkheads[bulkheadName] = new SemaphoreSlim(limit);
        }

        var bulkhead = _bulkheads[bulkheadName];

        await bulkhead.WaitAsync();

        try
        {
            return await operation();
        }
        finally
        {
            bulkhead.Release();
        }
    }
}

// Service Isolation Example
public class IsolatedService
{
    private readonly BulkheadIsolation _bulkhead;

    public async Task<User> GetUserAsync(Guid userId)
    {
        return await _bulkhead.ExecuteInBulkheadAsync("user-service", async () =>
        {
            // User service operation
            return new User();
        });
    }

    public async Task<Product> GetProductAsync(Guid productId)
    {
        return await _bulkhead.ExecuteInBulkheadAsync("product-service", async () =>
        {
            // Product service operation
            return new Product();
        });
    }
}

29. What is Idempotency?

An operation is idempotent when repeating the same request has the same intended effect as performing it once. Create operations can become idempotent by using a client-provided idempotency key with a defined key scope, payload matching rule, retention time, and replayed-result behavior.

30. What is a Distributed System?

A distributed system is a system whose components are located on different networked computers that coordinate their actions by passing messages to each other.

// Distributed System Node
public class DistributedNode
{
    private readonly string _nodeId;
    private readonly List<string> _peers;
    private readonly IMessageBus _messageBus;
    private readonly IStateManager _stateManager;

    public async Task StartAsync()
    {
        await _messageBus.SubscribeAsync(HandleMessageAsync);
        await BroadcastNodeInfoAsync();
    }

    public async Task BroadcastMessageAsync(Message message)
    {
        foreach (var peer in _peers)
        {
            await _messageBus.SendAsync(peer, message);
        }
    }

    private async Task HandleMessageAsync(Message message)
    {
        switch (message.Type)
        {
            case MessageType.StateUpdate:
                await _stateManager.UpdateStateAsync(message.Data);
                break;
            case MessageType.Heartbeat:
                await HandleHeartbeatAsync(message);
                break;
        }
    }
}

// Distributed Consensus
public class DistributedConsensus
{
    private readonly List<IDistributedNode> _nodes;
    private readonly IConsensusAlgorithm _algorithm;

    public async Task<bool> ReachConsensusAsync(Proposal proposal)
    {
        var votes = new List<Vote>();

        foreach (var node in _nodes)
        {
            var vote = await node.VoteAsync(proposal);
            votes.Add(vote);
        }

        return await _algorithm.EvaluateConsensusAsync(votes);
    }
}

31. What is a Database Index?

A database index is a data structure that improves the speed of data retrieval operations on a database table.

// Index Implementation Example
public class DatabaseIndex<TKey, TValue>
{
    private readonly Dictionary<TKey, List<long>> _index;
    private readonly IComparer<TKey> _comparer;

    public void AddIndex(TKey key, long recordId)
    {
        if (!_index.ContainsKey(key))
        {
            _index[key] = new List<long>();
        }

        _index[key].Add(recordId);
    }

    public List<long> FindRecords(TKey key)
    {
        return _index.GetValueOrDefault(key, new List<long>());
    }

    public List<long> FindRange(TKey start, TKey end)
    {
        var result = new List<long>();

        foreach (var kvp in _index)
        {
            if (_comparer.Compare(kvp.Key, start) >= 0 && 
                _comparer.Compare(kvp.Key, end) <= 0)
            {
                result.AddRange(kvp.Value);
            }
        }

        return result;
    }
}

32. What is a Primary Key?

A primary key is a column or set of columns that uniquely identifies each row in a database table.

// Primary Key Implementation
public class PrimaryKeyConstraint<T>
{
    private readonly HashSet<T> _existingKeys;
    private readonly Func<object, T> _keyExtractor;

    public bool ValidatePrimaryKey(object entity)
    {
        var key = _keyExtractor(entity);

        if (_existingKeys.Contains(key))
        {
            return false; // Duplicate key violation
        }

        _existingKeys.Add(key);
        return true;
    }
}

// Entity with Primary Key
public class User
{
    [PrimaryKey]
    public Guid Id { get; set; }

    public string Name { get; set; }
    public string Email { get; set; }
}

33. What is a Foreign Key?

A foreign key is a column or set of columns that creates a link between data in two tables, enforcing referential integrity.

// Foreign Key Implementation
public class ForeignKeyConstraint<TKey, TEntity>
{
    private readonly HashSet<TKey> _referencedKeys;
    private readonly Func<TEntity, TKey> _foreignKeyExtractor;

    public bool ValidateForeignKey(TEntity entity)
    {
        var foreignKey = _foreignKeyExtractor(entity);
        return _referencedKeys.Contains(foreignKey);
    }
}

// Entity with Foreign Key
public class Order
{
    public Guid Id { get; set; }

    [ForeignKey("User")]
    public Guid UserId { get; set; }

    public User User { get; set; }
}

34. What is Normalization?

Normalization is the process of organizing data in a database to reduce redundancy and improve data integrity.

// Normalized Database Design
public class NormalizedDatabase
{
    // First Normal Form (1NF) - Atomic values
    public class User
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
        public string Email { get; set; }
    }

    // Second Normal Form (2NF) - No partial dependencies
    public class Order
    {
        public Guid Id { get; set; }
        public Guid UserId { get; set; }
        public DateTime OrderDate { get; set; }
    }

    public class OrderItem
    {
        public Guid Id { get; set; }
        public Guid OrderId { get; set; }
        public Guid ProductId { get; set; }
        public int Quantity { get; set; }
        public decimal Price { get; set; }
    }

    // Third Normal Form (3NF) - No transitive dependencies
    public class Product
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
        public Guid CategoryId { get; set; }
    }

    public class Category
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
    }
}

35. What is Denormalization?

Denormalization is the process of adding redundant data to improve query performance at the cost of some data integrity.

// Denormalized Design for Performance
public class DenormalizedOrder
{
    public Guid Id { get; set; }
    public Guid UserId { get; set; }

    // Denormalized user data for faster queries
    public string UserName { get; set; }
    public string UserEmail { get; set; }

    public List<OrderItem> Items { get; set; }

    // Denormalized totals for faster calculations
    public decimal TotalAmount { get; set; }
    public int TotalItems { get; set; }
}

public class OrderItem
{
    public Guid Id { get; set; }
    public Guid ProductId { get; set; }

    // Denormalized product data
    public string ProductName { get; set; }
    public decimal ProductPrice { get; set; }

    public int Quantity { get; set; }
    public decimal TotalPrice { get; set; }
}

36. What is a NoSQL Database?

NoSQL databases are non-relational databases designed for distributed data stores and big data applications.

// NoSQL Document Store Example
public class DocumentStore
{
    private readonly Dictionary<string, Dictionary<string, object>> _collections;

    public async Task<bool> InsertDocumentAsync(string collection, string id, object document)
    {
        if (!_collections.ContainsKey(collection))
        {
            _collections[collection] = new Dictionary<string, object>();
        }

        _collections[collection][id] = document;
        return true;
    }

    public async Task<object> GetDocumentAsync(string collection, string id)
    {
        if (_collections.TryGetValue(collection, out var docs))
        {
            docs.TryGetValue(id, out var document);
            return document;
        }

        return null;
    }

    public async Task<List<object>> QueryDocumentsAsync(string collection, Func<object, bool> predicate)
    {
        if (_collections.TryGetValue(collection, out var docs))
        {
            return docs.Values.Where(predicate).ToList();
        }

        return new List<object>();
    }
}

37. What is a Relational Database?

A relational database is a database that organizes data into tables with rows and columns, with relationships between tables.

// Relational Database Implementation
public class RelationalDatabase
{
    private readonly Dictionary<string, Table> _tables;

    public async Task<bool> CreateTableAsync(string tableName, List<Column> columns)
    {
        _tables[tableName] = new Table { Name = tableName, Columns = columns, Rows = new List<Row>() };
        return true;
    }

    public async Task<bool> InsertRowAsync(string tableName, Row row)
    {
        if (_tables.TryGetValue(tableName, out var table))
        {
            table.Rows.Add(row);
            return true;
        }

        return false;
    }

    public async Task<List<Row>> ExecuteQueryAsync(string sql)
    {
        // Simple SQL parser and executor
        var query = ParseSql(sql);
        return await ExecuteQueryAsync(query);
    }
}

public class Table
{
    public string Name { get; set; }
    public List<Column> Columns { get; set; }
    public List<Row> Rows { get; set; }
}

public class Row
{
    public Dictionary<string, object> Values { get; set; } = new Dictionary<string, object>();
}

38. What is a Key-Value Store?

A key-value store is a simple data storage system that stores data as a collection of key-value pairs.

// Key-Value Store Implementation
public class KeyValueStore
{
    private readonly Dictionary<string, object> _store;
    private readonly object _lock = new object();

    public async Task<bool> SetAsync(string key, object value)
    {
        lock (_lock)
        {
            _store[key] = value;
        }
        return true;
    }

    public async Task<object> GetAsync(string key)
    {
        lock (_lock)
        {
            _store.TryGetValue(key, out var value);
            return value;
        }
    }

    public async Task<bool> DeleteAsync(string key)
    {
        lock (_lock)
        {
            return _store.Remove(key);
        }
    }

    public async Task<List<string>> GetKeysAsync(string pattern = null)
    {
        lock (_lock)
        {
            if (string.IsNullOrEmpty(pattern))
            {
                return _store.Keys.ToList();
            }

            return _store.Keys.Where(k => k.Contains(pattern)).ToList();
        }
    }
}

Technical Architecture Interview Questions & Answers

Database Types

39. What is a document store?

A document store is a NoSQL database that stores data in document format (typically JSON, BSON, or XML). Documents are self-contained units that can have different structures and fields.

Key Characteristics: - Schema-flexible - Document-oriented - Supports nested data structures - Good for semi-structured data

Example C# with MongoDB:

using MongoDB.Driver;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;

public class User
{
    [BsonId]
    public ObjectId Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
    public Address Address { get; set; }
    public List<string> Tags { get; set; }
}

public class Address
{
    public string Street { get; set; }
    public string City { get; set; }
    public string Country { get; set; }
}

public class DocumentStoreExample
{
    private IMongoCollection<User> _users;

    public DocumentStoreExample()
    {
        var client = new MongoClient("mongodb://localhost:27017");
        var database = client.GetDatabase("testdb");
        _users = database.GetCollection<User>("users");
    }

    public async Task InsertUserAsync(User user)
    {
        await _users.InsertOneAsync(user);
    }

    public async Task<List<User>> GetUsersByCityAsync(string city)
    {
        var filter = Builders<User>.Filter.Eq("Address.City", city);
        return await _users.Find(filter).ToListAsync();
    }
}

40. What is a graph database?

A graph database stores data as nodes (entities) and edges (relationships) between them. It's optimized for complex relationships and graph traversal operations.

Key Characteristics: - Relationship-centric - Excellent for complex queries - Supports graph algorithms - Good for social networks, recommendation systems

Example C# with Neo4j:

using Neo4j.Driver;

public class GraphDatabaseExample
{
    private IDriver _driver;

    public GraphDatabaseExample()
    {
        _driver = GraphDatabase.Driver("bolt://localhost:7687", 
            AuthTokens.Basic("neo4j", "password"));
    }

    public async Task CreatePersonAsync(string name, int age)
    {
        using var session = _driver.AsyncSession();
        await session.ExecuteWriteAsync(async tx =>
        {
            var query = "CREATE (p:Person {name: $name, age: $age})";
            await tx.RunAsync(query, new { name, age });
        });
    }

    public async Task CreateFriendshipAsync(string person1, string person2)
    {
        using var session = _driver.AsyncSession();
        await session.ExecuteWriteAsync(async tx =>
        {
            var query = @"
                MATCH (p1:Person {name: $person1})
                MATCH (p2:Person {name: $person2})
                CREATE (p1)-[:FRIENDS_WITH]->(p2)";
            await tx.RunAsync(query, new { person1, person2 });
        });
    }

    public async Task<List<string>> GetFriendsOfFriendsAsync(string personName)
    {
        using var session = _driver.AsyncSession();
        var result = await session.ExecuteReadAsync(async tx =>
        {
            var query = @"
                MATCH (p:Person {name: $name})-[:FRIENDS_WITH]-()-[:FRIENDS_WITH]-(friend)
                RETURN DISTINCT friend.name as name";
            var cursor = await tx.RunAsync(query, new { name = personName });
            return await cursor.ToListAsync();
        });

        return result.Select(r => r["name"].As<string>()).ToList();
    }
}

41. What is a time-series database?

A time-series database is optimized for storing and querying time-stamped data points. It's designed for high-volume, time-ordered data like metrics, IoT data, and financial data.

Key Characteristics: - Time-indexed data - High write throughput - Efficient time-range queries - Data compression and retention policies

Example C# with InfluxDB:

using InfluxDB.Client;
using InfluxDB.Client.Api.Domain;
using InfluxDB.Client.Writes;

public class TimeSeriesDatabaseExample
{
    private InfluxDBClient _client;
    private const string Bucket = "sensors";
    private const string Org = "myorg";

    public TimeSeriesDatabaseExample()
    {
        _client = new InfluxDBClient("http://localhost:8086", "token");
    }

    public async Task WriteSensorDataAsync(string sensorId, double temperature, double humidity)
    {
        var point = PointData
            .Measurement("sensor_readings")
            .Tag("sensor_id", sensorId)
            .Field("temperature", temperature)
            .Field("humidity", humidity)
            .Timestamp(DateTime.UtcNow, WritePrecision.Ms);

        var writeApi = _client.GetWriteApiAsync();
        await writeApi.WritePointAsync(point, Bucket, Org);
    }

    public async Task<List<SensorReading>> GetSensorDataAsync(string sensorId, DateTime start, DateTime end)
    {
        var query = $@"
            from(bucket: ""{Bucket}"")
            |> range(start: {start:yyyy-MM-ddTHH:mm:ssZ}, stop: {end:yyyy-MM-ddTHH:mm:ssZ})
            |> filter(fn: (r) => r[""_measurement""] == ""sensor_readings"")
            |> filter(fn: (r) => r[""sensor_id""] == ""{sensorId}"")";

        var queryApi = _client.GetQueryApi();
        var tables = await queryApi.QueryAsync(query, Org);

        var readings = new List<SensorReading>();
        foreach (var table in tables)
        {
            foreach (var record in table.Records)
            {
                readings.Add(new SensorReading
                {
                    Timestamp = record.GetTime().GetValueOrDefault(),
                    SensorId = record.GetValueByKey("sensor_id").ToString(),
                    Field = record.GetField(),
                    Value = Convert.ToDouble(record.GetValue())
                });
            }
        }

        return readings;
    }
}

public class SensorReading
{
    public DateTime Timestamp { get; set; }
    public string SensorId { get; set; }
    public string Field { get; set; }
    public double Value { get; set; }
}

42. What is a data warehouse?

A data warehouse is a centralized repository that stores integrated data from multiple sources for analytical processing and business intelligence.

Key Characteristics: - Subject-oriented - Integrated - Time-variant - Non-volatile - Optimized for read operations

Example C# with SQL Server Analysis Services:

using Microsoft.AnalysisServices.AdomdClient;

public class DataWarehouseExample
{
    private string _connectionString = "Data Source=localhost;Catalog=AdventureWorksDW;";

    public async Task<List<SalesData>> GetSalesByRegionAsync(int year)
    {
        var salesData = new List<SalesData>();

        using var connection = new AdomdConnection(_connectionString);
        await connection.OpenAsync();

        var query = $@"
            SELECT 
                [Geography].[English Country Region Name] as Region,
                [Measures].[Internet Sales Amount] as SalesAmount
            FROM [Adventure Works]
            WHERE [Date].[Calendar Year] = {year}";

        using var command = new AdomdCommand(query, connection);
        using var reader = await command.ExecuteReaderAsync();

        while (await reader.ReadAsync())
        {
            salesData.Add(new SalesData
            {
                Region = reader["Region"].ToString(),
                SalesAmount = Convert.ToDecimal(reader["SalesAmount"])
            });
        }

        return salesData;
    }

    public async Task<Dictionary<string, decimal>> GetTopProductsAsync(int topCount)
    {
        var products = new Dictionary<string, decimal>();

        using var connection = new AdomdConnection(_connectionString);
        await connection.OpenAsync();

        var query = $@"
            SELECT TOP {topCount}
                [Product].[English Product Name] as ProductName,
                [Measures].[Internet Sales Amount] as SalesAmount
            FROM [Adventure Works]
            ORDER BY [Measures].[Internet Sales Amount] DESC";

        using var command = new AdomdCommand(query, connection);
        using var reader = await command.ExecuteReaderAsync();

        while (await reader.ReadAsync())
        {
            products.Add(
                reader["ProductName"].ToString(),
                Convert.ToDecimal(reader["SalesAmount"])
            );
        }

        return products;
    }
}

public class SalesData
{
    public string Region { get; set; }
    public decimal SalesAmount { get; set; }
}

43. What is OLTP vs OLAP?

OLTP (Online Transaction Processing): - Designed for transaction-oriented applications - Optimized for fast, small transactions - Normalized data structure - Real-time processing - ACID compliance

OLAP (Online Analytical Processing): - Designed for complex queries and analysis - Optimized for read-heavy operations - Denormalized data structure (star/snowflake schema) - Batch processing - Focus on aggregations and reporting

Example C# showing the difference:

// OLTP Example - Transaction Processing
public class OLTPExample
{
    private readonly SqlConnection _connection;

    public OLTPExample(string connectionString)
    {
        _connection = new SqlConnection(connectionString);
    }

    public async Task<bool> ProcessOrderAsync(Order order)
    {
        using var transaction = _connection.BeginTransaction();
        try
        {
            // Insert order
            var orderId = await InsertOrderAsync(order, transaction);

            // Update inventory
            foreach (var item in order.Items)
            {
                await UpdateInventoryAsync(item.ProductId, item.Quantity, transaction);
            }

            // Process payment
            await ProcessPaymentAsync(order.PaymentInfo, transaction);

            transaction.Commit();
            return true;
        }
        catch
        {
            transaction.Rollback();
            return false;
        }
    }

    private async Task<int> InsertOrderAsync(Order order, SqlTransaction transaction)
    {
        var query = @"
            INSERT INTO Orders (CustomerId, OrderDate, TotalAmount)
            VALUES (@CustomerId, @OrderDate, @TotalAmount);
            SELECT SCOPE_IDENTITY();";

        using var command = new SqlCommand(query, _connection, transaction);
        command.Parameters.AddWithValue("@CustomerId", order.CustomerId);
        command.Parameters.AddWithValue("@OrderDate", order.OrderDate);
        command.Parameters.AddWithValue("@TotalAmount", order.TotalAmount);

        return Convert.ToInt32(await command.ExecuteScalarAsync());
    }
}

// OLAP Example - Analytical Processing
public class OLAPExample
{
    private readonly SqlConnection _connection;

    public OLAPExample(string connectionString)
    {
        _connection = new SqlConnection(connectionString);
    }

    public async Task<List<SalesAnalysis>> GetSalesAnalysisAsync(int year)
    {
        var query = @"
            SELECT 
                p.ProductName,
                c.CategoryName,
                r.RegionName,
                SUM(f.SalesAmount) as TotalSales,
                AVG(f.SalesAmount) as AvgSales,
                COUNT(*) as TransactionCount
            FROM FactSales f
            JOIN DimProduct p ON f.ProductKey = p.ProductKey
            JOIN DimCategory c ON p.CategoryKey = c.CategoryKey
            JOIN DimRegion r ON f.RegionKey = r.RegionKey
            JOIN DimDate d ON f.DateKey = d.DateKey
            WHERE d.Year = @Year
            GROUP BY p.ProductName, c.CategoryName, r.RegionName
            ORDER BY TotalSales DESC";

        using var command = new SqlCommand(query, _connection);
        command.Parameters.AddWithValue("@Year", year);

        var results = new List<SalesAnalysis>();
        await _connection.OpenAsync();

        using var reader = await command.ExecuteReaderAsync();
        while (await reader.ReadAsync())
        {
            results.Add(new SalesAnalysis
            {
                ProductName = reader["ProductName"].ToString(),
                CategoryName = reader["CategoryName"].ToString(),
                RegionName = reader["RegionName"].ToString(),
                TotalSales = Convert.ToDecimal(reader["TotalSales"]),
                AvgSales = Convert.ToDecimal(reader["AvgSales"]),
                TransactionCount = Convert.ToInt32(reader["TransactionCount"])
            });
        }

        return results;
    }
}

public class SalesAnalysis
{
    public string ProductName { get; set; }
    public string CategoryName { get; set; }
    public string RegionName { get; set; }
    public decimal TotalSales { get; set; }
    public decimal AvgSales { get; set; }
    public int TransactionCount { get; set; }
}

Caching Strategies

44. What is a cache eviction policy?

A cache eviction policy determines which items to remove from cache when it reaches capacity. Common policies include LRU, LFU, FIFO, and TTL.

Example C# Implementation:

public interface ICacheEvictionPolicy<TKey, TValue>
{
    void Add(TKey key, TValue value);
    TValue Get(TKey key);
    void Remove(TKey key);
    void Clear();
}

public abstract class BaseCache<TKey, TValue> : ICacheEvictionPolicy<TKey, TValue>
{
    protected readonly int _capacity;
    protected readonly Dictionary<TKey, TValue> _cache;

    protected BaseCache(int capacity)
    {
        _capacity = capacity;
        _cache = new Dictionary<TKey, TValue>();
    }

    public abstract void Add(TKey key, TValue value);

    public virtual TValue Get(TKey key)
    {
        return _cache.TryGetValue(key, out var value) ? value : default(TValue);
    }

    public virtual void Remove(TKey key)
    {
        _cache.Remove(key);
    }

    public virtual void Clear()
    {
        _cache.Clear();
    }

    protected bool IsFull => _cache.Count >= _capacity;
}

45. What is LRU cache?

LRU (Least Recently Used) cache removes the least recently accessed item when the cache is full.

C# Implementation:

public class LRUCache<TKey, TValue> : BaseCache<TKey, TValue>
{
    private readonly LinkedList<TKey> _accessOrder;
    private readonly Dictionary<TKey, LinkedListNode<TKey>> _keyToNode;

    public LRUCache(int capacity) : base(capacity)
    {
        _accessOrder = new LinkedList<TKey>();
        _keyToNode = new Dictionary<TKey, LinkedListNode<TKey>>();
    }

    public override void Add(TKey key, TValue value)
    {
        if (_cache.ContainsKey(key))
        {
            // Update existing key
            _cache[key] = value;
            UpdateAccessOrder(key);
        }
        else
        {
            if (IsFull)
            {
                // Remove least recently used item
                var lruKey = _accessOrder.Last.Value;
                Remove(lruKey);
            }

            // Add new item
            _cache[key] = value;
            var node = _accessOrder.AddFirst(key);
            _keyToNode[key] = node;
        }
    }

    public override TValue Get(TKey key)
    {
        if (_cache.TryGetValue(key, out var value))
        {
            UpdateAccessOrder(key);
            return value;
        }
        return default(TValue);
    }

    public override void Remove(TKey key)
    {
        if (_cache.Remove(key))
        {
            if (_keyToNode.TryGetValue(key, out var node))
            {
                _accessOrder.Remove(node);
                _keyToNode.Remove(key);
            }
        }
    }

    private void UpdateAccessOrder(TKey key)
    {
        if (_keyToNode.TryGetValue(key, out var node))
        {
            _accessOrder.Remove(node);
            _accessOrder.AddFirst(node);
        }
    }
}

// Usage Example
public class CacheExample
{
    public void DemonstrateLRU()
    {
        var cache = new LRUCache<string, int>(3);

        cache.Add("A", 1);
        cache.Add("B", 2);
        cache.Add("C", 3);

        // Access A to make it most recently used
        var value = cache.Get("A");

        // Adding D will evict B (least recently used)
        cache.Add("D", 4);

        // B should no longer be in cache
        var bValue = cache.Get("B"); // Returns 0 (default)
    }
}

46. What is LFU cache?

LFU (Least Frequently Used) cache removes the item that has been accessed the least number of times.

C# Implementation:

public class LFUCache<TKey, TValue> : BaseCache<TKey, TValue>
{
    private readonly Dictionary<TKey, int> _frequency;
    private readonly Dictionary<int, HashSet<TKey>> _frequencyToKeys;
    private int _minFrequency;

    public LFUCache(int capacity) : base(capacity)
    {
        _frequency = new Dictionary<TKey, int>();
        _frequencyToKeys = new Dictionary<int, HashSet<TKey>>();
        _minFrequency = 0;
    }

    public override void Add(TKey key, TValue value)
    {
        if (_cache.ContainsKey(key))
        {
            // Update existing key
            _cache[key] = value;
            IncrementFrequency(key);
        }
        else
        {
            if (IsFull)
            {
                // Remove least frequently used item
                var lfuKey = _frequencyToKeys[_minFrequency].First();
                Remove(lfuKey);
            }

            // Add new item
            _cache[key] = value;
            _frequency[key] = 1;
            _minFrequency = 1;

            if (!_frequencyToKeys.ContainsKey(1))
                _frequencyToKeys[1] = new HashSet<TKey>();
            _frequencyToKeys[1].Add(key);
        }
    }

    public override TValue Get(TKey key)
    {
        if (_cache.TryGetValue(key, out var value))
        {
            IncrementFrequency(key);
            return value;
        }
        return default(TValue);
    }

    public override void Remove(TKey key)
    {
        if (_cache.Remove(key))
        {
            var freq = _frequency[key];
            _frequency.Remove(key);
            _frequencyToKeys[freq].Remove(key);

            if (_frequencyToKeys[freq].Count == 0)
            {
                _frequencyToKeys.Remove(freq);
                if (freq == _minFrequency)
                    _minFrequency++;
            }
        }
    }

    private void IncrementFrequency(TKey key)
    {
        var currentFreq = _frequency[key];
        _frequency[key] = currentFreq + 1;

        _frequencyToKeys[currentFreq].Remove(key);
        if (_frequencyToKeys[currentFreq].Count == 0)
        {
            _frequencyToKeys.Remove(currentFreq);
            if (currentFreq == _minFrequency)
                _minFrequency++;
        }

        if (!_frequencyToKeys.ContainsKey(currentFreq + 1))
            _frequencyToKeys[currentFreq + 1] = new HashSet<TKey>();
        _frequencyToKeys[currentFreq + 1].Add(key);
    }
}

47. What is write-through cache?

Write-through cache writes data to both cache and underlying storage simultaneously, ensuring consistency but potentially slower writes.

C# Implementation:

public class WriteThroughCache<TKey, TValue>
{
    private readonly Dictionary<TKey, TValue> _cache;
    private readonly IStorageProvider<TKey, TValue> _storage;

    public WriteThroughCache(IStorageProvider<TKey, TValue> storage)
    {
        _cache = new Dictionary<TKey, TValue>();
        _storage = storage;
    }

    public async Task<TValue> GetAsync(TKey key)
    {
        if (_cache.TryGetValue(key, out var value))
        {
            return value;
        }

        // Cache miss - load from storage
        value = await _storage.GetAsync(key);
        if (value != null)
        {
            _cache[key] = value;
        }
        return value;
    }

    public async Task SetAsync(TKey key, TValue value)
    {
        // Write to both cache and storage simultaneously
        _cache[key] = value;
        await _storage.SetAsync(key, value);
    }

    public async Task RemoveAsync(TKey key)
    {
        _cache.Remove(key);
        await _storage.RemoveAsync(key);
    }
}

public interface IStorageProvider<TKey, TValue>
{
    Task<TValue> GetAsync(TKey key);
    Task SetAsync(TKey key, TValue value);
    Task RemoveAsync(TKey key);
}

public class FileStorageProvider<TKey, TValue> : IStorageProvider<TKey, TValue>
{
    private readonly string _filePath;

    public FileStorageProvider(string filePath)
    {
        _filePath = filePath;
    }

    public async Task<TValue> GetAsync(TKey key)
    {
        // Implementation for reading from file
        return default(TValue);
    }

    public async Task SetAsync(TKey key, TValue value)
    {
        // Implementation for writing to file
        await Task.CompletedTask;
    }

    public async Task RemoveAsync(TKey key)
    {
        // Implementation for removing from file
        await Task.CompletedTask;
    }
}

48. What is write-back cache?

Write-back cache writes data to cache first and later writes to storage in batches, providing better performance but potential data loss risk.

C# Implementation:

public class WriteBackCache<TKey, TValue>
{
    private readonly Dictionary<TKey, TValue> _cache;
    private readonly Dictionary<TKey, TValue> _dirtyItems;
    private readonly IStorageProvider<TKey, TValue> _storage;
    private readonly Timer _flushTimer;
    private readonly object _lock = new object();

    public WriteBackCache(IStorageProvider<TKey, TValue> storage, TimeSpan flushInterval)
    {
        _cache = new Dictionary<TKey, TValue>();
        _dirtyItems = new Dictionary<TKey, TValue>();
        _storage = storage;
        _flushTimer = new Timer(FlushDirtyItems, null, flushInterval, flushInterval);
    }

    public async Task<TValue> GetAsync(TKey key)
    {
        lock (_lock)
        {
            if (_cache.TryGetValue(key, out var value))
            {
                return value;
            }
        }

        // Cache miss - load from storage
        var storageValue = await _storage.GetAsync(key);
        if (storageValue != null)
        {
            lock (_lock)
            {
                _cache[key] = storageValue;
            }
        }
        return storageValue;
    }

    public void Set(TKey key, TValue value)
    {
        lock (_lock)
        {
            _cache[key] = value;
            _dirtyItems[key] = value; // Mark as dirty
        }
    }

    public async Task FlushAsync()
    {
        Dictionary<TKey, TValue> itemsToFlush;

        lock (_lock)
        {
            itemsToFlush = new Dictionary<TKey, TValue>(_dirtyItems);
            _dirtyItems.Clear();
        }

        foreach (var item in itemsToFlush)
        {
            await _storage.SetAsync(item.Key, item.Value);
        }
    }

    private async void FlushDirtyItems(object state)
    {
        await FlushAsync();
    }

    public void Dispose()
    {
        _flushTimer?.Dispose();
        FlushAsync().Wait(); // Final flush
    }
}

49. What is a bloom filter?

A Bloom filter is a probabilistic data structure that tests whether an element is a member of a set. It may have false positives but never false negatives.

C# Implementation:

public class BloomFilter
{
    private readonly bool[] _bitArray;
    private readonly int _size;
    private readonly int _hashCount;
    private readonly HashFunction[] _hashFunctions;

    public BloomFilter(int size, int hashCount)
    {
        _size = size;
        _hashCount = hashCount;
        _bitArray = new bool[size];
        _hashFunctions = GenerateHashFunctions(hashCount);
    }

    public void Add(string item)
    {
        var hashes = GetHashes(item);
        foreach (var hash in hashes)
        {
            _bitArray[hash % _size] = true;
        }
    }

    public bool Contains(string item)
    {
        var hashes = GetHashes(item);
        return hashes.All(hash => _bitArray[hash % _size]);
    }

    private int[] GetHashes(string item)
    {
        var hashes = new int[_hashCount];
        for (int i = 0; i < _hashCount; i++)
        {
            hashes[i] = _hashFunctions[i](item);
        }
        return hashes;
    }

    private HashFunction[] GenerateHashFunctions(int count)
    {
        var functions = new HashFunction[count];
        var random = new Random(42); // Fixed seed for consistency

        for (int i = 0; i < count; i++)
        {
            var seed = random.Next();
            functions[i] = str => Math.Abs(str.GetHashCode() ^ seed);
        }

        return functions;
    }

    private delegate int HashFunction(string input);
}

// Usage Example
public class BloomFilterExample
{
    public void DemonstrateBloomFilter()
    {
        var bloomFilter = new BloomFilter(1000, 3);

        // Add items
        bloomFilter.Add("apple");
        bloomFilter.Add("banana");
        bloomFilter.Add("cherry");

        // Check if items exist
        Console.WriteLine(bloomFilter.Contains("apple"));   // True
        Console.WriteLine(bloomFilter.Contains("banana"));  // True
        Console.WriteLine(bloomFilter.Contains("orange"));  // False (probably)
    }
}

50. What is a consistent hash?

Consistent hashing is a technique that minimizes the number of keys that need to be remapped when a hash table is resized or when nodes are added/removed from a distributed system.

C# Implementation:

public class ConsistentHash<T>
{
    private readonly SortedDictionary<int, T> _circle;
    private readonly int _virtualNodes;
    private readonly HashFunction _hashFunction;

    public ConsistentHash(int virtualNodes = 150)
    {
        _circle = new SortedDictionary<int, T>();
        _virtualNodes = virtualNodes;
        _hashFunction = MurmurHash2;
    }

    public void AddNode(T node)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var virtualNodeName = $"{node}-{i}";
            var hash = _hashFunction(virtualNodeName);
            _circle[hash] = node;
        }
    }

    public void RemoveNode(T node)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var virtualNodeName = $"{node}-{i}";
            var hash = _hashFunction(virtualNodeName);
            _circle.Remove(hash);
        }
    }

    public T GetNode(string key)
    {
        if (_circle.Count == 0)
            throw new InvalidOperationException("No nodes available");

        var hash = _hashFunction(key);

        // Find the first node with hash >= key hash
        var node = _circle.FirstOrDefault(x => x.Key >= hash);

        // If no node found, wrap around to the first node
        if (node.Value == null)
        {
            node = _circle.First();
        }

        return node.Value;
    }

    private int MurmurHash2(string key)
    {
        // Simplified MurmurHash2 implementation
        uint h = 0x5bd1e995;
        uint k = (uint)key.GetHashCode();

        k *= h;
        k ^= k >> 13;
        k *= h;
        h ^= k >> 15;

        return (int)h;
    }

    private delegate int HashFunction(string input);
}

// Usage Example
public class ConsistentHashExample
{
    public void DemonstrateConsistentHash()
    {
        var consistentHash = new ConsistentHash<string>();

        // Add nodes
        consistentHash.AddNode("server1");
        consistentHash.AddNode("server2");
        consistentHash.AddNode("server3");

        // Get node for keys
        var node1 = consistentHash.GetNode("user1");
        var node2 = consistentHash.GetNode("user2");
        var node3 = consistentHash.GetNode("user3");

        Console.WriteLine($"user1 -> {node1}");
        Console.WriteLine($"user2 -> {node2}");
        Console.WriteLine($"user3 -> {node3}");

        // Remove a node
        consistentHash.RemoveNode("server2");

        // Keys will be redistributed
        var newNode1 = consistentHash.GetNode("user1");
        Console.WriteLine($"user1 -> {newNode1} (after server2 removal)");
    }
}

51. What is a quorum?

Explanation: A quorum is a minimum number of nodes or participants required to make a decision or perform an operation in a distributed system. It ensures that the system can continue operating even if some nodes fail, while maintaining consistency and availability.

Key Concepts: - Majority Quorum: Requires more than 50% of nodes to agree - Read Quorum (R): Minimum nodes needed to read data - Write Quorum (W): Minimum nodes needed to write data - Quorum Formula: R + W > N (where N is total nodes)

public class QuorumManager
{
    private readonly List<Node> _nodes;
    private readonly int _totalNodes;
    private readonly int _readQuorum;
    private readonly int _writeQuorum;

    public QuorumManager(int totalNodes)
    {
        _totalNodes = totalNodes;
        _nodes = new List<Node>();
        _readQuorum = (totalNodes / 2) + 1;
        _writeQuorum = (totalNodes / 2) + 1;
    }

    public bool CanRead()
    {
        var availableNodes = _nodes.Count(n => n.IsHealthy);
        return availableNodes >= _readQuorum;
    }

    public bool CanWrite()
    {
        var availableNodes = _nodes.Count(n => n.IsHealthy);
        return availableNodes >= _writeQuorum;
    }
}

public class Node
{
    public string Id { get; set; }
    public bool IsHealthy { get; set; }
}

52. What is leader election?

Explanation: Leader election is a process in distributed systems where nodes select one node as the "leader" to coordinate operations, make decisions, or manage resources. The leader is responsible for maintaining consistency and coordinating activities across the cluster.

Common Algorithms: - Bully Algorithm: Highest ID wins - Ring Algorithm: Token passing in a ring topology - Paxos/Raft: Consensus-based election

public class LeaderElection
{
    private readonly List<Node> _nodes;
    private Node _currentLeader;
    private readonly object _lock = new object();

    public LeaderElection(List<Node> nodes)
    {
        _nodes = nodes;
    }

    public async Task<Node> ElectLeaderAsync()
    {
        // Bully Algorithm implementation
        var candidates = _nodes.Where(n => n.IsHealthy).ToList();
        var leader = candidates.OrderByDescending(n => n.Id).First();

        lock (_lock)
        {
            _currentLeader = leader;
        }

        // Notify all nodes about the new leader
        await NotifyLeaderChangeAsync(leader);
        return leader;
    }

    private async Task NotifyLeaderChangeAsync(Node leader)
    {
        var tasks = _nodes.Select(node => 
            node.NotifyLeaderChangeAsync(leader));
        await Task.WhenAll(tasks);
    }
}

53. What is a heartbeat in distributed systems?

Explanation: A heartbeat is a periodic signal sent between nodes in a distributed system to indicate that a node is alive and functioning. It's used for failure detection, health monitoring, and maintaining system awareness.

public class HeartbeatManager
{
    private readonly Timer _heartbeatTimer;
    private readonly Dictionary<string, DateTime> _lastHeartbeats;
    private readonly TimeSpan _heartbeatInterval = TimeSpan.FromSeconds(5);
    private readonly TimeSpan _failureThreshold = TimeSpan.FromSeconds(15);

    public HeartbeatManager()
    {
        _lastHeartbeats = new Dictionary<string, DateTime>();
        _heartbeatTimer = new Timer(CheckHeartbeats, null, 
            TimeSpan.Zero, _heartbeatInterval);
    }

    public void SendHeartbeat(string nodeId)
    {
        lock (_lastHeartbeats)
        {
            _lastHeartbeats[nodeId] = DateTime.UtcNow;
        }
    }

    private void CheckHeartbeats(object state)
    {
        var now = DateTime.UtcNow;
        var failedNodes = new List<string>();

        lock (_lastHeartbeats)
        {
            foreach (var kvp in _lastHeartbeats)
            {
                if (now - kvp.Value > _failureThreshold)
                {
                    failedNodes.Add(kvp.Key);
                }
            }
        }

        foreach (var failedNode in failedNodes)
        {
            OnNodeFailure(failedNode);
        }
    }

    private void OnNodeFailure(string nodeId)
    {
        Console.WriteLine($"Node {nodeId} is considered failed");
        // Trigger failover or recovery procedures
    }
}

54. What is a gossip protocol?

Explanation: A gossip protocol is a communication pattern where nodes randomly share information with other nodes, similar to how rumors spread. It's used for information dissemination, membership management, and failure detection in large distributed systems.

public class GossipProtocol
{
    private readonly Dictionary<string, NodeInfo> _membership;
    private readonly Random _random = new Random();
    private readonly Timer _gossipTimer;

    public GossipProtocol()
    {
        _membership = new Dictionary<string, NodeInfo>();
        _gossipTimer = new Timer(Gossip, null, 
            TimeSpan.Zero, TimeSpan.FromSeconds(1));
    }

    public void AddNode(string nodeId, string address)
    {
        _membership[nodeId] = new NodeInfo
        {
            Id = nodeId,
            Address = address,
            Heartbeat = 0,
            LastSeen = DateTime.UtcNow
        };
    }

    private void Gossip(object state)
    {
        var nodes = _membership.Keys.ToList();
        if (nodes.Count < 2) return;

        // Select random node to gossip with
        var targetNode = nodes[_random.Next(nodes.Count)];
        var myInfo = _membership[GetCurrentNodeId()];

        // Send membership information
        SendMembershipInfo(targetNode, myInfo);
    }

    public void ReceiveGossip(string fromNode, NodeInfo info)
    {
        lock (_membership)
        {
            if (!_membership.ContainsKey(info.Id) || 
                _membership[info.Id].Heartbeat < info.Heartbeat)
            {
                _membership[info.Id] = info;
            }
        }
    }

    private string GetCurrentNodeId() => "current-node-id";
    private void SendMembershipInfo(string targetNode, NodeInfo info) { }
}

public class NodeInfo
{
    public string Id { get; set; }
    public string Address { get; set; }
    public int Heartbeat { get; set; }
    public DateTime LastSeen { get; set; }
}

55. What is a distributed lock?

Explanation: A distributed lock is a synchronization mechanism that allows only one process or node to access a shared resource at a time across a distributed system. It's used to prevent race conditions and ensure data consistency.

public class DistributedLock
{
    private readonly IDistributedCache _cache;
    private readonly string _lockKey;
    private readonly TimeSpan _lockTimeout;

    public DistributedLock(IDistributedCache cache, string lockKey, TimeSpan timeout)
    {
        _cache = cache;
        _lockKey = lockKey;
        _lockTimeout = timeout;
    }

    public async Task<bool> AcquireLockAsync(string lockId)
    {
        var lockValue = new LockInfo
        {
            LockId = lockId,
            AcquiredAt = DateTime.UtcNow,
            ExpiresAt = DateTime.UtcNow.Add(_lockTimeout)
        };

        var options = new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = _lockTimeout
        };

        try
        {
            await _cache.SetStringAsync(_lockKey, 
                JsonSerializer.Serialize(lockValue), options);
            return true;
        }
        catch
        {
            return false;
        }
    }

    public async Task ReleaseLockAsync(string lockId)
    {
        var existingLock = await _cache.GetStringAsync(_lockKey);
        if (existingLock != null)
        {
            var lockInfo = JsonSerializer.Deserialize<LockInfo>(existingLock);
            if (lockInfo.LockId == lockId)
            {
                await _cache.RemoveAsync(_lockKey);
            }
        }
    }
}

public class LockInfo
{
    public string LockId { get; set; }
    public DateTime AcquiredAt { get; set; }
    public DateTime ExpiresAt { get; set; }
}

56. What is a two-phase commit?

Explanation: Two-Phase Commit (2PC) is a distributed algorithm that ensures all nodes in a distributed system either commit or abort a transaction. It consists of two phases: prepare phase and commit phase.

public class TwoPhaseCommit
{
    private readonly List<IParticipant> _participants;
    private readonly ILogger<TwoPhaseCommit> _logger;

    public TwoPhaseCommit(List<IParticipant> participants, ILogger<TwoPhaseCommit> logger)
    {
        _participants = participants;
        _logger = logger;
    }

    public async Task<bool> ExecuteTransactionAsync(Transaction transaction)
    {
        // Phase 1: Prepare
        var prepareResults = new List<bool>();
        foreach (var participant in _participants)
        {
            try
            {
                var canCommit = await participant.PrepareAsync(transaction);
                prepareResults.Add(canCommit);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Prepare failed for participant");
                prepareResults.Add(false);
            }
        }

        // Check if all participants can commit
        if (prepareResults.All(r => r))
        {
            // Phase 2: Commit
            await CommitTransactionAsync(transaction);
            return true;
        }
        else
        {
            // Phase 2: Abort
            await AbortTransactionAsync(transaction);
            return false;
        }
    }

    private async Task CommitTransactionAsync(Transaction transaction)
    {
        var tasks = _participants.Select(p => p.CommitAsync(transaction));
        await Task.WhenAll(tasks);
    }

    private async Task AbortTransactionAsync(Transaction transaction)
    {
        var tasks = _participants.Select(p => p.AbortAsync(transaction));
        await Task.WhenAll(tasks);
    }
}

public interface IParticipant
{
    Task<bool> PrepareAsync(Transaction transaction);
    Task CommitAsync(Transaction transaction);
    Task AbortAsync(Transaction transaction);
}

public class Transaction
{
    public string Id { get; set; }
    public List<Operation> Operations { get; set; }
}

57. What is a three-phase commit?

Explanation: Three-Phase Commit (3PC) is an improvement over 2PC that reduces blocking by adding a pre-commit phase. It handles coordinator failures better and provides non-blocking recovery.

public class ThreePhaseCommit
{
    private readonly List<IParticipant> _participants;
    private readonly ILogger<ThreePhaseCommit> _logger;

    public ThreePhaseCommit(List<IParticipant> participants, ILogger<ThreePhaseCommit> logger)
    {
        _participants = participants;
        _logger = logger;
    }

    public async Task<bool> ExecuteTransactionAsync(Transaction transaction)
    {
        // Phase 1: CanCommit
        var canCommitResults = new List<bool>();
        foreach (var participant in _participants)
        {
            try
            {
                var canCommit = await participant.CanCommitAsync(transaction);
                canCommitResults.Add(canCommit);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "CanCommit failed for participant");
                canCommitResults.Add(false);
            }
        }

        if (!canCommitResults.All(r => r))
        {
            await AbortTransactionAsync(transaction);
            return false;
        }

        // Phase 2: PreCommit
        var preCommitResults = new List<bool>();
        foreach (var participant in _participants)
        {
            try
            {
                var preCommitted = await participant.PreCommitAsync(transaction);
                preCommitResults.Add(preCommitted);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "PreCommit failed for participant");
                preCommitResults.Add(false);
            }
        }

        if (!preCommitResults.All(r => r))
        {
            await AbortTransactionAsync(transaction);
            return false;
        }

        // Phase 3: DoCommit
        await CommitTransactionAsync(transaction);
        return true;
    }

    private async Task CommitTransactionAsync(Transaction transaction)
    {
        var tasks = _participants.Select(p => p.DoCommitAsync(transaction));
        await Task.WhenAll(tasks);
    }

    private async Task AbortTransactionAsync(Transaction transaction)
    {
        var tasks = _participants.Select(p => p.AbortAsync(transaction));
        await Task.WhenAll(tasks);
    }
}

public interface IParticipant
{
    Task<bool> CanCommitAsync(Transaction transaction);
    Task<bool> PreCommitAsync(Transaction transaction);
    Task DoCommitAsync(Transaction transaction);
    Task AbortAsync(Transaction transaction);
}

58. What is a saga pattern?

Explanation: The Saga pattern is a sequence of local transactions where each transaction updates data within a single service and publishes an event or message to trigger the next transaction. If one transaction fails, the saga executes compensating transactions to undo the impact of preceding transactions.

public class OrderSaga
{
    private readonly IServiceProvider _serviceProvider;
    private readonly ILogger<OrderSaga> _logger;

    public OrderSaga(IServiceProvider serviceProvider, ILogger<OrderSaga> logger)
    {
        _serviceProvider = serviceProvider;
        _logger = logger;
    }

    public async Task<bool> ExecuteOrderSagaAsync(Order order)
    {
        var sagaSteps = new List<ISagaStep>
        {
            new CreateOrderStep(),
            new ReserveInventoryStep(),
            new ProcessPaymentStep(),
            new ShipOrderStep()
        };

        var executedSteps = new Stack<ISagaStep>();

        try
        {
            foreach (var step in sagaSteps)
            {
                var success = await step.ExecuteAsync(order);
                if (!success)
                {
                    await CompensateAsync(executedSteps, order);
                    return false;
                }
                executedSteps.Push(step);
            }
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Saga execution failed");
            await CompensateAsync(executedSteps, order);
            return false;
        }
    }

    private async Task CompensateAsync(Stack<ISagaStep> executedSteps, Order order)
    {
        while (executedSteps.Count > 0)
        {
            var step = executedSteps.Pop();
            await step.CompensateAsync(order);
        }
    }
}

public interface ISagaStep
{
    Task<bool> ExecuteAsync(Order order);
    Task CompensateAsync(Order order);
}

public class CreateOrderStep : ISagaStep
{
    public async Task<bool> ExecuteAsync(Order order)
    {
        // Create order logic
        return true;
    }

    public async Task CompensateAsync(Order order)
    {
        // Cancel order logic
    }
}

59. What is eventual consistency vs strong consistency?

Explanation: - Strong Consistency: All nodes see the same data at the same time. Any read operation returns the most recent write. - Eventual Consistency: All nodes will eventually see the same data, but there may be temporary inconsistencies during replication.

// Strong Consistency Example
public class StrongConsistentDatabase
{
    private readonly object _lock = new object();
    private Dictionary<string, string> _data = new Dictionary<string, string>();

    public void Write(string key, string value)
    {
        lock (_lock)
        {
            _data[key] = value;
        }
    }

    public string Read(string key)
    {
        lock (_lock)
        {
            return _data.TryGetValue(key, out var value) ? value : null;
        }
    }
}

// Eventual Consistency Example
public class EventualConsistentDatabase
{
    private readonly List<Replica> _replicas;
    private readonly IMessageQueue _messageQueue;

    public EventualConsistentDatabase(List<Replica> replicas, IMessageQueue messageQueue)
    {
        _replicas = replicas;
        _messageQueue = messageQueue;
    }

    public async Task WriteAsync(string key, string value)
    {
        // Write to primary replica
        await _replicas[0].WriteAsync(key, value);

        // Queue replication message
        await _messageQueue.PublishAsync(new ReplicationMessage
        {
            Key = key,
            Value = value,
            Timestamp = DateTime.UtcNow
        });
    }

    public async Task<string> ReadAsync(string key)
    {
        // Read from any replica (may be stale)
        var randomReplica = _replicas[new Random().Next(_replicas.Count)];
        return await randomReplica.ReadAsync(key);
    }
}

60. What is a read replica?

Explanation: A read replica is a copy of a database that is used primarily for read operations. It receives updates from the primary database through replication and helps distribute read load, improving performance and availability.

public class ReadReplicaManager
{
    private readonly Database _primaryDatabase;
    private readonly List<Database> _readReplicas;
    private readonly ILoadBalancer _loadBalancer;

    public ReadReplicaManager(Database primaryDatabase, List<Database> readReplicas, ILoadBalancer loadBalancer)
    {
        _primaryDatabase = primaryDatabase;
        _readReplicas = readReplicas;
        _loadBalancer = loadBalancer;
    }

    public async Task<T> ReadAsync<T>(string query)
    {
        // Route read operations to replicas
        var replica = _loadBalancer.SelectReplica(_readReplicas);
        return await replica.ExecuteQueryAsync<T>(query);
    }

    public async Task WriteAsync<T>(string command, T data)
    {
        // All writes go to primary
        await _primaryDatabase.ExecuteCommandAsync(command, data);

        // Trigger replication to replicas
        await ReplicateToReplicasAsync(command, data);
    }

    private async Task ReplicateToReplicasAsync<T>(string command, T data)
    {
        var tasks = _readReplicas.Select(replica => 
            replica.ExecuteCommandAsync(command, data));
        await Task.WhenAll(tasks);
    }
}

61. What is master-slave replication?

“Master-slave” is legacy terminology; use primary/replica. A primary accepts writes and replicas copy changes. Read replicas improve read capacity but can return stale data, so define which reads require primary/consistent routing.

62. What is master-master replication?

Multi-primary replication allows writes in more than one location but needs conflict detection/resolution, careful schema evolution, and partition behavior. It is not automatically more available or simpler than a single-primary design.

63. What is a failover?

Explanation: Failover is the process of automatically switching from a failed primary system to a backup system to maintain service availability. It's a critical component of high availability architectures.

public class FailoverManager
{
    private readonly List<IService> _services;
    private readonly IHealthChecker _healthChecker;
    private readonly ILogger<FailoverManager> _logger;
    private IService _activeService;
    private readonly Timer _healthCheckTimer;

    public FailoverManager(List<IService> services, IHealthChecker healthChecker, ILogger<FailoverManager> logger)
    {
        _services = services;
        _healthChecker = healthChecker;
        _logger = logger;
        _activeService = services.First();

        _healthCheckTimer = new Timer(CheckHealth, null, 
            TimeSpan.Zero, TimeSpan.FromSeconds(5));
    }

    private async void CheckHealth(object state)
    {
        if (_activeService == null) return;

        var isHealthy = await _healthChecker.IsHealthyAsync(_activeService);
        if (!isHealthy)
        {
            await PerformFailoverAsync();
        }
    }

    private async Task PerformFailoverAsync()
    {
        _logger.LogWarning("Initiating failover from {ActiveService}", _activeService.Id);

        // Find healthy backup service
        var backupService = _services.FirstOrDefault(s => 
            s != _activeService && _healthChecker.IsHealthyAsync(s).Result);

        if (backupService != null)
        {
            // Switch to backup
            var oldService = _activeService;
            _activeService = backupService;

            // Notify about failover
            await OnFailoverCompletedAsync(oldService, _activeService);

            _logger.LogInformation("Failover completed to {NewService}", _activeService.Id);
        }
        else
        {
            _logger.LogError("No healthy backup service available for failover");
        }
    }

    public async Task<T> ExecuteAsync<T>(Func<IService, Task<T>> operation)
    {
        return await operation(_activeService);
    }

    private async Task OnFailoverCompletedAsync(IService oldService, IService newService)
    {
        // Notify monitoring systems, update load balancers, etc.
    }
}

64. What is disaster recovery?

Explanation: Disaster recovery is a set of policies, tools, and procedures that enable the recovery or continuation of technology infrastructure and systems following a natural or human-induced disaster.

public class DisasterRecoveryManager
{
    private readonly IBackupService _backupService;
    private readonly IRecoveryService _recoveryService;
    private readonly IMonitoringService _monitoringService;
    private readonly ILogger<DisasterRecoveryManager> _logger;

    public DisasterRecoveryManager(
        IBackupService backupService,
        IRecoveryService recoveryService,
        IMonitoringService monitoringService,
        ILogger<DisasterRecoveryManager> logger)
    {
        _backupService = backupService;
        _recoveryService = recoveryService;
        _monitoringService = monitoringService;
        _logger = logger;
    }

    public async Task<RecoveryPlan> CreateRecoveryPlanAsync()
    {
        var plan = new RecoveryPlan
        {
            Id = Guid.NewGuid(),
            CreatedAt = DateTime.UtcNow,
            Steps = new List<RecoveryStep>
            {
                new RecoveryStep { Name = "Assess Damage", Priority = 1 },
                new RecoveryStep { Name = "Restore Infrastructure", Priority = 2 },
                new RecoveryStep { Name = "Restore Data", Priority = 3 },
                new RecoveryStep { Name = "Verify Systems", Priority = 4 },
                new RecoveryStep { Name = "Resume Operations", Priority = 5 }
            }
        };

        return plan;
    }

    public async Task<bool> ExecuteRecoveryAsync(RecoveryPlan plan)
    {
        try
        {
            _logger.LogInformation("Starting disaster recovery with plan {PlanId}", plan.Id);

            foreach (var step in plan.Steps.OrderBy(s => s.Priority))
            {
                _logger.LogInformation("Executing recovery step: {StepName}", step.Name);

                var success = await ExecuteRecoveryStepAsync(step);
                if (!success)
                {
                    _logger.LogError("Recovery step {StepName} failed", step.Name);
                    return false;
                }
            }

            _logger.LogInformation("Disaster recovery completed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Disaster recovery failed");
            return false;
        }
    }

    private async Task<bool> ExecuteRecoveryStepAsync(RecoveryStep step)
    {
        switch (step.Name)
        {
            case "Assess Damage":
                return await AssessDamageAsync();
            case "Restore Infrastructure":
                return await RestoreInfrastructureAsync();
            case "Restore Data":
                return await RestoreDataAsync();
            case "Verify Systems":
                return await VerifySystemsAsync();
            case "Resume Operations":
                return await ResumeOperationsAsync();
            default:
                return false;
        }
    }

    private async Task<bool> AssessDamageAsync()
    {
        var healthStatus = await _monitoringService.GetSystemHealthAsync();
        return healthStatus.OverallHealth != HealthStatus.Critical;
    }

    private async Task<bool> RestoreInfrastructureAsync()
    {
        return await _recoveryService.RestoreInfrastructureAsync();
    }

    private async Task<bool> RestoreDataAsync()
    {
        var latestBackup = await _backupService.GetLatestBackupAsync();
        return await _recoveryService.RestoreDataAsync(latestBackup);
    }

    private async Task<bool> VerifySystemsAsync()
    {
        return await _monitoringService.VerifyAllSystemsAsync();
    }

    private async Task<bool> ResumeOperationsAsync()
    {
        return await _recoveryService.ResumeOperationsAsync();
    }
}

public class RecoveryPlan
{
    public Guid Id { get; set; }
    public DateTime CreatedAt { get; set; }
    public List<RecoveryStep> Steps { get; set; }
}

public class RecoveryStep
{
    public string Name { get; set; }
    public int Priority { get; set; }
}

65. What is a hot backup vs cold backup?

Explanation: - Hot Backup: A backup taken while the system is running and actively serving requests. It captures the current state without stopping operations. - Cold Backup: A backup taken when the system is completely stopped. It provides a consistent snapshot but requires downtime.

public class BackupManager
{
    private readonly IDatabase _database;
    private readonly IStorageService _storageService;
    private readonly ILogger<BackupManager> _logger;

    public BackupManager(IDatabase database, IStorageService storageService, ILogger<BackupManager> logger)
    {
        _database = database;
        _storageService = storageService;
        _logger = logger;
    }

    public async Task<BackupResult> CreateHotBackupAsync()
    {
        try
        {
            _logger.LogInformation("Starting hot backup");

            // Create backup without stopping the database
            var backupData = await _database.CreateSnapshotAsync();

            // Store backup data
            var backupId = Guid.NewGuid();
            await _storageService.StoreBackupAsync(backupId, backupData);

            var result = new BackupResult
            {
                Id = backupId,
                Type = BackupType.Hot,
                CreatedAt = DateTime.UtcNow,
                Size = backupData.Length,
                Status = BackupStatus.Completed
            };

            _logger.LogInformation("Hot backup completed: {BackupId}", backupId);
            return result;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Hot backup failed");
            return new BackupResult { Status = BackupStatus.Failed };
        }
    }

    public async Task<BackupResult> CreateColdBackupAsync()
    {
        try
        {
            _logger.LogInformation("Starting cold backup");

            // Stop the database
            await _database.StopAsync();

            // Create backup
            var backupData = await _database.CreateSnapshotAsync();

            // Store backup data
            var backupId = Guid.NewGuid();
            await _storageService.StoreBackupAsync(backupId, backupData);

            // Restart the database
            await _database.StartAsync();

            var result = new BackupResult
            {
                Id = backupId,
                Type = BackupType.Cold,
                CreatedAt = DateTime.UtcNow,
                Size = backupData.Length,
                Status = BackupStatus.Completed
            };

            _logger.LogInformation("Cold backup completed: {BackupId}", backupId);
            return result;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Cold backup failed");

            // Ensure database is restarted even if backup fails
            try
            {
                await _database.StartAsync();
            }
            catch (Exception restartEx)
            {
                _logger.LogError(restartEx, "Failed to restart database after cold backup failure");
            }

            return new BackupResult { Status = BackupStatus.Failed };
        }
    }
}

public enum BackupType
{
    Hot,
    Cold
}

public enum BackupStatus
{
    InProgress,
    Completed,
    Failed
}

public class BackupResult
{
    public Guid Id { get; set; }
    public BackupType Type { get; set; }
    public DateTime CreatedAt { get; set; }
    public long Size { get; set; }
    public BackupStatus Status { get; set; }
}

66. What is a blue-green deployment?

Explanation: Blue-green deployment is a deployment strategy where you maintain two identical production environments (blue and green). One environment serves live traffic while the other is used for testing new releases. When ready, traffic is switched from one environment to the other.

public class BlueGreenDeployment
{
    private readonly ILoadBalancer _loadBalancer;
    private readonly IDeploymentService _deploymentService;
    private readonly IHealthChecker _healthChecker;
    private readonly ILogger<BlueGreenDeployment> _logger;

    public BlueGreenDeployment(
        ILoadBalancer loadBalancer,
        IDeploymentService deploymentService,
        IHealthChecker healthChecker,
        ILogger<BlueGreenDeployment> logger)
    {
        _loadBalancer = loadBalancer;
        _deploymentService = deploymentService;
        _healthChecker = healthChecker;
        _logger = logger;
    }

    public async Task<bool> DeployAsync(string version)
    {
        try
        {
            _logger.LogInformation("Starting blue-green deployment for version {Version}", version);

            // Determine current active environment
            var currentActive = await _loadBalancer.GetActiveEnvironmentAsync();
            var targetEnvironment = currentActive == Environment.Blue ? Environment.Green : Environment.Blue;

            // Deploy to inactive environment
            _logger.LogInformation("Deploying to {TargetEnvironment}", targetEnvironment);
            var deploymentSuccess = await _deploymentService.DeployAsync(targetEnvironment, version);

            if (!deploymentSuccess)
            {
                _logger.LogError("Deployment to {TargetEnvironment} failed", targetEnvironment);
                return false;
            }

            // Health check the new deployment
            var healthCheck = await _healthChecker.CheckHealthAsync(targetEnvironment);
            if (!healthCheck.IsHealthy)
            {
                _logger.LogError("Health check failed for {TargetEnvironment}", targetEnvironment);
                return false;
            }

            // Switch traffic to new environment
            _logger.LogInformation("Switching traffic from {Current} to {Target}", currentActive, targetEnvironment);
            await _loadBalancer.SwitchTrafficAsync(targetEnvironment);

            // Verify switch was successful
            var verification = await _loadBalancer.GetActiveEnvironmentAsync();
            if (verification != targetEnvironment)
            {
                _logger.LogError("Traffic switch verification failed");
                return false;
            }

            _logger.LogInformation("Blue-green deployment completed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Blue-green deployment failed");
            return false;
        }
    }

    public async Task<bool> RollbackAsync()
    {
        try
        {
            var currentActive = await _loadBalancer.GetActiveEnvironmentAsync();
            var rollbackEnvironment = currentActive == Environment.Blue ? Environment.Green : Environment.Blue;

            _logger.LogInformation("Rolling back to {RollbackEnvironment}", rollbackEnvironment);

            // Switch traffic back
            await _loadBalancer.SwitchTrafficAsync(rollbackEnvironment);

            _logger.LogInformation("Rollback completed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Rollback failed");
            return false;
        }
    }
}

public enum Environment
{
    Blue,
    Green
}

67. What is a canary release?

Explanation: A canary release is a deployment strategy where a new version is deployed to a small subset of users or servers first. This allows for testing in production with minimal risk, and the release can be gradually expanded or rolled back based on monitoring and feedback.

public class CanaryRelease
{
    private readonly ILoadBalancer _loadBalancer;
    private readonly IDeploymentService _deploymentService;
    private readonly IMonitoringService _monitoringService;
    private readonly ILogger<CanaryRelease> _logger;

    public CanaryRelease(
        ILoadBalancer loadBalancer,
        IDeploymentService deploymentService,
        IMonitoringService monitoringService,
        ILogger<CanaryRelease> logger)
    {
        _loadBalancer = loadBalancer;
        _deploymentService = deploymentService;
        _monitoringService = monitoringService;
        _logger = logger;
    }

    public async Task<bool> DeployCanaryAsync(string version, double initialPercentage = 5.0)
    {
        try
        {
            _logger.LogInformation("Starting canary release for version {Version} with {Percentage}% traffic", 
                version, initialPercentage);

            // Deploy to canary servers
            var canaryServers = await _loadBalancer.GetCanaryServersAsync();
            foreach (var server in canaryServers)
            {
                await _deploymentService.DeployAsync(server, version);
            }

            // Route small percentage of traffic to canary
            await _loadBalancer.SetCanaryTrafficPercentageAsync(initialPercentage);

            // Start monitoring
            await StartCanaryMonitoringAsync(version);

            _logger.LogInformation("Canary release initiated successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Canary release failed");
            return false;
        }
    }

    public async Task<bool> ExpandCanaryAsync(double newPercentage)
    {
        try
        {
            _logger.LogInformation("Expanding canary to {Percentage}%", newPercentage);

            // Check current metrics
            var metrics = await _monitoringService.GetCanaryMetricsAsync();
            if (!metrics.IsHealthy)
            {
                _logger.LogWarning("Canary metrics indicate issues, not expanding");
                return false;
            }

            // Increase traffic percentage
            await _loadBalancer.SetCanaryTrafficPercentageAsync(newPercentage);

            _logger.LogInformation("Canary expanded successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Canary expansion failed");
            return false;
        }
    }

    public async Task<bool> PromoteCanaryAsync()
    {
        try
        {
            _logger.LogInformation("Promoting canary to full release");

            // Deploy to all servers
            var allServers = await _loadBalancer.GetAllServersAsync();
            foreach (var server in allServers)
            {
                await _deploymentService.DeployAsync(server, GetCanaryVersion());
            }

            // Route all traffic to new version
            await _loadBalancer.SetCanaryTrafficPercentageAsync(100.0);

            // Stop canary monitoring
            await StopCanaryMonitoringAsync();

            _logger.LogInformation("Canary promoted successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Canary promotion failed");
            return false;
        }
    }

    public async Task<bool> RollbackCanaryAsync()
    {
        try
        {
            _logger.LogInformation("Rolling back canary release");

            // Route all traffic back to stable version
            await _loadBalancer.SetCanaryTrafficPercentageAsync(0.0);

            // Stop canary monitoring
            await StopCanaryMonitoringAsync();

            _logger.LogInformation("Canary rollback completed");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Canary rollback failed");
            return false;
        }
    }

    private async Task StartCanaryMonitoringAsync(string version)
    {
        // Start monitoring canary metrics
        await _monitoringService.StartCanaryMonitoringAsync(version);
    }

    private async Task StopCanaryMonitoringAsync()
    {
        await _monitoringService.StopCanaryMonitoringAsync();
    }

    private string GetCanaryVersion()
    {
        // Get the current canary version
        return "canary-version";
    }
}

68. What is a rolling update?

Explanation: A rolling update is a deployment strategy where new versions are deployed to servers one at a time or in small batches, while keeping the service available throughout the deployment process. This minimizes downtime and allows for gradual rollout.

public class RollingUpdate
{
    private readonly ILoadBalancer _loadBalancer;
    private readonly IDeploymentService _deploymentService;
    private readonly IHealthChecker _healthChecker;
    private readonly ILogger<RollingUpdate> _logger;

    public RollingUpdate(
        ILoadBalancer loadBalancer,
        IDeploymentService deploymentService,
        IHealthChecker healthChecker,
        ILogger<RollingUpdate> logger)
    {
        _loadBalancer = loadBalancer;
        _deploymentService = deploymentService;
        _healthChecker = healthChecker;
        _logger = logger;
    }

    public async Task<bool> DeployAsync(string version, int batchSize = 1)
    {
        try
        {
            _logger.LogInformation("Starting rolling update for version {Version} with batch size {BatchSize}", 
                version, batchSize);

            var servers = await _loadBalancer.GetAllServersAsync();
            var totalServers = servers.Count;
            var updatedServers = 0;

            for (int i = 0; i < totalServers; i += batchSize)
            {
                var batch = servers.Skip(i).Take(batchSize).ToList();

                _logger.LogInformation("Updating batch {BatchNumber}: {ServerCount} servers", 
                    (i / batchSize) + 1, batch.Count);

                // Remove servers from load balancer
                foreach (var server in batch)
                {
                    await _loadBalancer.RemoveServerAsync(server);
                }

                // Deploy to batch
                var deploymentTasks = batch.Select(server => 
                    _deploymentService.DeployAsync(server, version));
                await Task.WhenAll(deploymentTasks);

                // Health check batch
                var healthTasks = batch.Select(server => 
                    _healthChecker.CheckHealthAsync(server));
                var healthResults = await Task.WhenAll(healthTasks);

                if (healthResults.All(r => r.IsHealthy))
                {
                    // Add servers back to load balancer
                    foreach (var server in batch)
                    {
                        await _loadBalancer.AddServerAsync(server);
                    }

                    updatedServers += batch.Count;
                    _logger.LogInformation("Batch updated successfully. Progress: {Updated}/{Total}", 
                        updatedServers, totalServers);
                }
                else
                {
                    _logger.LogError("Health check failed for batch");
                    await RollbackBatchAsync(batch);
                    return false;
                }

                // Wait between batches
                if (i + batchSize < totalServers)
                {
                    await Task.Delay(TimeSpan.FromSeconds(30));
                }
            }

            _logger.LogInformation("Rolling update completed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Rolling update failed");
            return false;
        }
    }

    private async Task RollbackBatchAsync(List<Server> batch)
    {
        _logger.LogInformation("Rolling back batch");

        foreach (var server in batch)
        {
            await _deploymentService.RollbackAsync(server);
            await _loadBalancer.AddServerAsync(server);
        }
    }

    public async Task<bool> RollbackAsync()
    {
        try
        {
            _logger.LogInformation("Starting rolling rollback");

            var servers = await _loadBalancer.GetAllServersAsync();
            var batchSize = 1;

            for (int i = 0; i < servers.Count; i += batchSize)
            {
                var batch = servers.Skip(i).Take(batchSize).ToList();

                foreach (var server in batch)
                {
                    await _loadBalancer.RemoveServerAsync(server);
                    await _deploymentService.RollbackAsync(server);
                    await _loadBalancer.AddServerAsync(server);
                }

                if (i + batchSize < servers.Count)
                {
                    await Task.Delay(TimeSpan.FromSeconds(30));
                }
            }

            _logger.LogInformation("Rolling rollback completed");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Rolling rollback failed");
            return false;
        }
    }
}

69. What is a stateless service?

A stateless service is a service that doesn't maintain any state or data between requests. Each request is processed independently without relying on information from previous requests.

Key Characteristics: - No memory of previous interactions - Can be easily scaled horizontally - High availability and fault tolerance - Each request contains all necessary information

Example in C#:

public class StatelessUserService
{
    public User GetUser(int userId)
    {
        // No state maintained between calls
        // All data comes from external source (database)
        return _userRepository.GetById(userId);
    }

    public void ProcessOrder(Order order)
    {
        // Each order is processed independently
        // No dependency on previous orders
        _orderProcessor.Process(order);
    }
}

70. What is a stateful service?

A stateful service maintains state or data between requests. It remembers information from previous interactions and uses it to process subsequent requests.

Key Characteristics: - Maintains session data or application state - State can be in-memory or persistent - More complex to scale and manage - Provides better performance for state-dependent operations

Example in C#:

public class StatefulShoppingCartService
{
    private Dictionary<string, List<CartItem>> _userCarts = new();

    public void AddItem(string userId, CartItem item)
    {
        if (!_userCarts.ContainsKey(userId))
        {
            _userCarts[userId] = new List<CartItem>();
        }

        // Maintains state between requests
        _userCarts[userId].Add(item);
    }

    public List<CartItem> GetCart(string userId)
    {
        return _userCarts.GetValueOrDefault(userId, new List<CartItem>());
    }
}

71. What is a session store?

A session store is a mechanism to store session data (user state) that persists across multiple requests. It can be in-memory, distributed cache, or database.

Types: - In-memory session store - Distributed cache (Redis, Memcached) - Database-based session store

Example in C#:

public class RedisSessionStore
{
    private readonly IDatabase _redis;

    public RedisSessionStore(IConnectionMultiplexer redis)
    {
        _redis = redis.GetDatabase();
    }

    public async Task SetSessionDataAsync(string sessionId, string key, string value)
    {
        await _redis.StringSetAsync($"session:{sessionId}:{key}", value, TimeSpan.FromMinutes(30));
    }

    public async Task<string> GetSessionDataAsync(string sessionId, string key)
    {
        return await _redis.StringGetAsync($"session:{sessionId}:{key}");
    }
}

72. What is sticky session?

Sticky session (session affinity) ensures that all requests from a specific client are routed to the same server instance, maintaining session state.

Use Cases: - When using stateful services - Maintaining user sessions - Ensuring data consistency

Example in C#:

public class StickySessionMiddleware
{
    private readonly RequestDelegate _next;

    public StickySessionMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var sessionId = context.Session.Id;

        // Route to specific server based on session ID
        var serverId = GetServerIdFromSessionId(sessionId);

        // Set routing header
        context.Request.Headers["X-Server-ID"] = serverId;

        await _next(context);
    }

    private string GetServerIdFromSessionId(string sessionId)
    {
        // Hash-based routing to ensure consistency
        var hash = sessionId.GetHashCode();
        return $"server-{Math.Abs(hash) % 3}"; // 3 servers
    }
}

73. What is a web socket?

WebSocket is a persistent, full-duplex connection established through an HTTP upgrade handshake. It is useful for low-latency bidirectional events, but requires connection management, backpressure, authentication renewal, routing, and reconnection design.

74. What is long polling?

Long polling is a technique where the client sends a request to the server and the server holds the connection open until new data is available, then responds.

Characteristics: - Reduces server load compared to short polling - Provides near real-time updates - Works well with HTTP

Example in C#:

public class LongPollingController : ControllerBase
{
    [HttpGet("updates")]
    public async Task<IActionResult> GetUpdates([FromQuery] long lastUpdateTime)
    {
        var timeout = TimeSpan.FromSeconds(30);
        var startTime = DateTime.UtcNow;

        while (DateTime.UtcNow - startTime < timeout)
        {
            var updates = await GetNewUpdates(lastUpdateTime);

            if (updates.Any())
            {
                return Ok(updates);
            }

            await Task.Delay(1000); // Check every second
        }

        return NoContent(); // No updates available
    }

    private async Task<List<Update>> GetNewUpdates(long lastUpdateTime)
    {
        // Check for new updates since lastUpdateTime
        return await _updateService.GetUpdatesSince(lastUpdateTime);
    }
}

75. What is short polling?

Short polling is a technique where the client repeatedly sends requests to the server at regular intervals to check for new data.

Characteristics: - Simple to implement - Higher server load - May miss real-time updates - Works with any HTTP client

Example in C#:

public class ShortPollingService
{
    private readonly HttpClient _httpClient;

    public ShortPollingService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task StartPolling()
    {
        while (true)
        {
            try
            {
                var response = await _httpClient.GetAsync("/api/updates");

                if (response.IsSuccessStatusCode)
                {
                    var updates = await response.Content.ReadFromJsonAsync<List<Update>>();
                    ProcessUpdates(updates);
                }
            }
            catch (Exception ex)
            {
                // Handle error
            }

            await Task.Delay(5000); // Poll every 5 seconds
        }
    }

    private void ProcessUpdates(List<Update> updates)
    {
        // Process received updates
    }
}

76. What is REST?

REST (Representational State Transfer) is an architectural style for designing networked applications using HTTP methods and stateless operations.

Key Principles: - Stateless - Client-server architecture - Cacheable - Uniform interface - Layered system

Example in C#:

[ApiController]
[Route("api/[controller]")]
public class UsersController : ControllerBase
{
    [HttpGet]
    public async Task<ActionResult<IEnumerable<User>>> GetUsers()
    {
        var users = await _userService.GetAllUsers();
        return Ok(users);
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<User>> GetUser(int id)
    {
        var user = await _userService.GetUserById(id);
        if (user == null) return NotFound();
        return Ok(user);
    }

    [HttpPost]
    public async Task<ActionResult<User>> CreateUser([FromBody] CreateUserRequest request)
    {
        var user = await _userService.CreateUser(request);
        return CreatedAtAction(nameof(GetUser), new { id = user.Id }, user);
    }

    [HttpPut("{id}")]
    public async Task<IActionResult> UpdateUser(int id, [FromBody] UpdateUserRequest request)
    {
        var success = await _userService.UpdateUser(id, request);
        if (!success) return NotFound();
        return NoContent();
    }

    [HttpDelete("{id}")]
    public async Task<IActionResult> DeleteUser(int id)
    {
        var success = await _userService.DeleteUser(id);
        if (!success) return NotFound();
        return NoContent();
    }
}

77. What is gRPC?

gRPC is a high-performance RPC (Remote Procedure Call) framework that uses HTTP/2 and Protocol Buffers for efficient communication between services.

Characteristics: - Strongly typed contracts - High performance - Bidirectional streaming - Code generation

Example in C#:

// Proto file (user.proto)
/*
syntax = "proto3";
package user;

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
  rpc CreateUser (CreateUserRequest) returns (User);
  rpc GetUsers (GetUsersRequest) returns (stream User);
}

message GetUserRequest {
  int32 id = 1;
}

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
}
*/

// Server implementation
public class UserService : User.UserBase
{
    public override async Task<User> GetUser(GetUserRequest request, ServerCallContext context)
    {
        var user = await _userRepository.GetByIdAsync(request.Id);
        return new User
        {
            Id = user.Id,
            Name = user.Name,
            Email = user.Email
        };
    }

    public override async Task GetUsers(GetUsersRequest request, IServerStreamWriter<User> responseStream, ServerCallContext context)
    {
        var users = await _userRepository.GetAllAsync();

        foreach (var user in users)
        {
            await responseStream.WriteAsync(new User
            {
                Id = user.Id,
                Name = user.Name,
                Email = user.Email
            });
        }
    }
}

78. What is GraphQL?

GraphQL is a query language and runtime for APIs that allows clients to request exactly the data they need, nothing more and nothing less.

Characteristics: - Single endpoint - Strongly typed schema - Introspection - Real-time subscriptions

Example in C#:

public class UserType : ObjectGraphType<User>
{
    public UserType()
    {
        Field(x => x.Id);
        Field(x => x.Name);
        Field(x => x.Email);
        Field<ListGraphType<PostType>>("posts", resolve: context => 
        {
            return _postService.GetPostsByUserId(context.Source.Id);
        });
    }
}

public class UserQuery : ObjectGraphType
{
    public UserQuery(IUserService userService)
    {
        Field<UserType>("user",
            arguments: new QueryArguments(new QueryArgument<IntGraphType> { Name = "id" }),
            resolve: context => userService.GetUserById(context.GetArgument<int>("id")));

        Field<ListGraphType<UserType>>("users",
            resolve: context => userService.GetAllUsers());
    }
}

// Schema setup
public class UserSchema : Schema
{
    public UserSchema(IServiceProvider provider) : base(provider)
    {
        Query = provider.GetRequiredService<UserQuery>();
    }
}

79. What is API versioning?

API versioning is a strategy to manage changes to APIs while maintaining backward compatibility and allowing clients to use different versions.

Common Strategies: - URL versioning (/api/v1/users) - Header versioning (Accept: application/vnd.api+json;version=1) - Query parameter versioning (/api/users?version=1)

Example in C#:

[ApiVersion("1.0")]
[ApiVersion("2.0")]
[Route("api/v{version:apiVersion}/[controller]")]
[ApiController]
public class UsersController : ControllerBase
{
    [HttpGet]
    [MapToApiVersion("1.0")]
    public async Task<ActionResult<IEnumerable<UserV1>>> GetUsersV1()
    {
        var users = await _userService.GetAllUsers();
        return Ok(users.Select(u => new UserV1 { Id = u.Id, Name = u.Name }));
    }

    [HttpGet]
    [MapToApiVersion("2.0")]
    public async Task<ActionResult<IEnumerable<UserV2>>> GetUsersV2()
    {
        var users = await _userService.GetAllUsers();
        return Ok(users.Select(u => new UserV2 
        { 
            Id = u.Id, 
            Name = u.Name, 
            Email = u.Email,
            CreatedAt = u.CreatedAt
        }));
    }
}

// Startup configuration
public void ConfigureServices(IServiceCollection services)
{
    services.AddApiVersioning(options =>
    {
        options.DefaultApiVersion = new ApiVersion(1, 0);
        options.AssumeDefaultVersionWhenUnspecified = true;
        options.ReportApiVersions = true;
    });

    services.AddVersionedApiExplorer(options =>
    {
        options.GroupNameFormat = "'v'VVV";
        options.SubstituteApiVersionInUrl = true;
    });
}

80. What is a service mesh?

A service mesh is a dedicated infrastructure layer that handles service-to-service communication, providing features like service discovery, load balancing, failure recovery, metrics, and monitoring.

Components: - Data plane (sidecar proxies) - Control plane (management) - Service discovery - Load balancing

Example in C# (using Envoy proxy configuration):

// Service mesh configuration example
public class ServiceMeshConfiguration
{
    public void ConfigureServiceMesh()
    {
        // Service discovery
        var serviceRegistry = new ServiceRegistry();
        serviceRegistry.RegisterService("user-service", "10.0.0.1:8080");
        serviceRegistry.RegisterService("order-service", "10.0.0.2:8080");

        // Load balancing
        var loadBalancer = new RoundRobinLoadBalancer();

        // Circuit breaker
        var circuitBreaker = new CircuitBreakerPolicy
        {
            FailureThreshold = 5,
            RecoveryTimeout = TimeSpan.FromMinutes(1)
        };

        // Retry policy
        var retryPolicy = new RetryPolicy
        {
            MaxRetries = 3,
            BackoffDelay = TimeSpan.FromSeconds(1)
        };
    }
}

// Service communication with mesh
public class UserService
{
    private readonly IServiceMeshClient _meshClient;

    public async Task<Order> GetUserOrders(int userId)
    {
        // Service mesh handles discovery, load balancing, retries, etc.
        return await _meshClient.CallServiceAsync<Order>("order-service", 
            $"/api/orders/user/{userId}");
    }
}

81. What is observability?

Observability is the ability to understand the internal state of a system by examining its outputs, typically through logs, metrics, and traces.

Three Pillars: - Logging - Metrics - Tracing

Example in C#:

public class ObservableService
{
    private readonly ILogger<ObservableService> _logger;
    private readonly IMetrics _metrics;
    private readonly ITracer _tracer;

    public async Task<User> GetUser(int userId)
    {
        using var span = _tracer.StartSpan("get_user");
        span.SetTag("user_id", userId);

        try
        {
            _logger.LogInformation("Fetching user with ID: {UserId}", userId);
            _metrics.IncrementCounter("user_requests_total");

            var user = await _userRepository.GetByIdAsync(userId);

            if (user != null)
            {
                _metrics.IncrementCounter("user_found_total");
                span.SetTag("user_found", true);
                return user;
            }
            else
            {
                _metrics.IncrementCounter("user_not_found_total");
                span.SetTag("user_found", false);
                _logger.LogWarning("User not found: {UserId}", userId);
                return null;
            }
        }
        catch (Exception ex)
        {
            _metrics.IncrementCounter("user_errors_total");
            span.SetTag("error", true);
            span.SetTag("error.message", ex.Message);
            _logger.LogError(ex, "Error fetching user: {UserId}", userId);
            throw;
        }
    }
}

82. What is monitoring?

Monitoring is the process of collecting, analyzing, and using information to track a system's performance and health over time.

Types: - Infrastructure monitoring - Application monitoring - Business monitoring

Example in C#:

public class SystemMonitor
{
    private readonly IMetrics _metrics;
    private readonly IHealthCheck _healthCheck;

    public void MonitorSystemHealth()
    {
        // CPU usage
        var cpuUsage = GetCpuUsage();
        _metrics.Gauge("system.cpu.usage", cpuUsage);

        // Memory usage
        var memoryUsage = GetMemoryUsage();
        _metrics.Gauge("system.memory.usage", memoryUsage);

        // Database connections
        var dbConnections = GetDatabaseConnections();
        _metrics.Gauge("database.connections.active", dbConnections);

        // Response times
        var avgResponseTime = GetAverageResponseTime();
        _metrics.Gauge("api.response_time.avg", avgResponseTime);
    }

    public async Task<HealthReport> CheckHealthAsync()
    {
        var healthChecks = new List<IHealthCheck>
        {
            new DatabaseHealthCheck(),
            new RedisHealthCheck(),
            new ExternalApiHealthCheck()
        };

        var results = new List<HealthCheckResult>();

        foreach (var check in healthChecks)
        {
            var result = await check.CheckHealthAsync();
            results.Add(result);
        }

        return new HealthReport(results);
    }
}

83. What is logging?

Logging is the process of recording events, messages, and data about application execution for debugging, monitoring, and auditing purposes.

Log Levels: - Trace, Debug, Info, Warning, Error, Critical

Example in C#:

public class LoggingService
{
    private readonly ILogger<LoggingService> _logger;

    public async Task ProcessOrder(Order order)
    {
        _logger.LogInformation("Starting order processing for order {OrderId}", order.Id);

        try
        {
            // Validate order
            _logger.LogDebug("Validating order {OrderId}", order.Id);
            await ValidateOrder(order);

            // Process payment
            _logger.LogInformation("Processing payment for order {OrderId}, Amount: {Amount}", 
                order.Id, order.TotalAmount);
            await ProcessPayment(order);

            // Update inventory
            _logger.LogDebug("Updating inventory for order {OrderId}", order.Id);
            await UpdateInventory(order);

            _logger.LogInformation("Order {OrderId} processed successfully", order.Id);
        }
        catch (PaymentException ex)
        {
            _logger.LogError(ex, "Payment failed for order {OrderId}: {ErrorMessage}", 
                order.Id, ex.Message);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogCritical(ex, "Unexpected error processing order {OrderId}", order.Id);
            throw;
        }
    }

    public void LogStructuredData()
    {
        var logData = new
        {
            UserId = 123,
            Action = "login",
            Timestamp = DateTime.UtcNow,
            IpAddress = "192.168.1.1"
        };

        _logger.LogInformation("User activity: {@LogData}", logData);
    }
}

84. What is tracing?

Tracing is the process of tracking requests as they flow through a distributed system, helping to understand the path and performance of requests.

Concepts: - Spans (individual operations) - Traces (complete request flows) - Context propagation

Example in C#:

public class TracingService
{
    private readonly ITracer _tracer;

    public async Task<Order> ProcessOrder(OrderRequest request)
    {
        using var span = _tracer.StartSpan("process_order");
        span.SetTag("order_id", request.OrderId);
        span.SetTag("customer_id", request.CustomerId);

        try
        {
            // Validate order
            using (var validateSpan = _tracer.StartSpan("validate_order", span.Context))
            {
                await ValidateOrder(request);
                validateSpan.SetTag("validation_result", "success");
            }

            // Process payment
            using (var paymentSpan = _tracer.StartSpan("process_payment", span.Context))
            {
                var paymentResult = await ProcessPayment(request);
                paymentSpan.SetTag("payment_method", request.PaymentMethod);
                paymentSpan.SetTag("payment_status", paymentResult.Status);
            }

            // Create order
            using (var createSpan = _tracer.StartSpan("create_order", span.Context))
            {
                var order = await CreateOrder(request);
                createSpan.SetTag("order_created", true);
                return order;
            }
        }
        catch (Exception ex)
        {
            span.SetTag("error", true);
            span.SetTag("error.message", ex.Message);
            span.Log(new Dictionary<string, object>
            {
                ["event"] = "error",
                ["error.kind"] = ex.GetType().Name,
                ["error.message"] = ex.Message,
                ["stack"] = ex.StackTrace
            });
            throw;
        }
    }

    public async Task<Order> GetOrder(int orderId)
    {
        using var span = _tracer.StartSpan("get_order");
        span.SetTag("order_id", orderId);

        // Inject trace context into HTTP request
        var httpClient = new HttpClient();
        _tracer.Inject(span.Context, BuiltinFormats.HttpHeaders, new HttpHeadersCarrier(httpClient.DefaultRequestHeaders));

        var response = await httpClient.GetAsync($"https://api.orders.com/orders/{orderId}");
        var order = await response.Content.ReadFromJsonAsync<Order>();

        span.SetTag("order_found", order != null);
        return order;
    }
}

85. What is alerting?

Alerting is the process of notifying stakeholders when certain conditions or thresholds are met, typically for system issues or performance problems.

Types: - Threshold-based alerts - Anomaly detection - Business metrics alerts

Example in C#:

public class AlertingService
{
    private readonly IAlertManager _alertManager;
    private readonly IMetrics _metrics;

    public void MonitorAndAlert()
    {
        // CPU usage alert
        var cpuUsage = GetCpuUsage();
        if (cpuUsage > 80)
        {
            _alertManager.SendAlert(new Alert
            {
                Severity = AlertSeverity.Warning,
                Title = "High CPU Usage",
                Message = $"CPU usage is {cpuUsage}%",
                Metric = "system.cpu.usage",
                Value = cpuUsage,
                Threshold = 80
            });
        }

        // Error rate alert
        var errorRate = GetErrorRate();
        if (errorRate > 5)
        {
            _alertManager.SendAlert(new Alert
            {
                Severity = AlertSeverity.Critical,
                Title = "High Error Rate",
                Message = $"Error rate is {errorRate}%",
                Metric = "api.error_rate",
                Value = errorRate,
                Threshold = 5
            });
        }

        // Response time alert
        var avgResponseTime = GetAverageResponseTime();
        if (avgResponseTime > TimeSpan.FromSeconds(2))
        {
            _alertManager.SendAlert(new Alert
            {
                Severity = AlertSeverity.Warning,
                Title = "Slow Response Time",
                Message = $"Average response time is {avgResponseTime.TotalMilliseconds}ms",
                Metric = "api.response_time.avg",
                Value = avgResponseTime.TotalMilliseconds,
                Threshold = 2000
            });
        }
    }

    public async Task SendAlertAsync(Alert alert)
    {
        // Send to multiple channels
        await Task.WhenAll(
            SendEmailAlert(alert),
            SendSlackAlert(alert),
            SendPagerDutyAlert(alert)
        );
    }
}

86. What is autoscaling?

Autoscaling is the automatic adjustment of computing resources based on demand, ensuring optimal performance and cost efficiency.

Types: - Horizontal scaling (add/remove instances) - Vertical scaling (increase/decrease instance size)

Example in C#:

public class AutoScalingService
{
    private readonly IMetrics _metrics;
    private readonly ICloudProvider _cloudProvider;

    public async Task MonitorAndScale()
    {
        var cpuUsage = await GetAverageCpuUsage();
        var memoryUsage = await GetAverageMemoryUsage();
        var requestCount = await GetRequestCount();

        var currentInstances = await _cloudProvider.GetInstanceCount();
        var targetInstances = CalculateTargetInstances(cpuUsage, memoryUsage, requestCount);

        if (targetInstances > currentInstances)
        {
            await ScaleUp(targetInstances - currentInstances);
        }
        else if (targetInstances < currentInstances)
        {
            await ScaleDown(currentInstances - targetInstances);
        }
    }

    private int CalculateTargetInstances(double cpuUsage, double memoryUsage, int requestCount)
    {
        // Scale based on CPU usage
        var cpuBasedInstances = Math.Ceiling(cpuUsage / 70.0); // 70% CPU per instance

        // Scale based on memory usage
        var memoryBasedInstances = Math.Ceiling(memoryUsage / 80.0); // 80% memory per instance

        // Scale based on request count
        var requestBasedInstances = Math.Ceiling(requestCount / 1000.0); // 1000 requests per instance

        // Take the maximum and apply min/max bounds
        var targetInstances = Math.Max(cpuBasedInstances, Math.Max(memoryBasedInstances, requestBasedInstances));

        return Math.Max(1, Math.Min(10, (int)targetInstances)); // Min 1, Max 10 instances
    }

    private async Task ScaleUp(int additionalInstances)
    {
        _logger.LogInformation("Scaling up by {Count} instances", additionalInstances);

        for (int i = 0; i < additionalInstances; i++)
        {
            await _cloudProvider.CreateInstance(new InstanceConfig
            {
                InstanceType = "t3.medium",
                ImageId = "ami-12345678",
                SecurityGroups = new[] { "web-sg" }
            });
        }
    }

    private async Task ScaleDown(int instancesToRemove)
    {
        _logger.LogInformation("Scaling down by {Count} instances", instancesToRemove);

        var instances = await _cloudProvider.GetInstances();
        var instancesToTerminate = instances
            .Where(i => i.State == InstanceState.Running)
            .OrderBy(i => i.LaunchTime)
            .Take(instancesToRemove);

        foreach (var instance in instancesToTerminate)
        {
            await _cloudProvider.TerminateInstance(instance.Id);
        }
    }
}

87. What is a dead letter queue?

A dead letter queue (DLQ) is a queue that stores messages that cannot be processed successfully after multiple attempts, allowing for analysis and reprocessing.

Use Cases: - Failed message processing - Poison messages - Retry exhaustion - Manual intervention

Example in C#:

public class DeadLetterQueueService
{
    private readonly IQueueService _queueService;
    private readonly ILogger<DeadLetterQueueService> _logger;

    public async Task ProcessMessage(Message message)
    {
        var retryCount = 0;
        const int maxRetries = 3;

        while (retryCount < maxRetries)
        {
            try
            {
                await ProcessMessageInternal(message);
                return; // Success, exit retry loop
            }
            catch (Exception ex)
            {
                retryCount++;
                _logger.LogWarning(ex, "Failed to process message {MessageId}, attempt {RetryCount}", 
                    message.Id, retryCount);

                if (retryCount >= maxRetries)
                {
                    await SendToDeadLetterQueue(message, ex);
                }
                else
                {
                    await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, retryCount))); // Exponential backoff
                }
            }
        }
    }

    private async Task SendToDeadLetterQueue(Message message, Exception error)
    {
        var dlqMessage = new DeadLetterMessage
        {
            OriginalMessage = message,
            ErrorMessage = error.Message,
            ErrorType = error.GetType().Name,
            FailedAt = DateTime.UtcNow,
            RetryCount = 3
        };

        await _queueService.SendMessageAsync("dead-letter-queue", dlqMessage);

        _logger.LogError("Message {MessageId} sent to dead letter queue after {RetryCount} failed attempts", 
            message.Id, 3);
    }

    public async Task ReprocessDeadLetterMessages()
    {
        var dlqMessages = await _queueService.ReceiveMessagesAsync("dead-letter-queue");

        foreach (var dlqMessage in dlqMessages)
        {
            try
            {
                // Attempt to reprocess the original message
                await ProcessMessageInternal(dlqMessage.OriginalMessage);

                // Remove from DLQ if successful
                await _queueService.DeleteMessageAsync("dead-letter-queue", dlqMessage.Id);

                _logger.LogInformation("Successfully reprocessed message {MessageId} from DLQ", 
                    dlqMessage.OriginalMessage.Id);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed to reprocess message {MessageId} from DLQ", 
                    dlqMessage.OriginalMessage.Id);
            }
        }
    }
}

88. What is a poison message?

A poison message is a message that causes a processing failure every time it's attempted, often due to malformed data, missing dependencies, or system issues.

Characteristics: - Always fails processing - Consumes resources - Can block message processing - Requires manual intervention

Example in C#:

public class PoisonMessageHandler
{
    private readonly ILogger<PoisonMessageHandler> _logger;

    public async Task ProcessMessage(Message message)
    {
        try
        {
            // Validate message format
            if (!IsValidMessage(message))
            {
                throw new InvalidMessageException("Message format is invalid");
            }

            // Check for poison message patterns
            if (IsPoisonMessage(message))
            {
                await HandlePoisonMessage(message);
                return;
            }

            await ProcessValidMessage(message);
        }
        catch (Exception ex)
        {
            await HandleProcessingError(message, ex);
        }
    }

    private bool IsPoisonMessage(Message message)
    {
        // Check for known poison message patterns
        return message.Content.Contains("malformed_data") ||
               message.Content.Contains("invalid_json") ||
               message.Size > 1024 * 1024; // Messages larger than 1MB
    }

    private async Task HandlePoisonMessage(Message message)
    {
        _logger.LogWarning("Detected poison message: {MessageId}", message.Id);

        // Log detailed information for analysis
        _logger.LogInformation("Poison message details: {@MessageDetails}", new
        {
            MessageId = message.Id,
            Content = message.Content,
            Size = message.Size,
            Timestamp = message.Timestamp
        });

        // Send to poison message queue for manual review
        await SendToPoisonMessageQueue(message);

        // Don't retry poison messages
        throw new PoisonMessageException($"Message {message.Id} identified as poison message");
    }

    private async Task HandleProcessingError(Message message, Exception error)
    {
        var retryCount = GetRetryCount(message);

        if (retryCount >= 3)
        {
            // Check if this is becoming a poison message
            if (IsBecomingPoisonMessage(message, error))
            {
                await HandlePoisonMessage(message);
            }
            else
            {
                await SendToDeadLetterQueue(message, error);
            }
        }
        else
        {
            // Increment retry count and retry
            IncrementRetryCount(message);
            throw; // Re-throw to trigger retry
        }
    }

    private bool IsBecomingPoisonMessage(Message message, Exception error)
    {
        // Check if the same message has failed multiple times with the same error
        var errorHistory = GetErrorHistory(message.Id);
        var sameErrorCount = errorHistory.Count(e => e.ErrorType == error.GetType().Name);

        return sameErrorCount >= 2; // Same error twice indicates potential poison message
    }
}

89. What is a data lake?

A data lake is a centralized repository that stores structured, semi-structured, and unstructured data in its raw format, allowing for flexible data processing and analytics.

Characteristics: - Raw data storage - Schema-on-read - Scalable storage - Multiple data formats

Example in C#:

public class DataLakeService
{
    private readonly IBlobStorage _blobStorage;
    private readonly IDataCatalog _dataCatalog;

    public async Task IngestData(DataIngestionRequest request)
    {
        var dataPath = GenerateDataPath(request.DataType, request.Timestamp);

        // Store raw data
        await _blobStorage.UploadAsync(dataPath, request.RawData);

        // Extract metadata
        var metadata = new DataMetadata
        {
            DataType = request.DataType,
            Source = request.Source,
            Timestamp = request.Timestamp,
            Size = request.RawData.Length,
            Format = request.Format,
            Schema = ExtractSchema(request.RawData)
        };

        // Store metadata in catalog
        await _dataCatalog.StoreMetadataAsync(dataPath, metadata);

        _logger.LogInformation("Data ingested to lake: {DataPath}, Size: {Size} bytes", 
            dataPath, request.RawData.Length);
    }

    public async Task<DataQueryResult> QueryData(DataQueryRequest request)
    {
        // Find relevant data files
        var dataFiles = await _dataCatalog.FindDataAsync(request.Criteria);

        var results = new List<object>();

        foreach (var file in dataFiles)
        {
            var data = await _blobStorage.DownloadAsync(file.Path);
            var processedData = ProcessData(data, file.Metadata.Schema, request.Transformations);
            results.AddRange(processedData);
        }

        return new DataQueryResult
        {
            Data = results,
            TotalRecords = results.Count,
            DataSources = dataFiles.Select(f => f.Path).ToList()
        };
    }

    public async Task TransformData(DataTransformationRequest request)
    {
        var sourceData = await QueryData(request.SourceQuery);

        // Apply transformations
        var transformedData = ApplyTransformations(sourceData.Data, request.Transformations);

        // Store transformed data
        var outputPath = GenerateTransformedDataPath(request.OutputName, DateTime.UtcNow);
        await _blobStorage.UploadAsync(outputPath, transformedData);

        // Update catalog
        var outputMetadata = new DataMetadata
        {
            DataType = "transformed",
            Source = "data_lake_transformation",
            Timestamp = DateTime.UtcNow,
            Size = transformedData.Length,
            Format = "parquet",
            Schema = request.OutputSchema
        };

        await _dataCatalog.StoreMetadataAsync(outputPath, outputMetadata);
    }

    private string GenerateDataPath(string dataType, DateTime timestamp)
    {
        return $"raw/{dataType}/{timestamp:yyyy/MM/dd}/{timestamp:HH}/{Guid.NewGuid()}.json";
    }

    private string GenerateTransformedDataPath(string outputName, DateTime timestamp)
    {
        return $"transformed/{outputName}/{timestamp:yyyy/MM/dd}/{timestamp:HH}.parquet";
    }
}

90. What is a data pipeline?

A data pipeline is a series of processes that extract, transform, and load (ETL) data from various sources to a destination, enabling data processing and analytics.

Components: - Data extraction - Data transformation - Data loading - Orchestration

Example in C#:

public class DataPipeline
{
    private readonly IDataExtractor _extractor;
    private readonly IDataTransformer _transformer;
    private readonly IDataLoader _loader;
    private readonly IPipelineOrchestrator _orchestrator;

    public async Task ExecutePipeline(PipelineConfig config)
    {
        var pipelineId = Guid.NewGuid();
        _logger.LogInformation("Starting pipeline execution: {PipelineId}", pipelineId);

        try
        {
            // Extract data from sources
            var extractedData = await ExtractData(config.Sources);

            // Transform data
            var transformedData = await TransformData(extractedData, config.Transformations);

            // Load data to destination
            await LoadData(transformedData, config.Destination);

            _logger.LogInformation("Pipeline completed successfully: {PipelineId}", pipelineId);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Pipeline failed: {PipelineId}", pipelineId);
            await HandlePipelineFailure(pipelineId, ex);
            throw;
        }
    }

    private async Task<List<DataRecord>> ExtractData(List<DataSource> sources)
    {
        var allData = new List<DataRecord>();

        foreach (var source in sources)
        {
            _logger.LogInformation("Extracting data from source: {SourceName}", source.Name);

            var data = await _extractor.ExtractAsync(source);
            allData.AddRange(data);

            _logger.LogInformation("Extracted {RecordCount} records from {SourceName}", 
                data.Count, source.Name);
        }

        return allData;
    }

    private async Task<List<TransformedRecord>> TransformData(List<DataRecord> data, List<Transformation> transformations)
    {
        _logger.LogInformation("Transforming {RecordCount} records", data.Count);

        var transformedData = new List<TransformedRecord>();

        foreach (var record in data)
        {
            var transformedRecord = record;

            foreach (var transformation in transformations)
            {
                transformedRecord = await ApplyTransformation(transformedRecord, transformation);
            }

            transformedData.Add(transformedRecord);
        }

        _logger.LogInformation("Transformed {RecordCount} records", transformedData.Count);
        return transformedData;
    }

    private async Task LoadData(List<TransformedRecord> data, DataDestination destination)
    {
        _logger.LogInformation("Loading {RecordCount} records to {Destination}", 
            data.Count, destination.Name);

        await _loader.LoadAsync(data, destination);

        _logger.LogInformation("Successfully loaded {RecordCount} records to {Destination}", 
            data.Count, destination.Name);
    }

    private async Task<TransformedRecord> ApplyTransformation(DataRecord record, Transformation transformation)
    {
        switch (transformation.Type)
        {
            case TransformationType.Filter:
                return await ApplyFilter(record, transformation);

            case TransformationType.Map:
                return await ApplyMap(record, transformation);

            case TransformationType.Aggregate:
                return await ApplyAggregate(record, transformation);

            case TransformationType.Join:
                return await ApplyJoin(record, transformation);

            default:
                throw new NotSupportedException($"Transformation type {transformation.Type} not supported");
        }
    }

    public async Task SchedulePipeline(PipelineSchedule schedule)
    {
        await _orchestrator.SchedulePipelineAsync(new PipelineJob
        {
            PipelineConfig = schedule.PipelineConfig,
            Schedule = schedule.CronExpression,
            Enabled = true
        });
    }

    public async Task MonitorPipeline(Guid pipelineId)
    {
        var status = await _orchestrator.GetPipelineStatusAsync(pipelineId);

        _logger.LogInformation("Pipeline {PipelineId} status: {Status}, " +
            "Records processed: {RecordsProcessed}, Duration: {Duration}", 
            pipelineId, status.State, status.RecordsProcessed, status.Duration);

        if (status.State == PipelineState.Failed)
        {
            await HandlePipelineFailure(pipelineId, status.Error);
        }
    }
}

91. What is a pub/sub system?

Explanation: A publish/subscribe (pub/sub) system is a messaging pattern where senders (publishers) don't send messages directly to specific receivers (subscribers). Instead, publishers categorize messages into topics/queues, and subscribers express interest in one or more topics and receive messages from those topics. This decouples publishers from subscribers, enabling asynchronous communication.

Key Benefits: - Loose coupling between components - Scalability (multiple subscribers can receive the same message) - Asynchronous processing - Fault tolerance

C# Example using Azure Service Bus:

using Azure.Messaging.ServiceBus;
using System.Text.Json;

public class PubSubService
{
    private readonly ServiceBusClient _client;
    private readonly ServiceBusSender _sender;
    private readonly ServiceBusProcessor _processor;

    public PubSubService(string connectionString, string topicName, string subscriptionName)
    {
        _client = new ServiceBusClient(connectionString);
        _sender = _client.CreateSender(topicName);

        var options = new ServiceBusProcessorOptions
        {
            MaxConcurrentCalls = 1,
            AutoCompleteMessages = false
        };

        _processor = _client.CreateProcessor(topicName, subscriptionName, options);
    }

    // Publisher
    public async Task PublishMessageAsync<T>(T message, string correlationId = null)
    {
        var jsonMessage = JsonSerializer.Serialize(message);
        var serviceBusMessage = new ServiceBusMessage(jsonMessage)
        {
            CorrelationId = correlationId ?? Guid.NewGuid().ToString()
        };

        await _sender.SendMessageAsync(serviceBusMessage);
    }

    // Subscriber
    public async Task StartSubscribingAsync(Func<ProcessMessageEventArgs, Task> messageHandler)
    {
        _processor.ProcessMessageAsync += messageHandler;
        _processor.ProcessErrorAsync += ErrorHandler;

        await _processor.StartProcessingAsync();
    }

    private Task ErrorHandler(ProcessErrorEventArgs args)
    {
        Console.WriteLine($"Error processing message: {args.Exception.Message}");
        return Task.CompletedTask;
    }

    public async Task StopAsync()
    {
        await _processor.StopProcessingAsync();
        await _processor.DisposeAsync();
        await _sender.DisposeAsync();
        await _client.DisposeAsync();
    }
}

// Usage Example
public class OrderService
{
    private readonly PubSubService _pubSub;

    public OrderService(PubSubService pubSub)
    {
        _pubSub = pubSub;
    }

    public async Task PlaceOrderAsync(Order order)
    {
        // Process order
        await ProcessOrder(order);

        // Publish order placed event
        await _pubSub.PublishMessageAsync(new OrderPlacedEvent
        {
            OrderId = order.Id,
            CustomerId = order.CustomerId,
            Amount = order.TotalAmount,
            Timestamp = DateTime.UtcNow
        });
    }
}

92. What is a push vs pull model?

Explanation: These are two different approaches for data transfer between systems:

Push Model: The sender actively pushes data to the receiver when it's available. The receiver is passive and waits for incoming data.

Pull Model: The receiver actively requests/pulls data from the sender when it's ready to process. The sender is passive and waits for requests.

C# Examples:

Push Model (Webhooks):

public class WebhookService
{
    private readonly HttpClient _httpClient;
    private readonly ILogger<WebhookService> _logger;

    public WebhookService(HttpClient httpClient, ILogger<WebhookService> logger)
    {
        _httpClient = httpClient;
        _logger = logger;
    }

    public async Task PushNotificationAsync(string webhookUrl, object payload)
    {
        try
        {
            var json = JsonSerializer.Serialize(payload);
            var content = new StringContent(json, Encoding.UTF8, "application/json");

            var response = await _httpClient.PostAsync(webhookUrl, content);

            if (!response.IsSuccessStatusCode)
            {
                _logger.LogError($"Webhook failed: {response.StatusCode}");
                // Implement retry logic
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error pushing webhook notification");
        }
    }
}

// Webhook Receiver (Push Model)
[ApiController]
[Route("api/[controller]")]
public class WebhookController : ControllerBase
{
    [HttpPost("order-updated")]
    public async Task<IActionResult> ReceiveOrderUpdate([FromBody] OrderUpdatePayload payload)
    {
        // Process the pushed data
        await ProcessOrderUpdate(payload);
        return Ok();
    }
}

Pull Model (Polling):

public class PollingService : BackgroundService
{
    private readonly HttpClient _httpClient;
    private readonly ILogger<PollingService> _logger;
    private readonly TimeSpan _pollingInterval = TimeSpan.FromSeconds(30);

    public PollingService(HttpClient httpClient, ILogger<PollingService> logger)
    {
        _httpClient = httpClient;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await PullDataAsync();
                await Task.Delay(_pollingInterval, stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error during polling");
                await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);
            }
        }
    }

    private async Task PullDataAsync()
    {
        var response = await _httpClient.GetAsync("/api/orders/pending");

        if (response.IsSuccessStatusCode)
        {
            var orders = await response.Content.ReadFromJsonAsync<List<Order>>();
            await ProcessOrders(orders);
        }
    }

    private async Task ProcessOrders(List<Order> orders)
    {
        foreach (var order in orders)
        {
            await ProcessOrder(order);
        }
    }
}

93. What is a distributed transaction?

Explanation: A distributed transaction is a transaction that spans multiple databases, services, or systems. It ensures that all operations across these distributed resources either all succeed (commit) or all fail (rollback), maintaining ACID properties across the distributed system.

Challenges: - Network failures - Partial failures - Performance overhead - Complexity in coordination

C# Example using Saga Pattern:

public class OrderSaga
{
    private readonly IOrderService _orderService;
    private readonly IPaymentService _paymentService;
    private readonly IInventoryService _inventoryService;
    private readonly ILogger<OrderSaga> _logger;

    public OrderSaga(
        IOrderService orderService,
        IPaymentService paymentService,
        IInventoryService inventoryService,
        ILogger<OrderSaga> logger)
    {
        _orderService = orderService;
        _paymentService = paymentService;
        _inventoryService = inventoryService;
        _logger = logger;
    }

    public async Task<bool> ProcessOrderAsync(OrderRequest request)
    {
        var sagaId = Guid.NewGuid();
        var compensations = new Stack<Func<Task>>();

        try
        {
            // Step 1: Create Order
            var order = await _orderService.CreateOrderAsync(request);
            compensations.Push(async () => await _orderService.CancelOrderAsync(order.Id));

            // Step 2: Reserve Inventory
            await _inventoryService.ReserveInventoryAsync(order.Items);
            compensations.Push(async () => await _inventoryService.ReleaseInventoryAsync(order.Items));

            // Step 3: Process Payment
            var payment = await _paymentService.ProcessPaymentAsync(order.TotalAmount, request.PaymentInfo);
            compensations.Push(async () => await _paymentService.RefundPaymentAsync(payment.Id));

            // Step 4: Confirm Order
            await _orderService.ConfirmOrderAsync(order.Id);

            _logger.LogInformation($"Saga {sagaId} completed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, $"Saga {sagaId} failed, executing compensations");

            // Execute compensations in reverse order
            while (compensations.Count > 0)
            {
                var compensation = compensations.Pop();
                try
                {
                    await compensation();
                }
                catch (Exception compEx)
                {
                    _logger.LogError(compEx, "Compensation failed");
                }
            }

            return false;
        }
    }
}

// Two-Phase Commit Example
public class TwoPhaseCommitCoordinator
{
    private readonly List<IParticipant> _participants;
    private readonly ILogger<TwoPhaseCommitCoordinator> _logger;

    public TwoPhaseCommitCoordinator(List<IParticipant> participants, ILogger<TwoPhaseCommitCoordinator> logger)
    {
        _participants = participants;
        _logger = logger;
    }

    public async Task<bool> ExecuteTransactionAsync()
    {
        var transactionId = Guid.NewGuid();

        try
        {
            // Phase 1: Prepare
            _logger.LogInformation($"Starting 2PC transaction {transactionId}");

            var prepareTasks = _participants.Select(p => p.PrepareAsync(transactionId));
            var prepareResults = await Task.WhenAll(prepareTasks);

            if (prepareResults.Any(r => !r))
            {
                await RollbackAsync(transactionId);
                return false;
            }

            // Phase 2: Commit
            var commitTasks = _participants.Select(p => p.CommitAsync(transactionId));
            await Task.WhenAll(commitTasks);

            _logger.LogInformation($"2PC transaction {transactionId} committed successfully");
            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, $"2PC transaction {transactionId} failed");
            await RollbackAsync(transactionId);
            return false;
        }
    }

    private async Task RollbackAsync(Guid transactionId)
    {
        var rollbackTasks = _participants.Select(p => p.RollbackAsync(transactionId));
        await Task.WhenAll(rollbackTasks);
    }
}

public interface IParticipant
{
    Task<bool> PrepareAsync(Guid transactionId);
    Task CommitAsync(Guid transactionId);
    Task RollbackAsync(Guid transactionId);
}

94. What is a rate limiter?

Explanation: A rate limiter controls the rate of requests a client can make to a service within a specified time window. It prevents abuse, ensures fair usage, and protects system resources from being overwhelmed.

Types: - Fixed Window Counter - Sliding Window Counter - Token Bucket - Leaky Bucket

C# Example:

public class RateLimiter
{
    private readonly Dictionary<string, Queue<DateTime>> _requestTimestamps;
    private readonly object _lock = new object();
    private readonly int _maxRequests;
    private readonly TimeSpan _windowSize;

    public RateLimiter(int maxRequests, TimeSpan windowSize)
    {
        _maxRequests = maxRequests;
        _windowSize = windowSize;
        _requestTimestamps = new Dictionary<string, Queue<DateTime>>();
    }

    public bool IsAllowed(string clientId)
    {
        lock (_lock)
        {
            var now = DateTime.UtcNow;

            if (!_requestTimestamps.ContainsKey(clientId))
            {
                _requestTimestamps[clientId] = new Queue<DateTime>();
            }

            var queue = _requestTimestamps[clientId];

            // Remove expired timestamps
            while (queue.Count > 0 && now - queue.Peek() > _windowSize)
            {
                queue.Dequeue();
            }

            // Check if under limit
            if (queue.Count < _maxRequests)
            {
                queue.Enqueue(now);
                return true;
            }

            return false;
        }
    }
}

// Token Bucket Implementation
public class TokenBucketRateLimiter
{
    private readonly Dictionary<string, TokenBucket> _buckets;
    private readonly object _lock = new object();
    private readonly int _capacity;
    private readonly double _refillRate; // tokens per second

    public TokenBucketRateLimiter(int capacity, double refillRate)
    {
        _capacity = capacity;
        _refillRate = refillRate;
        _buckets = new Dictionary<string, TokenBucket>();
    }

    public bool IsAllowed(string clientId, int tokens = 1)
    {
        lock (_lock)
        {
            if (!_buckets.ContainsKey(clientId))
            {
                _buckets[clientId] = new TokenBucket(_capacity, _refillRate);
            }

            return _buckets[clientId].TryConsume(tokens);
        }
    }

    private class TokenBucket
    {
        private double _tokens;
        private DateTime _lastRefill;
        private readonly int _capacity;
        private readonly double _refillRate;

        public TokenBucket(int capacity, double refillRate)
        {
            _capacity = capacity;
            _refillRate = refillRate;
            _tokens = capacity;
            _lastRefill = DateTime.UtcNow;
        }

        public bool TryConsume(int tokens)
        {
            Refill();

            if (_tokens >= tokens)
            {
                _tokens -= tokens;
                return true;
            }

            return false;
        }

        private void Refill()
        {
            var now = DateTime.UtcNow;
            var timePassed = (now - _lastRefill).TotalSeconds;
            var tokensToAdd = timePassed * _refillRate;

            _tokens = Math.Min(_capacity, _tokens + tokensToAdd);
            _lastRefill = now;
        }
    }
}

// ASP.NET Core Middleware
public class RateLimitingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly RateLimiter _rateLimiter;

    public RateLimitingMiddleware(RequestDelegate next, RateLimiter rateLimiter)
    {
        _next = next;
        _rateLimiter = rateLimiter;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var clientId = GetClientId(context);

        if (!_rateLimiter.IsAllowed(clientId))
        {
            context.Response.StatusCode = 429; // Too Many Requests
            await context.Response.WriteAsync("Rate limit exceeded");
            return;
        }

        await _next(context);
    }

    private string GetClientId(HttpContext context)
    {
        // Extract client ID from IP, API key, or user ID
        return context.Connection.RemoteIpAddress?.ToString() ?? "unknown";
    }
}

95. What is a scheduler?

Explanation: A scheduler is a system component that manages and executes tasks at predetermined times or intervals. It handles job queuing, execution timing, retry logic, and resource allocation.

C# Example:

public class JobScheduler : BackgroundService
{
    private readonly ILogger<JobScheduler> _logger;
    private readonly IServiceProvider _serviceProvider;
    private readonly ConcurrentDictionary<string, Timer> _scheduledJobs;
    private readonly ConcurrentQueue<ScheduledJob> _jobQueue;

    public JobScheduler(ILogger<JobScheduler> logger, IServiceProvider serviceProvider)
    {
        _logger = logger;
        _serviceProvider = serviceProvider;
        _scheduledJobs = new ConcurrentDictionary<string, Timer>();
        _jobQueue = new ConcurrentQueue<ScheduledJob>();
    }

    public void ScheduleJob(string jobId, Func<Task> job, TimeSpan interval, bool runImmediately = false)
    {
        var scheduledJob = new ScheduledJob
        {
            Id = jobId,
            Job = job,
            Interval = interval,
            NextRun = runImmediately ? DateTime.UtcNow : DateTime.UtcNow.Add(interval)
        };

        _jobQueue.Enqueue(scheduledJob);

        if (runImmediately)
        {
            _ = ExecuteJobAsync(scheduledJob);
        }
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await ProcessJobQueueAsync();
                await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error in job scheduler");
            }
        }
    }

    private async Task ProcessJobQueueAsync()
    {
        var now = DateTime.UtcNow;
        var jobsToRun = new List<ScheduledJob>();

        // Find jobs that need to run
        while (_jobQueue.TryDequeue(out var job))
        {
            if (job.NextRun <= now)
            {
                jobsToRun.Add(job);
            }
            else
            {
                _jobQueue.Enqueue(job);
            }
        }

        // Execute jobs
        foreach (var job in jobsToRun)
        {
            await ExecuteJobAsync(job);

            // Reschedule for next run
            job.NextRun = DateTime.UtcNow.Add(job.Interval);
            _jobQueue.Enqueue(job);
        }
    }

    private async Task ExecuteJobAsync(ScheduledJob job)
    {
        try
        {
            _logger.LogInformation($"Executing job: {job.Id}");
            await job.Job();
            _logger.LogInformation($"Job completed: {job.Id}");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, $"Error executing job: {job.Id}");
        }
    }

    private class ScheduledJob
    {
        public string Id { get; set; }
        public Func<Task> Job { get; set; }
        public TimeSpan Interval { get; set; }
        public DateTime NextRun { get; set; }
    }
}

// Advanced Scheduler with Priority and Dependencies
public class AdvancedScheduler
{
    private readonly PriorityQueue<JobTask, int> _priorityQueue;
    private readonly Dictionary<string, JobTask> _jobRegistry;
    private readonly SemaphoreSlim _semaphore;

    public AdvancedScheduler(int maxConcurrency = 5)
    {
        _priorityQueue = new PriorityQueue<JobTask, int>();
        _jobRegistry = new Dictionary<string, JobTask>();
        _semaphore = new SemaphoreSlim(maxConcurrency);
    }

    public void ScheduleJob(string jobId, Func<Task> job, int priority = 0, 
        List<string> dependencies = null, TimeSpan? delay = null)
    {
        var jobTask = new JobTask
        {
            Id = jobId,
            Job = job,
            Priority = priority,
            Dependencies = dependencies ?? new List<string>(),
            ScheduledTime = DateTime.UtcNow.Add(delay ?? TimeSpan.Zero)
        };

        _jobRegistry[jobId] = jobTask;
        _priorityQueue.Enqueue(jobTask, priority);
    }

    public async Task ExecuteJobsAsync()
    {
        while (_priorityQueue.Count > 0)
        {
            var job = _priorityQueue.Dequeue();

            if (job.ScheduledTime > DateTime.UtcNow)
            {
                // Re-queue if not ready
                _priorityQueue.Enqueue(job, job.Priority);
                await Task.Delay(1000);
                continue;
            }

            if (await CanExecuteJobAsync(job))
            {
                await _semaphore.WaitAsync();
                _ = ExecuteJobWithSemaphoreAsync(job);
            }
        }
    }

    private async Task<bool> CanExecuteJobAsync(JobTask job)
    {
        foreach (var dependency in job.Dependencies)
        {
            if (_jobRegistry.TryGetValue(dependency, out var depJob) && !depJob.IsCompleted)
            {
                return false;
            }
        }
        return true;
    }

    private async Task ExecuteJobWithSemaphoreAsync(JobTask job)
    {
        try
        {
            await job.Job();
            job.IsCompleted = true;
        }
        finally
        {
            _semaphore.Release();
        }
    }

    private class JobTask
    {
        public string Id { get; set; }
        public Func<Task> Job { get; set; }
        public int Priority { get; set; }
        public List<string> Dependencies { get; set; }
        public DateTime ScheduledTime { get; set; }
        public bool IsCompleted { get; set; }
    }
}

96. What is a cron job?

Explanation: A cron job is a time-based job scheduler that runs commands or scripts at specified intervals. It uses cron expressions to define when jobs should execute (e.g., every day at 2 AM, every Monday, etc.).

Cron Expression Format: * * * * * (minute hour day month day-of-week)

C# Example:

public class CronScheduler : BackgroundService
{
    private readonly ILogger<CronScheduler> _logger;
    private readonly IServiceProvider _serviceProvider;
    private readonly List<CronJob> _cronJobs;

    public CronScheduler(ILogger<CronScheduler> logger, IServiceProvider serviceProvider)
    {
        _logger = logger;
        _serviceProvider = serviceProvider;
        _cronJobs = new List<CronJob>();
    }

    public void AddCronJob(string name, string cronExpression, Func<Task> job)
    {
        var cronJob = new CronJob
        {
            Name = name,
            CronExpression = cronExpression,
            Job = job,
            NextRun = CronHelper.GetNextOccurrence(cronExpression)
        };

        _cronJobs.Add(cronJob);
        _logger.LogInformation($"Added cron job: {name} with expression: {cronExpression}");
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                var now = DateTime.UtcNow;
                var jobsToRun = _cronJobs.Where(j => j.NextRun <= now).ToList();

                foreach (var job in jobsToRun)
                {
                    _ = ExecuteJobAsync(job);
                    job.NextRun = CronHelper.GetNextOccurrence(job.CronExpression);
                }

                await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error in cron scheduler");
            }
        }
    }

    private async Task ExecuteJobAsync(CronJob job)
    {
        try
        {
            _logger.LogInformation($"Executing cron job: {job.Name}");
            await job.Job();
            _logger.LogInformation($"Cron job completed: {job.Name}");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, $"Error executing cron job: {job.Name}");
        }
    }

    private class CronJob
    {
        public string Name { get; set; }
        public string CronExpression { get; set; }
        public Func<Task> Job { get; set; }
        public DateTime NextRun { get; set; }
    }
}

public static class CronHelper
{
    public static DateTime GetNextOccurrence(string cronExpression)
    {
        var parts = cronExpression.Split(' ');
        if (parts.Length != 5)
            throw new ArgumentException("Invalid cron expression");

        var minute = ParseField(parts[0], 0, 59);
        var hour = ParseField(parts[1], 0, 23);
        var day = ParseField(parts[2], 1, 31);
        var month = ParseField(parts[3], 1, 12);
        var dayOfWeek = ParseField(parts[4], 0, 6);

        var now = DateTime.UtcNow;
        var next = now;

        // Simple implementation - find next occurrence
        while (true)
        {
            if (minute.Contains(next.Minute) &&
                hour.Contains(next.Hour) &&
                day.Contains(next.Day) &&
                month.Contains(next.Month) &&
                dayOfWeek.Contains((int)next.DayOfWeek))
            {
                return next;
            }

            next = next.AddMinutes(1);
        }
    }

    private static List<int> ParseField(string field, int min, int max)
    {
        if (field == "*")
            return Enumerable.Range(min, max - min + 1).ToList();

        if (field.Contains(","))
            return field.Split(',').Select(int.Parse).ToList();

        if (field.Contains("-"))
        {
            var range = field.Split('-');
            var start = int.Parse(range[0]);
            var end = int.Parse(range[1]);
            return Enumerable.Range(start, end - start + 1).ToList();
        }

        return new List<int> { int.Parse(field) };
    }
}

// Usage Example
public class CronJobService
{
    private readonly CronScheduler _scheduler;

    public CronJobService(CronScheduler scheduler)
    {
        _scheduler = scheduler;
        SetupJobs();
    }

    private void SetupJobs()
    {
        // Run every day at 2 AM
        _scheduler.AddCronJob("DailyBackup", "0 2 * * *", DailyBackupJob);

        // Run every Monday at 9 AM
        _scheduler.AddCronJob("WeeklyReport", "0 9 * * 1", WeeklyReportJob);

        // Run every 5 minutes
        _scheduler.AddCronJob("HealthCheck", "*/5 * * * *", HealthCheckJob);
    }

    private async Task DailyBackupJob()
    {
        // Implement daily backup logic
        await Task.Delay(1000);
    }

    private async Task WeeklyReportJob()
    {
        // Implement weekly report generation
        await Task.Delay(1000);
    }

    private async Task HealthCheckJob()
    {
        // Implement health check logic
        await Task.Delay(1000);
    }
}

97. What is a batch processing system?

Explanation: A batch processing system processes large volumes of data in groups (batches) rather than processing individual records in real-time. It's optimized for throughput rather than latency and typically runs during off-peak hours.

Characteristics: - High throughput - Fault tolerance - Scalability - Resource optimization

C# Example:

public class BatchProcessor<T>
{
    private readonly ILogger<BatchProcessor<T>> _logger;
    private readonly int _batchSize;
    private readonly int _maxConcurrency;
    private readonly SemaphoreSlim _semaphore;

    public BatchProcessor(int batchSize = 1000, int maxConcurrency = 5)
    {
        _batchSize = batchSize;
        _maxConcurrency = maxConcurrency;
        _semaphore = new SemaphoreSlim(maxConcurrency);
        _logger = LoggerFactory.Create(builder => builder.AddConsole())
            .CreateLogger<BatchProcessor<T>>();
    }

    public async Task ProcessBatchAsync(IEnumerable<T> items, Func<List<T>, Task> processor)
    {
        var batches = items.Chunk(_batchSize).ToList();
        _logger.LogInformation($"Processing {batches.Count} batches of {_batchSize} items each");

        var tasks = new List<Task>();
        var processedCount = 0;
        var failedCount = 0;

        foreach (var batch in batches)
        {
            await _semaphore.WaitAsync();

            var task = Task.Run(async () =>
            {
                try
                {
                    await processor(batch.ToList());
                    Interlocked.Add(ref processedCount, batch.Length);
                    _logger.LogInformation($"Processed batch of {batch.Length} items");
                }
                catch (Exception ex)
                {
                    Interlocked.Add(ref failedCount, batch.Length);
                    _logger.LogError(ex, $"Error processing batch of {batch.Length} items");
                }
                finally
                {
                    _semaphore.Release();
                }
            });

            tasks.Add(task);
        }

        await Task.WhenAll(tasks);
        _logger.LogInformation($"Batch processing completed. Processed: {processedCount}, Failed: {failedCount}");
    }
}

// ETL (Extract, Transform, Load) Batch Processing
public class ETLBatchProcessor
{
    private readonly IDataExtractor _extractor;
    private readonly IDataTransformer _transformer;
    private readonly IDataLoader _loader;
    private readonly ILogger<ETLBatchProcessor> _logger;

    public ETLBatchProcessor(
        IDataExtractor extractor,
        IDataTransformer transformer,
        IDataLoader loader,
        ILogger<ETLBatchProcessor> logger)
    {
        _extractor = extractor;
        _transformer = transformer;
        _loader = loader;
        _logger = logger;
    }

    public async Task ProcessETLBatchAsync(string source, string destination)
    {
        var stopwatch = Stopwatch.StartNew();

        try
        {
            _logger.LogInformation($"Starting ETL batch process from {source} to {destination}");

            // Extract
            var rawData = await _extractor.ExtractAsync(source);
            _logger.LogInformation($"Extracted {rawData.Count} records");

            // Transform
            var transformedData = await _transformer.TransformAsync(rawData);
            _logger.LogInformation($"Transformed {transformedData.Count} records");

            // Load
            await _loader.LoadAsync(destination, transformedData);
            _logger.LogInformation($"Loaded {transformedData.Count} records to {destination}");

            stopwatch.Stop();
            _logger.LogInformation($"ETL batch process completed in {stopwatch.ElapsedMilliseconds}ms");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "ETL batch process failed");
            throw;
        }
    }
}

// Batch Job with Checkpointing
public class CheckpointedBatchProcessor<T>
{
    private readonly string _checkpointFile;
    private readonly ILogger<CheckpointedBatchProcessor<T>> _logger;

    public CheckpointedBatchProcessor(string checkpointFile)
    {
        _checkpointFile = checkpointFile;
        _logger = LoggerFactory.Create(builder => builder.AddConsole())
            .CreateLogger<CheckpointedBatchProcessor<T>>();
    }

    public async Task ProcessWithCheckpointAsync(
        IEnumerable<T> items,
        Func<T, Task> processor,
        Func<T, string> keySelector)
    {
        var processedKeys = await LoadCheckpointAsync();
        var itemsList = items.ToList();
        var processedCount = 0;

        foreach (var item in itemsList)
        {
            var key = keySelector(item);

            if (processedKeys.Contains(key))
            {
                _logger.LogInformation($"Skipping already processed item: {key}");
                continue;
            }

            try
            {
                await processor(item);
                processedKeys.Add(key);
                processedCount++;

                if (processedCount % 100 == 0)
                {
                    await SaveCheckpointAsync(processedKeys);
                    _logger.LogInformation($"Checkpoint saved at {processedCount} items");
                }
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, $"Error processing item {key}");
            }
        }

        await SaveCheckpointAsync(processedKeys);
        _logger.LogInformation($"Batch processing completed. Total processed: {processedCount}");
    }

    private async Task<HashSet<string>> LoadCheckpointAsync()
    {
        if (!File.Exists(_checkpointFile))
            return new HashSet<string>();

        var checkpointData = await File.ReadAllTextAsync(_checkpointFile);
        return JsonSerializer.Deserialize<HashSet<string>>(checkpointData) ?? new HashSet<string>();
    }

    private async Task SaveCheckpointAsync(HashSet<string> processedKeys)
    {
        var checkpointData = JsonSerializer.Serialize(processedKeys);
        await File.WriteAllTextAsync(_checkpointFile, checkpointData);
    }
}

// Usage Example
public class OrderBatchProcessor
{
    private readonly BatchProcessor<Order> _batchProcessor;
    private readonly CheckpointedBatchProcessor<Order> _checkpointedProcessor;

    public OrderBatchProcessor()
    {
        _batchProcessor = new BatchProcessor<Order>(batchSize: 500);
        _checkpointedProcessor = new CheckpointedBatchProcessor<Order>("order_checkpoint.json");
    }

    public async Task ProcessOrdersAsync(List<Order> orders)
    {
        await _batchProcessor.ProcessBatchAsync(orders, async batch =>
        {
            // Process batch of orders
            foreach (var order in batch)
            {
                await ProcessOrderAsync(order);
            }
        });
    }

    public async Task ProcessOrdersWithCheckpointAsync(List<Order> orders)
    {
        await _checkpointedProcessor.ProcessWithCheckpointAsync(
            orders,
            ProcessOrderAsync,
            order => order.Id.ToString());
    }

    private async Task ProcessOrderAsync(Order order)
    {
        // Simulate order processing
        await Task.Delay(100);
    }
}

98. What is a stream processing system?

Explanation: A stream processing system processes data in real-time as it flows through the system, rather than processing batches. It handles continuous data streams and provides low-latency processing with high throughput.

Key Concepts: - Event-driven processing - Real-time analytics - Windowing operations - State management

C# Example:

public class StreamProcessor<T>
{
    private readonly Channel<T> _inputChannel;
    private readonly List<IStreamOperator<T>> _operators;
    private readonly ILogger<StreamProcessor<T>> _logger;

    public StreamProcessor(int bufferSize = 1000)
    {
        _inputChannel = Channel.CreateBounded<T>(bufferSize);
        _operators = new List<IStreamOperator<T>>();
        _logger = LoggerFactory.Create(builder => builder.AddConsole())
            .CreateLogger<StreamProcessor<T>>();
    }

    public void AddOperator(IStreamOperator<T> streamOperator)
    {
        _operators.Add(streamOperator);
    }

    public async Task StartProcessingAsync(CancellationToken cancellationToken = default)
    {
        _logger.LogInformation("Starting stream processing");

        var processingTask = Task.Run(async () =>
        {
            await foreach (var item in _inputChannel.Reader.ReadAllAsync(cancellationToken))
            {
                await ProcessItemAsync(item);
            }
        }, cancellationToken);

        await processingTask;
    }

    public async Task AddItemAsync(T item)
    {
        await _inputChannel.Writer.WriteAsync(item);
    }

    private async Task ProcessItemAsync(T item)
    {
        try
        {
            var processedItem = item;

            foreach (var streamOperator in _operators)
            {
                processedItem = await streamOperator.ProcessAsync(processedItem);

                if (processedItem == null)
                    break; // Item filtered out
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error processing stream item");
        }
    }
}

public interface IStreamOperator<T>
{
    Task<T> ProcessAsync(T item);
}

// Filter Operator
public class FilterOperator<T> : IStreamOperator<T>
{
    private readonly Func<T, bool> _predicate;

    public FilterOperator(Func<T, bool> predicate)
    {
        _predicate = predicate;
    }

    public Task<T> ProcessAsync(T item)
    {
        return Task.FromResult(_predicate(item) ? item : default(T));
    }
}

// Map Operator
public class MapOperator<TInput, TOutput> : IStreamOperator<TInput>
{
    private readonly Func<TInput, TOutput> _mapper;
    private readonly Action<TOutput> _outputHandler;

    public MapOperator(Func<TInput, TOutput> mapper, Action<TOutput> outputHandler)
    {
        _mapper = mapper;
        _outputHandler = outputHandler;
    }

    public Task<TInput> ProcessAsync(TInput item)
    {
        var output = _mapper(item);
        _outputHandler(output);
        return Task.FromResult(item);
    }
}

// Window Operator
public class SlidingWindowOperator<T> : IStreamOperator<T>
{
    private readonly Queue<T> _window;
    private readonly int _windowSize;
    private readonly TimeSpan _slideInterval;
    private readonly Action<IEnumerable<T>> _windowHandler;
    private DateTime _lastSlide;

    public SlidingWindowOperator(int windowSize, TimeSpan slideInterval, Action<IEnumerable<T>> windowHandler)
    {
        _window = new Queue<T>();
        _windowSize = windowSize;
        _slideInterval = slideInterval;
        _windowHandler = windowHandler;
        _lastSlide = DateTime.UtcNow;
    }

    public Task<T> ProcessAsync(T item)
    {
        _window.Enqueue(item);

        // Remove old items if window is full
        while (_window.Count > _windowSize)
        {
            _window.Dequeue();
        }

        // Check if it's time to slide the window
        var now = DateTime.UtcNow;
        if (now - _lastSlide >= _slideInterval)
        {
            _windowHandler(_window.ToList());
            _lastSlide = now;
        }

        return Task.FromResult(item);
    }
}

// Stateful Stream Processing
public class StatefulStreamProcessor<T, TState>
{
    private readonly Dictionary<string, TState> _stateStore;
    private readonly Func<T, string> _keySelector;
    private readonly Func<T, TState, TState> _stateUpdater;
    private readonly Action<string, TState> _stateHandler;

    public StatefulStreamProcessor(
        Func<T, string> keySelector,
        Func<T, TState, TState> stateUpdater,
        Action<string, TState> stateHandler)
    {
        _stateStore = new Dictionary<string, TState>();
        _keySelector = keySelector;
        _stateUpdater = stateUpdater;
        _stateHandler = stateHandler;
    }

    public Task<T> ProcessAsync(T item)
    {
        var key = _keySelector(item);

        if (!_stateStore.TryGetValue(key, out var currentState))
        {
            currentState = default(TState);
        }

        var newState = _stateUpdater(item, currentState);
        _stateStore[key] = newState;

        _stateHandler(key, newState);

        return Task.FromResult(item);
    }
}

// Usage Example
public class OrderStreamProcessor
{
    private readonly StreamProcessor<OrderEvent> _streamProcessor;
    private readonly List<Order> _processedOrders;

    public OrderStreamProcessor()
    {
        _streamProcessor = new StreamProcessor<OrderEvent>();
        _processedOrders = new List<Order>();
        SetupOperators();
    }

    private void SetupOperators()
    {
        // Filter high-value orders
        _streamProcessor.AddOperator(new FilterOperator<OrderEvent>(order => order.Amount > 1000));

        // Map to Order objects
        _streamProcessor.AddOperator(new MapOperator<OrderEvent, Order>(
            orderEvent => new Order { Id = orderEvent.OrderId, Amount = orderEvent.Amount },
            order => _processedOrders.Add(order)
        ));

        // Sliding window for analytics
        _streamProcessor.AddOperator(new SlidingWindowOperator<OrderEvent>(
            windowSize: 100,
            slideInterval: TimeSpan.FromMinutes(5),
            windowHandler: ProcessWindow
        ));

        // Stateful processing for customer order count
        _streamProcessor.AddOperator(new StatefulStreamProcessor<OrderEvent, int>(
            keySelector: order => order.CustomerId,
            stateUpdater: (order, currentCount) => currentCount + 1,
            stateHandler: (customerId, count) => Console.WriteLine($"Customer {customerId} has {count} orders")
        ));
    }

    private void ProcessWindow(IEnumerable<OrderEvent> orders)
    {
        var totalAmount = orders.Sum(o => o.Amount);
        Console.WriteLine($"Window total: {totalAmount}");
    }

    public async Task StartAsync()
    {
        await _streamProcessor.StartProcessingAsync();
    }

    public async Task AddOrderEventAsync(OrderEvent orderEvent)
    {
        await _streamProcessor.AddItemAsync(orderEvent);
    }
}

public class OrderEvent
{
    public string OrderId { get; set; }
    public string CustomerId { get; set; }
    public decimal Amount { get; set; }
    public DateTime Timestamp { get; set; }
}

I'll act as a technical architect and provide comprehensive answers to these interview questions, including explanations and C# code examples where relevant.

99. What is a Content Delivery Network (CDN)?

Explanation

A Content Delivery Network (CDN) is a distributed network of servers strategically located across different geographical regions that work together to deliver web content to users more efficiently. The primary goal is to reduce latency and improve performance by serving content from servers that are geographically closer to the end users.

Key Components and Benefits:

1. Edge Servers: Servers located at the "edge" of the network, closer to end users
2. Origin Server: The original server where content is hosted
3. Load Balancing: Distributes traffic across multiple servers
4. Caching: Stores frequently accessed content at edge locations
5. Geographic Distribution: Reduces latency by serving content from nearby locations

Benefits:

• Reduced Latency: Faster content delivery
• Improved Scalability: Handles high traffic loads
• Better Reliability: Redundancy and failover capabilities
• Cost Reduction: Reduces bandwidth costs on origin servers
• Security: DDoS protection and security features

C# Implementation Example

Here's a simple C# example demonstrating how to integrate with a CDN:

Let me create the file in a proper directory structure:

Ran tool I see we're at the root directory. Let me create a proper workspace directory for our examples:

mkdir CDNExamples

Let me provide the C# code examples directly in the response:

C# CDN Integration Example

using System;
using System.Net.Http;
using System.Threading.Tasks;
using System.Text.Json;
using System.Collections.Generic;

namespace CDNExample
{
    public class CDNConfiguration
    {
        public string BaseUrl { get; set; }
        public string ApiKey { get; set; }
        public int CacheTimeoutMinutes { get; set; }
        public List<string> EdgeLocations { get; set; }
    }

    public class CDNService
    {
        private readonly HttpClient _httpClient;
        private readonly CDNConfiguration _config;
        private readonly Dictionary<string, DateTime> _cache;

        public CDNService(CDNConfiguration config)
        {
            _config = config;
            _httpClient = new HttpClient();
            _httpClient.DefaultRequestHeaders.Add("X-API-Key", config.ApiKey);
            _cache = new Dictionary<string, DateTime>();
        }

        public async Task<string> GetContentAsync(string contentPath)
        {
            // Check if content is cached locally
            if (IsCached(contentPath))
            {
                Console.WriteLine($"Serving {contentPath} from local cache");
                return await GetFromCache(contentPath);
            }

            // Get optimal edge server
            string edgeServer = GetOptimalEdgeServer();
            string cdnUrl = $"{edgeServer}/{contentPath}";

            try
            {
                var response = await _httpClient.GetAsync(cdnUrl);
                response.EnsureSuccessStatusCode();

                string content = await response.Content.ReadAsStringAsync();

                // Cache the content
                CacheContent(contentPath, content);

                Console.WriteLine($"Served {contentPath} from CDN edge server: {edgeServer}");
                return content;
            }
            catch (HttpRequestException ex)
            {
                Console.WriteLine($"CDN request failed: {ex.Message}");
                // Fallback to origin server
                return await GetFromOriginServer(contentPath);
            }
        }

        private string GetOptimalEdgeServer()
        {
            // In a real implementation, this would use geolocation or load balancing
            Random random = new Random();
            int index = random.Next(_config.EdgeLocations.Count);
            return _config.EdgeLocations[index];
        }

        private bool IsCached(string contentPath)
        {
            return _cache.ContainsKey(contentPath) && 
                   DateTime.Now.Subtract(_cache[contentPath]).TotalMinutes < _config.CacheTimeoutMinutes;
        }

        private void CacheContent(string contentPath, string content)
        {
            _cache[contentPath] = DateTime.Now;
            Console.WriteLine($"Cached content: {contentPath}");
        }

        private async Task<string> GetFromCache(string contentPath)
        {
            await Task.Delay(10); // Simulate cache access time
            return $"Cached content for {contentPath}";
        }

        private async Task<string> GetFromOriginServer(string contentPath)
        {
            Console.WriteLine($"Fallback to origin server for {contentPath}");
            await Task.Delay(100);
            return $"Origin content for {contentPath}";
        }

        public async Task<bool> PurgeCacheAsync(string contentPath)
        {
            try
            {
                string purgeUrl = $"{_config.BaseUrl}/purge/{contentPath}";
                var response = await _httpClient.PostAsync(purgeUrl, null);
                response.EnsureSuccessStatusCode();

                _cache.Remove(contentPath);
                Console.WriteLine($"Purged cache for {contentPath}");
                return true;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Failed to purge cache: {ex.Message}");
                return false;
            }
        }
    }
}

---

100. How do you design a highly available system?

Start by defining availability, latency, consistency, recovery, cost, and security objectives. Remove single points of failure with redundancy, use health checks and tested failover, make dependencies time out and degrade safely, protect data with backups and recovery drills, and measure the resulting service-level indicators. High availability is a property of the whole dependency chain, including operations and deployment.

Official references

• Google SRE workbook
• AWS Well-Architected Reliability Pillar
• CAP theorem overview