Why distributes database

 Why Distributed Database

Before starting to understand the core concept of distributed database system, first try to understand why distributed database system is needed.

Scalability:

  • If your data volume, read load, write load grows bigger then a single machine capability, you can potentially spread load across multiple machines.

High Availability or Fault Tolerance:

  • If your application needs to continue working even if one machine or network or entire data centre goes down, you can use multiple machines to give you redundancy. when one fail another can take over.

Latency:

  • If you have users around the world, you might want to have servers at various locations world wide so that each user can be served from a datacenter that is geographically closer to them.
  • That avoids the users having to wait for network packets to travel from longer distance.

Scaling to Higher load:

  • Vertical Scaling (Shared memory architecture)
    • Cost grows faster than linearly
    • limited fault tolerance
    • limited to single geographically location
  • Horizontal Scaling ( Shared nothing architecture)
    • Low cost machines
    • distribute data across multiple geographic regions, hence reduce latency
    • high availability
    • additional complexity
  • Replication (Redundancy)
    • Keeping a copy of same data on several different nodes, potentially in different locations
    • Replication provides redundancy: If some nodes are unavailable, the data can still be served from remaining nodes.
    • It improve performance
  • Partitioning (Data Sharding):
    • Splitting a big database into smaller subsets called partitions so that different partitions can be assigned to different nodes.
Next is details of Replications.

Comments

Post a Comment

Popular posts from this blog

Distributed Lock with Redlock

 Distributed Lock with Redlock Distributed lock systems are used to ensure that only one process or thread can access a particular resource at a time. Redlock is a distributed lock system that was introduced by Redis. The goal of Redlock is to provide a reliable and fault-tolerant distributed lock service.   The Redlock system is designed to work in a distributed environment where multiple instances of a service or application can run concurrently. Each instance of the application can acquire a lock and perform some operations on the resource. The Redlock system ensures that only one instance of the application has access to the resource at a time.   Components of Redlock :   Lock Manager : The Lock Manager is responsible for managing the lock state and deciding which instance of the application can acquire the lock. It receives requests from the applications to acquire or release the lock.   The Lock Manager uses a distributed consensus algorithm to en...
Read more

Distributed Transaction

Distributed Transaction A distributed transaction across multiple database services involves multiple databases that are geographically distributed across  different locations .       Here are some important steps to consider when designing a system for distributed transactions across multiple database services:     Transaction Protocol      Transactional consistency      Concurrency control      Reliability and Fault-tolerance      Performance   Transactional Protocol A distributed transaction protocol is a set of rules and procedures that define how a distributed transaction should be executed across multiple databases .   It is used for Atomic commit across databases    There are several distributed transaction protocols to choose from, each with its own advantages and disadvantages.     Two-Phase Commit (2PC ) :   It is a widely-used protocol that ensures at...
Read more

Storage Engine

Image
Database Storage Engine Two kinds of database storage engine are most popular: Log Structured Storage Engine (SSTable, LSMTree) Page Oriented Storage Engine (B-Tree) Log Structured Storage Engine  Most of the databases internally use a log, which is an append only data file. Log file is an append only sequence of records It is not human readable, binary file Issues Since log file is append only file, so it keeps growing, hence resulting performance issue. While reading data, every time it looks from beginning Performance degrades if number of records are more in the database To mitigate these issues, a new concept came out called Indexing. Index In order to efficiently find the value for a particular key in the database, we need a different data structure, this is called an Index. In General the idea behind them is to keep some additional metadata on the side, which acts as a signpost and helps you to locate the data you want. If you want to search some data in a several different ...
Read more