CnosDB is a distributed time series database system, where the metadata cluster is one of the core components responsible for managing the metadata information of the entire cluster. Metadata includes database catalogs, DDL operations, tenant and user permissions, data sharding distribution, cluster node information, and more. The metadata cluster achieves high availability through the Raft consensus protocol and provides RESTful API services externally.
Metadata Storage
The metadata cluster primarily stores the following metadata information:
- Database Catalog Information: Including definitions and metadata for database, table, view, index, and other objects.
- DDL Operation Resource Management: Records various DDL operations performed on database objects, such as create, modify, delete, etc.
- Tenant and User Information and Permissions: Stores tenant and user information in the system as well as their permission settings.
- Data Sharding and Distribution: Records the sharding and distribution of data within the cluster.
- Cluster Node Information: Stores information about each node in the cluster, such as node ID, address, status, etc.
CnosDB Cluster Management
The metadata cluster provides the following cluster management functionalities:
- Node Health Check: Periodically checks the health status of each node in the cluster.
- Node Registration: When a new node joins the cluster, the metadata cluster is responsible for registering that node.
- Distributed Lock: Provides a distributed locking mechanism for various distributed operations in the cluster, ensuring the consistency of operations.
Change Notification
The metadata cluster offers a powerful Watch mechanism that can notify changes in metadata in real-time to every node in the cluster. The specific process is as follows:
- Each CnosDB node subscribes to change events from the metadata cluster.
- When metadata changes occur, the metadata cluster asynchronously notifies all subscribed nodes
- Nodes that receive notifications will synchronize the changes to their local cache.
- Subsequent requests will prioritize using the local cache, while requests with high consistency requirements will pass through to the metadata cluster to fetch the latest data.
Backup and Restore
The metadata cluster provides complete backup and restore functionalities:
- Data Backup: Metadata information can be exported using the `cnosdb-cli dump-ddl` command and saved as DDL statements.
- Data Restore: The metadata information backed up can be imported back into the metadata cluster using the `cnosdb-cli restore-dump-ddl` command.