Time Series Databases in hydropower -

Hello, after Prof. Wang Hongzhi's introduction to the frontier field of database in the last issue, I wonder if you have a deeper understanding of the database field, let's return to the Time Series Database in this issue and talk about the application of Time Series Database in the field of hydropower plant.

This article is only a personal opinion, please bear with meif there is any bias

🤠🤠🤠

Time Series Databases in hydropower

Challenges Faced by Traditional Hydropower Monitoring System

In the process of automatic control of hydropower plant, the operation data monitoring system of hydropower plant plays an important role. It provides strong guarantee for operation analysis, trend judgment and accident treatment. The data monitored by hydropower station has the characteristics of high collection frequency and time series sequencing. Thus, various problems occur if using traditional RDB (such as MySQL, SQL Server, Oracle, etc.) for its storage.

The first drawback is the high storage cost but low write throughput. The measuring points of large hydropower stations are so large that even if only 10,000 measuring points are collected every second, it will occupy about 10-20 GB of disk storage space per day. The capacity of common disk arrays is difficult to meet the storage requirements of the above-mentioned massive data, and the compression of time-series data is not ideal in relational databases, which results in an increasing cost over time. In addition, under the traditional scheme, large amount of data writing consumes a long time and is difficult to meet the write requirements of tens of millions of levels.The other problem is the poor query performance. Although sub-database, index optimization and other technical operations can be carried out on relational database, with the continuous growth of data storage, the query efficiency will significantly reduce so that it is difficult to obtain the required data in the second or even millisecond level. At the same time, the sub-table strategy will increase the complexity of query business. For example, if you use monthly sub-tables, querying cross-month data requires multiple SQL or joint queries to get the desired results.

Using Time Series Database to Solve the Bottleneck of Traditional Database

Since most of the data stored in the monitoring system of hydropower plants is time series data, it is a better choice to build the monitoring system of hydropower plants based on the time series database. Compared with relational databases, temporal databases have obvious performance advantages in data storage and query.

The efficient compression ratio of time series database can save much storage space. This is mainly due to its uses of the column storage method: Each column stores a set of ordered data, which is converted into single row “array” data. Then, according to its data type, each array is compressed separately using a specific compression algorithm.

The query performancethe can also be ensured in time series database. It is known that time series data is usually recorded every second or every millisecond, which leads to a rapid growth of data. For traditional relational databases, due to the massive use of B+trees for indexing, when the amount of data reaches a certain level, its write performance will decline significantly. However, time series database can be automatically partitioned. With continuous data writing, it can automatically create partitions with time stamps, store time series data in partitions, and ensure that the index of each partition is maintained at a smaller scale, thus maintaining the write performance. When querying, you can also quickly locate the required data partition to ensure query performance.

Providing Better Serivice Based on Time Series Database

The time series database of hydropower station monitoring system is mainly used for curve query, report query and other services. Specifically, according to relevant query requests, query the corresponding data in the time series database, and generate the corresponding result set to return. For example, the report query function is mainly used to sample and output time series data. It also needs to fulfill statistical functions such as the average, maximum, minimum with optional intervals. Moreover, it requires the front-end page to display the required statistics values via configuring the statistical function for the measuring points on the configuration interface. At present, it seems that the aggregate query function provided by the time series database can fully meet the statistical functions required by the hydropower station monitoring system within 1 second. Additionally, the curve query function enables the system to provide real-time data of any historical period from the time sequence database. Hence, the power supply station personnel can query real-time curves of multiple measuring points at the same time for analysis and comparison.

This paper refers to the article “Design and Implementation of Hydroelectric Power Station Monitoring System Based on Time-Series Library” by Lijuan Bian et al. in the journal “Hydroelectric Power Station Design”.