is an ACID-compliant relational database management system, contained in a relatively small library written in C
The project developers SQLite have started testing an experimental HCtree backend which supports row-level locking and provides a high level of parallelism in query processing.
Regarding the new backend in which they are working, it is mentioned that this aims to improve the efficiency of using SQLite in client-server systems They have to process a large number of concurrent write requests to the database.
All the b-tree structures originally used in SQLite to store data They are not designed for this type of load., which limits SQLite to writing to only one stream. As an experiment, the developers started to develop an alternative solution that uses HCtree structures for storage, more suitable for parallelizing write operations.
The HC-tree (hctree) project is an attempt to develop a new database backend that improves on regular SQLite as follows:
Improved concurrency: Using the begin-concurrent extension changes this so that it can be executed concurrently using optimistic locking at the page level. This improves concurrency somewhat, but page-level locking can detect conflicts between logically independent transactions, and COMMIT operations still need to be serialized.
Replication support: Stock SQLite supports the session extension, which allows the content of a committed transaction to be serialized for transmission and application to a second database. Hctree integrates this into the database backend and adds support for applying such transactions to follower databases in leader-follower configurations. In this case, transactions received from a leading database can be applied faster and with greater concurrency than those originally applied to the leading database, because no transaction validation.
Elimination of database size limitations: Stock SQLite uses 32-bit page numbers. Using the default page size of 4 KiB, this leads to a maximum database size of 2^44 bytes or 16 TiB.
To handle several operations at the same time, HCtree logs use a transaction separation mechanism which uses page level locks, similar to MVCC (multiple version containment control), but uses transaction controls based on key ranges and key ranges instead of page sets.
Read and write operations are performed relative to the database snapshot, the changes of which become visible to the main database only after the transaction completes.
Clients can use three operations to open transactions:
- «BEGIN«: Transactions do not take into account the access data of other customers. If write operations are performed within a transaction, the transaction can only be committed if there were no other write operations to the database during the time of its execution.
«BEGIN CONCURRENT«: Transactions collect information about the access of other clients. If write operations are performed within a transaction, the transaction can be committed if other transactions have been committed to the database since the snapshot was created.
«BEGIN EXCLUSIVE«: after opening a transaction, blocks the operations of other transactions until it is completed.
HCtree supports master-slave replication, which allows you to transfer transactions to another database and keep secondary databases in sync with the primary database.
HCtree also removes the database size limit: instead of 32-bit data page identifiers, HCtree uses 48-bit identifiers, which increases the maximum database size from 16 tebibytes to 1 exbibyte ( million tebibytes).
SQLite performance with the HCtree backend is expected to be at least as good as the classic single-threaded backend. SQLite clients with HCtree support will be able to access both HC-tree databases and legacy SQLite databases.
Source: https://sqlite.org/