Multi-cluster Storage for Row Records

Worksheets are a core module in HAP applications. According to Application Data Storage Architecture, each worksheet corresponds to an independent Collection in MongoDB.

Design Background

A single MongoDB instance has an upper limit on the number of Collections it can create. Although this limit can be adjusted, an excessive number of Collections increases instance resource consumption and O&M pressure.

As the number of worksheets grows, if databases continue to be separated only within the same cluster, resource consumption remains concentrated in the current cluster. In the long run, this approach is not suitable for horizontal scaling. Therefore, HAP uses multi-cluster storage for worksheet row records.

Implementation

The system uses a routing table to record the relationship between worksheets and row record storage clusters. The routing information mainly includes:

• Table ID
• Row record database cluster instance number

When writing or querying worksheet row records, the system first queries the routing table by table ID, determines the MongoDB cluster instance corresponding to the worksheet, and then accesses the corresponding row record Collection.

Advantages

Multi-cluster storage for row records can distribute worksheet data across multiple MongoDB clusters, reduce the number of Collections and resource pressure on a single cluster, and provide clearer boundaries for later scaling, migration, and fault isolation.