How to Enable the Aggregation Table Feature

caution

The Aggregation Table feature (v5.6.0+) relies on the Flink MongoDB CDC connector as a functional dependency. The following prerequisites must be met to use this feature properly:

1. Deploy Flink services. Refer to: How to deploy Flink

2. MongoDB must be upgraded to 4.4+. Refer to: Upgrade MongoDB in standalone mode to 4.4 or Upgrade MongoDB in cluster mode to 4.4

   • If the standalone mode uses the built-in MongoDB and the current mingdaoyun-sc container version is ​​≥​ 3.0.0, the requirement is already met.

3. MongoDB must be configured in replica set mode (if already in replica set mode, skip this). Refer to: Convert a single-node MongoDB instance to replica set

   • The built-in MongoDB in standalone mode is not in replica set mode by default.

Using Built-in MongoDB Without Authentication Enabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.2.0 image.

After meeting the prerequisites, no additional configuration is required to use the Aggregation Table feature. 👏 👏 👏

Using Built-in MongoDB with Authentication Enabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.2.0 image.

1. Use the root account to connect to the admin database and create roles and users for the aggregation table database:

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'ReplaceThisWithThePasswordOfAggUser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS to the configuration file (if there are additional options, ensure proper concatenation) and restart the service. For example, in standalone mode yaml:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-hap:7.1.0
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@sc:27017/mdaggregationwsrows?authSource=admin"

Using External MongoDB Without Authentication Enabled

Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS to the configuration file and restart the service. For example, in standalone mode yaml:

services:
  app:
    image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-hap:7.1.0
    environment: &app-environment
      ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://ServiceNameOrIP:27017/mdaggregationwsrows" 
      # For multi-node
      # ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://ServiceNameOrIP1:27017,ServiceNameOrIP2:27017,ServiceNameOrIP3:27017/mdaggregationwsrows"

Using External MongoDB with Authentication Enabled

1. Use the root account to connect to the admin database and create roles and users for the aggregation table database:

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'ReplaceThisWithThePasswordOfAggUser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS to the configuration file (if there are additional options, ensure proper concatenation) and restart the service. For example, in standalone mode yaml:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-hap:7.1.0
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@ServiceNameOrIP:27017/mdaggregationwsrows?authSource=admin"
          # For multi-node
          # ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@ServiceNameOrIP1:27017,ServiceNameOrIP2:27017,ServiceNameOrIP3:27017/mdaggregationwsrows?authSource=admin"