How to enable aggregate table

The aggregate table (v5.6.0+) relies on the MongoDB CDC connector of Flink to function properly. Before you can use this feature, ensure that the following prerequisites are met:

1. Deploy the data integration service. See: How to Enable Data Integration

2. MongoDB must be upgraded to version 4.4 or higher. Refer to: Upgrade MongoDB to 4.4 in Standalone Mode or Upgrade MongoDB to 4.4 in Cluster Mode

   • If you are using the built-in MongoDB in standalone mode and the current running mingdaoyun-sc container version is ≥ 3.0.0, the requirements are already met.

3. Adjust MongoDB to replica set mode (if it is already a replica set, you can ignore this step). See: Convert MongoDB Single Node to Replica Set

   • The built-in MongoDB in standalone mode is by default not in replica set mode.

Built-in MongoDB with authentication disabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.1.0 mirror.

Once the base conditions are met, no other configuration is required to use the aggregate table.👏 👏 👏

Built-in MongoDB with authentication enabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.1.0 mirror.

1. Connect to the admin database with the root account and create the roles and users for the aggregate table database

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'Here modify it to the password of the agguser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file (splice them if there are more options parameters), and restart the service. Take the standalone mode yaml as an example:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:6.4.0
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@sc:27017/mdaggregationwsrows?authSource=admin"

External MongoDB with authentication disabled

Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file, and restart the service. Take the standalone mode yaml as an example:

services:
  app:
  image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:6.4.0
  environment: &app-environment
    ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://service name or IP:27017/mdaggregationwsrows" 
    #Multi-node
    #ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://service name or IP1:27017, service name or IP2:27017, service name or IP3:27017/mdaggregationwsrows" 

External MongoDB with authentication enabled

1. Connect to the admin database with the root account and create the roles and users for the aggregate table database

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'Here modify it to the password of the agguser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file (splice them if there are more options parameters), and restart the service. Take the standalone mode yaml as an example:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:6.4.0
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@service name or IP:27017/mdaggregationwsrows?authSource=admin" 
          #Multi-node
          #ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@service name or IP1:27017, service name or IP2:27017, service name or IP3:27017/mdaggregationwsrows?authSource=admin"