How to enable aggregate table

The aggregate table (v5.6.0+) relies on the MongoDB CDC connector of Flink to function properly. Before you can use this feature, ensure that the following prerequisites are met:

1. Deploy the data pipeline service. How to enable data pipeline
2. MongoDB needs to be upgraded to v4.4+. Upgrade standalone mode MongoDB to 4.4 or Upgrade cluster mode MongoDB to 4.4
3. MongoDB needs to be configured to a replica set mode (if already a replica set, this step can be skipped). Convert MongoDB single node to replica set

Built-in MongoDB with authentication disabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.0.0 mirror.

Once the base conditions are met, no other configuration is required to use the aggregate table.👏 👏 👏

Built-in MongoDB with authentication enabled

Built-in MongoDB refers to the MongoDB component included in the mingdaoyun-sc:3.0.0 mirror.

1. Connect to the admin database with the root account and create the roles and users for the aggregate table database

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'Here modify it to the password of the agguser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file (splice them if there are more options parameters), and restart the service. Take the standalone mode yaml as an example:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.8.1
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@sc:27017/mdaggregationwsrows?authSource=admin"

External MongoDB with authentication disabled

Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file, and restart the service. Take the standalone mode yaml as an example:

services:
  app:
  image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.8.1
  environment: &app-environment
    ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://service name or IP:27017/mdaggregationwsrows" 
    #Multi-node
    #ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://service name or IP1:27017, service name or IP2:27017, service name or IP3:27017/mdaggregationwsrows" 

External MongoDB with authentication enabled

1. Connect to the admin database with the root account and create the roles and users for the aggregate table database

   use admin;
   db.createRole({
     role: "aggrole",
     privileges: [
       {
         resource: { db: "", collection: "" },
         actions: [
           "splitVector",
           "listDatabases",
           "listCollections",
           "collStats",
           "find",
           "changeStream"
         ]
       }
     ],
     roles: [{ role: 'read', db: 'config' }]
   });

   db.createUser({
     user: 'agguser',
     pwd: 'Here modify it to the password of the agguser',
     roles: [
       { role: 'aggrole', db: 'admin' },
       { role: 'readWrite', db: 'mdaggregationwsrows' }
     ]
   });

2. Add the environment variable ENV_MONGODB_URI_AGGREGATIONWSROWS in the configuration file (splice them if there are more options parameters), and restart the service. Take the standalone mode yaml as an example:

    services:
      app:
        image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.8.1
        environment: &app-environment
          ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@service name or IP:27017/mdaggregationwsrows?authSource=admin" 
          #Multi-node
          #ENV_MONGODB_URI_AGGREGATIONWSROWS: "mongodb://agguser:******@service name or IP1:27017, service name or IP2:27017, service name or IP3:27017/mdaggregationwsrows?authSource=admin"