Data Source Management
Register monitored hosts, middleware, and other targets on the "Data Sources" page. Resource monitoring menus and available alert rule types are displayed dynamically based on enabled data sources. For Kubernetes collection, see Kubernetes Cluster Monitoring.

Supported Types
| Type | Multiple Instances | Connection Method |
|---|---|---|
| Prometheus | Built-in singleton | Built-in metrics database of the Ops Platform; read-only and cannot be disabled or deleted |
| Host | Supported | Host address (port connectivity and related monitoring) |
| MySQL | Supported | Node addresses (multiple allowed; each node collected separately)/username/password |
| MongoDB | Supported | Connection URI, replica sets supported |
| Redis | Supported | Node addresses (multiple allowed; list every Sentinel/Cluster node)/password/DB |
| Kafka | Supported | Broker addresses, SASL supported |
| Elasticsearch | Supported | Node addresses (multiple allowed) + HTTP Basic authentication |
| Custom page | Supported | Any Web UI address (such as Flink or Langfuse). Registered pages appear in the "Custom" sidebar group; see Custom Pages below |
Multiple MySQL, Redis, or Elasticsearch nodes can be entered in one data source, with one host:port per line or separated by commas. Each node uses a separate collector process, so an unavailable node does not affect other nodes. Redis failover requires no configuration change.
Purposes
Each data source can select one or more purposes, which determine the feature scope it participates in:
| Purpose | Effect |
|---|---|
| View Charts | Displays the instance dashboard in the corresponding "Resource Monitoring" menu |
| Alert | Allows alert rules to be created for the instance |
| Slow Query Diagnostics | MongoDB only: enables slow query analysis and index diagnostics. If profiling is disabled on the target database, the Ops Platform runs db.runCommand({profile:1, slowms:100}); the threshold is controlled by ENV_GATEWAY_PROFILE_SLOW_MS. Indexes are not created automatically; click "One-click Create" on the diagnostics page to create them. Diagnostic history is stored in the Ops Platform's own ops-mongo |
Account Permissions for Monitored Targets
Before registration, prepare accounts with the required permissions for each monitored target. The Ops Platform primarily performs read-only monitoring by default:
| Target | Required Permissions |
|---|---|
| MySQL | SELECT, SHOW DATABASES, PROCESS, REPLICATION CLIENT |
| MongoDB | root recommended; see the minimum-permission example below |
| Redis | Data read permission |
| Kafka | If authentication is enabled, permission to read cluster metadata |
| Elasticsearch | Read/write index permissions |
| Custom page | Reverse proxy mode: the target address must be reachable from the Ops Platform server. Direct iframe mode: the target must be reachable from the browser and must not restrict embedding via X-Frame-Options/frame-ancestors |
Minimum MongoDB permissions (readWrite/dbAdmin are used for index creation during slow query diagnostics):
use admin
db.createUser({
user: "opsuser", pwd: "your_secure_password",
roles: [
{ role: "clusterMonitor", db: "admin" },
{ role: "readAnyDatabase", db: "admin" },
{ role: "clusterManager", db: "admin" },
{ role: "dbAdmin", db: "mdwsrows" }, { role: "readWrite", db: "mdwsrows" },
{ role: "dbAdmin", db: "mdservicedata" }, { role: "readWrite", db: "mdservicedata" },
{ role: "dbAdmin", db: "mdworksheet" }, { role: "readWrite", db: "mdworksheet" },
{ role: "dbAdmin", db: "mdworkflow" }, { role: "readWrite", db: "mdworkflow" }
]
})
Elasticsearch Account Fields
If the connection succeeds but no metrics are collected, check the username. "Test Connection" checks both connection and collection status.
| Item | Value |
|---|---|
| URL | http://ES地址:9200, including the protocol |
| Username | The built-in Elasticsearch username in private deployment is md |
| Password | ES password set during HAP/HDP deployment |
Use the "Collection Status" column or "Test Connection" to check whether data has been collected. Hover over "No Data" to view the reason.
Data Source Operations
- Create: Click "New Data Source", select a type, and enter connection information.
- Test Connection: Verifies target connectivity and authentication, and checks collection status. After a successful connection, confirm that metrics have been collected; results display the status and reason.
- Connectivity Check: Click the upper-right button to check listed data sources in batch. Results appear in the "Status" column.
- Enable or disable: Toggle the list switch. After disabling, the corresponding resource monitoring menu and alert types are hidden, and existing alert rules stop scheduling.
- Edit: Saved passwords are masked. Leave a password blank to keep it unchanged.
Connection tests use actual credentials, including MongoDB ping, MySQL login, Kafka metadata retrieval, Redis AUTH, and Elasticsearch authentication requests.
Passwords and connection URIs are AES-encrypted with ENV_ALERT_CRYPTO_KEY before database storage; APIs return masked values. Set this key during first deployment and keep it safe. Changing it prevents saved credentials from being decrypted and requires re-entry.
How Configuration Takes Effect
Add, modify, enable, or disable monitored targets on the Data Sources page. Changes take effect immediately without restarting containers.
Since 1.5.0,
ops.yamlno longer participates in data source configuration. It only describes the Ops Platform's own dependencies, not monitoring targets.
For batch registration, click "Batch Import" in the upper-right corner, download the YAML template, modify the connection addresses, and upload it. We recommend selecting "Validate Only" first to check file format and probe each connection without writing data.
Import identifies data sources by "type + name", so the same file can be imported repeatedly. When updating an existing data source, leave the password blank to retain the original password.
Export Configuration (Backup/Migration)
Click "Export Configuration" in the upper-right corner to export all data sources as YAML. The format matches batch import and supports backups, reinstall recovery, and cross-environment migration.
| Option | Password | Purpose |
|---|---|---|
| Export configuration (without passwords) | Empty | Daily backups and import back to the same Ops Platform. Empty does not clear passwords; import reuses stored passwords by "type + name" |
| Export configuration (with plaintext passwords) | Plaintext | Migration to another Ops Platform, or recovery when the original Ops Platform database is unavailable. The file contains plaintext passwords; store it properly and never commit it to a code repository |
Exports without passwords also remove credentials from MongoDB URIs (mongodb://user:pass@host → mongodb://host). Supply credentials when restoring to another environment. See the file header comments for details.
Collection Status
Each data source in the list includes "Collection Status", used to determine whether collection is running. Start with this status when investigating most collection problems:
| Status | Meaning | Common Handling |
|---|---|---|
| Collecting | Normal; metrics are being written | — |
| No Data | Collector process is running, but no metrics have been obtained | Usually account/password/permission issues. Hover to view the original reason |
| Stopped | Collector process is not running | Hover to view the exit reason (unreachable address, wrong port, authentication failure, etc.); the process retries automatically |
| Not Collected | Collection has not started for this data source | Check whether the purpose includes "View Charts" and whether the data source is enabled |
| Unknown | Status is temporarily unavailable | Confirm that the ops-agent (middleware) or ops-prometheus (host) container has started |
Hover over a red status to view the failure reason (such as 401 unable to authenticate user or
connection refused) and troubleshooting commands. When dashboards have no data, check collection status first.
node_exporter runs on the monitored host and is scraped directly by the Ops Platform Prometheus, without going through ops-agent. Host collection status is determined from Prometheus scrape results:
- A newly registered host may briefly show "Not Collected". The scrape target takes effect after about 30 seconds, and the status changes to "Collecting".
- If the service at the address is not node_exporter or the port is unreachable, the status is "No Data".
- If the Ops Platform Prometheus cannot be queried, "Unknown" indicates that collection status is temporarily undetermined.
The troubleshooting command shown on hover probes the host address (curl http://主机:端口/metrics); it is not an instruction to enter the agent container.
Custom Pages (External Web UI)
Custom pages add Flink Web UI, Langfuse, internal dashboards, and other pages to the "Custom" sidebar group without collecting metrics. Select "Custom page" in "New Data Source" to add multiple pages.
| Field | Description |
|---|---|
| Name | Shown in the "Custom" sidebar group; the path /custom/<id> is generated from the name and remains unchanged after creation |
| Page URL | The http(s) address of the target Web UI |
| Embedding mode | Reverse proxy: forwards requests through the Ops Platform with authentication. The Ops Platform server must be able to reach the target; suitable for services using relative asset paths, such as Flink. Direct iframe: the browser loads the target directly, so the target must be reachable from the browser and allow iframe embedding. Applications using absolute paths, such as Langfuse, require this mode |
If the page is blank, check the browser console. For asset 404s in reverse proxy mode, try direct iframe. For X-Frame-Options or frame-ancestors restrictions in direct iframe mode, adjust the target restrictions or try reverse proxy.
The address is probed on save. Login pages returning 401 or 403 still count as reachable. Failed probes do not prevent saving; open the page from the menu once connectivity is available.
After upgrading from 1.5.11 or earlier, the former singleton "Flink" entry is automatically migrated to a custom page in reverse proxy mode. Its sidebar entry and URL remain unchanged; no manual action is required.
Custom pages are also included in "Export Configuration" and "Batch Import". YAML example:
- kind: customPage
name: Langfuse
url: http://langfuse:3000
mode: iframe # proxy reverse proxy / iframe direct iframe
enabled: true
After Registration
After data sources are registered and collection status is normal:
- Resource Monitoring: View dashboards for each instance
- Alert Rules: Configure alerts for instances
- Slow Query Diagnostics: MongoDB only; select the purpose first