FAQ
For cluster deployment, use the corresponding kubectl commands. Container and workload names use the ops- prefix, such as kubectl -n hap-ops logs deploy/ops-gateway.
The access port is 30881, not 48881.
Startup and Access
Containers keep restarting? Check logs first with docker compose -f ops.yaml logs <service-name>. Common causes:
- Incorrect connection information: verify it on the "Data Sources" page. Collection configuration is based on the UI.
- Port conflict:
48881/59100/4317/4318is already occupied. Change the host port mapping or stop the occupying process. - Insufficient memory: check resource usage with
docker stats. Reserve at least 8G memory on the host.
The :48881 page keeps loading?
- Run
docker ps | grep gatewayto confirmgatewayis Up. - If the browser console shows 401, log in with the correct
ENV_OPS_TOKEN. - If access goes through a reverse proxy with a sub-path,
ENV_OPS_SUB_PATHmust be configured.
Grafana panels keep loading or return 404 after login?
- Reverse proxy mode:
ENV_OPS_SUB_PATHmust match the actual proxy path. For example, proxying/mdisrequires/mdis. - Direct port mode: do not configure
ENV_OPS_SUB_PATH.
Will gateway keep failing if it starts before ops-mongo? No. If ops-server cannot connect to ops-mongo, it retries in the background with backoff. After mongo is ready, it connects automatically. Use docker logs <gateway-container> | grep alert; when [alert] alert subsystem mounted at /api/alert/ appears, it is normal and no manual container restart is required.
No Monitoring Data
The "Data Sources" page includes a Collection Status column. Each data source directly shows whether collection is active:
- Collecting: normal.
- No data: the collection process is running but has not collected data, usually because the account, password, or permissions are incorrect.
- Stopped: the collection process is not running.
- Not collected: collection was not started for this data source, often because the "Diagram" purpose is not selected.
Hover over a red status label to view the failure reason. The message is a localized troubleshooting hint, such as "port unreachable or remote service not started" or "incorrect account or password: the built-in private deployment ES username is md, not elastic", and includes the retry count. Exporter logs and Go stacks are no longer shown directly. Most collection issues can be located from this column first.
Host panel has no data?
- Confirm
nodeagentis Up. - Confirm that the host data source IP on the "Data Sources" page is reachable from the prometheus container:
docker exec <prometheus-container> wget -O- http://IP:59100/metrics.
Middleware panels have no data? Continue troubleshooting based on the "Collection Status" column:
| Status | Meaning | Action |
|---|---|---|
| No data | Connected successfully but no metrics were collected | Usually an account/password/permission issue. Elasticsearch is common; see the next item |
| Stopped | Exporter process is not running | Hover to view the reason. The process retries automatically. If it remains "Stopped" and the retry count increases, the target is continuously unreachable, usually because of an incorrect address, blocked port, or firewall |
| Not collected | Collection is not started for this data source | Check whether Diagram is selected under "Purpose" and whether the data source is enabled |
| Unknown | agent has not reported status | Confirm the agent container is Up |
Elasticsearch shows "No data" but connection test succeeds? The most common cause is incorrect credentials. The built-in ES username in private deployment is md, not elastic. Correct the username and password on the "Data Sources" page. The change takes effect immediately and no container restart is required.
When underlying logs are needed: run docker logs <agent-container>. 10-50 seconds after startup, agent prints a collection self-check table with one line per item and a pass/fail marker plus reason. When an exporter exits unexpectedly, it also prints [supervise] xxx exited (code=N) and the last few log lines.
Panel reports Datasource ${DS_PROMETHEUS} was not found?
This historical issue has been fixed in the 1.4.0 single image. If it still occurs, confirm that the image tag is not earlier than 1.2.5.
Create or Edit Data Sources
Can an unreachable connection address still be saved? Yes. A failed connection test does not block saving. When the target is not yet deployed or the network is temporarily unreachable, the configuration can be saved first. A yellow prompt at the top of the form shows the reason, and the save button changes to "Save Anyway". Confirm again to save. After saving, the Ops Platform keeps retrying and starts collection automatically after the target becomes reachable. Progress can be viewed in the "Collection Status" column on the "Data Sources" list.
When is connection detection executed? It runs automatically about 0.6 seconds after a connection field is modified. No manual trigger is required. If saving happens before detection completes, the Ops Platform completes detection before deciding the save flow; validation is not skipped. Host data sources are also included in automatic detection.
What does "connection cannot be checked temporarily" mean? It means the detection service has not responded yet. It does not mean the connection configuration is wrong. The configuration can be saved, and status can be checked later in the "Collection Status" column.
Collection Configuration Changes Do Not Take Effect
Data source changes in the UI are not reflected in collection? Normally, changes take effect within 30 seconds. If they do not, check the agent container logs first.
If [sd] failed to pull xxx keeps appearing, the agent cannot access the service discovery API. The common causes are ENV_OPS_SD_URL pointing to gateway port 48881, which is nginx and does not have the /api/sd route, or the Kubernetes Service exposing only 48881 without exposing 8081. In this case, exporters cannot start and all middleware monitoring stays empty.
Changes to middleware connection variables in ops.yaml do not take effect? Since 1.5.0, monitored targets are no longer configured in ops.yaml.
ENV_KAFKA_ENDPOINTS/ENV_ELASTICSEARCH_*/ENV_REDIS_*/ENV_MYSQL_*/ENV_MONGODB_URI and similar variables are no longer read. Startup logs explicitly list and warn about them. All connection information must be changed on the UI "Data Sources" page.
If these deprecated variables are still present in ops.yaml, startup logs list them one by one and mark them ignored. Delete the related configuration lines.
Logs Are Not Visible
No HAP/HDP microservice logs in "Service Logs"? Check in order:
- HAP/HDP services must configure
ENV_LOKI_URL. It should point to the MDIS Loki endpoint, such ashttp://<MDIS-host>:3100. If this variable is not configured, logservice does not write logs to Loki because the installer setsStoreInLokitofalse. The Ops Platform cannot query service logs in that case. Restart the corresponding product services after changing it. - Confirm that product service containers can access this address. For cross-host deployment, check whether port 3100 is open.
- Microservice logs are written only after business requests occur. Trigger an operation such as login or sending a verification code, then refresh.
ENV_LOKI_URL variables are different- HAP/HDP service-side
ENV_LOKI_URL= write-side switch. If not configured, logs are not written. Default is empty. - MDIS-side
ENV_LOKI_URL= query address. Default ishttp://ops-loki:3100, and no configuration is required in the standard topology.
Configure the former and avoid setting it on the wrong side.
A container is missing from the container console dropdown? Alloy collects only logs from containers created after it starts, and does not backfill historical logs. Containers with no recent stdout output may not appear. Trigger output in the container or restart it, then check again.
Object Storage (MinIO/COS)
In standalone deployment, only logs (Loki) need object storage consideration. Standalone deployment does not provide tracing capability, so ops-tempo does not generate trace data and does not need object storage. Tracing plus object storage belongs to the cluster deployment scenario.
ops-loki cannot start after object storage is configured? Check the first error log with docker logs <loki-container>:
dial tcp: lookup xxx: no such host: endpoint uses a name that cannot be resolved inside the container network. If MinIO is not in the same Compose network, such as another stack or a host deployment, use host IP + host mapped port instead of the other container name.connection refused: port is incorrect. The host mapped port may not be 9000; use the actual value fromdocker ps.SignatureDoesNotMatch/AccessDenied: AK/SK or bucket permission issue.- Path style: use
ENV_S3_FORCE_PATH_STYLE=truefor MinIO. Tencent Cloud COS must befalsebecause it supports only virtual-hosted style.
Historical logs are not visible after switching to object storage? Loki does not migrate old data when the backend is switched. Old logs remain in the previous local volume. Generate new logs to see data in the new backend.
Upgrade
Version does not take effect after upgrade? Since 1.4.0, all components share the ops-allinone image. To upgrade, modify the tag in x-ops-image at the top of ops.yaml, run docker compose -f ops.yaml pull, and then run docker compose -f ops.yaml up -d. Add --force-recreate if containers must be recreated forcibly.
Will historical data remain after upgrade? The storage volumes for Loki, Prometheus, Tempo, and ops-mongo are not changed, so monitoring history and alert configuration are retained. Unsaved manual edits in Grafana are lost, while provisioned dashboards are reloaded automatically.
Should existing data volume paths be changed according to the new documentation during upgrade? No. The new deployment documentation places data volumes under volume/data/mdis/, but that layout applies only to new installations. If an existing instance directly changes volume paths in ops.yaml, containers mount empty directories and appear as if monitoring history, alert rules, and data source configuration are all lost. The data is still in the original path and can be recovered by restoring the original configuration.
What happens to collection configuration after upgrading from 1.4.x? Data sources already configured in the UI remain unchanged and monitoring is not interrupted.
Middleware ENV_* variables in ops.yaml are no longer read since 1.5.0. They can be deleted. Keeping them does not affect runtime, but startup logs warn that they are deprecated.
Post-Deployment Self-Check
The delivery package includes an end-to-end regression script. Run it once after deployment to confirm the main paths: login, resource pages, Grafana charts, data source create/delete and connection tests, PromQL preview, alert flow, notification channels, logs, and trace queries.
pip install playwright && playwright install chromium
# Direct port access
python3 mdis_regression.py --base http://<host-ip>:48881 --token <ENV_OPS_TOKEN>
# Reverse proxy sub-path
python3 mdis_regression.py --base http://<host-ip>:9080 --subpath /mdis --token <ENV_OPS_TOKEN>
Exit code 0 means all checks passed. A non-zero exit code means some checks failed, and the script prints the failure list at the end.