FAQs
The commands on this page mainly apply to Docker Compose deployments. For Kubernetes, use corresponding kubectl commands, such as kubectl -n hap-ops logs deploy/ops-gateway. The example access ports are 48881 for Docker Compose and 30881 for Kubernetes.
Startup and Access
Containers Keep Restarting
Check logs with docker compose -f ops.yaml logs 服务名. Common causes:
- Incorrect connection information: Check the collection configuration on the Data Sources page.
- Port conflict: If 48881, 59100, 4317, or 4318 is occupied, change the host port mapping or stop the process using the port.
- Insufficient memory: Check usage with
docker stats. It is recommended to reserve 8G or more of memory on the host.
The Page Keeps Loading
If the page at :48881 keeps loading, check the following:
- Run
docker ps | grep gatewayto confirmgatewayhas started. - If the browser console reports 401, log in with the correct
ENV_OPS_TOKEN. - When accessing through a reverse proxy, confirm
ENV_OPS_SUB_PATHis configured.
Grafana Panels Keep Loading or Return 404 After Login
Check the reverse proxy configuration:
- Reverse proxy:
ENV_OPS_SUB_PATHmust match the actual proxy path. For example, use/mdiswhen proxying/mdis. - Direct port access: The URL path must match
ENV_OPS_SUB_PATH. With the deployment example's default value, access/mdis/web/.
gateway Starts Before ops-mongo
If ops-server cannot connect to ops-mongo, it retries in the background with backoff and connects automatically when MongoDB is ready. Check logs with docker logs gateway容器 | grep 告警子系统. When [alert] 告警子系统已挂载 /api/alert/ appears, mounting succeeded; no manual container restart is required.
No Monitoring Data
The Data Sources page has a Collection Status column showing whether each data source is being collected:
- Collecting: Collection is normal.
- No data: The collection process is running but has not obtained data, possibly due to incorrect credentials or insufficient permissions.
- Stopped: The collection process is not running.
- Not collected: Collection has not started for this data source, for example because View Charts is not selected under Purpose.
Hover over a status label to view the failure reason and retry information. Start troubleshooting collection issues from this column.
The Host Panel Has No Data
- Confirm
nodeagentis Up. - Confirm the host data source IP on the Data Sources page is reachable from the prometheus container:
docker exec prometheus容器 wget -O- http://IP:59100/metrics.
Middleware Panels Have No Data
Troubleshoot according to the Collection Status column:
| Status | Meaning | Action |
|---|---|---|
| No data | Connection succeeded but no metrics were obtained | Usually a credentials or permissions issue, especially for ES. See below |
| Stopped | The exporter process is not running | Hover to view the cause. The process retries automatically. If it stays Stopped while the retry count increases, the target remains unreachable due to an incorrect address, blocked port, or firewall |
| Not collected | Collection has not started for this data source | Check whether View Charts is selected under Purpose and the data source is enabled |
| Unknown | agent has not reported | Confirm the agent container is Up |
Elasticsearch Connection Test Succeeds but Shows No Data
Check the username and password. The built-in Elasticsearch username in private deployments is md. Changes to credentials on the Data Sources page take effect immediately without restarting containers.
For underlying logs, run docker logs agent容器. Within 10–50 seconds of startup, agent prints a collection self-check table with ✓/✗ and a reason for each item. If an exporter exits unexpectedly, it also prints [supervise] xxx 退出(code=N) and the last few log lines.
A Panel Reports Datasource ${DS_PROMETHEUS} was not found
This historical issue was fixed in the 1.4.0 single image. If it persists, confirm the image tag is 1.2.5 or later.
Create or Edit Data Sources
The Connection Address Is Temporarily Unreachable
Connection failure does not block saving the configuration. If the target is not deployed yet or the network is temporarily unreachable, the form displays the reason at the top and the button changes to Save Anyway. Confirm again to save. The Ops Platform keeps retrying and starts collection once connected. Track progress in the Collection Status column of the Data Sources list.
Connection Check Timing
Checks run automatically about 0.6 seconds after changing connection fields. No manual trigger is needed. If you save before the check finishes, the Ops Platform completes it before deciding how to save; validation is not skipped. Host data sources also support automatic checks.
The Check Reports That the Connection Cannot Be Checked Temporarily
This means the check service has not responded yet, rather than that the connection settings are incorrect. Save the configuration and check Collection Status later.
Collection Configuration Changes Do Not Take Effect
Collection Does Not Change After Editing a Data Source
Configuration normally takes effect within 30 seconds. If collection does not change, first check the agent container logs.
Repeated [sd] 拉取 xxx 失败 messages mean agent cannot reach the service discovery API. Common causes include ENV_OPS_SD_URL pointing to gateway port 48881, or a Kubernetes Service exposing only 48881 without 8081. Port 48881 is used by nginx and does not provide /api/sd. Exporters cannot start in this case, leaving middleware monitoring empty.
Middleware Connection Changes in ops.yaml Do Not Take Effect
Since 1.5.0, monitored targets are no longer configured through ops.yaml.
ENV_KAFKA_ENDPOINTS/ENV_ELASTICSEARCH_*/ENV_REDIS_*/ENV_MYSQL_*/ENV_MONGODB_URI
These variables are no longer read, as explicitly listed in the startup logs. Change all connection information on the UI Data Sources page.
Startup logs list each deprecated variable still in ops.yaml and indicate that it is ignored. Delete the related configuration lines.
Logs Are Not Visible
HAP/HDP Microservice Logs Are Missing from Service Logs
Check in this order:
- HAP/HDP services must configure
ENV_LOKI_URLpointing to MDIS Loki, such ashttp://MDIS主机:3100. Without this variable, logservice does not write logs to Loki because the installer setsStoreInLokitofalse; the Ops Platform cannot query service logs. Restart the corresponding product services after changing it. - Confirm the product service containers can access this address. For deployment across hosts, check that port 3100 is allowed.
- Microservices write logs only when business calls occur. Trigger an action such as login or sending a verification code, then refresh.
The two ENV_LOKI_URL variables have different purposes:
- HAP/HDP service-side
ENV_LOKI_URL: Enables writing. When unset, no logs are written. Empty by default. - MDIS-side
ENV_LOKI_URL: Query address, defaulting tohttp://ops-loki:3100. No configuration is needed in the standard topology.
Configure the former on the correct side.
Containers Are Missing from the Container Console
Alloy collects only new container logs after startup; it does not backfill historical logs. A container with no recent stdout output may not appear. Trigger output or restart the container, then check again.
Object Storage (MinIO/COS)
In standalone deployments, only Loki logs need object storage consideration. Standalone deployments do not support tracing, so ops-tempo generates no trace data; object storage is neither needed nor recommended for it. Tracing with object storage applies to cluster deployments.
ops-loki Cannot Start After Configuring Object Storage
Check the first error in docker logs loki容器:
dial tcp: lookup xxx: no such host: The endpoint uses a name that cannot resolve in the container network. If MinIO is not on the same Compose network, for example because it runs in a separate stack or on the host, use the host IP and host-mapped port instead of its container name.connection refused: Incorrect port. The host-mapped port may not be 9000; use the actual value shown bydocker ps.SignatureDoesNotMatch/AccessDenied: AK/SK or bucket permissions issue.- Path style: Use
ENV_S3_FORCE_PATH_STYLE=truefor MinIO. Tencent Cloud COS must usefalsebecause it supports only virtual-hosted style.
Historical Logs Disappear After Switching to Object Storage
Loki does not migrate old data when switching storage backends. Historical logs remain in the original local volume. Newly generated logs can be viewed in the new backend.
Upgrade
The Version Does Not Change After Upgrade
Since 1.4.0, all components share the ops-allinone image. Change the tag in x-ops-image at the top of ops.yaml, run docker compose -f ops.yaml pull, then docker compose -f ops.yaml up -d. Add --force-recreate if forced container recreation is needed.
Historical Data After Upgrade
Monitoring history and alert configuration remain if the Loki, Prometheus, Tempo, and ops-mongo storage volumes are unchanged. Unsaved manual edits in Grafana are lost; provisioned dashboards reload automatically.
Changing Data Volume Paths During Upgrade
The volume/data/mdis/ directory layout in the new deployment guide applies only to new installations. Existing instances do not need to change volume paths. Directly changing paths in ops.yaml mounts empty directories, making monitoring history, alert rules, and data source configuration unavailable. Actual data remains at the original paths and becomes accessible after restoring the original configuration.
Collection Configuration After Upgrading from 1.4.x
Data sources already configured in the UI remain unchanged, and monitoring is not interrupted.
Middleware ENV_* variables in ops.yaml are no longer read since 1.5.0 and can be deleted. Keeping them does not affect operation;
startup logs simply report that they are deprecated.
Post-Deployment Self-Check
The delivery package includes an end-to-end regression script. Run it once after deployment to verify the main paths: login, resource pages, Grafana charts, data source creation/deletion and connection tests, PromQL preview, the alert flow, notification channels, and log/trace queries:
pip install playwright && playwright install chromium
# Direct port access
python3 mdis_regression.py --base http://<主机IP>:48881 --token <ENV_OPS_TOKEN>
# Reverse proxy sub-path
python3 mdis_regression.py --base http://<主机IP>:9080 --subpath /mdis --token <ENV_OPS_TOKEN>
Exit code 0 means all checks passed. A nonzero code means some checks failed; the script prints a failure list at the end.