Kafka Connection Security Configuration Guide
Kafka is used in all three images of this product: mingdaoyun-hap, mingdaoyun-hdp, and mingdaoyun-flink. Any adjustments to the security configuration must be applied to all of them accordingly.
Connection Configuration Parameters
| Environment Variable | Description | Default Value |
|---|---|---|
ENV_KAFKA_ENDPOINTS | Kafka broker address list. Multiple addresses should be separated by commas | 127.0.0.1:9092 |
ENV_KAFKA_SECURITY_PROTOCOL | Kafka security protocol mode | Empty |
ENV_KAFKA_SASL_MECHANISM | SASL authentication mechanism | Empty |
ENV_KAFKA_SASL_USERNAME | SASL username | Empty |
ENV_KAFKA_SASL_PASSWORD | SASL password | Empty |
ENV_KAFKA_SSL_CA_LOCATION | Kafka CA certificate path | Empty |
ENV_KAFKA_SSL_CLIENT_CA_LOCATION | Kafka client certificate path | Empty |
ENV_KAFKA_SSL_CLIENT_KEY_LOCATION | Kafka client private key path | Empty |
Security Protocol Modes
Kafka supports multiple security authentication modes, including Plaintext、SaslPlaintext、Ssl、SaslSsl.
1. Plaintext (Default Mode)
No security authentication mechanism is enabled. Communication is performed in plaintext, and no SASL or SSL-related parameters are required.
2. SaslPlaintext (SASL Authentication Enabled)
Uses the SASL mechanism for authentication, but data transmission remains in plaintext.
The following environment variables must be added to the configuration file:
| Environment Variable | Description |
|---|---|
ENV_KAFKA_SECURITY_PROTOCOL | Set to SaslPlaintext |
ENV_KAFKA_SASL_MECHANISM | SASL authentication mechanism, fixed as PLAIN |
ENV_KAFKA_SASL_USERNAME | SASL username |
ENV_KAFKA_SASL_PASSWORD | SASL password |
3. Ssl (TLS Encryption Enabled)
Uses TLS/SSL for encrypted communication but does not enable SASL authentication.
The following environment variables must be added to the configuration file:
| Environment Variable | Description |
|---|---|
ENV_KAFKA_SECURITY_PROTOCOL | Set to Ssl |
ENV_KAFKA_SSL_CA_LOCATION | CA root certificate path |
ENV_KAFKA_SSL_CLIENT_CA_LOCATION | Client certificate path |
ENV_KAFKA_SSL_CLIENT_KEY_LOCATION | Client private key path |
4. SaslSsl (SASL + TLS Enabled)
Enables both SASL authentication and SSL encrypted communication (recommended security mode).
The following environment variables must be added to the configuration file:
| Environment Variable | Description |
|---|---|
ENV_KAFKA_SECURITY_PROTOCOL | Set to SaslSsl |
ENV_KAFKA_SASL_MECHANISM | SASL authentication mechanism, fixed as PLAIN |
ENV_KAFKA_SASL_USERNAME | SASL username |
ENV_KAFKA_SASL_PASSWORD | SASL password |
ENV_KAFKA_SSL_CA_LOCATION | CA root certificate (PEM format) |
ENV_KAFKA_SSL_CLIENT_CA_LOCATION | Client certificate (PEM format) |
ENV_KAFKA_SSL_CLIENT_KEY_LOCATION | Client private key (PEM format) |
TLS Certificate Configuration
When using Ssl or SaslSsl, TLS certificates must be provided inside the container.
Certificate Files
Prepare the following PEM certificate files:
| File | Description |
|---|---|
ca.crt | Kafka CA root certificate |
client.crt | Client certificate |
client.key | Client private key |
Directory example:
volume/kafka/
├─ ca.crt
├─ client.crt
└─ client.key
Docker Certificate Mount Example
The certificates need to be mounted into the container:
volumes:
- ./volume/kafka/ca.crt:/usr/local/kafka/ca.crt
- ./volume/kafka/client.crt:/usr/local/kafka/client.crt
- ./volume/kafka/client.key:/usr/local/kafka/client.key
The corresponding environment variable values should be as follows (the environment variable paths must match the paths inside the container):
ENV_KAFKA_SSL_CA_LOCATION: "/usr/local/kafka/ca.crt"
ENV_KAFKA_SSL_CLIENT_CA_LOCATION: "/usr/local/kafka/client.crt"
ENV_KAFKA_SSL_CLIENT_KEY_LOCATION: "/usr/local/kafka/client.key"