I have 3 node kafka cluster, In one node the kafka service is not coming up

Kafka Connect
1

Hi,
The kafka service is not coming up in one of the node suddenly with below errors.
[root@liankafk01.xat.c ansible_rdu]# systemctl status kafka.service
● kafka.service - Kafka Service
Loaded: loaded (/etc/systemd/system/kafka.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Thu 2024-03-21 03:58:11 EDT; 794ms ago
Docs: http://kafka.apache.org
Process: 5981 ExecStop=/opt/local/kafka/bin/kafka-server-stop.sh (code=exited, status=1/FAILURE)
Process: 5977 ExecStart=/opt/local/kafka/bin/kafka-server-start.sh /opt/local/kafka/conf/server.properties (code=exited, status=0/SUCCESS)
Main PID: 5977 (code=exited, status=0/SUCCESS)

Mar 21 03:58:11 liankafk01.xat.c systemd[1]: Unit kafka.service entered failed state.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: kafka.service failed.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: kafka.service holdoff time over, scheduling restart.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: Stopped Kafka Service.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: start request repeated too quickly for kafka.service
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: Failed to start Kafka Service.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: Unit kafka.service entered failed state.
Mar 21 03:58:11 liankafk01.xat.c systemd[1]: kafka.service failed.

(pkg/logs/launchers/file/launcher.go:305 in handleTailingModeChange) | Tailing mode changed for file:/var/log/secure. Was: end: Now: beginning
| INFO | (pkg/logs/launchers/file/launcher.go:283 in startNewTailer) | Starting a new tailer for: /var/log/secure (offset: 0, whence: 0) for tailer key
| INFO | (pkg/logs/tailers/file/tailer_nix.go:30 in setup) | Opening /var/log/secure for tailer key /var/log/secure
| WARN | (pkg/logs/launchers/file/launcher.go:286 in startNewTailer) | open /var/log/secure: permission denied
| INFO | (pkg/logs/launchers/file/launcher.go:305 in handleTailingModeChange) | Tailing mode changed for file:/var/log/secure. Was: end: Now: beginning
| INFO | (pkg/logs/launchers/file/launcher.go:283 in startNewTailer) | Starting a new tailer for: /var/log/secure (offset: 0, whence: 0) for tailer key
| INFO | (pkg/logs/tailers/file/tailer_nix.go:30 in setup) | Opening /var/log/secure for tailer key /var/log/secure
| WARN | (pkg/logs/launchers/file/launcher.go:286 in startNewTailer) | open /var/log/secure: permission denied
| WARN | (pkg/collector/python/datadog_agent.go:131 in LogMessage) | disk:67cc0574430a16ba | (disk.py:136) | Unable to get disk metrics for /var/lib/nfs/rpc_pipefs: [Errno 13] Permission denied: ‘/var/lib/nfs/rpc_pipefs’ [Errno 13] Permission denied: ‘/var/lib/nfs/rpc_pipefs’. You can exclude this mountpoint in the settings if it is invalid.
| TRACE | INFO | (pkg/trace/info/stats.go:91 in LogAndResetStats) | No data received

2

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.