💁‍♂️ Check the timestamp of the latest message on Confluent Cloud from the CLI

Confluent Cloud
1

Here’s a nifty way of checking the timestamp of the latest message written to a given topic on Confluent Cloud:

docker run --rm --tty edenhill/kafkacat:1.6.0 \
       kafkacat -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X ssl.ca.location=./etc/ssl/cert.pem -X api.version.request=true \
                -b BROKER.confluent.cloud:9092 \
                -X sasl.username="API_USER" \
                -X sasl.password="API_PASSWORD" \
                -t TOPIC -C -c1 -o-1 -J | \
  jq '.ts' | \
  sed -E 's/[0-9]{3}$//g' | \
  xargs -Ifoo date -j -f %s foo

Mon  1 Mar 2021 11:04:48 GMT

It’s doing this:

  • With kafkacat, read (-C) one message (-c1), starting with the last message (-o-1) on the topic, outputting the message payload and metadata as JSON (-J).
  • Pipe it through jq to strip out just the timestamp on the Kafka message metadata
  • Uses sed to strips off the last three digits ([0-9]{3}$) to get the microseconds into milliseconds
  • Uses xargs to pass the value as an argument to date making it human-readable.

This is on the Mac with its version of date.
If you want it on Linux use xargs -Ifoo date --date=@foo instead, otherwise you’ll get the error date: invalid option -- 'j'

2 Likes