Protobuf Timestamp rendering in Cloud UI

Confluent Cloud
1

My name we have been using Confluent Cloud for about a year and a half. After the first few tests with JSON we have been using protobuf for schema registering and when using the UI to inspect messages there has been no issue with any rendering of the messages. I would say in the past month or two this is no longer the case and messages that contain a reference to a google.protobuf.timestamp are only displayed in bytes. We currently use a Confluent managed sink to Azure Cosmos and that seems to work fine, so I dont believe that the registry or messgaes are incorrect as they come out as ms epoch ints. I don’t believe our schema registry or production methods have changed, but it is possible. Really just looking for direction in where to look to get this sorted as we do use the feature regularly.

Our schema looks like this….

syntax = "proto3";
package testing;

import "google/protobuf/timestamp.proto";
import "testing.enums.proto";

message Flag {
  string id = 1;
  string partition_key = 2;
  .testing.Flag.Payload payload = 3;

  message Payload {
    string id = 1;
    .google.protobuf.Timestamp flag_start_time = 2;
    .testing.EnumValue1 flag_state = 3;
    uint32 value = 4;

If I go into the evolve schema interface on Confluent Cloud it shows that I have a reference set.

image
image1170×245 14.7 KB

Yet when I go to the UI and try to view the message all I get is bytes…

{
  "type": "Buffer",
  "data": [ 0, 0, 1, 134, 212, 0, 10...

Anyone else seeing this behavior?

2

So I did some more testing and the issue persists.

starting with a basic message, this is what is displayed in the schema registry.

syntax = "proto3";
package test;

message WeatherString {
  string name = 1;
  int64 temp = 2;
  string sample_time_string = 3;
}

The value is properly deserialized in the UI and reports as JSON. now granted, I put temp as an int and its coming out a string but, hey its close.

{
"name": "test",
"temp": "68",
"sampleTimeString": "later than now"
}

So if I add in a google time stamp…

syntax = "proto3";

package test;
import "google/protobuf/timestamp.proto";

message Weather {
  string name = 1;
  int64 temp = 2;
  .google.protobuf.Timestamp sample_time = 3;

}

Confluent adds google/protobuf/timestamp.proto


syntax = "proto3";

package google.protobuf;
option java_package = "com.google.protobuf";
option java_outer_classname = "TimestampProto";
option java_multiple_files = true;
option go_package = "google.golang.org/protobuf/types/known/timestamppb";
option cc_enable_arenas = true;
option objc_class_prefix = "GPB";
option csharp_namespace = "Google.Protobuf.WellKnownTypes";

message Timestamp {
  int64 seconds = 1 [json_name = "seconds"];
  int32 nanos = 2 [json_name = "nanos"];
}

I produce the sample message


%python

msg = weather_pb2.Weather()
msg.name = "test"
msg.temp = 68
# msg.sample_time_string = "later than now"
msg.sample_time.FromSeconds(1675893509)

and we get bytes

{"type":"Buffer","data":[0,0,1,134,165,0,10,4,116,101,115,116,16,68,26,6,8,133,182,144,159,6]}

Anyone have any ideas…

edit: formating, sorry

3

This was fixed in the UI at some point. Honestly I stoped looking for it after a while.

  "utcCrossingTime": {
    "seconds": "1675017840",
    "nanos": 206000000
  },