# DESCRIBE throws Failed to deserialise object error **URL:** \ **Category:** ksqlDB\ **Created:** [23 February 2021 21:46 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002 "2021-02-23T21:46:38Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [23 February 2021 21:46 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/1 "2021-02-23T21:46:39Z")
Hello, everyone. I am currently trying to ingest into my kafka topics some .csv files containg medical data. I was successful in getting the rows into my topics with the FilePulse Source connector, but when I try to create a Stream on them and then use the DESCRIBE command, I get this error: ```auto Failed to deserialise object Caused by: Cannot construct instance of `io.confluent.ksql.rest.entity.SourceDescription`, problem: queryOffsetSummaries at [Source: (byte[])"[{"@type":"sourceDescription","statementText":"describe ECG_STREAM;","sourceDescription":{"name":"ECG_STREAM","windowType":null,"readQue ries":[],"writeQueries":[],"fields":[{"name":"TIMESTAMP","schema":{"type":"BIGIN T","fields":null,"memberSchema":null}},{"name":"INX","schema":{"type":"STRING"," fields":null,"memberSchema":null}},{"name":"SAMPLENUM","schema":{"type":"STRING" ,"fields":null,"memberSchema":null}},{"name":"FREQUENCY","schema":{"type":"INTEG ER","fields":null,"memberSchema":null}},{"na"[truncated 760 bytes]; line: 1, column: 1244] (through reference chain: io.confluent.ksql.rest.entity.KsqlEntityList[0]->io.confluent.ksql.rest.entity.S ourceDescriptionEntity["sourceDescription"]) Caused by: queryOffsetSummaries ``` My connector configuration is pretty simple: - name = BR\_Ingestion - connector.class = io.streamthoughts.kafka.connect.filepulse.source.FilePulseSourceConnector - tasks.max = 1 - topic = file\_BR - offset.strategy = name - filters = ParseLine, brIntervalToInt, breathRate, TimestampToInt - internal.kafka.reporter.bootstrap.servers = kafka1:19092 - fs.scan.filters = - fs.scan.directory.path = /tmp/input\_files/BR/ - fs.cleanup.policy.class = io.streamthoughts.kafka.connect.filepulse.clean.LogCleanupPolicy - filters.brIntervalToInt.field = brInterval - skip.headers = 1 - filters.ParseLine.extractColumnName = headers - filters.ParseLine.type = io.streamthoughts.kafka.connect.filepulse.filter.DelimitedRowFilter - filters.brIntervalToInt.to = INTEGER - filters.brIntervalToInt.type = io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter - filters.ParseLine.separator = ; - filters.ParseLine.trimColumn = true - filters.breathRate.type = io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter - filters.breathRate.field = breathRate - filters.breathRate.to = INTEGER - filters.TimestampToInt.type = io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter - filters.TimestampToInt.field = timestamp - filters.TimestampToInt.to = LONG I also tried to specify the key and value converter to the Avro class, and explicitly set the key/value.converter.schema.registry.url as an additional property, but haven’t managed to solve the issue. Researching for the error also didn’t show any hope of a solution. Am I missing something elementary in my configuration? If someone could help me, I would really be thankful. Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 07:55 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/2 "2021-02-24T07:55:23Z")
Hi @Steno, what version of ksqlDB server and CLI are you running? How have you deployed them? ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 08:13 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/3 "2021-02-24T08:13:48Z")
Hello, @rmoff , thank you very much for answering. Both my ksqlDB Server and CLI are version 6.0.1 and are deployed as docker containers. I do not know if it could be useful, but here is one of the messages that are inside on the topics I am trying to base my streams on: > rowtime: 2021/02/24 07:23:57.857 Z, key: , value: {“timestamp”: 1579604593, “inx”: “212”, “sampleNum”: “773”, “Frequency”: 128, “HeartRate”: 112, “SensorStatus”: “255”, “RRLength”: 531, “Samples”: “-301”} A key value, and schema, has not been set and has been left untouched. The value I used to define the schema registry url should be correct, as I have also landoop’s api that shows me the schemas correctly. Is the problem the key? EDIT: I also tried to set the key at ingestion-time and manually create its schema, but the error remains the same. Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 09:11 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/4 "2021-02-24T09:11:20Z")
Could you share: 1. The full output of `PRINT your_topic FROM BEGINNING LIMIT 1;` 2. The full `CREATE STREAM` statement that you’re running ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 09:16 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/5 "2021-02-24T09:16:12Z")
Hello, Absolutely, here they are: ```sql ksql> print file_ECG from beginning limit 1; Key format: ¯\_(ツ)_/¯ - no data processed Value format: AVRO rowtime: 2021/02/24 07:23:56.412 Z, key: , value: {"timestamp": 1579604534, "inx": "20", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "67"} ``` ```auto ksql> CREATE STREAM ECG_STREAM WITH(KAFKA_TOPIC='file_ECG',VALUE_FORMAT='AVRO'); ``` Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 09:25 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/6 "2021-02-24T09:25:17Z")
Hmm that kinda looks OK. Is it only the `DESCRIBE` that fails? Does it work if you run this: ```sql SET 'auto.offset.reset' = 'earliest'; SELECT * FROM ECG_STREAM EMIT CHANGES; ``` ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 09:54 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/7 "2021-02-24T09:54:19Z")
Hello, So, in order: 1. Running the SET command and then the SELECT command, worked properly. Here a sample of the output: ```sql ksql> SET 'auto.offset.reset' = 'earliest'; Successfully changed local property 'auto.offset.reset' from 'earliest' to 'earliest'. ksql> SELECT * FROM ECG_STREAM EMIT CHANGES; |TIMESTAMP |INX |SAMPLENUM |FREQUENCY |HEARTRATE |SENSORSTATUS |RRLENGTH |SAMPLES | |1579604973 |328 |1151 |128 |163 |255 |367 |-34 | |1579604973 |332 |1151 |128 |163 |255 |367 |-249 | ``` (I cleaned the input up a bit, or it would have been unreadable) 1. Running the SET command and then trying the DESCRIBE command again, still ends up with the usual error. ```sql ksql> SET 'auto.offset.reset' = 'earliest'; Successfully changed local property 'auto.offset.reset' from 'earliest' to 'earliest'. ksql> DESCRIBE ECG_STREAM; Failed to deserialise object Caused by: Cannot construct instance of `io.confluent.ksql.rest.entity.SourceDescription`, problem: queryOffsetSummaries at [Source: (byte[])"[{"@type":"sourceDescription","statementText":"DESCRIBE ECG_STREAM;","sourceDescription":{"name":"ECG_STREAM","windowType":null,"readQue ries":[],"writeQueries":[],"fields":[{"name":"TIMESTAMP","schema":{"type":"BIGIN T","fields":null,"memberSchema":null}},{"name":"INX","schema":{"type":"STRING"," fields":null,"memberSchema":null}},{"name":"SAMPLENUM","schema":{"type":"STRING" ,"fields":null,"memberSchema":null}},{"name":"FREQUENCY","schema":{"type":"INTEG ER","fields":null,"memberSchema":null}},{"na"[truncated 760 bytes]; line: 1, column: 1244] (through reference chain: io.confluent.ksql.rest.entity.KsqlEntityList[0]->io.confluent.ksql.rest.entity.S ourceDescriptionEntity["sourceDescription"]) Caused by: queryOffsetSummaries ``` So it looks like the DESCRIBE commands is the one with issues, and with all streams generated, even from different topics. Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 17:55 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/8 "2021-02-24T17:55:32Z")
Do you have access to Docker to give ksqlDB 0.15 a try? And/or are you able to share a sample of the CSV file so that I can try to reproduce the issue from here? ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 19:31 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/9 "2021-02-24T19:31:22Z")
Hello, I could test ksqlDB 0.15 in the next days to see if something changes, and will revert with the feedback on the attempt back as soon as I have the results In the meanwhile, as you asked, here’s the one of the files that I am working with. As the data inside is completely anonymous, there was no problem in uploading the entire file as-is: > Input file: [Gofile - Your all-in-one storage solution](https://gofile.io/d/iJx9aM) Also, hoping it might help in reproducing the issue, I have also uploaded the configuration file corresponding the FilePulse source connector I used to ingest the file into my topic: > FilePulse link: [Gofile - Your all-in-one storage solution](https://gofile.io/d/l1rEno) Thank you. Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 21:24 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/10 "2021-02-24T21:24:23Z")
- The bad news: I couldn’t reproduce your error 😕 - To good news: using that file and config I could ingest the data and get it to work just fine using ksqlDB 0.15 🙂 I used [this Docker Compose](https://github.com/confluentinc/demo-scene/blob/d0df77b181d001d4452f176fb62698d3837ed307/csv-to-kafka/docker-compose.yml) to spin up Kafka Connect etc, and then this connector config, which is the same as yours except I fiddled with `fs.scan.filters` to get it to pick up the file: ```json curl -i -X PUT -H "Accept:application/json" \ -H "Content-Type:application/json" http://localhost:8083/connectors/ECG_Ingestion/config \ -d '{ "connector.class" : "io.streamthoughts.kafka.connect.filepulse.source.FilePulseSourceConnector", "fs.cleanup.policy.class" : "io.streamthoughts.kafka.connect.filepulse.clean.LogCleanupPolicy", "fs.scan.directory.path" : "/data/", "fs.scan.filters" : "io.streamthoughts.kafka.connect.filepulse.scanner.local.filter.RegexFileListFilter", "file.filter.regex.pattern" : "file_ECG.csv", "offset.strategy" : "name", "skip.headers" : "1", "tasks.max" : "1", "topic" : "file_ECG", "internal.kafka.reporter.bootstrap.servers": "broker:29092", "value.converter.schema.registry.url" : "http://schema-registry:8081/", "value.converter" : "io.confluent.connect.avro.AvroConverter", "key.converter.schema.registry.url" : "http://schema-registry:8081/", "key.converter" : "io.confluent.connect.avro.AvroConverter", "filters" : "ParseLine, HeartRateToInt, FrequencyToInt, RRLengthToInt, TimestampToInt, SetKey, KeyConversion", "filters.ParseLine.extractColumnName" : "headers", "filters.ParseLine.separator" : ";", "filters.ParseLine.trimColumn" : "true", "filters.ParseLine.type" : "io.streamthoughts.kafka.connect.filepulse.filter.DelimitedRowFilter", "filters.SetKey.field" : "$key", "filters.SetKey.type" : "io.streamthoughts.kafka.connect.filepulse.filter.AppendFilter", "filters.SetKey.value" : "{{ uppercase($value.sampleNum)}}", "filters.FrequencyToInt.field" : "Frequency", "filters.FrequencyToInt.to" : "INTEGER", "filters.FrequencyToInt.type" : "io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter", "filters.HeartRateToInt.field" : "HeartRate", "filters.HeartRateToInt.to" : "INTEGER", "filters.HeartRateToInt.type" : "io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter", "filters.KeyConversion.field" : "$key", "filters.KeyConversion.to" : "STRING", "filters.KeyConversion.type" : "io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter", "filters.RRLengthToInt.field" : "RRLength", "filters.RRLengthToInt.to" : "INTEGER", "filters.RRLengthToInt.type" : "io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter", "filters.TimestampToInt.field" : "timestamp", "filters.TimestampToInt.to" : "LONG", "filters.TimestampToInt.type" : "io.streamthoughts.kafka.connect.filepulse.filter.ConvertFilter" }' ``` From there, I could then see the data in the Kafka topic from ksqlDB: ```sql ksql> SHOW TOPICS; Kafka Topic | Partitions | Partition Replicas ------------------------------------------------------------------------- confluent_rmoff_01ksql_processing_log | 1 | 1 connect-file-pulse-status | 1 | 1 docker-connect-configs | 1 | 1 docker-connect-offsets | 25 | 1 docker-connect-status | 5 | 1 file_ECG | 1 | 1 ------------------------------------------------------------------------- ksql> PRINT file_ECG FROM BEGINNING LIMIT 5; Key format: AVRO or HOPPING(KAFKA_STRING) or TUMBLING(KAFKA_STRING) or KAFKA_STRING Value format: AVRO rowtime: 2021/02/24 21:16:47.681 Z, key: 715, value: {"timestamp": 1579604534, "inx": "20", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "67"}, partition: 0 rowtime: 2021/02/24 21:16:47.681 Z, key: 715, value: {"timestamp": 1579604534, "inx": "24", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "134"}, partition: 0 rowtime: 2021/02/24 21:16:47.681 Z, key: 715, value: {"timestamp": 1579604534, "inx": "28", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "214"}, partition: 0 rowtime: 2021/02/24 21:16:47.681 Z, key: 715, value: {"timestamp": 1579604534, "inx": "32", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "315"}, partition: 0 rowtime: 2021/02/24 21:16:47.681 Z, key: 715, value: {"timestamp": 1579604534, "inx": "36", "sampleNum": "715", "Frequency": 128, "HeartRate": 108, "SensorStatus": "255", "RRLength": 554, "Samples": "449"}, partition: 0 Topic printing ceased ksql> ``` and then create the stream, describe it, and query it: ```sql ksql> CREATE STREAM ECG_STREAM WITH(KAFKA_TOPIC='file_ECG',FORMAT='AVRO'); Message ---------------- Stream created ---------------- ksql> DESCRIBE ECG_STREAM; Name : ECG_STREAM Field | Type --------------------------------------- ROWKEY | VARCHAR(STRING) (key) TIMESTAMP | BIGINT INX | VARCHAR(STRING) SAMPLENUM | VARCHAR(STRING) FREQUENCY | INTEGER HEARTRATE | INTEGER SENSORSTATUS | VARCHAR(STRING) RRLENGTH | INTEGER SAMPLES | VARCHAR(STRING) --------------------------------------- For runtime statistics and query details run: DESCRIBE EXTENDED ; ksql> SET 'auto.offset.reset' = 'earliest'; > Successfully changed local property 'auto.offset.reset' from 'earliest' to 'earliest'. ksql> SELECT * FROM ECG_STREAM EMIT CHANGES LIMIT 5; +------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+ |ROWKEY |TIMESTAMP |INX |SAMPLENUM |FREQUENCY |HEARTRATE |SENSORSTATUS |RRLENGTH |SAMPLES | +------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+ |715 |1579604534 |20 |715 |128 |108 |255 |554 |67 | |715 |1579604534 |24 |715 |128 |108 |255 |554 |134 | |715 |1579604534 |28 |715 |128 |108 |255 |554 |214 | |715 |1579604534 |32 |715 |128 |108 |255 |554 |315 | |715 |1579604534 |36 |715 |128 |108 |255 |554 |449 | Limit Reached Query terminated ``` One thing comparing my output with yours is that the `PRINT` output shows your keys as null, whereas the connector should be setting them. Perhaps something strange happened at ingest, or there are messages on the topic from earlier attempts. ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 22:01 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/11 "2021-02-24T22:01:27Z")
Hello, Thank you very much, I will definitely try with the 0.15 version. Do you suggest it as a more stable version in general? Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [24 February 2021 22:14 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/12 "2021-02-24T22:14:31Z")
The issue you reported is not one I’ve seen before - so either it’s a weird bug in the version you’ve got, or something’s up with your install. Either way, trying the new version should definitely help 🙂 ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [24 February 2021 22:24 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/13 "2021-02-24T22:24:50Z")
I understand, I will definitely try this version then: what I was using before was _confluentinc/cp-ksqldb-server:6.0.1_. Thank you very much for your help, I really appreciated it! Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [25 February 2021 06:37 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/14 "2021-02-25T06:37:34Z")
One more thing - could you post the splash screen from when you launch the CLI? ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [25 February 2021 11:06 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/15 "2021-02-25T11:06:57Z")
Hello, So, I have managed to make a few more tests and got ahold of the 0.15.0 version. With the same ‘yml code’, this were the results: - With the original cp-ksqldb-server:6.0.1 image, the error is as I have reported - With the ksqldb-server:0.15.0 image, that specific error disappears (I have another one with the construction of the kafka consumer, but I want to research on it). It does seem like that specific version is causing some issue, but it could also be my installation being messed up, as I am not very experienced yet. As you have asked, here is the splash screen of the CLI: ![immagine](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/56c1d18d338775a7499c92769ad033b597c59bbe.png) Best regards, ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [25 February 2021 11:20 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/16 "2021-02-25T11:20:44Z")
Ah yes, this is the problem ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/32b7a587b17c238263fca1a6a8898f40556db59a.png) It looks like you’ve mixed CLI and server versions, and that will cause the kind of behaviour you’ve been seeing. Can you reproduce the `DESCRIBE` problem if you also use 6.0.1 CLI (_not_ 0.15) ? ---
**Author:** ![Steno](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@Steno](https://forum.confluent.io/u/Steno)\ **Post date:** [25 February 2021 13:57 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/17 "2021-02-25T13:57:23Z")
Hello, Yes, this solved the issue, I did not thought that it was related to difference between server’s and cli’s versions. Thank you very much for your help @rmoff , you have been extremely helpful and kind! ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\ **Post date:** [4 March 2021 13:57 UTC](https://forum.confluent.io/t/describe-throws-failed-to-deserialise-object-error/1002/18 "2021-03-04T13:57:38Z")
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.