bytewax.connectors.kafka.serde

Serializers and deserializers for Kafka messages.

Classes

class PlainAvroSerializer(

schema: Union[str, Schema],

named_schemas: Optional[Dict] = None,

)

Bases:

• Serializer

Unframed Avro serializer. Encodes into raw Avro.

This is in comparison to confluent_kafka.schema_registry.avro.AvroSerializer which prepends magic bytes to the Avro payload which specify the schema ID. This serializer will not prepend those magic bytes. If downstream deserializers expect those magic bytes, use AvroSerializer instead.

Initialization

Init.

Parameters:

• schema – Selected schema to use.

• named_schemas – Other schemas the selected schema references. See documentation for fastavro._schema_py.parse_schema.

class PlainAvroDeserializer(

schema: Union[str, Schema],

named_schemas: Optional[Dict] = None,

)

Bases:

• Deserializer

Unframed Avro deserializer. Decodes from raw Avro.

Requires you to manually specify the schema to use.

This is in comparison to confluent_kafka.schema_registry.avro.AvroDeserializer which expects magic bytes in the output proceeding the actual Avro payload. This deserializer can not handle those bytes and will throw an exception. If upstream serializers are including magic bytes, use AvroDeserializer instead.

Initialization

Init.

Parameters:

• schema – Selected schema to use.

• named_schemas – Other schemas the selected schema references. See documentation for fastavro._schema_py.parse_schema.