bytewax.connectors.kafka.registry

Schema registries connection.

This module offers two preconfigured schema registries:

• ConfluentSchemaRegistry

• RedpandaSchemaRegistry

Subclass “SchemaRegistry” to implement support for your any registry.

Data

logger

‘getLogger(…)’

Classes

class SchemaRef

Info used to retrieve a schema from a schema registry.

Specify the subject and optionally version. If no version is specified, it defaults to the latest schema.

subject: str

version: Optional[int]

class ConfluentSchemaRegistry(

client: SchemaRegistryClient,

)

Confluent’s schema registry for Kafka’s input and output connectors.

Initialization

Init.

Args: client: Configured confluent_kafka.schema_registry.SchemaRegistryClient

serializer(

schema_ref: Union[int, SchemaRef],

) → SchemaSerializer[Dict, bytes]

Confluent avro serializer.

Specify either the schema_id or a SchemaRef instance.

deserializer() → SchemaDeserializer[bytewax.connectors.kafka._types.MaybeStrBytes, fastavro.types.AvroMessage]

Confluent avro deserializer.

schema_ref is not needed since Confluent cloud adds the schema_id as metadata in each message. The client will automatically fetch and cache the schemas needed.

class RedpandaSchemaRegistry(base_url: str = 'http://localhost:8081')

Redpanda’s schema registry client.

Initialization

Init.

Args: base_url: Base url of redpanda’s schema registry instance

serializer(

schema_ref: Union[int, SchemaRef],

) → SchemaSerializer[Dict, bytes]

Fastavro serializer.

Specify either the schema_id or a SchemaRef instance.

deserializer(

schema_ref: Union[int, SchemaRef],

) → SchemaDeserializer[bytewax.connectors.kafka._types.MaybeStrBytes, fastavro.types.AvroMessage]

Fastavro deserializer.

Specify either the schema_id or a SchemaRef instance.