bytewax.connectors.kafka.registry#

Schema registries connection.

This module offers two preconfigured schema registries:

  • ConfluentSchemaRegistry

  • RedpandaSchemaRegistry

Subclass “SchemaRegistry” to implement support for your any registry.

Data#

logger#

‘getLogger(…)’

Classes#

class SchemaRef#

Info used to retrieve a schema from a schema registry.

Specify the subject and optionally version. If no version is specified, it defaults to the latest schema.

subject: str#
version: Optional[int]#
class ConfluentSchemaRegistry(
client: SchemaRegistryClient,
)#

Confluent’s schema registry for Kafka’s input and output connectors.

Initialization

Init.

Args: client: Configured confluent_kafka.schema_registry.SchemaRegistryClient

serializer(
schema_ref: Union[int, SchemaRef],
) SchemaSerializer[Dict, bytes]#

Confluent avro serializer.

Specify either the schema_id or a SchemaRef instance.

deserializer() SchemaDeserializer[bytewax.connectors.kafka._types.MaybeStrBytes, fastavro.types.AvroMessage]#

Confluent avro deserializer.

schema_ref is not needed since Confluent cloud adds the schema_id as metadata in each message. The client will automatically fetch and cache the schemas needed.

class RedpandaSchemaRegistry(base_url: str = 'http://localhost:8081')#

Redpanda’s schema registry client.

Initialization

Init.

Args: base_url: Base url of redpanda’s schema registry instance

serializer(
schema_ref: Union[int, SchemaRef],
) SchemaSerializer[Dict, bytes]#

Fastavro serializer.

Specify either the schema_id or a SchemaRef instance.

deserializer(
schema_ref: Union[int, SchemaRef],
) SchemaDeserializer[bytewax.connectors.kafka._types.MaybeStrBytes, fastavro.types.AvroMessage]#

Fastavro deserializer.

Specify either the schema_id or a SchemaRef instance.

Join our community Slack channel

Need some help? Join our community!

If you have any trouble with the process or have ideas about how to improve this document, come talk to us in the #questions-answered Slack channel!

Join now