bytewax.recovery#

Failure recovery.

Classes#

class RecoveryConfig(db_dir, backup_interval=None, snapshot_serde=None)#

Configuration settings for recovery.

Parameters:
  • db_dir (Path) – Local filesystem directory to search for recovery database partitions.

  • backup_interval (Optional[timedelta]) – Amount of system time to wait to permanently delete a state snapshot after it is no longer needed. You should set this to the interval at which you are backing up the recovery partitions off of the workers into archival storage (e.g. S3). Defaults to zero duration.

  • snapshot_serde (Optional[Serde]) – Format to use when encoding state snapshot objects in the recovery partitions. Defaults to JsonPickleSerde.

Initialization

property db_dir#
property backup_interval#
property snapshot_serde#

Functions#

init_db_dir(db_dir, count)#

Create and init a set of empty recovery partitions.

Parameters:
  • db_dir (Path) – Local directory to create partitions in.

  • count (int) – Number of partitions to create.

Join our community Slack channel

Need some help? Join our community!

If you have any trouble with the process or have ideas about how to improve this document, come talk to us in the #questions-answered Slack channel!

Join now