Scaling and Resource Types
Bytewax provides two ways to scale bees - replicas and sizes. Replicas allow you to scale bees horizontally (replicate the number of bees running), whereas the size allows you to scale vertically, increasing CPU and Memory available to a bee. The best practice is to scale horizontally wherever possible, but the size should be just enough to fit the runtime objects (model, batch etc.)
Bee replicas let you define how many bee processes are spawned. Because replicas will be spread across multiple servers, they also provide resiliency for physical crashes of a single underlying server. It is recommended to have at least 2 replicas at all times for redundancy and the default is 3.
Bee size provides the means of vertically scaling - that is, how much memory and CPU time is reserved for each Bee replica. This should be configured to a safe minimum required for the process running. It’s generally better to have more smaller bees than a few large ones. A fraction of a CPU core means that one physical CPU core can be shared between multiple processes.
The current available sizes are shown below. If you need something else for your particular swarm, please contact us.
- memory: 500MB
- cpu: 1/8 core
- memory: 1GB
- cpu: 1/4 core
- memory: 4GB
- cpu: 1
- memory: 8GB
- cpu: 2
We currently are supporting CPU Bees, if you require GPU Bees for your use case, please contact us.