Define a Swarm
What is a Swarm?
A Bytewax Swarm is a sequence of processing steps. Each individual processing step in a Swarm is what we call a Bee. In other words, a Swarm is a collection of Bees arranged into a topology or a directed acyclic graph (DAG).
A swarm is written in a declarative YAML format and here we will define the flow data takes through the swarm.
--- name: hello-world bees: - name: hello type: python input: type: gateway name: hello-world spec: image: ghcr.io/bytewax/hello-world:v0.5.0 file: hello.py bee: hello - name: goodbye type: python input: type: bee name: hello spec: image: ghcr.io/bytewax/hello-world:v0.5.0 file: hello.py bee: goodbye
Let’s take a look at our Swarm definition. At the top of the swarm definition, we give our Swarm the name hello-world. Below that we have a list of Bees. The first of which is our Bee named
hello bee is a bee of type
python and includes an image, a file, and a bee. The final Bee is the goodbye Bee, which is another Python Bee.
In order for our first Bee to receive messages, we’ll need to configure it to listen to the gateway for messages, in our first bee definition, we configure the
gateway, and specify what message
name we are listening for.
Defining our first Bee
To complete our
hello Bee, we’ll give our Bee a name, and a type. This Bee is written in Python, which is why we have set the type parameter of this Bee to
Each Bee in a Swarm needs to configure a way to receive messages. For our first bee, we set the
type field of
gateway. For our
goodbye Bee input, we’ve set the
bee, and the
hello. This Bee will now listen for messages that are sent from the
A Bee spec is configurable information that is specific to the type of Bee. The
goodbye Bees are using the
type and the spec is specific to it. The
type has three required fields:
file argument tells Bytewax where to find the file that this Bee is defined in. The
image argument is the Docker image that contains the code and dependencies for this Bee, and finally, the
bee argument tells Bytewax which function to call, but more on that in the next section.