Define a Swarm

What is a Swarm?

A Bytewax Swarm is a sequence of processing steps. Each individual processing step in a Swarm is what we call a Bee. In other words, a Swarm is a collection of Bees arranged into a topology or a directed acyclic graph (DAG).

A swarm is written in a declarative YAML format and here we will define the flow data takes through the swarm.

---
name: hello-world
bees:
  - name: hello
    type: python
    input:
      type: gateway
      name: hello-world
    spec:
      image: ghcr.io/bytewax/hello-world:v0.5.0
      file: hello.py
      bee: hello 
      
  - name: goodbye
    type: python
    input: 
      type: bee
      name: hello
    spec:
      image: ghcr.io/bytewax/hello-world:v0.5.0
      file: hello.py
      bee: goodbye

Let’s take a look at our Swarm definition. At the top of the swarm definition, we give our Swarm the name hello-world. Below that we have a list of Bees. The first of which is our Bee named hello. Our hello bee is a bee of type python and includes an image, a file, and a bee. The final Bee is the goodbye Bee, which is another Python Bee.

Gateway Input

In order for our first Bee to receive messages, we’ll need to configure it to listen to the gateway for messages, in our first bee definition, we configure the input type as gateway, and specify what message name we are listening for.

Defining our first Bee

To complete our hello Bee, we’ll give our Bee a name, and a type. This Bee is written in Python, which is why we have set the type parameter of this Bee to python.

Bee input

Each Bee in a Swarm needs to configure a way to receive messages. For our first bee, we set the type field of input to gateway. For our goodbye Bee input, we’ve set the type to bee, and the name to hello. This Bee will now listen for messages that are sent from the hello Bee.

Bee spec

A Bee spec is configurable information that is specific to the type of Bee. The hello and goodbye Bees are using the python Bee type and the spec is specific to it. The python Bee type has three required fields: file, image and bee. The file argument tells Bytewax where to find the file that this Bee is defined in. The image argument is the Docker image that contains the code and dependencies for this Bee, and finally, the bee argument tells Bytewax which function to call, but more on that in the next section.