Simple Routing for Elastic Beanstalk Worker tier

Elastic Beanstalk offers both a Web tier and a Worker tier. This allows developers to build reasonably complex applications without having to maintain moving pieces. Offloading heavy-duty workloads to the worker in order to keep the web tier responsive is as easy as putting a message on a queue.

One annoyance that I have with Beanstalk is that there is no way to direct a message to a specific endpoint, hence leaving a single endpoint the responsibility of distributing the messages to all their handlers and potentially leading to brittle code. But it doesn’t have to be that way.

Implementation

SQS messages have attributes, attributes can be set by the sender and are read by the receiver. The idea is to use a known attribute to attach routing metadata to the message.

Constants

Constants are the base of any decently built C# application. I did not want to depart from to this rule and hence added some constants:

public static class RoutingConstants
{
    public const string HeaderName = "Task";
    public const string HeaderType = "String";
}

These constansts will be used to add routing metadata to the SQS message.

We can then define our routes via some more constants:

public static class WorkerConstants
{
    public const string DoSomeWorkTaskName = "do-some-work";
    public const string DoSomeOtherWorkTaskName = "do-some-other-work";
}

Note: those two class will have to be referenced by the sender and the Worker.

Sending the message

The sender will most likely be the Web tier but it could be any system being able to send a message to a SQS queue.

var sendMessageRequest = new SendMessageRequest();
// Abbreviated: set properties on sendMessageRequest, such as the MessageBody and the QueueUrl

// We're using RoutingConstants.HeaderName as the MessageAttribute key
// and WorkerConstants.DoSomeWorkTaskName as the MessageAttribute value
sendMessageRequest.MessageAttributes.Add(
    RoutingConstants.HeaderName,
    new MessageAttributeValue {StringValue = WorkerConstants.DoSomeWorkTaskName, DataType = RoutingConstants.HeaderType});

// Abbreviated: send the message

Middleware

In the Worker, the routing is implemented via a Middleware:

public static class HeaderRoutingMiddleware
{
    // Elastic Beanstalk prefixes the SQS messages properties' name with "X-Aws-Sqsd-Attr-"
    private static readonly string TaskHeaderName = $"X-Aws-Sqsd-Attr-{RoutingConstants.HeaderName}";

    public Task Invoke(HttpContext context)
    {
        // We get the value of the routing header
        StringValues task = context.Request.Headers[TaskHeaderName];

        // And set it as the path
        context.Request.Path = $"/{task.Single()}";

        return _next(context);
    }
}

Note: don’t forget to add HeaderRoutingMiddleware to the IApplicationBuilder.

Controller

The last piece of the puzzle is defining the expected route on the Controller:

// This is important, we do not want a prefix in front of the action's route
[Route("")]
public class SomeController : Controller
{
  // The route has to match the value given to the MessageAttribute
  [HttpPost(WorkerConstants.DoSomeWorkTaskName)]
  public async Task<IActionResult> SomeMethod(SomeModel model)
  {
      // Abbreviated for clarity
  }
}

Simple routing

I used Simple Routing in production over the last few months and am now confident that it does what it’s supposed to do. This is why I decided to release it under a MIT license to allow others to benefit from my work.

Simple Routing is available:

• On NuGet as the package BeanstalkWorker.SimpleRouting
• A GitHub release
• As source on Github
  • The implementation is so simple that you can just copy the classes into your own solution if that works better for you

Demo

The Simple Routing solution contains a SampleWeb app, you can either:

• Send “work” - Send/Work
• Send “nothing” - Send/Nothing

Send messages

GET http://localhost:5000/Send/Work HTTP/1.1
Host: localhost:5000

GET http://localhost:5000/Send/Nothing HTTP/1.1
Host: localhost:5000

Peek at the messages

Now let’s look at the messages in the SQS queue:

• The Work message

• The Nothing message

Handle the messages

Launch the SampleWorker app. When running in ElasticBeanstalk the Sqsd daemon reads SQS messages from the SQS queue and POST the content to your Worker. But we’re running the Worker on our machine and the Sqsd daemon is not available. This is why I wrote Beanstalk Seeder.

Beanstalk Seeder emulates the SQS Daemon surrounding an Elastic Beanstalk Worker Tier so that you can replicate the interaction between a Web Tier and a Worker Tier on your machine.

Handling the Work message

• Beanstalk Seeder

• Worker

Handling the Nothing message

• Beanstalk Seeder

• Worker

I wrote a detailed guide in the GitHub repository. Give it a try and let me know if it works for you.