Idea object

ECS Concepts – Part 7: Messaging

by Cory on May 13, 2018 11:38 PM

Messaging is cool, trust me. It’s gonna have some complicated template code though so hang tight.

If this feels like a digression because I haven’t gotten around to talking about systems yet, bear with me. I want to explain all the ingredients that the System class depends on before I put it all together. And besides, I don’t really have a fleshed out System class because I don’t really have a clue what to write… The development I do there is going to depend on adjustments I make after having creating some actually useful system examples.

Meet the Cast

My messaging getup requires a few classes and a bunch of moving parts. First is an interface I call Messageable. Every class that wants to send and/or receive messages needs to inherit from this and implement the virtual methods. The Messageable interface gives you templated functions that let you send and receive messages as well as register message handlers. By message handler, I mean a pointer to a function that contains code to perform specific actions when a message is received. In the code I define an alias called MessageHandlerT for this.

The Messageable interface also gives each inheritor a Mailbox instance member. This Mailbox class has two things—a queue to hold received messages and a mapping of message types to message handlers.

Lastly, there is the Message class. This class acts as the base class for anything that can be passed around as a message by Messageables and stored in their Mailboxes. Users who want to pass messages can instantiate new instances of existing message classes or define their own message child classes to pass around without modifying any of the other messaging classes. This is an important requirement I wanted for my messaging system, as I will explain in the next section.

Envisioned Usage

In my ECS, I currently have it coded so that every System instance is Messageable. The EntitySubscription component of every system is also Messageable. So is every Entity and even the Scene itself. Yes, that means every Entity, even if there are twenty thousand of them, has it’s own mailbox and when messages are broadcast, it goes to every single one of them. This sounds expensive as you would guess and I think it is but I haven’t done any testing to figure out how expensive. So far it’s not too bad but I’ll cross that bridge when I get to it… 

Let’s say we are making a GraphicalSystem. And that the scene to which this system belongs can receive input events from the operating system whenever the program window is resized. If I want the GraphicalSystem to automatically resize the viewport to match the new window size, I could hard-code this relationship by having the scene manipulate the system whenever this event comes up...

OR…!

I could have the scene broadcast a message and have the systems respond to it. This is much more flexible, allowing me to redefine behavior at a finer granularity AND letting me reuse the code for different kinds of events. If I want to go the messaging route, I define a WindowResizeMessage class that derives from Message and I code the scene to broadcast this message whenever it receives a window resize event. Our graphical system will receive the message, since it is registered with the scene. In this system’s initialization code, I made sure to install a message handler for WindowResizeMessage in which the function pointed to by the handler will take the camera and resize it with the integer height and width provided as members from the WindowResizeMessage reference.

In most other cases, the recipients probably won’t have a message handler installed for WindowResizeMessage so those recipients will simply discard the message because it isn’t relevant to them.

Note that the only restriction I have on message types is that it derives from a base Message class. It should be easy to derive from Message and create new message types without having to modify any of the message delivery code. If you’ve been using C++ long enough, this probably screams TEMPLATES to you and that’s exactly what I had to use. Now that the usage example is out there, let’s dig into the code.

The Messageable Interface, In Depth

The Messageable interface creates a Mailbox instance and has a bunch of functions for sending, receiving, and handling messages.

class Messageable {
public:
// constructors omitted
// copy constructors omitted (copy constructors are needed so that
// ObjectPools can properly hold classes that derive from Messageable
// and the copy constructor body needs to copy the Mailbox
// instance too)

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void receive_message(std::shared_ptr<MsgT> message);

protected:
template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr,
typename... Args
>
void send_message(Args&&... args);

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr,
typename... Args
>
void send_message_sync(Args&&... args);

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void install_message_handler(const Mailbox::MessageHandlerT<MsgT>& handler);

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr,
typename LambdaT
>
void install_message_handler(LambdaT handler);

private:
Mailbox mailbox_;

virtual void send_message_helper(MessagePtr message);

// customization hooks omitted
};

Ok, first of all, for those of you who read the code above in detail before skipping here, a few things:

  • Mailbox::MessageHandlerT is an alias inside the Mailbox class that evaluates to std::function<void(MsgT&)>, a function pointer that takes functions that don’t have a return value and take 1 parameter, a reference to a message child class instance. It is important to note that this alias has a template parameter called MsgT. The code looks like this:

    template <
    typename MsgT,
    typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
    >
    using MessageHandlerT = std::function<void(MsgT&)>;
  • MessagePtr is another alias in the Mailbox file that evaluates to std::shared_ptr<Message>. Shared pointers can be used polymorphically, so for instance, if I had a class ChildMessage that was a child of Message, an std::shared_ptr<ChildMessage> could be assigned to a MessagePtr. So you can treat them like raw pointers in that respect.

  • In most of the template function headers, you can see the following lines:

    template <
    typename MsgT,
    typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
    >
    The first argument is a plain vanilla template parameter. This is whatever message type you want to send across. This can be any class value and it is on purpose because I don’t want to have to force people to change code just to introduce new message types. Just like with Component, it should be extremely easy to add new message types without having to modify any message delivery code.

    The second line is an std::enable_if clause. If you recall from the Component post, I’m doing the same thing where I want to enforce the MsgT parameter to be a child class of Message. If this is not true, you will get an (unfortunately inscrutable) compiler error complaining about this.

The important parts of the code listing from the user point of view are the send_message() and install_message_handler() template member functions. Let me explain the member functions in order of importance from the message passing user point of view:

send_message()

This function lets users send messages asynchronously from a Messageable class. This function is what the user calls most of the time to have messages broadcast to other messageables. This function should be called anywhere inside the class definition when a message needs to be sent. Here’s an example:

Entity* Scene::create_entity(const std::string& id) {
Handle entity_handle;

// entity factory code omitted
// assume entity_handle = <factory code to create new entity>

// broadcast this message to everyone to let everyone know
// a new entity is here
this->send_message<EntityCreatedMessage>(entity_handle);
}

This will be by far the most commonly called message related thing that the user should concern themselves with.

send_message_sync()

This is the same as the previous call except that the messages delivered are marked synchronous so that recipients stuff the message into their Mailbox to handle later (in the next game loop update for my ECS). 

install_message_handler()

This is used to specify actions to take when certain messages are received. You install a message handler on a Messageable when you want that recipient to perform any kind of actions when a certain kind of message is received. The actions performed are specified via an anonymous lambda function or a pointer to an existing function or bound member function. This is probably the second part that a message passing user should be concerned with. Here’s a usage example:

void System::init(Game& game) {
// some shit here

this->install_message_handler<ComponentAddedMessage>([this](ComponentAddedMessage& msg) {
// when we receive a message that says a component was
// added to an entity write code here to update the
// filtered entities list in case we need to add/remove
// this entity
});

// some more shit here
}

So now that the message handler is installed, when this system above receives a message of type ComponentAddedMessage, it will run this function here. If a message type is received and there is no message handler that matches that message type, the recipient will ignore the message. You can install message handlers at any time inside the System code (this function is protected) but I recommend doing it in the system’s init() method so it’s all in one place.

I provided a lambda function here, as you can see from the cryptic [this]... syntax inside the function call parenthesis, but you can supply anything that can go into a function pointer. If you look at the code listing for the Messageable class earlier, you’ll notice there are two overloads of the message handler register function. The example with the lambda function here uses the second overload.

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void install_message_handler(const Mailbox::MessageHandlerT<MsgT>& handler);

Overload 1: This is used when provided with actual function pointers and stuff

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr,
typename LambdaT
>
void install_message_handler(LambdaT handler);

Overload 2: Lamba function signature. I’m counting on the fact that the static compiler checks will fail if you give a class for LambdaT that doesn’t have an operator() overload

If you supply a function pointer (say for instance, you want to run a function that has already been defined elsewhere, or a member function) the first overload will be used. I did this because C++ has this weird technicality where it cannot implicitly convert lambda functions into std::function for certain cases where you use template parameters in the lambda function header. 

send_message_helper()

The name of this function is less than descriptive or flashy but it’s extremely important. Maybe I should consider changing the name of this to something else… 

This function is a customizable hook that the messaging system integrator needs to implement in order to connect all the messageable things together. Basically, if you inherit from Messageable, you’ll need to implement this, which I think is a little messy of me especially since I didn’t name this in a way that suggests that it’s a customization hook. The contents of this function should essentially call receive_message (see below) on the messageables you want to send this message to. This is as specific as I want to get now, but if you want to know more details, I go over this in the “Integration into the ECS” section at the bottom.

This function should have been a pure virtual function but anything that can go in my object pool needs to be constructable, so instead this function will throw a runtime error unless it’s overridden by the inheritor.

receive_message()

The receive_message() function is called externally by whoever delivers a message to this class, usually by other Messageables’ send_message_helper functions. It contains code to either run a message handler on the provided message or stuff it in the Mailbox, depending on whether the message was sent asynchronously or not. In the ECS, the Scene class will call receive_message() on all its registered systems and entities to deliver the message when anyone sends a message out. Systems will also call receive_message() to pass along messages to their EntitySubscriptions.

handle_queued_messages()

This is a convenience function that clients can use. It iterates through the mailbox queue and executes message handlers on each before clearing the queue. There should be no need to mess with this unless you make a new kind of Messageable child class. Every system calls this once per update loop. Each scene also calls this once per update loops and it also loops through all active entities and calls handle_queued_messages() on each of them too.

Mailbox, a Close Examination

The Mailbox should not be touched directly by anything other than the Messageable interface. I made the Mailbox class responsible for doing two things. The first is most obvious—storing received messages to be processed later. The data structure to handle this is simple. I have a vector of Message shared pointers. The second task is waaay more complicated. When the messages are processed, there needs to be some mechanism to specify what actions to take. The data structure here I use is an std::map indexed by class type and containing function pointers called message handlers. You can see these two definitions in the bottom of the Mailbox code sample. Let me cut them out and paste them here for convenience:

std::vector<MessagePtr> messages_;
std::map<std::type_index, std::shared_ptr<HandlerEntry>> handlers_;

Below is an approximation of the Mailbox class. I’m saying that because this code won’t compile because of a few issues related to forward declarations and stuff. I deliberately wrote it this way because I think it’s easier to understand. If you want the compile-able version, you’ll need to look up the file in the git repo.

class Mailbox {
public:
template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
using MessageHandlerT = std::function<void(MsgT&)>;
using Messages = std::vector<MessagePtr>;

// constructors omitted

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void handle(const MessageHandler<MsgT>& handler);

template <
typename MsgT,
typename LambdaT
>
void handle(LambdaT handler);

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void process(std::shared_ptr<MsgT> message);

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
void enqueue(std::shared_ptr<MsgT> message);

void process_queue();

private:
// polymorphic base class for MessageHandlers
class HandlerEntry {
public:
virtual void operator()(Message& msg) {}
};

// template child class to “remember” message handler type
template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
class HandlerEntryImpl : public HandlerEntry {
public:
virtual void operator()(Message& msg) {
handler_.operator()(static_cast<MsgT&>(msg));
}

private:
MessageHandlerT<MsgT> handler_;
};

std::vector<MessagePtr> messages_;
std::map<std::type_index, std::shared_ptr<HandlerEntry>> handlers_;
};

Putting a Message in the Mailbox (Or Processing it Immediately)

A Messageable will use the mailbox to store and process messages. As I stated above when talking about send_message() and send_message_async(), the Messageable will call the mailbox’s process() message if the message is asynchronous and the mailbox’s enqueue() message if it is synchronous. The process function looks up the message handler based on the message’s class type and executes it, providing a reference to the message instance as a parameter to the message handler.

For asynchronous messages, the Messageable will want to store it in the mailbox queue structure for later, which is what enqueue() does. When it comes time to handle these deferred messages, the Messageable can call process_queue(), which will iterate through the entire queue and call process() on all of them and then clear the queue. This is exactly what the process_queued_messages() in the Messageable interface does.

Configuring the Mailbox with Message Handlers

Now comes the hard part (to understand, that is. The user shouldn’t care how this works, only that it works). When a Messageable wants to put a new message handler into the mailbox, it must call the handle() function. This function takes a message handler function pointer and stuffs it into the mailbox’s handler map. The handler map is indexed using std::type_index and value for this std::type_index is taken from the MsgT parameter from the message handler signature. There’s a problem though. It seems like it would be easy to store all these handlers in the same map structure, but if you tried to implement your own, you’d quickly run into a problem. Fortunately, there is a solution to it!

Type Erasure for Storing Message Handlers in the Mailbox

Remember my second post about Entities and how they store Components? I brought up type erasure and said that it would show up in a more complicated implementation later. Well that time is now.

The Mailbox class happens to use the same std::type_index mechanism that the entities use to store components. However, this time I’m trying to use function pointers. The thing about function pointers is that two functions pointers are the same type only if their return type and every single argument type is the same. That’s a problem here because the message handler function types are parameterized with MsgT, which can be any child class of Message. Since the argument to all of these message handlers can be different, I can’t assume they will be the same type and that means I can’t just store them in the same map as is.

template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
using MessageHandlerT = std::function<void(MsgT&)>;

I could change MsgT in the alias above to just Message BUT I specifically don’t want that. In the body of that function pointer, I want the exact child type of the message to be accessible. That is so when I’m writing a message handler, I don’t have to do a boilerplate dynamic cast or have to worry about casting when storing/retrieving these things.

Take this code example—if message handler uses the Message base class like so, it would be easier to implement the Mailbox but things would be messier for the user:

using MessageHandlerT = std::function<void(Message&)>;

// elsewhere
handle([this](Message& msg) {
// user has to know to do this
WindowResizeMessage& wrm = dynamic_cast<WindowResizeMessage msg>(msg);

if (!wrm) {
return;
}

// not accessible from msg without doing cast to wrm
this->set_window(wrm.width, wrm.height);
});

In this message handler, the user is only given a base class pointer and has to do the dynamic cast themselves, make sure it was successful, and then it can access the child class members. This is inconvenient, error prone, and repetitive code. This is the perfect kind of thing that the messaging framework should take care of. 

Take another look at the type definition for the message handler map:

std::map<std::type_index, std::shared_ptr<HandlerEntry>> handlers_;

This structure is a standard C++ map that uses std::type_index as the index and a shared pointer to a class called HandlerEntry. This HandlerEntry class is a private, nested class inside the Mailbox that contains a MessageHandlerT function pointer. So I lied when I said the map connects type indices to message handlers… There’s actually an extra step in between for HandlerEntry, which then contains the message handlers.

// polymorphic base class for MessageHandlers
class HandlerEntry {
public:
virtual void operator()(Message& msg) {}
};

There’s nothing in this definition besides the () operator. Note that it takes a message base reference. Notice also that there’s another nested, private class:

// template child class to “remember” message handler type
template <
typename MsgT,
typename std::enable_if<std::is_base_of<Message, MsgT>::value>::type* = nullptr
>
class HandlerEntryImpl : public HandlerEntry {
public:
virtual void operator()(Message& msg) {
handler_.operator()(static_cast<MsgT&>(msg));
}

private:
MessageHandlerT<MsgT> handler_;
};

Lookie here! This class inherits from HandlerEntry but it is a template class! This is how we’re going to store all these different function pointer types polymorphically. This is the type erasure part. In the code example above, the () operator is implemented as an override. Inside the operator, we do a static downcast of the provided message reference to MsgT. Note here that we used an enable_if to make sure MsgT is a child class of Message. This time, given a MsgT, we can supply the correct child message type to the function pointer because it is “remembered” by the template type of the HandlerEntryImpl class. That is why the HandlerEntry class was defined—to act as an “envelope” to the MessageHandlerT class and uses templates to store the child class type information. And lo! We can still use this HandlerEntryImpl class polymorphically because it inherits from HandlerEntry. This is like having a cake and eating it too, in my opinion.

But wait, Cory, what the fuck I still don’t understand, you just do a static cast to MsgT in HandlerEntryImpl’s () operator without knowing what the actual type of the msg parameter is? Haha, yes, that’s you. You are so naive and innocent, my child. Ok, let me tell you. It is true, that whoever calls the () operator on these HandlerEntry classes has to carefully supply the message reference as a parameter to this call such that the static cast will never fail. HOWEVER, REMEMBER THIS:

std::map<std::type_index, std::shared_ptr<HandlerEntry>> handlers_;

The handler entries of this map are only going to be touched by the Mailbox internals. So I carefully make sure that the MsgT supplied to the HandlerEntryImpl that goes in the second slot of this map always matches the std::type_index of MsgT in the corresponding index. So the static cast should be true all the time, given the way I’ve carefully laid out my code.

Also another note! This whole shebang works for me because the function pointers have void as a return type and the () operator does not return anything (and especially not a function pointer reference). If I wanted to get EntryHandler to return a reference to the message handlers so I could do something other than call the () operator on it, I would need to figure out a whole ‘nother thing to take care of that because that would mean returning different variable types from the same function. But I don’t need to worry about that, so let’s move on and maybe I can actually start the “game making” part of making a game one of these days.

Isn’t this cool??? Well I thought it was cool… I was excited because it seemed really cool when I figured it out. HMPH

The Message Class, and Some Example Children

Alright so all the hard stuff is out of the way. The last piece of the puzzle is what the actual messages look like.

class Message {
public:
// constructors omitted

virtual std::type_index type() const {
return std::type_index(typeid(*this));
}

void async(bool async) { this->async_ = async; }
bool async() const { return this->async_; }

private:
bool async_;
std::string id_;
};

This definition is very minimal. I don’t need to force much standardization here. I just have a string identifier for users to tag messages with and a boolean to set whether or not this message is asynchronous.

When users inherit from this class should add any members needed to capture useful information to be passed along. For example, if we go with WindowResizeMessage again, it would look something like this:

class WindowResizeMessage : public Message {
public:
WindowResizeMessage(int width, int height)
: Message(“WindowResizeMessage”)
, width(width)
, height(height)
{}

WindowResizeMessage(sf::Vector2f size)
: Message(“WindowResizeMessage”)
, width(size.x)
, height(size.y)
{}

int width;
int height;
};

I made the width and height public members because I’m treating this message like a struct where anyone can access the members freely. I also added in some convenient constructors.

Integration into the ECS

Ok so now that you know how the messaging system works in detail (and if not, you can re-read it as much as you want), let me describe how it’s integrated into my ECS.

The lifecycle of a message starts at the user’s perspective, when the user decides that they want to communicate with a system or entity or whatever that is outside the class they are currently coding in. The user calls a variant of the send_message functions, creating a message instance and populating it’s data members with useful information in the process. It is only possible to send messages from inside a Messageable object, which means this message sending invocation must be in Scene code, or Entity, System, and EntitySubscription code.

From there, the send_message functions call the Messageable interface’s send_message_helper function which, if you recall, is a virtual function that must be defined by any inheritors. For my ECS, the objective of this helper function is to pass this message to the Scene somehow so that it can broadcast the message to all the Messageables inside of it. So for anything that isn’t Scene, the contents of this function is basically to obtain a reference to the parent scene somehow (usually it’s stored in the class from the constructor), and then call receive_message() on the scene.

Once it’s at the scene, there is custom receive hooks that are written so that the Scene will now send the message to all of the entities and systems in the scene and then process the message for itself.

There’s no protection against infinite loops here. If you write a function to send a message inside the handler that handles the same message type, you could crash the engine. I think to solve this issue, I’ll need to write some of the extensions discussed in the next section…

For integration, that’s really all there is to it.

Possible Extensions

Compared to the actual design of the messaging system, the way it’s integrated into the ECS seems a bit oddly simple. I could come back and fuck with it if I decide that I need more powerful capabilities. I’m already thinking about two possible features that might come in handy.

  1. The first one is the fact that all messages are automatically broadcasted to every mailbox. This seems kind of wasteful I think where you have a lot of things shotgunning messages to 90% of things that don’t need it every frame. It would be nice to have an addition mode to send messages to a very specific recipient (or group of recipients?) if possible.

    I don’t know exactly how I would like to do it, but my thoughts on it so far would be to give each Messageable a unique ID to use for message sending. The entities already have a unique string id enforced, so I could use that, but the System, EntitySubscription, and Scene have no guarantees about their string id uniqueness.

    Then again, maybe we only need to support specific recipient addresses for messages between Entities? For example, maybe I want to send a message from a treasure chest entity to only the character entity. It would be wasteful to just broadcast the message. If this is the case, then Entity handle might also be a good “mailbox address” to use.

    Also, messages to and from Entities to Systems is something I need to do as well...
  2. There have already been a few cases where I want to have maybe a request/response protocol type thing to let systems “talk” to each other in more complicated ways. Currently, the messaging system allows things to send messages, but if you want to get a message from something else in response, it is difficult to link the sent message with a subsequent response.

    There are plenty of reasons you’d want to do that. For instance, maybe you’d want to use messages to do queries on systems or something. For example, if I had a scene graph system that contained spatial relationship information of my entities, I could use a messaging protocol to query the system for children or parents of a given node, etc.

    I feel like the first point I made about specific destination addresses is probably a prerequisite for this. We’d need two Messageables to have a point to point communication with each other. And then I’d probably need to modify the Message class to contain packet header metadata for requests and responses or something. I could even copy an existing communication protocol like UDP.

    I’m not going to worry about actually implementing this until I absolutely need it because I think I can get by with just broadcasting everything. I might also consider doing some of these modifications if performance becomes an issue.


This Thought is part of Game Programming

Game programming general topic. I eventually hope to split this into separate ideas exclusively about specific games that I make.

back to the

top