Event based synchronization of threads with main game loop

Typically, Game Engines want their telemetry data sent in a side thread which runs independently (asynchronously) from the main game loop. The telemetry thread packages up data as it comes in and executes a send via HTTP or websockets every 30 seconds or so. This usually works fine when you’re using Telemetry for post analysis of data. But if you want to do more real time processing of the Telemetry information coming in ( to provide in-game recommendation, load balance servers etc.), the data needs to be sent much more frequently – even lets say every 100 mili-seconds or so.

However, we had a recent Client who wanted to fire the Telemetry send function every Frame.

At a high level, a game loop consists of an infinite loop, which processes user input, updates the state of game objects/players and renders the updated state on the display screen for the user. It looks something like this:

while(true)
{
    ProcessInput();  // Input sources include keyboard, mouse, xbox controllers etc.
    Update(); // Update the state of various game objects based on user input
    Render();  // Render the updated state to the screen		
}

The game loop always keeps spinning without ever blocking for user input. Each execution of a game loop is called a Frame. An in-depth discussion of “Game Loop” and “Frames” is outside the scope of this post – please refer to this post if you’re interested in more details.

So, since the Telemetry processing and Game loop runs in separate threads, we needed to let one thread know that one execution of a game loop (A Frame) has happened such that it can start sending telemetry data. Once the Telemetry data has been sent, the Telemetry thread needs to let the Game loop know that it’s ready for sending the next batch and is waiting for the game loop to set the signal to send telemetry data again.

There are two ways we can achieve the above in code:

1. Set an atomic flag at the end of main loop – use a spinlock with sleep in Telemetry processing thread to check the variable and fire when the variable is set. After firing, reset the variable state for the main lop to modify this again.
2. Use a HANDLE based event: Set an event in the main loop and wait for the event to be set in Telemetry thread. Once the event is set, fire the cell update and then reset the event.

Option # 2 is preferable because it’ll consume less CPU cycles than the spinlock based solution. Also, if we set the spinlock to sleep for a while, we’ll incur additional thread swaps and might miss the exact timing of when to check for the signal.

So, here’s what we need to do to implement this in code:

1. Initialize the event right before initializing the Telemetry thread and before entering the main game loop
2. Get the event in the main game loop and set it at the end of each loop
3. Get the event in telemetry processing thread – fire send() if event is set and then reset the event.

The code below with annotations achieves just that.

// EventSynchronization.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <Windows.h>
#include <assert.h>
#include <thread>
#include <atomic>
#include <iostream>

using namespace std;

void SendTelemetryDataToAzure(void);

atomic<bool> shouldTerminate = false;

int main()
{
  //Create a Manual Reset Event
  HANDLE hEvent = CreateEvent(NULL, true, false, L"FIRE_TELEMETRY");

  //Create the Telemetry Processing Thread
  thread telemetryEndpointThread(SendTelemetryDataToAzure);

  //Simulate the Game Loop
  while (!shouldTerminate)
  {
    // ProcessUserInput() - Get input from game controller

    //Update() - Modify state of game objects based on user input

    //Signal Telemetry Thread
    // Note that this will be called per frame, which will ensure that we're not pumping telemetry data any faster 
    // than once per frame. However, the sending telemetry to azure can take upto 200 ms - which means that we might call
    // SetEvent() multiple times before a ResetEvent() is called by Telemetry thread. This is okay because calling SetEvent()
    // on an event that's already set has no effect.
    SetEvent(hEvent);

    //Test case - Game loop sleeps longer than Telemetry thread
    cout << "\nMain Thread is Rendering Game objects\n" << endl;
    Sleep(2000);

    //Render()
  }

  //Wait for any Telemetry data flush to happen
  telemetryEndpointThread.join();

    return 0;
}

void SendTelemetryDataToAzure()
{
  //Get the event - the event should have been created in main before telemetry thread initialization
  HANDLE hEvent = OpenEvent(EVENT_ALL_ACCESS, false, L"FIRE_TELEMETRY");
  
  if (!hEvent) 
  { 
    assert(false); 
  }

  //Loop to simulate constant calling of TelemetryProcessor::SendToAzure()

  for (int i = 0; i < 5; i++)
  {
    //Wait for the event to be set
    WaitForSingleObject(hEvent, INFINITE);

    //once Main loop signals us - send the Telemetry Event
    cout << "Signalled by Main Loop - sending event # "<< i << endl;

    //Simulate the time required to send the event over to Azure Telemetry Processing service
    Sleep(174);

    //Now reset the event - so that Main game loop can signal us in the next available frame
    ResetEvent(hEvent);
  }

  cout << "\nAll Telemetry Data has been sent ! We're done here." << endl;

  //Signal the main thread(game loop ) to terminate
  shouldTerminate = true;

}

The output from running the program is below:

Main Thread is Rendering Game objects
Signalled by Main Loop - sending event # 0


Main Thread is Rendering Game objects
Signalled by Main Loop - sending event # 1

Main Thread is Rendering Game objects
Signalled by Main Loop - sending event # 2

Main Thread is Rendering Game objects
Signalled by Main Loop - sending event # 3

Main Thread is Rendering Game objects
Signalled by Main Loop - sending event # 4

All Telemetry Data has been sent ! We're done here.

Notice that the Telemetry thread fires the send operation exactly 5 times, same as the number of times the game renders the screen, i.e., completes 5 frames.

Interested in learning more?

Game Programming Algorithms & Techniques gives a fantastic primer on platform agnostics game development and prepares you with the basic tools needed for Game Development. Once you have these tools and looking to create serious networked gaming applications, you can refer to Multiplayer Game Programming: Architecting Networked Games (Game Design) for an in-depth treatise on authoring real life online games.

Please share if you liked the article. 🙂