Custom Game Engine/Editor

About

This is a C++ game engine made from scratch in Vulkan. It's only a couple months in development, but already features the graphics fundamentals, such as a skybox, directional and point lighting, and texture sampling.

The engine follows the Entity-Component-System (ECS) model, which focuses on separating data (in Components) from behavior (in Systems). Entities are the containers for Components, and are usually only a single integer value (like an index into a global Entity list).

The graphics system takes advantage of bindless arrays of resources, so each batch of objects in the scene only needs indices into a set of global arrays instead of passing pointers around every frame. This is mostly more efficient than binding per-object data every frame.

The physics system supports basic collision detection and resolution for any convex shape. There is more work to be done to stabilize a physics object's motion, especially when affected by friction.

Recent Development

The most recent progress has been in the editor application, the frontend game development environment that runs on the engine. The engine itself is built as a DLL and the editor includes it as a dependency.

The application itself uses GTK4 for retained-mode GUI and interfaces with the engine in some complicated ways, as detailed below.

Choosing a framework for something as important as a frontend for a game engine isn't an easy task. I learned that GTK4 doesn't have native support for rendering Vulkan frames from another application, which meant that a workaround was required, since the main editor viewport needed to use the engine's Vulkan renderer to accurately preview the game world.

GTK does support OpenGL for this purpose, however. If Vulkan is set to export some of its rendering resources on the engine side after startup and OpenGL is setup to import those resources on the editor side, they can be used in tandem as a translation layer from Vulkan to OpenGL, and so Vulkan-rendered frames can be presented in a Gtk::GLArea. The code for doing this is given in the following snippets.

First, on the Vulkan side, we create the objects (image memory and synchronization objects) that need to be exported to OpenGL:

void GraphicsSystem::CreateExternalRenderSyncObjects()
{
// Create Vulkan semaphores with external handle support
VkExportSemaphoreCreateInfo exportInfo = {};
exportInfo.sType = VK_STRUCTURE_TYPE_EXPORT_SEMAPHORE_CREATE_INFO;
#ifdef _WIN32
exportInfo.handleTypes = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_BIT;
#else
exportInfo.handleTypes = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT;
#endif

VkSemaphoreCreateInfo semaphoreInfo = {};
semaphoreInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO;
semaphoreInfo.pNext = &exportInfo;

vkCreateSemaphore(device, &semaphoreInfo, nullptr, &externalRenderCompleteSemaphore);
vkCreateSemaphore(device, &semaphoreInfo, nullptr, &externalRenderReleaseSemaphore);

// Export semaphore handles
#ifdef _WIN32
VkSemaphoreGetWin32HandleInfoKHR handleInfo = {};
handleInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_GET_WIN32_HANDLE_INFO_KHR;
handleInfo.handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_BIT;

handleInfo.semaphore = Instance->externalRenderCompleteSemaphore;
vkGetSemaphoreWin32HandleKHR(Instance->device, &handleInfo, &externalRenderCompleteSemaphoreHandle);

handleInfo.semaphore = Instance->externalRenderReleaseSemaphore;
vkGetSemaphoreWin32HandleKHR(Instance->device, &handleInfo, &externalRenderReleaseSemaphoreHandle);
#else
VkSemaphoreGetFdInfoKHR handleInfo = {};
handleInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_GET_FD_INFO_KHR;
handleInfo.handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT;

handleInfo.semaphore = Instance->externalRenderCompleteSemaphore;
vkGetSemaphoreFdKHR(Instance->device, &handleInfo, &externalRenderCompleteSemaphoreFd);

handleInfo.semaphore = Instance->externalRenderReleaseSemaphore;
vkGetSemaphoreFdKHR(Instance->device, &handleInfo, &externalRenderReleaseSemaphoreFd);
#endif
}
void GraphicsSystem::CreateExternalRenderImage()
{
// Create Vulkan image with external memory
VkExternalMemoryImageCreateInfo externalInfo = {};
externalInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO;
#ifdef _WIN32
externalInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT; // Windows
#else
externalInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; // Linux
#endif

VkImageCreateInfo imageInfo = {};
imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
imageInfo.imageType = VK_IMAGE_TYPE_2D;
imageInfo.format = EXTERNAL_RENDER_IMAGE_FORMAT;
imageInfo.extent = { externalRenderImageExtent.width, externalRenderImageExtent.height, 1 };
imageInfo.mipLevels = 1;
imageInfo.arrayLayers = 1;
imageInfo.samples = VK_SAMPLE_COUNT_1_BIT;
imageInfo.usage = VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_SAMPLED_BIT;
imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
imageInfo.pNext = &externalInfo;

vkCreateImage(device, &imageInfo, nullptr, &externalRenderImage);

// Allocate exportable memory
VkMemoryRequirements memReqs;
vkGetImageMemoryRequirements(device, externalRenderImage, &memReqs);
externalRenderMemorySize = memReqs.size;

VkExportMemoryAllocateInfo exportInfo = {};
exportInfo.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO;
#ifdef _WIN32
exportInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT; // Windows
#else
exportInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; // Linux
#endif

VkMemoryAllocateInfo allocInfo = {};
allocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
allocInfo.allocationSize = memReqs.size;
allocInfo.memoryTypeIndex = FindMemoryType(memReqs.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
allocInfo.pNext = &exportInfo;

vkAllocateMemory(device, &allocInfo, nullptr, &externalRenderMemory);
vkBindImageMemory(device, externalRenderImage, externalRenderMemory, 0);

// Export memory handle
#ifdef _WIN32
VkMemoryGetWin32HandleInfoKHR handleInfo = {};
handleInfo.sType = VK_STRUCTURE_TYPE_MEMORY_GET_WIN32_HANDLE_INFO_KHR;

handleInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT;
handleInfo.memory = Instance->externalRenderMemory;

vkGetMemoryWin32HandleKHR(Instance->device, &handleInfo, &externalRenderMemoryHandle);
#else
VkMemoryGetFdInfoKHR handleInfo = {};
handleInfo.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR;

handleInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT;
handleInfo.memory = Instance->externalRenderMemory;

vkGetMemoryFdKHR(Instance->device, &handleInfo, &externalRenderMemoryFd);
#endif
}

Then, after initialization completes, the OpenGL-based GUI imports the memory and semaphores:

void VulkanViewport::ImportVulkanMemory()
{
size_t memorySize = GraphicsSystem::GetExternalRenderMemorySize();
#ifdef _WIN32
// Get the exported memory handle from Vulkan engine
void* memoryHandle = GraphicsSystem::GetExternalRenderMemoryHandle();

// Create OpenGL memory object
glCreateMemoryObjectsEXT(1, &memoryObject);
glImportMemoryWin32HandleEXT(memoryObject, memorySize, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, memoryHandle);
#else
// Get the exported memory handle from Vulkan engine
int memoryFd = GraphicsSystem::GetExternalRenderMemoryHandle();

// Create OpenGL memory object
glCreateMemoryObjectsEXT(1, &memoryObject);
glImportMemoryFdEXT(memoryObject, memorySize, GL_HANDLE_TYPE_OPAQUE_FD_EXT, memoryFd);
#endif

// Create OpenGL texture from imported memory
glCreateTextures(GL_TEXTURE_2D, 1, &sharedTexture);
glTextureStorageMem2DEXT(sharedTexture, 1, GL_RGBA8,
viewportWidth, viewportHeight,
memoryObject, 0);

// Set texture parameters
glTextureParameteri(sharedTexture, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTextureParameteri(sharedTexture, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTextureParameteri(sharedTexture, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTextureParameteri(sharedTexture, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}
void VulkanViewport::ImportVulkanSemaphores()
{
#ifdef _WIN32
// Get exported semaphore handles from Vulkan
void* renderCompleteHandle = GraphicsSystem::GetExternalRenderCompleteSemaphoreHandle();
void* renderReleaseHandle = GraphicsSystem::GetExternalRenderReleaseSemaphoreHandle();

// Import semaphores into OpenGL
glGenSemaphoresEXT(1, &waitSemaphore);
glGenSemaphoresEXT(1, &signalSemaphore);

glImportSemaphoreWin32HandleEXT(waitSemaphore, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, renderCompleteHandle);
glImportSemaphoreWin32HandleEXT(signalSemaphore, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, renderReleaseHandle);
#else
// Get exported semaphore handles from Vulkan
int renderCompleteFd = GraphicsSystem::GetExternalRenderCompleteSemaphoreHandle();
int renderReleaseFd = GraphicsSystem::GetExternalRenderReleaseSemaphoreHandle();

// Import semaphores into OpenGL
glGenSemaphoresEXT(1, &waitSemaphore);
glGenSemaphoresEXT(1, &signalSemaphore);

glImportSemaphoreFdEXT(waitSemaphore, GL_HANDLE_TYPE_OPAQUE_FD_EXT, renderCompleteFd);
glImportSemaphoreFdEXT(signalSemaphore, GL_HANDLE_TYPE_OPAQUE_FD_EXT, renderReleaseFd);
#endif
}

You'll notice that both Windows and Linux are supported. Because GTK is mostly intended for Linux development, there are some GUI development tools that are exclusive to the Linux platform, so sometimes it helps to run the editor in a Linux environment, which requires these additions.

In order for the editor GUI to directly access and modify component variables (demonstrated in the section below), it must identify the properties associated with each component type while the engine is running. For example, the TransformComponent has the variables location (3D vector), rotation (quaternion), and scale (3D vector), all of which can be set in the editor.

Static initialization is useful here, since we need this information to be publicly available.

#define REGISTER_COMPONENT_BEGIN(Type) \
struct RegisterComponent_##Type \
{ \
RegisterComponent_##Type() \
{ \
if(std::find(GetRegisteredComponentsList().begin(), GetRegisteredComponentsList().end(), typeid(Type)) != GetRegisteredComponentsList().end()) \
return; \
if(GetRegisteredComponentInfoMap().find(typeid(Type)) != GetRegisteredComponentInfoMap().end()) \
return; \
\
GetRegisteredComponentsList().push_back(typeid(Type)); \
GetRegisteredComponentInfoMap().emplace(typeid(Type), ComponentInfo( \
#Type, \
[](void* rawComponent) -> std::vector<std::pair<void*, ComponentVariableInfo>> \
{ \
Type* component = static_cast<Type*>(rawComponent); \
\
return \
{
#define COMPONENT_VAR(variableType, variableName) \
{ &component->variableName, { typeid(variableType), #variableName }},
#define REGISTER_COMPONENT_END(Type) \
}; \
} )); \
} \
}; \
static RegisterComponent_##Type registerComponent_##Type;

And when we need to access the set of component variables (along with their display names and types), we use the GET_COMPONENT_NAME() and GET_COMPONENT_VARS() macros, which just read from/write to the globally accessible map defined in GetRegisteredComponentInfoMap():

struct ComponentVariableInfo
{
std::type_index variableType;
std::string variableName;
};
struct ComponentInfo
{
ComponentInfo() = default;
ComponentInfo(std::string componentName, std::function<std::vector<std::pair<void*, ComponentVariableInfo>>(void*)> getVariables) :
componentName(componentName), getVariables(getVariables) {};

std::string componentName;
std::function<std::vector<std::pair<void*, ComponentVariableInfo>>(void*)> getVariables;
};

REDFORGE_API std::vector<std::type_index>& GetRegisteredComponentsList();
REDFORGE_API std::unordered_map<std::type_index, ComponentInfo>& GetRegisteredComponentInfoMap();

#define GET_COMPONENT_NAME(componentTypeID) GetRegisteredComponentInfoMap()[componentTypeID].componentName
#define GET_COMPONENT_VARS(componentTypeID, componentPtr) GetRegisteredComponentInfoMap()[componentTypeID].getVariables(componentPtr)

Notice that ComponentInfo includes a function pointer that returns a list of void*/ComponentVariableInfo pairs and takes a void*. This is the function that gives the variable list for some given void* to a component instance. GET_COMPONENT_VARS() uses this function pointer to get the raw pointer to a variable belonging to a specific component instance of the given type. This is how the editor keeps track of and alters the values of all component variables. Also, because components are never moved during their lifetime (since they're stored in a ComponentArray), we know the pointer is valid for as long as the component exists.

This setup allows components to be easily declared and registered like so:

REDFORGE_API struct TransformComponent
{
glm::vec3 location = { 0, 0, 0 };
glm::quat rotation = { 0, 0, 0, 1 };
glm::vec3 scale = { 1, 1, 1 };

public:
void MoveRelative(glm::vec3 vector);

glm::vec3 GetRight() const;
glm::vec3 GetUp() const;
glm::vec3 GetForward() const;

glm::mat4 GetMatrix() const;
glm::mat4 GetRotationMatrix() const;

glm::vec3 LocalToWorld_Point(const glm::vec3& vector, bool includeScale = false) const;
glm::vec3 LocalToWorld_Direction(const glm::vec3& vector, bool includeScale = false) const;
};

REGISTER_COMPONENT_BEGIN(TransformComponent)
COMPONENT_VAR(glm::vec3, location)
COMPONENT_VAR(glm::quat, rotation)
COMPONENT_VAR(glm::vec3, scale)
REGISTER_COMPONENT_END(TransformComponent)

---

For the Engine: Implementing transform hierarchies is a priority, since many common editor features rely on them. Even though this would theoretically just be another matrix multiplication step before rendering, it has greater implications for the bindless setup that currently exists. For example, how do we make sure that transform matrices are calculated in the correct order (from topmost parent to bottommost child)? Instead of changing render order, which would interfere with the current batching system (which groups instances by mesh), we could maintain a parent indices array, where each transform holds a parent index. The entry at that index is another index, which points to the parent's parent, and so on. The array is traversed, multiplying at each step, until an invalid parent index is reached (for example, 0xFFFFFFFF for a 32-bit unsigned integer). This would have to be done individually for each transform, but this shouldn't be an issue since they'll be run in parallel on the GPU during the vertex shader stage.

For the Editor: Scene serialization is one of the most important next steps for the editor, since an editor is useless without saving your progress, and building a game with pre-defined scene data is a feature of all game engines. It's likely that the above macros will be useful in achieve this, since they categorize and define all data contained in various component types. I have concerns about how certain complex component variable types will be serialized, though. For example, the InputComponent contains a series of function pointer variables that represent callbacks to be used when specific button and key events occur. Because function pointers are invalidated on program exit, serializing them isn't a trivial task and may require a convoluted workaround. In preparation for this, I'll continue to review the techniques used by Unity Events, for example, which abstract event callbacks by storing an entity reference and a function name that refers to a function for one of that entity's components.

Demos

The ECS model lends itself to simple frontends, since all variety in the game world comes from component variables, and all component variables are modified directly by systems. The editor has direct access to each entity's components as well, and so an entity's appearance and behavior can be modified live without requiring any component-specific implementation.

A single PhysicsSystem handles the simulated motion of all entities with a PhysicsComponent, including collision responses. All objects that should be included in collision detection/resolution logic should also have a ColliderComponent to define their bounds. Some examples are given below:

Code

The three most significant files are the PhysicsSystem, GraphicsSystem and EntityManager. There's a lot that isn't covered in the above sections that are included in the following code samples:

• PhysicsSystem.cpp
• GraphicsSystem.cpp
• EntityManager.cpp