Game Programming Concepts

In the first chapters we used low-level libraries to create multimedia and interact with hardware.

In the following chapters we are going to explore higher-level concepts of game programming: patterns and solutions applied to frequently in game programming.

Many ideas presented here come from imperative game programming and you can find more about them in standard literature. Others will be motivated by trying to address problems from a purely functional perspective.

The big picture

At the beginning, games needs to set up the video configuration, load graphics and audio, render some loading screen, present a menu to configure settings and games, load levels, communicate with other players and the server, unless there is no server, in which case AI should control other players.

Games need to produce a pleasing visual and auditory effect, simulating realistic physics and with perfect synchronisation between images and sounds, wrapping you in a cloud that protects you from your daily problems, unless your mum calls you, in which case it must let you pause the game, or even save it and resume it later.

You can play games using different kinds of inputs (mouse, keyboard) on any device (console, PC, mobile), and they are expected to work seamlessly and perform efficiently, without draining your device's battery and letting you download the latest season of Breaking Bad in the background. Your games will be featured on online listings open to comments, ratings and discussions, and for every supporter and lover you will get two demanding teenagers with very low tolerance for failure who will rate your app -10 because it did not have use their 3000 USD graphics card newest extension for particle effects.

As you read the above, you will probably understand that game development is, in part, just art, that game success has a component of luck, and that the complexity of the problem, the expectations and the level of testing and robustness required are very high. And if this is your reaction, honestly, I don't blame you:

To attack this problem we need to divide it into subproblems and address each one independently. In some cases the solution will be a library that we can import from every project. In other cases, it will be a template that we need to integrate with out code and adapt. Finally, some solutions will not be coded as templates, but rather as patterns or guidelines, and used by many existing modules and definitions in our code. In general we will provide a general pattern that captures the core idea and a default implementation that works well in many cases.

Most games need to address many of the following concerns:

• Asset management (locating assets, loading them, using them, freeing them, recovering from errors loading assets, showing info about loading progress).
• Physics and collisions (simulating a realistic world)
• Error handling (reporting useful information while keeping the game running).
• STARTED Cameras (projections, interaction).
• Asynchronicity (communciation of different threads running different subcomponents).
• Audio (background music, sound effects, audio synchronisation).
• Menus (configuring preferences, jumping to specific parts of the game, resuming the game).
• Preferences.
• PSEUDO-STARTED Animations.
• STARTED Backgrounds and scrolling effects.
• STARTED Z-axis (in 2.5, parallax or 3D).
• Reconfigurable input devices.
• Saving or interrupting the game.
• Time transformations and alterations: pausing
• Testing.
• STARTED On-screen HUDS.
• Network communication.
• AI.

Architectural patterns

State machine

During gameplay, your game transitions over several states. At the beginning, it will probably show a loading splash screen to let users know there should be a delay while assets are being loaded. Then it might show a menu and let users select and configure devices, quality and game settings select a level and start playing.

During gameplay, your game can be loading a level, playing, paused, showing won/game over screens, showing a menu (save, quit, etc.) or depending on the kind of game, waiting for you or other players, to make a move. It is easy to see then that your game is, in some respect, a state machine.

Simple state machines can be described by a diagram of states (circles) with directed links between them (arrows). Arrows can be labeled with inputs, or the events that would make the machine transition from one state to the next. An arrow from a state x to a state y labeled '0' could be interpreted as "if the user enters '0' while in state 'x', the game can/will transition to state 'y'". The initial state can be marked with an arrow pointing towards it, and final states are often circled twice.

⭐ Representation

There are other ways of representing state machines. The machine's output can also be included in the diagram. See REFERENCES for alternative representations and a description of other areas that use this abstraction.

Apart from the idea that your game switches from one state to the next, it is important to understand that both the input and the output may be completely different between one state and another.

Consider, for example, a menu. An abstract description of the possible input actions (without depending on any specific input device or keyboard layout) might be termed 'up', 'down', 'activate', 'back' and so forth. During another game stage, your game might need continuous information from the accelerometers of a wiimote.

While it is possible to use the part of the complete input that each part of the program is interested in, it pays to create a first layer of filtering that transforms the raw input into a more declarative description based on the context of the program.

The same is true about the output: it is best to structure each state's processing function so that the output is a declarative description of that particular state. For instance, while in the menu state, we could create a type to represent the possible input, one to represent the state while in the menu, and create one function that processes new input to produce an ever changing state.

⭐ Change

Such a function would need to have access to the current state to produce the new one. Different abstractions that address this are monads, arrows, continuations, stream processors, functional reactive programming and explicit state passing. We will explore some of them later (NEEDS REFERENCE).

PLACE FOR AN EXAMPLE WITHOUT FRP, ONE WITH FRP.

The game loop

Games are interactive visualizations and, as such, they need to process user input and produce continuous video and/or audio.

One possible way of structuring them, akin to mathematical functions and command line invocations, is to repeatedly detect the input, process it (based on the program's state at the time), present the output after that. This results in a sense-process-render loop often called the game loop.

⭐ Past, present and future

Game loops have been used for many years, and they continue to be used to this day. See, for example, REFERENCE and REFERENCE for an old example and a new one.

Input and output, in the real world, occur in continuous time. Yet your game gathers input at discrete points, and renders a little bit afterwards. A sequence of actions as required by a game loop creates a small delay between some effects and others, and also enforces a certain priority of some over others. For instance, in your game you might advance the physics (movement of elements based on velocities and accelerations) before or after you process the user's input. If you do it afterwards, then the physics at a given time will be independent of all the input since the last time the game was rendered until now. You might also run the players actions before or after the artificial intelligence that controls the opponents, in which case you are deciding on an order that might change the game's outcome completely. Depending on you game pace, this may or may not make a noticeable difference.

In its most basic form, a game loop could look as follows (REF TO ACTUAL GAME THAT USES THIS):

main :: IO ()
main = do
  initialise
  
  gameLoop initialState

gameLoop :: GameState -> IO ()
gameLoop oldState = do
  input      <- senseInput
  timePassed <- senseTime

  let newState = progress oldState gameLoop timePassed

  render newState

  -- Continue until some gameFinished :: GameState -> Bool
  -- function evaluates to False.
  if (gameFinished newState)  -- Could use Control.Monad.when/unless.
     then return ()
     else gameLoop newState

There is a lot that we can deduce from this schematic code already. For instance, while sensing input, time and rendering are effectful operations (the type of senseInput might be in the example above IO Input for some type Input), the actual game logic (and physics) is completely pure. This means we can define the game state as an abstract datatype and use pure functional programming to implement it, with the advantages that provides (simpler code, equational reasoning, compiler optimisations, higher-level descriptions, declarativeness, parallelism, testing).

⭐ FRP

The FRP implementation Yampa contains an execution function called reactimate that implements a game loop parameterised over an input provider, a processing function and an output consumer. The function runs as fast as possible by default, although the sensor or the consumer can choose to introduce artificial delays to lower CPU consumption or try to approximate a specific number of FPS.

This continuous game loop architecture is not without caveats, and is nowadays only used in very simple games with low performance demands. The main reason is that all the functions are synchronized: we sense once, we progress (logic and physics) once, we render once, and repeat.

Current screens render at 60 frames per second, while some physics simulations need to be executed at about 100 frames per second to look realistic, and some input devices produce data at only 25 frames per second (eg. a webcam). A fixed loop like the one above could lower the global performance of your game, while unnecessarily increasing the frequency with which some functions get executed.

There are multiple ways of addressing this problem, from executing some functions more than once per cycle to running them asynchronously. Introducing asynchronicity (threads) also increases game complexity, as it requires handling shared memory, signaling, delays, blocks and deadlocks.

⭐ OpenGL and HGL: unless you have something better to do...

OpenGL includes a function that can execute one game loop iteration when the thread is idle (not rendering).

NOTE: So what?

Also, while some input devices, like wiimotes, need to be polled regularly, others like kinects (and widget systems like Gtk, which also produce input) work using callbacks. Callbacks are asynchronous functions that get executed in response to certain events.

Therefore, an architecture based on polling may not adapt well to devices that should not or can not be polled regularly.

NOTE: Introduce async structure or point to chapter on async games.

Views

A common pattern in interactive applications is that the same entity that consumes or presents output also produces input. This may obviously true for players: they are he ultimate consumers and producers. With this abstraction in mind AI-controlled players can also been as pairs of consumer/producer: they take the game state, "show" it (to an intelligent agent/function), and produce a set of actions pertaining to some of the players.

In the game programming literature, this abstaction is known as a view. Other names for it are reactive value ¹.

Consuming the output and producing a new input may not be synchronous actions. Network players, for instance, can be abstracted as views: the game state (or a game update) is sent via the network, and input actions are transmited back to the server. The player may be receiving updates every few miliseconds (if latency is low) while only firing a few times per second, max.

Using views also helps abstract from polling vs callbacks, rendering asynchronously, having multiple cameras for different players, and even using widget systems.

Example of a view in a monadic game. Example of views in Arrowized FRP. Example of views in Keera Hails.

Cameras

Games often present scenes that do not fit on the screen. Mario Bros. is such a game. Counter strike is such a game. Any racing game is also of this kind.

In arcade 2D-based games, one can easily simulate this effect by moving all other elements (oponents, background, etc), when the player "moves" forward. The player moving forward, then, is just an illusion. But, as your logic and physics become more complex, and you add more than one player, this workaround becomes harder to implement.

While elements are shown in screen coordinates, the internal processing function should work with game, or world coordinates.

What's actually going on in these games is that you have a perspective, a point of view, a camera pointed to the game world that only shows you a part of it, from a specific angle. What's more is that camera is a portal also for the input: what looks like middle of the screen in screen coordinates may not represent the center of the world.

It's easy to see that, given a function f that transforms game world coordinates into screen coordinates, we have:

gameForUser   = capScene screenSize $ transformState f gameState
inputFromUser = transformInput f<sup>-1</sup> screenInput

IN 3D... Game Engine Architecture

In 2D... Whatever

PLACE FOR ANOTHER EXAMPLE

HUDs, toolbars and overlaid visuals

Heads-on displays are a general name for those information panels shown on top of the game with additional information.

PLACE FOR AN IMAGE

What's interesting about them is that:

• They can be enabled or disabled.
• They may be shown or not depending on the game state.
• They may be interactive (handle input apart from presenting output).

The last point is perhaps the most crucial one: while it is easy to display something in one area of the screen, we also need to:

• Transform screen coordinates into coordinates local for the displayed elements.
• Translate raw input events into more meaningful event descriptions.

For example, given an input event (eg. mouse click at position (x,y)), we need to determine wether it lies on the game area or a panel, and then translate the screen coordinates into local coordinates for that element. If it's the game area, all we need is to apply the inverse camera function. If the panel, we may just need to rebase them based on the position of the panel on the screen.

The general transformation is as follows:

PLACE FOR THE MATRIX

What we are describing here is true both for HUDs and any other permanent or temporary on-screen element that does not use the game camera as the main game area.

PLACE FOR AN EXAMPLE

Note that, after the transformation of the event coordinates, we still need to transform the low-level input event (click, etc.) into an declarative depiction of what happened (fire).

You can see this intermediate layer as a simple widget system, and may even consider creating an abstraction for containers and alignments that facilitates adapting to different screen sizes and screen rotations during gameplay. I recommend that you keep it as simple as you can, and as abstract as you can.

PLACE FOR AN EXAMPLE OF CONTAINERS

PLACE FOR AN EXAMPLE OF ALIGNMENT

Backgrounds, scrolling and parallax.

As we just saw in the chapter about cameras, many games present a portal to a world larger than it fits on the screen. Backgrounds can lighten an otherwise dull screen, but what really changes game appearance are moving backgrounds and a perception of depth.

First, we will explore backgrounds without depth (static and moving). Then we will explore backgrounds with depth (parallax). In the next section we will explore depth in a more general way (even in 2D games, often termed 2.5 as opposed to full 3D).

Static backgrounds

Implementing static backgrounds could not be more trivial. A static, uninteractive background can be presented both as an adhoc layer in the render (see first example) or integrated as one more visual element. What's important in the second case is that, unlike other visual elements, a static background is not affected by the game camera, but it is affected by transformations that have to do with laying different elements out on the screen.

A PICTURE IS WORTH A THOUSAND WORDS

PLACE FOR AN EXAMPLE

Moving backgrounds: a very large, but limited, world

In some cases, the game world is just slightly larger than the screen, but still limited in size. The background is therefore also larger than the screen, but finite. Presenting this kind of world is relatively simple, and it only requires that we crop part of it to the size of the screen. We can generalise this approach easily by using transformation/camera matrices applied to background coordinates.

What's important in this case is to make sure that players can never move out of the world boundaries, or there will be screen areas with no background.

PLACE FOR AN EXAMPLE

Moving backgrounds: unlimited worlds

Some games gave very large or even unlimited world sizes. In such cases, backgrounds tend to be designed so that each side matches the opposite perfectly, like in a rolling paper.

PLACE FOR AN ANIMATION

This was a common trick in animation, and you can see it often in older TV series:

PLACE FOR AN ANIMATION

There are several ways of implementing this. One way is to crop the background in two, and present one part and then the other:

PLACE FOR AN EXAMPLE

Depending on the underlying framework, you may get away without cropping: you can also paint the world twice, and then paint the rest of the scene relative to its position with respect to the background.

While I personally tend to favour the first approach, whether cropping or painting twice is performance-wise pretty much depends on the platform and the underlying graphics layer (some do not let you paint out of screen area, others easily facilitate rebasing the scene relative to a background).

Parallax:

Games can achieve a more appealing sense of perception by adding depth. Even if the action takes place in 2D, a common effect is to add several background elements moving at different speeds as characters move along the world (so as to make it look like some elements, the ones moving faster, are closer than others, moving slower).

PLACE FOR AN IMAGE

While this may seem like a relatively new trick because of its common presence on the web nowadays, there are old games around there that used this effect already:

PLACE FOR AN IMAGE

Implementing parallax, so as long as the background layers are all presented behind all the game action, is just as trivial as presenting one background, but having one camera transformation matrix for each background.

PLACE FOR AN EXAMPLE

Depth

2.5 dimensions

Even 2D games can achieve a sense of depth by introducing layers. Layers can be presented without any modification to make them look closer, or with a camera transformation.

PLACE FOR TWO IMAGES

Implementing these kinds of transformations is relatively trivial. First, your game elements need to have a third dimension. Unlike the x and y dimensions, which may have fine precision approximating the continuous, the z dimension can easily be a integer if all you want is to layer your scene. Different layers can then be presented to users one by one.

Layering your scene is actually a form of 3D with an integer z coordinate, and as such it introduces some of the casuistics of 3D:

• If I click on the game area, which element of which layer am I trying to interact with?

• If interactive layers are presented using parallax effects, how do I adapt the input?

• If interactive layers are presented using parallax effects, how does that affect the physics?

TO BE COMPLETED

3D

TO BE COMPLETED. Readers will mainly be refered to books and other material.

Animations

Animations can be extremely appealing for players. Animations can be achieved either continuous transformation, or, in the case of sprite-based graphics, by a mapping from continuous time to discrete time.

Sprite-based animations

Transitions

Handling input during transitions

Physics and collisions

Like with most areas of game programming, you could devote tons of books only to simulating physics and collisions. Some games do not have a physics engine at all (board games), but even the simplest arcade games perform basic physics simulations in some way or another.

PLACE FOR AN IMAGE

References

1. Keera Hails http://github.com/keerastudios/keera-hails