5  Gameplay

In the previous chapters we covered mechanics, the building blocks of gameplay, and assembling them into systems so that together they form chains, loops, and interlocked mechanisms. In this section we look at how these elements together produce gameplay.

Gameplay is what happens when we participate in an ongoing interaction with the game’s mechanics and systems and potentially other players in multiplayer games. Gameplay is the dynamic activity that emerges out of player’s interaction with mechanical elements, and participating in this interaction creates a certain experience for the player.

In this chapter we look at how we can systematically think about and analyze this gameplay. In the previous chapter we discussed feedback loops and systems. Now we will turn our attention to a different kind of a loop, in which the players and the game itself interact in the form of cyclical, repeating activities.

In the first part of the chapter we learn how to analyze games in terms of cyclical gameplay loops with various frequencies and how those loops interact with systems. Then in the second part of the chapter we look at player motivations for taking part in these loops and how we can support player motivation through gameplay design.

Figure 5.1

Recap of the basic model

Motivating Example: The Sims

Let’s say we are playing a session of The Sims 4. This is a game in which we control the lives of artificial people set in a simulated replica of a small town. We have complete control over our simulated people, but we need to keep them happy and fulfill their various needs, which are usually numerous and at odds with each other.

We are playing Alice and Bob’s house, and Bob came back from work to see that Alice is throwing a party. Bob is exhausted and his energy is close to crashing. But at the same time, he feels unhappy because his social score is low and meeting someone new would really cheer him up.

We need to decide what he should do. Meeting someone new would be good, but he is also very tired. On one hand, he risks passing out from tiredness and oversleeping for work the next day. On the other hand, meeting new people will make him very happy and therefore more positive and efficient, a sure way to get that promotion that he has been working on. His life ambition is to become a CEO one day and a promotion would get him much closer. So, we make a decision—Bob will have to buck up, go out there, and socialize. This is for his own good. Sleep can wait for another time.

Figure 5.2

Screenshot from The Sims 4

Just that one decision point asked us to consider our choices and goals on several levels.

  • Short-term: the need to maintain our sim’s immediate well-being and urgent needs
  • Medium-term: the need to make sure today and tomorrow are taken care of
  • Long-term: the need to work towards the larger goals of becoming a CEO

As we play, we must consider goals and activities on different time scales at the same time. This is not accidental. Games deliberately present challenges on multiple levels simultaneously, and the various levels support each other. Without short-term challenges, there would be nothing interesting to do right here and now, and without long-term goals, there would be no reason to keep playing over a longer period.

Working towards goals on different time scales typically means engaging with different systems in the game. In The Sims, short-term planning is based on the “wants and needs” system, which gives sims different needs that need to be managed. For example, a hungry sim will need to find food very quickly otherwise bad things will happen. Medium-term planning is based on the “jobs” system and the game economy, in which sims can find jobs with different requirements, salaries, and work hours so that they can make the money to afford the things they need to succeed in the long term. Long-term planning is based on the “life goals” system in which a sim’s long-term goal might be to become a CEO or a famous musician. Setting them up for success requires planning in advance and often spending money and time on things that have very delayed payoffs, such as taking evening classes or investing time in skill practice.

Gameplay Loops

We have seen some activities in which the player engages repeatedly. We will call these gameplay loops since they are cyclical activities—the player keeps making decisions, acting on them, and then coming back to the same decision points.

In our Sims example, our smallest gameplay loop was action selection. We pick the next action to take (go to sleep, go socialize, go make dinner), wait for it to happen, see the effects (more energy, happier sim), and repeat. This happens very frequently, on the order of several times a minute.

Figure 5.3

Player actions produce reactions from game systems and from other players to which the player must react, forming a continuous loop

Then there are larger gameplay loops present as well at the same time. For example, every morning in the game, we need to make sure the sim is well rested, going to work, and making money. This requires different kinds of decisions maybe every few minutes of real time. Finally, on a multiday basis, we need to make sure they are improving their skills and advancing their career. These kinds of larger decisions do not come as frequently, maybe every ten or twenty minutes, but they also require attention.

Loop Frequencies

Gameplay loops have different frequencies. A fast loop turns over every few seconds or minutes of game time, and a very slow loop (like working towards a career goal) only requires infrequent decisions.

For other examples of gameplay loops, we can look briefly at a dungeon crawler game like Diablo, described in the previous chapter.

  • Every few seconds (also known as the micro level): move, attack, defend, cast spell, equip, take, drop
  • Every minute or two: find enemies, battle enemies, collect loot, locate treasure
  • Every five to ten minutes: venture out from camp, clear out a section, return and sell loot, heal
  • Every fifteen to thirty minutes: clear out the entire area, change up crew, level up characters
  • Every few hours: explore an entire region of the game world, advance the story

Figure 5.4

Comparison of a sequence of fast loops only with a combination of layered loops of varying frequencies

Loops with different frequencies are active at the same time. The player’s attention will be most focused on actions on the micro level, focusing on the immediate situation. But these actions also have macro consequences, so players must keep all levels in mind as they play so they can slowly advance long-term loops while they interact with the shorter ones.

As designers, we must plan for all loops to be active and design the game with multiple frequencies of decision making in mind.

Onion Diagrams

A popular way to diagram gameplay loops visually is using onion diagrams. Figure 5.5 shows an example onion diagram from a (simplified) description of Monopoly, which will also illustrate the reason for this name.

Figure 5.5

Simplified onion diagram for Monopoly

The fastest, smallest loop starts out in the center, then ones with longer and longer periods get displayed around it. We can imagine that as the player iterates through the smallest loop over and over, they will also progress through the outer layers as well but less quickly and only partially.

A few more examples from Diablo and The Sims, also simplified, are shown in figure 5.6.

Figure 5.6

Simplified onion diagrams for Diablo and The Sims games

Onion diagrams are mainly a communication tool. They make it very easy to document and communicate to others what kinds of gameplay loops exist in the design.

Since a game will also want to have a good distribution of loops at various frequencies, an onion diagram will easily show if we have a gap someplace that we need to fill in with interesting activity.

The Core Loop

Designing interlocked activities is complicated, so it can be beneficial to start by focusing on the core loop first and evolve the design from there. This is the “minimum viable” level of activity, the smallest kind of a loop that’s going to be meaningful and enjoyable to the player and give them a reason to keep playing.

In some game types, the smallest loop or micro loop is also the core loop because micro-level activities already provide challenge that keeps the player interested. This is more common in action games, for example in a driving game just mastering physical movement and keeping the car on the road while passing the others to get into the lead might be challenging enough.

In other games, however, the core loop will need to be slightly more complex than just micro actions. This is particularly the case in more systems-heavy games in which micro actions are not sufficiently challenging and not enough to motivate the player to keep playing. In a game like The Sims, just keeping track of physical needs might not be challenging, and the core loop is higher up on the level of “go to work, make money, come back, buy some stuff for the house, repeat.” In Diablo, it might be “go out and fight, collect loot, come back to camp, sell it, buy or upgrade gear, repeat.”

The core loop requires a lot of attention because the player will be interacting with it all the time, so we need to make sure they enjoy it even as their attention shifts towards harder problems with a longer time horizon. This problem is amplified by players’ varying skill levels and advanced players may not be as drawn by the core loop as novice players.

Layering

The core loop is also a convenient starting point for building a larger game on top of it—for example, using the following steps:

  1. Start with a micro loop that involves doing small things in the game. This typically will not be very interesting but will be interactive and lets the designer test out the basics.
  2. Explore and grow the micro loop to find the core loop. This is the point where the game starts being enjoyable to play over the short term on the level of minutes. Once identified, we spend some time iterating on the core loop to improve it and to explore ideas about how to extend it.
  3. At this point, we can start layering on longer term loops. If we have not designed those yet, exploring the core loop should give us ideas about larger loops that make the player interested over a longer time. These larger loops should support smaller ones by interacting with them or their outputs.

Figure 5.7

Onion diagrams from figure 5.6 with the core loops highlighted

In The Sims, for example, we described an engaging core loop of making and spending money and managing the sim’s physical needs. But this loop by itself has relatively short-term appeal. Adding larger loops that challenge it, such as adding family members or having a career to advance in, will change up the parameters of how to play the core loop and keep it fresh and interesting in addition to adding long-term challenges.

Advancing in one loop does not have to advance other loops, and it might be more interesting if the various loops are slightly at odds. Some game types seem to delight in making the layers work against each other, which gives the player a considerable challenge. Factorio, for example, is a resource production and logistics game in which the player manages complex production chains and spatial layout of the “factory” is of utmost importance. However, the player’s objectives in early game will push them towards layouts that become a serious problem later in the game and need to be worked around or undone and rebuilt. Figuring out how to balance short term early game goals with future long-term objectives becomes a major source of fun and often frustration.

Loops and Systems

Gameplay loops come from engaging with mechanics and systems as well as other players. Different frequencies are often supported by different game systems that work together as already highlighted at the beginning of this section in The Sims example.

For another example, we come back to dungeon crawlers. We have already mentioned the loop of leaving camp, clearing out an area, and coming back with loot. Then each time we come back to base camp, we have a variety of goals we can try to advance: economic development (selling and trading loot), developing our characters (leveling them up or training), crafting (such as upgrading weapons), and so on. These in turn serve to advance different longer-term goals as well: amassing wealth, character and story advancement, or ensuring that we have much better weapons for later battles.

Just like we listed out activities for each loop, we can turn it around and match loops to systems and mechanics, for example as listed in table 5.1 below.

Table 5.1

Examples of game loops and systems that support them

Frequency

Gameplay loops

Example of related systems

every minute or two

find enemies, battle enemies, collect loot, locate treasure

exploration, combat, items, stats, currencies, inventory

every five to ten minutes

venture out from camp, clear out a section, return and sell loot, heal

exploration, items, inventory, economy, crafting, upgrades, stats, skills

every fifteen to thirty minutes

clear out the entire area, change up crew, level up characters

inventory, crafting, character specialization, collectibles

every few hours

explore an entire region of the game world, advance the story

character specialization, crew management, campaign/story arc

We can see that various loops and frequencies often involve specific systems (as opposed to having systems that work across all frequencies). This is because different systems have different behaviors and are tuned in different ways. It is difficult to devise a single system that would present the player with interesting challenges at all the various frequencies from micro- to long-term.

However, loops often share systems with their “neighbors.” For example, inventory and stats participate in several loops across different frequencies. This is desirable because it means that a single action that advances one loop can also affect (advance or revert) progress in other loops, which makes those actions more strategically interesting.

Finally, some of these loops focus on more than interaction; they also pull in narrative and storytelling elements. In a story-driven RPG like Dragon Age: Origins, some of the shorter loops tend to focus on combat, leveling up, and other such systems. However, the long-term player motivation comes from a gripping narrative that drives the player to want to learn what will happen next and want to take actions that will affect the unfolding story. We will come back to the analysis of these kinds of narrative elements later on, in chapter 6, “Macrostructure.”

Player Motivation

We have looked at how mechanics and systems produce gameplay by giving the player a variety of activities and challenges that arise out of interaction with them. The player interacts with them repeatedly, forming loops of repeated activity with different frequencies. By layering challenges with different time horizons together we give the player a stream of gameplay which challenges them in the short as well as long term.

However, all this is premised on the assumption that the player will want to engage in the loops to begin with. If the loops are not interesting to the player, they will not want to play the game and they will not enjoy our elegantly designed machine.

So, what is it that motivates players to play our game? In chapter 2, “Player Experience,” we already talked about large-scale player motivations—the desire for particular types of experience, such as challenge, strategy, or community. We also talked about how personality types might affect what players enjoy and how to take all of this knowledge into account in overall game design.

In this section, we turn our attention to small-scale motivation: what motivates players to play the game on a minute-by-minute or hour-by-hour basis and the shared psychological principles behind it.

Intrinsic and Extrinsic Motivation

First, we can talk about two broad types of motivation.

  • Intrinsic motivation: when the player is inherently interested in the activity and its outcome. In this case, there is something about the activity that the player finds appealing—for example, dancing in Dance Dance Revolution or learning to fly around the world in Microsoft Flight Simulator may be intrinsically enjoyable.
  • Extrinsic motivation: when the player is driven by sources external to the activity, such as by desiring just the outcome but not necessarily the activity itself. Grinding to get great items or competing in a match to get the top prize may be examples of being motivated extrinsically.

These types of motivations are not exclusive and most activities fit somewhere on the spectrum between being intrinsically and extrinsically motivated. For example, an activity like playing a competitive multiplayer game usually mixes goal-driven motivation (getting a top position on the leaderboard) with some intrinsic pleasure of the game itself. Similarly, a racing game can combine the intrinsic rush of driving quickly and effortlessly with the extrinsic reward of competing for the top prize in a race, both of which are rewarding in different ways (Hodent 2017, 65).

The details of intrinsic and extrinsic motivations are specific to particular players. Looking from the outside, we cannot tell how much someone is playing to get a reward, or playing for the enjoyment of it, or some combination of both. So, although a game cannot shape a player’s intrinsic motivation, it should support it whenever possible. We cannot make the player enjoy dancing in a virtual competition in Dance Dance Revolution—this motivation has to come from within. But if they do, or they think they do, we can build a scaffolding that will make them grow in skill and enjoyment.

On the other hand, extrinsic motivations are easier to manage artificially. Getting more points, getting a high score or a cash reward, winning in a competition against others—these kinds of motivations are popular across many types of games. They speak to the more basic human desires to get better, to compare ourselves against others, and to see the rewards of hard work.

Ideally, games combine intrinsic and extrinsic motivations. A game that provides extrinsic rewards but is not itself interesting is going to turn into a job. On the other hand, an activity that is enjoyable but does not provide feedback about performance or some extrinsic rewards might feel too undirected and “loose” for some types of players. Games are most widely approachable if they can both fulfill the player’s intrinsic desire for a specific kind of activity with the extrinsic “carrot” that also rewards them for getting better and better at what they already enjoy intrinsically.

Intrinsic Motivation: Flow and Learning

What is it that makes players interested in a game? Why do they enjoy engaging with it repeatedly, minute after minute and hour after hour? Everybody has their own reasons and motivations, but there are some commonalities that are broadly shared.

In this section we focus on the idea that gaining mastery of a skill or a domain is inherently interesting to players and serves as a powerful motivation behind our desire to play. Reaction to challenges is not the same across all players. It is based very much on where we are mentally when we encounter the particular challenge, whether we are able to meet the challenge, or if we are improving and learning from it. But even so, we as humans find learning and mastering challenges to be intrinsically interesting if it matches our interests and skill levels.

Surprisingly, this interest in a challenge is true regardless of whether the challenges are useful life skills or “useless” pastimes such as video games. The connection between ability and mastery on one hand and games on the other has been noted since the earliest theoretical writings on games (Caillois 1962).

Flow Theory

The concept of flow is central to understanding the joy of mastery. Commonly it is described as the feeling of “getting into the zone” when doing something. Anyone who has gotten lost in a game or an activity for hours at a time and not realized how much time had passed has experienced that intense, euphoric feeling.

People who get “into the zone” often report similar psychological effects, losing track of time being the most common one, as well as hyper focus on the single task at hand and losing track of oneself and one’s presence in the world. When we are in the zone, it is just us interacting with the activity without anything getting in the way. This effect can happen in a variety of contexts: mental challenges, creative activities, as well as with physical challenges such as with athletes pushing themselves beyond their skill level.

The psychological theory of flow posits that getting absorbed in activity like that is tied directly to the person’s skill and how well the challenge of the activity matches this skill (Csikszentmihalyi, Abuhamdeh and Nakamura 2005). If the activity is too easy the person will be bored, and when it is too hard, they will get frustrated or anxious. But if the subject is focused on the task and challenged well for their skill level, they will enter an optimal kind of state where they are fully engaged with the activity, as illustrated in figure 5.8.

Figure 5.8

Illustration of the player’s flow channel in which the challenge level is correctly balanced against player’s skill level

Csikszentmihalyi et al. (2005) postulate three core conditions of getting into the flow state.

  1. A clear set of goals that channel attention towards specific purposes
  2. Balance between perceived challenges and user’s perceived skills in order to absorb one’s attention into the task
  3. Clear and immediate feedback, which tells the person how to adjust their actions

They also mention in passing one more optional element that supports getting into the flow state: congruence between task-specific goals and longer-term, more abstract goals.

For players, this state of flow is highly desirable and games often actively trigger it. As we have seen, games employ a variety of techniques to satisfy conditions 1, 3, and 4. Game content and the variety of systems provide a large variety of clear and actionable goals for the player to choose from at different scales and frequencies which are set up to support each other. Also, feedback on how well the player did is plentiful, including using progression mechanics to guide the player along.

But one thing that games cannot easily do by themselves is balancing the challenge to match the player’s skill. This is something that is still a difficult design challenge. Most often, we resolve it by having the player pick their “difficulty level,” but that is often unsatisfactory (the player does not know or care to adjust the difficulty level correctly and will not experience the right level of challenge).

In multiplayer games, the problem seems easier. We may try to match the player automatically against an opponent of similar skill, using scores or Elo ranking as proxy for skill measurement. This works well over a longer period of time, but is harder to tune correctly for new players. Many multiplayer games suffer from a “learning cliff” (as opposed to a learning curve) during which brand new players do not get correctly matched and are expected to accept a period of frustrating losses as a part of the learning experience.

In the end, reaching the feeling of flow is central to successful gameplay. It requires the whole variety of game building blocks to work together—mechanics and systems that provide challenges for the player, systems and content that gives them goals, and finally the enjoyment of interacting with all of these elements and getting a good experience out of the challenge, exploration, and experimentation.

Learning and Challenge Escalation

Koster’s A Theory of Fun (2004) presents a related but slightly different argument. He posits that the enjoyment or “fun” of playing a game comes from learning, from acquiring mastery of activities in the game, and especially acquiring the skill to predict outcomes within the game. For example, this might mean learning how to navigate a particularly tricky level or figuring out how to solve a given type of a challenge or perhaps following a narrative and figuring out what will happen next. Additionally, this learning must happen in a safe environment in which stakes are low but challenges are still interesting so that the player has room to experiment and have fun while learning.

This model comes from a different perspective than flow theory (from the intuition of game design practitioners rather than from psychological studies), but it reaches similar conclusions: that enjoyment arises out of learning, out of understanding how something works, and out of having the space to figure it out.

But if learning how to predict or “figure out” the game is part of the fun, what happens once the player figures it out? One implicit consequence is that if a game has static or limited content, the gameplay will inevitably become boring once the player learns it and is able to predict how the game will behave. In the context of the previous discussion of loops and activities, this motivates why we need multiple gameplay loops at different frequencies as well as content arcs that modulate those loops. Once the player learns the small loops quickly, they will no longer hold any secrets. To keep the player engaged, we need to vary them. For example, repeat smaller loops but escalate them (such as by matching the player up against increasingly advanced opponents), stack on larger loops that are more difficult (as with systems-heavy games), or perhaps stack on loops that present very different challenges (such as adding a long-term narrative arc).

The major difficulty here is that the player’s knowledge and skill keep increasing, so we need to keep the loops carefully in sync with player’s changing skill. If the challenge jumps too quickly, this may end up being frustrating, and although some players like this kind of a feeling, others hate it. Conversely, if the challenge increases slower than skill, the game may be too relaxed and ultimately boring, and, again, some players enjoy relaxing activities that are below their skill level and others are infuriated by them. And sometimes the same player might have both reactions to different kinds of activities within the same game, depending on their mood or goals (whether they want to be challenged or relaxed).

Learning to Overcome Uncertainty

We have said that learning and skill mastery are crucial to enjoyment, and Koster argues that learning is intrinsically fun. This seems intuitive but with a caveat—clearly not all types of learning are equally fun. We can easily come up with counterexamples, such as a game oriented around learning mathematics or mastering a foreign language. Though the activity could be challenging and motivating, would we say that learning math is fun as a game?

Figure 5.9

Examples of activities that leave the flow channel owing to presenting too much challenge or not enough challenge for the skill level

Costikyan’s Uncertainty in Games (2013) argues that learning is not enough and that one unique characteristic of games is that they are frameworks for learning how to manage uncertainty. When we play sports or esports, we slowly learn how to control our own movement and attention. In sports as well as in chess, we learn how to anticipate what our opponents will do. In racing or strategy games, we learn how to analyze and anticipate the behavior of a complex mechanical system. The behavior of these systems is uncertain, and we enjoy figuring out how they work and how to control them in a play environment that supports trying, failing, and experimenting.

Costikyan’s taxonomy enumerates the following major and minor types of uncertainty.

  • Randomness: dealing with an unpredictable random process, like dice or a deck of cards. Knowing your odds at the blackjack table or figuring out where you might land with the next dice roll in Monopoly are some examples of why reasoning about randomness is crucial. Randomness is a very common kind of uncertainty, easy to introduce into various game rules, and an easy way to introduce dramatic tension and “shake up” predictable gameplay. In chapter 3, “Mechanics,” we discussed two types of randomness: stationary randomness like die rolls and nonstationary randomness like a shuffled deck of cards. These challenge players in different ways, testing their understanding of probability distributions and their ability to predict future outcomes from a history of previous ones.
  • Skill uncertainty: not being able to act as one intends and training or learning to overcome it. Inability to act is a big source of uncertainty and negative emotions for new players, but it decreases as they learn how to play the game. Overcoming skill uncertainty is often required to attain mastery at a particular game or sport. Since the player’s own ability is the source of uncertainty, we can increase or decrease it by changing the game’s difficulty or challenging the player’s ability to train and learn. We can consider them as three sub-types:
    • Performance uncertainty: the player has difficulty performing the tasks with skill or precision. Action games or sports that require the player to train to get good are a good example.
    • Perception uncertainty: the player has difficulty seeing or sensing what they need. Games with overwhelming or subtle displays, from RTS games on one hand to hidden objects games on the other, are examples.
    • Solver’s uncertainty: the player has difficulty identifying the right solution to the problem at hand. Puzzle games, whether casual crosswords or more involved adventure games, exceed at this kind of uncertainty.
  • Player unpredictability: not knowing what your opponents will do. This is a significant source of uncertainty in multiplayer games, since the player does not know what the enemies or collaborators will do, and learning to figure them out is a major challenge and a strategic advantage. This is present in a variety of multiplayer games as well as sports. Player unpredictability also applies when the other player is a computer-controlled AI character or gameplay system (for example, friendly and unfriendly nonplayer characters). The challenge is similar: the player is challenged to figure out how the AI works in order to improve their chances of success.
  • Complexity: dealing with situations that are hard to figure out. This kind of uncertainty is common in strategy games: chess, war games, management games, and others along these lines. We can distinguish two types:
    • Analytic complexity: reasoning about the current game state taxes the player’s mental abilities. This is very common for games with large action spaces and states with high branching factor, such as chess or war games, or games that exhibit emergent or chaotic behavior. Analytically complex games tax the player’s ability to infer the future from the present, and this kind of uncertainty can be improved by letting the players experiment, explore, and slowly build up a mental model of how the world works.
    • Hidden information: the player is missing all the data they need to make a good decision, such as in strategy games that employ fog of war or games like poker.
  • Anticipation: trying to anticipate what happens next in the overall game. Sometimes the uncertainty comes simply from not knowing what the designer has in store for us. With this kind of uncertainty, players will be challenged to prepare themselves for whatever might happen next at the whim of the designer. Players often enjoy anticipating the future and then seeing how their predictions turned out and if they were able to prepare accordingly. This level of uncertainty can be modulated by keeping the changes reasonable within the logic of the game universe and giving players appropriate clues about how things are going to change over time. We can distinguish three subtypes:
    • Narrative anticipation: the player does not know what comes next in the story so their ability to prepare will be challenged. This is particularly common in story-driven games where the player’s trajectory through the game is deliberately full of interesting turns.
    • Game evolution: the game simply changes over time and challenges players to prepare. This is common with games such as Magic the Gathering or social games such as CityVille, as well as esports games like Hearthstone, in which the game is guaranteed to evolve and players need to anticipate it to keep up with their friends or competitors.
    • Real life uncertainty: the game challenges players to adjust their real life to the demands of the game in order to advance and win. This is common in games with significant multiplayer elements, such as MMOs, esports, or other competitive multiplayer types in which participation in guild raids, scheduled competitions, and other such activities impact players’ real-life schedules.

The various types of uncertainty are used to challenge the player and make this challenge enjoyable. Games often employ multiple types of uncertainty at the same time—for example, a CCG like Magic the Gathering might interleave random processes (cards), hidden information, player unpredictability, high amounts of analytic complexity (arising from the complex combat and resource mechanics), as well as significant game evolution and real-life uncertainty elements.

Dominant Strategies and “Solving the Game”

Just like “enjoyment,” the player’s feeling of challenge is also subjective and based on their previous experience. Children like to play tic-tac-toe because it challenges their ability to analytically predict how it will unfold. But for adults, the game holds no secrets—and without that uncertainty, the enjoyment is also gone. A challenge that becomes predictable and rote loses a lot of its appeal.

Enjoyment of a challenge sits in the peculiar valley between full unpredictability and full predictability. If a game is completely unpredictable, like roulette or other games of chance, that reduces the enjoyment coming from gameplay (although players may still play for different reasons, like the thrill of gambling one’s money on a lucky draw). Conversely, if the game is predictable, it also loses appeal, for the opposite reason—if we already know how it will turn out, why even play?

To ward off predictability, designers pay special attention to dominant strategies in games, strategies which clearly bring out better results than others. A simple example of a dominant strategy in tic-tac-toe is to start the game by putting the first mark in the center of the board rather than somewhere else, which guarantees that any opponent moves can be countered and each game can be won or tied but not lost.

Dominant strategies can also be subtler. For example, when the game Civilization V first came out, different civilizations had different unique buffs, but the French civilization had a particularly powerful buff that allowed them faster border expansion compared to others. Playing as the French and concentrating on specific upgrades early on, one could easily outgrow other empires mid-game, which then led to easy victory. (As expected, this unbalanced tuning got viciously nerfed in a subsequent expansion pack.)

Dominant strategies affect enjoyment of a challenge, but they are difficult to find analytically. Finding them typically involves a lot of playtesting before the release, as well as adding analytics to games after the release and monitoring for specific patterns of play that cause unexpected scores or win ratios.

Loops and Challenges

Do smaller loops always offer smaller challenges? As we have already hinted, this is not necessarily the case. For example, in action games participating in the micro loop starts out difficult. Driving the car while passing and not crashing, shooting at alien space ships while avoiding their missiles—these take a lot of practice to get good at them. Arcade games, from Space Invaders or Asteroids to the more recent Flappy Bird, made punishingly difficult micro loops their hallmark. In comparison, larger loops in these games are easier (or nonexistent) and participating in them is a more abstract, aspirational goal for more advanced players.

We see the opposite in strategy games or board games. The more strategically oriented the game, the easier the micro loop tends to be, as moving pieces on the game board can be tedious or require some calculations but is not typically meant to be challenging. The larger challenge is in the medium- and long-term strategy of knowing what should be moved, where to allocate resources, and how to play the long game.

Games are likely to mix the two approaches. For example, squad-based tactical FPS games such as the Call of Duty series mix challenging micro loops that demand physical dexterity and fast reaction times with longer term loops based on the tactical situation of the entire squad as well as a solid amount of metagame.

Figure 5.10 illustrates this situation using onion diagrams based on an example from Koster (2012).

Figure 5.10

In some games the small loops are more challenging than the large loops, but in other games it is the opposite, after (Koster 2012)

Extrinsic Motivation: Work and Rewards

So far, we have focused on intrinsic motivation coming from internal enjoyment of the activity. In contrast, when the player is driven by the outcome of the activity, such as rewards or acclaim, rather than the activity itself, we can say that they are motivated extrinsically. These kinds of motivations are also common in games and supported by progression mechanics as well as other game design elements.

Here we introduce some ways in which players find extrinsic rewards motivating and what games do to support them and to tie extrinsic and intrinsic motivations together.

Progression and Rewards

It is very common for games to reward progression with achievements and leaderboards. Many platforms (including consoles or the Steam desktop platform) support them natively and make it easy for the player to see their virtual trophy wall or score board and maybe even show it off to other players.

The family of progression mechanics includes these kinds of rewards. These mechanics encompass the different ways for giving the player feedback on their progress. As discussed previously in chapter 3, “Mechanics,” these are elements such as:

  • Score or XP (increases as the player accomplishes various goals)
  • Levels (earned as the player reaches specific milestones)
  • Achievements (earned for reaching specific uncommon goals)
  • Leaderboards (show how players’ scores, levels, etc., stack up against those of other players)

Progression mechanics can provide a variety of extrinsic goals for players. They are very commonly used in games because they provide that extra bit of motivation. As the player’s intrinsic motivation tends to wax and wane over time, the additional extrinsic “carrot” helps to tide them over. Additionally, the rewards themselves can be enjoyable on a meta level. For example, getting a high score can bring acclaim and jealousy among friends.

Some mechanics can also serve a double duty as rewards in addition to their regular role. For example, amassing cash, gold, resources, or units can provide feedback about one’s progress and feel like a reward separately from how those resources can be used later in the game. Similarly, the game’s developing story can be interesting by itself and also give the player a sense of their progress.

However, it is important to note that extrinsic rewards by themselves are not sufficient to make a game enjoyable. They provide motivation, but the game activities have to be enjoyable for the player as well.

Reward Schedules

Assuming that we grant extrinsic rewards, we should consider when to do so. What is the best way to grant them to the player? At what kind of pace, or based on how much effort? What will be the most appealing to players?

To answer questions like these, we can turn to psychological studies of rewards and of how rewards motivate people in general, not just in context of games. One widely known group of approaches is operant conditioning, which is the study of how different types of predictable rewards (or punishments) affect behavior.12

For example, in some classic experiments, handlers would teach a pigeon to peck at a marked target spot and get food as a reward, and then they would vary how often the rewards get dropped to see if that affected the pigeon’s motivation and persistence. The term reward schedule describes a particular plan for how rewards get generated over time based on some chosen principles (for a simple example, we could give out one unit of food every five pecks, or one unit of food every minute).

It is commonly known that animals can usually be trained to do simple things for food, but one of the discoveries was that specific kinds of reward schedules would result in very different training results, some better than others. Another interesting discovery was that by spacing out the rewards, subjects could be trained to do increasing amounts of work in exchange for their rewards (up to a point where they lose interest).

Since its heyday in the middle of the 20th century, conditioning has been thoroughly criticized in psychology and education as a massively reductionist and inadequate view of human learning or motivation. This criticism is fitting. There is much more to human behavior than learning from rewards. We are intelligent creatures and our motivations are complex. But even so, reinforcing behavior through rewards does work on us even if in limited ways.

In game design, reinforcement and reward schedules are very commonly used, although not without controversy. They are criticized for encouraging players to focus on extrinsic rewards over their intrinsic motivations (Hodent 2017, 61–66) or for being overtly manipulative. Some critics also raise a moral panic, accusing them of exploiting human weaknesses for corporate profit and comparing them to gambling (Juul 2010).

However, virtually all video games use extrinsic rewards, and most commonly they use variable ratio schedules (described next) because they work very well in motivating the player. It is useful to understand how they work and how they are limited so that we can use them in positive ways.

Types of Schedules

Many kinds of reward schedules have been studied, but the basic schedules we will discuss here are as follows:

  • Continuous: reward the subject for each action directly
  • Fixed interval: reward the subject every n seconds while they are performing actions
  • Fixed ratio: reward the subject every n actions
  • Variable interval: reward the subject at randomized points in time while performing actions
  • Variable ratio: reward the subject every randomized number of actions

These schedules can be compared based on two criteria:

  • Response rate: how many actions are performed over time, or in other words, how hard the subject is “working” to get their rewards
  • Resistance to extinction: how long does the subject keep performing actions and waiting for rewards even if those rewards are no longer coming

The results are startlingly consistent across different experiments and subject types. Results of how trained animal subjects perform can be summarized in a chart in figure 5.11. Lines represent subject actions, and dots represent rewards.

Figure 5.11

Response rates of different reward schedules, patterned after Walker (1975, 81). Lines represent subject actions; dots represent rewards.

To describe these results in more detail, here is how the schedules stack up with the most effective schedule first.

Variable ratio: for example, reward the pigeon for pecking but vary how many actions are required for a reward. Maybe pecking ten times triggers the first reward, the next one after twenty times, then down to five, and so on. This ratio has the highest response rate (the pigeon works the hardest) and slowest extinction rate (it takes longer for the pigeon to give up) than other schedules. It rewards the subject’s active participation, and the variable schedule teaches the subject to not give up too quickly if the rewards are not coming for a while.13

Variable interval: reward the pigeon for time spent pecking, but vary the time delay between rewards. Maybe sometimes ten seconds of pecking will be enough for a reward, at other times fifteen or twenty seconds. This reward is tied to randomized time intervals rather than amount of work, and it also results in slow extinction (the pigeon does not give up easily) but less work per unit of time (the pigeon does not work as hard), since the reward is not tied to the level of activity.

Fixed ratio: reward the pigeon every twenty pecks. With rewards spaced out predictably based on the amount of activity, the activity level is good but extinction is faster (the pigeon gives up when the reward stops coming). Also, it is more likely that activity will slow down right after a reward.

Fixed interval: reward the pigeon every twenty seconds if it keeps pecking. Giving out rewards on a steady schedule shows faster extinction and lower reinforcement (pigeon does not work hard). Also, work slows down after a reward but then speeds up as expected reward time approaches.

Continuous schedule: reward the pigeon with every peck. This schedule (not shown on the chart), has the fastest extinction and lowest reinforcement of the set, as subjects get satiated quickly and lose interest if the rewards stop.

Although these results are best known from animal studies, they have been repeated in various ways across a variety of test subjects including humans. Something about the anticipation of a somewhat unpredictable future reward has clear effect on human and animal subjects alike, and variable schedules are especially good at teaching their subject not to give up even if their reward is not forthcoming.

Game Examples

To see how reward schedules can be successfully used in games, let’s go back to our ongoing example of dungeon crawlers—and specifically, games from the Diablo series.

The core loop is that we go out and kill monsters, collect loot, come back to camp, and sell loot. Loot is collected after combat. Monsters randomly drop loot and gold when killed. Levels also contain scattered treasure chests with random loot. Additionally, there are different types of loot. Tougher monsters drop better loot on average, and both monsters and chests also occasionally drop rare, collectible items.

If we consider combat as “work” (i.e., successfully finding and killing monsters over time), we can see that loot is a reward for this work. But what kind of a reward schedule is it on?

Monsters take a randomized amount of work to kill and upon success they drop a randomized amount of reward, which is proportional to how tough the monster was. This is a variable ratio reward schedule: rewarding the player for their actions with some built-in randomness, the details of which are opaque to the player.

Sometimes monsters also drop rare collectibles that have much higher value, more like rare payouts from slot machines. This is a secondary reward schedule layered on the first one. It is also a variable ratio schedule, like the first one but with much higher payouts for much higher amounts of work.

But combat is not the only way to get treasure: you can also explore to find treasure chests. Various locations hide treasure chests that drop different types of loot than monsters do. This requires finding those treasure chests, which in turn requires exploration and risk-taking (like fighting monsters along the way), and it might also require extra skills or tools to open them. Just like with monsters, the drops follow a variable ratio schedule but in exchange for a different type of activity.

Similar to monsters, some chests randomly contain very high value drops, so they also have a secondary variable ratio schedule layered on top, one that is slower but with much higher rewards.

So, in summary, this dungeon crawler loop employs four different reward schedules.

Rewarding combat

  • High-frequency variable ratio
  • Low-frequency variable ratio

Rewarding exploration and risk taking

  • High-frequency variable ratio
  • Low-frequency variable ratio

Interestingly, combat loops and exploration loops have their own multiple-reward schedules, and they are tuned differently to keep things interesting and unpredictable. This is a great example of chaining multiple concurrent types of schedules to reward the player for both varying their activity and also for advancing their skills in all of them.

Changing Workload

Another aspect of reinforced behavior is that, once learned, it can become a basis for new kinds of behaviors—either more complex work or a higher workload for the same kind of a reward.

Loot boxes. A recent controversy in game development is the use of loot boxes, which are surprise “treasure chests” that the player can buy with real money, or sometimes earn through playing. Each loot box contains some number of randomized rewards, a combination of low-value ones along with a chance of getting a rare or high-value reward.

These treasure chests have been criticized for being dangerously like gambling. First, loot boxes are perfect examples of random ratio reward scheduling in which the player puts in “work” for a random chance of getting one of the rare items, which is a behavior with high reinforcement and low extinction rates. Second, the “work” usually boils down to spending real money for each chance to open a new loot box, which can lead to excessive spending and other problems associated with compulsive behavior.

Multiple countries are already investigating or regulating loot boxes based on their similarity to gambling. How this might affect the use of variable ratio rewards in games in general remains to be seen.

This effect is intuitively known by teachers and trainers. Teach the student something simple and reward them consistently, and once that is mastered, you can start increasing how much they have to work for the reward (for example, to build up endurance) or you can use that as a building block to make the work more complex (for example, by adding more complications).

This is commonly done in games as well. Perhaps most typical example are level curves, which are the formulas for how character experience points (XP) translate into increases in character level. In many games, a player earns XP through in-game actions, such as killing monsters or exploring areas and collecting items. Characters also have levels which increase upon reaching appropriate XP milestones and unlock new abilities and rewards. And very often, those milestones are not spaced out evenly but require ever-increasing amounts of XP to reach.

For an example, consider the following XP/level chart for Diablo III, summarized in table 5.2.14 It shows a very typical, sharply increasing level curve, similar to what can be observed in many games.

Table 5.2

XP/leveling chart for Diablo III

Character level

XP required to reach level

1

0

2

280

3

2,700

4

4,500

5

6,600

…

10

19,200

…

20

57,200

…

30

115,200

…

40

420,000

…

50

2,080,000

Source: Excerpted from DiabloWiki (2018)

In a level curve like this, every time we reach a new level, the XP required to reach the next level also increases. This has an interesting effect on the player—they have to either work harder to get the next reward or they have to work smarter to get more XP with the same amount of work (for example, by crafting better weapons or finding more profitable enemies). In some cases, this might make them want to play the game more, but in others the increasing amount of work will demotivate them, depending entirely on the combination of how the rewards were tuned and the player’s own motivations.

Related Topic: Gamification

Gamification is closely related to extrinsic rewards. It is the application of progression mechanics to domains outside of games. For example, online discussion boards like Stack Overflow reward participants for their contributions with points and badges and eventually level-based ability unlocks (that is, the user needs a high enough score to be able to post new questions).

Since gamification provides extrinsic rewards, it can definitely serve as a “carrot” to drive desirable behavior but with caveats. First, the reward needs to be meaningful and valuable to the participant. Leveling up to unlock a new ability is more meaningful than leveling up and merely getting a higher number in the player’s profile. Secondly, the activity itself needs to be interesting as well. Just getting points without enjoying the activity behind it is neither interesting nor meaningful. A chore is still a chore even if you add points to it, and a leaderboard for who gets the most points is still a leaderboard for chores. Rewards do not drive motivation just by themselves.

Gameplay Loop Design Heuristics

Let’s say we have an idea for a game, and we even worked out roughly what the player is going to be doing and what kinds of experience they will be having. Now we want to figure out how to design the gameplay loops that will drive it. How do we get started?

At the end of chapter 4, “Systems,” we talked about using user stories to guide system design. This technique of starting with user stories is broadly useful, and we can use it to guide gameplay loop design as well.

From User Stories to Gameplay Loops

Consider our previous example involving a tower defense game. We can start by asking, “what will the player be doing over and over again?” Can we find a core loop just from a narrative description?

We can take a stab at it as follows:

Player builds out defenses, then waves of creeps spawn and attack the player’s base. Killed creeps drop coins, which can be used to repair the defenses and build better ones, before the next, more powerful wave of creeps comes to attack.

This level of description is good: the player’s actions are going to be building, defending, and then buying/upgrading/repairing. This kind of a loop sounds interesting already, and it could be a viable core loop (and playtesting will confirm whether it is engaging enough).

From this description we can also imagine smaller loops, for example:

During defending, creeps drop coins. The player must pay attention and collect them as they drop, otherwise the coins will roll off the game board.

This describes a very fast, reflex-oriented action loop. Is it interesting and enjoyable? That is for the designer to decide from experience or playtesting and potentially replace it with a different one if this one does not work well.

We can imagine larger loops in a similar way, for example:

Every wave of creeps grows more and more powerful. But the player can spend large numbers of coins on spells that change what creeps get spawned—for example, there could be spells to make them spawn weaker or slower or with a lower attack rate.

Now this describes a potential long-term loop, which lets the player strategize. If they built up a lot of attack towers, for example, they might want to invest in a spell that makes the creeps weaker to make their tower specialization even more useful. And since the coin cost is high, this is a long-term plan that needs to be worked towards and prioritized vis-à-vis spending coins on other things such as more towers or necessary repairs.

Playtesting Loops

Once gameplay loops are designed “on paper,” it is very important to implement them as soon as possible and actually see how they feel in real life. The implementation can be in-game or as a stand-alone prototype, depending on whether the loops interact with other systems or can be separated out.

The reason for immediate implementation is that gameplay loops often feel different live than how they read on paper. Narrative description can easily bias the reader to imagine a different experience, a better experience, than the one actually produced by the implementation. And so, it is important to figure out the actual experience of gameplay quickly before we start layering more loops on top. Skilled designers can sometimes infer some of this live behavior just from experience, but even so, there is no substitute for actual working implementation.

Summary

With this discussion of gameplay loops, we are finally ready to close the loop on player experience. Here are the key takeaways from this chapter:

  • We consider gameplay in the light of the dynamic experience of players interacting with the mechanics of the game and with each other and how this interaction evolves over time.
  • The basic unit of analysis are gameplay loops, which are activities in which players engager repeatedly. Games typically consist of a variety of repeating activities with various challenges and decision points. A fast or small gameplay loop is one that requires frequent decisions and attention compared to a slow or large loop in which decisions are less frequent. The core loop is the smallest loop that is meaningfully enjoyable to the player in a given game, but commonly it is not the same as the fastest loop.
  • Games often use layers of loops with different speeds to keep the player engrossed by having to juggle decision making at different time scales. Onion diagrams are used to visualize this layering.
  • A variety of motivations drives players to participate in these loops. We look broadly at intrinsic and extrinsic motivations.
    • The experience of flow and “getting lost in the game” is an important intrinsic motivator, and games are well suited to elicit this experience by providing a layering of challenges, decision points, and feedback. Learning skill mastery and learning how to overcome uncertain game behaviors are other types of strong intrinsic motivators.
    • Players are also motivated extrinsically by rewards such as progression mechanics (level ups, rewards, high scores). The relationships between rewards and behavior are well studied in psychology, and the concept of reward schedules can be used to understand what kind of extrinsic rewards will produce what kind of participation in gameplay loops.
  • Since gameplay loops emerge from players interacting with mechanics and systems, their design is inevitably linked. Like we did with systems design, we can start with user stories and see what kinds of interaction loops and systems emerge from analyzing them.

As we saw, gameplay loops operate on various time scales from seconds to minutes to hours and more. And in the next chapter we turn specifically to those long-term effects to see what kinds of structures emerge when we look at the entire game from start to finish, or even the activity of playing and replaying many sessions over an extended period of time.

Further Reading

Gameplay Loops

The topic of gameplay loops is often subsumed under the topic of systems, and more detailed discussion can be found in related books such as Advanced Game Design (Sellers 2017).

However, gameplay loops as a standalone concept are well known in game design. Perhaps the earliest preserved mention of them is in “Formal Abstract Design Tools” (Church 1999), and perhaps the earliest mention of onion diagrams is in Will Wright’s talk “Lessons in Game Design” (Wright 2003, starting at timestamp 13:30). A more recent discussion in “Loops and Arcs” (Cook 2012) is worth reading for the connection between loops and nonrepeating activities.

Motivation

For readers interested in the broad topic of player psychology, two books stand out because of their focus on games specifically, with plenty of practical examples and advice: Getting Gamers (Madigan 2015) and The Gamer’s Brain (Hodent 2017).

On the topic of player motivation, we have mentioned in passing that intrinsic and extrinsic motivations are two aspects that can be mixed together and are not binary opposites. There is much more to be said on this topic that is outside of the scope of this text. Readers interested in the psychological perspective can find a good introduction in studies on self-determination theory, for example the work by Ryan and Deci (2000).

On the topic of intrinsic motivation specifically, the theory of flow continues to be a strong influence in game design. For a short research overview please see “Flow” (Csikszentmihalyi, Abuhamdeh and Nakamura 2005) or, for a popular science overview, Flow: The Psychology of Optimal Experience (Csikszentmihalyi 1990).

On the topic of extrinsic motivation and reward schedules, any number of psychology textbooks will provide a good introduction, and a great example of one with a free online version is Learning and Reinforcement (Walker 1975), or the aforementioned Hodent (2017) text which provides a games-oriented introduction. Also, Addiction by Design (Schüll 2012) contains a great analysis of extrinsic rewards from the perspective of gambling and the potential for addiction.

Finally, on the game design theory of fun and the intrinsic joy of learning, Koster’s book A Theory of Fun (Koster 2004) introduces the topic, and “Theory of Fun: 10 Years Later” (Koster 2012) presents a retrospective and examines lessons learned since the initial publication.

Individual Exercises

5.1. Gameplay Loops

Consider some popular game that you know well, either computer or physical.

  1. Draw an onion diagram for that game. Identify as many activity loops with different frequencies as you can.
  2. What is the smallest (highest frequency) activity loop where players engage in the same activity over and over again? Now what is the core loop or the smallest loop that is enjoyable even in the absence of larger ones? Is the core loop the same as the smallest activity loop?

5.2. Loops and Systems

Consider the game and loops from exercise 5.1. What kinds of game mechanics or systems do the different loops work with? Do different loops share any of those systems?

5.3. Flow

Describe your own experience of being “in the zone” while playing a game, whether getting lost in the game for a few minutes or for few hours.

  1. What was the game, and what was the activity that was so engrossing?
  2. How do you think the game achieved that? List the three core conditions of getting into the flow state and describe whether and how the game fulfilled them.

5.4. Fun of Learning

Describe two games you know, one where small loops are harder to learn and another where large loops are harder to learn. In both cases, describe how the enjoyment of the game changes as you learn the “easy” loops. Is it enjoyable to be challenged in a different way by the remaining loops?

5.5. Extrinsic Rewards

Describe some game you know well that successfully uses extrinsic rewards on schedules. What are the different kinds of activities that create rewards, and what schedules are being used? Does the game ramp up the workload required to get rewards as the player progresses?

Group Exercises

G5.1. Identifying and Changing Loops in a Board Game

In this exercise we will identify game loops in the board game Settlers of Catan and explore modifying them. (This exercise assumes you have access to this specific board game. The instructor may suggest substituting a different game in its place.)

  1. Play the game Settlers of Catan through at least once to remind yourself of gameplay. Then, identify gameplay loops by listing out all the various actions you can take in the game (e.g., trade resource cards, build a settlement, move the robber, etc.) and how frequently you get to perform these actions. Each group member should do their own list separately at first. Then have everybody compare their lists to produce a final list for the entire group.
  2. Now identify the longest, least frequent loops and remove them from the game by changing or removing the rules or game elements. For example, you can consider removing the robber piece and soldier cards, removing the ability to build cities, and other infrequent elements. Playtest this variant through at least once.
  3. Describe how these changes affected your experience of gameplay. Does this feel the same as playing the original? If not, how is it different?
  4. Try to “restore” the game by replacing the low-frequency loops you removed with different ones of your own devising. For each element you removed, find a new replacement that will also operate on a similar timescale. Playtest this variant at least once through as well.
  5. Describe how the game feels with this new ruleset compared to your previous variant and compared to the original rules.

    To complete the exercise, describe in detail:

  6. The loops you identified in the beginning
  7. The details of what you removed and how that affected gameplay
  8. The details of what you added and how that affected gameplay

  1. 12.  Other types of conditioning also exist. For example, classic conditioning studies how events that co-occur become associated together in the subject’s mind, such as the famous example of dogs learning to associate the sound of a bell with an imminent meal. We do not address them here, but please see (Hodent 2017) for more details and examples.

  2. 13.  There also exists a related random ratio schedule, in which each action may be rewarded separately based on random chance, such as the roll of dice rather than randomizing the total number of required actions. This schedule is commonly seen in gambling, such as slot machines (Schüll 2012, 344). However, this distinction is subtle and the two schedules are similar enough for our purposes, so we will use “variable ratio” as a shortcut for both.

  3. 14.  Although this particular data has been contributed by the community of players and can contain inaccuracies in the specific values, the general shape of the level curve is characteristic of games of this type.