Let’s Program A Prisoner’s Dilemma Part 6: Eye For An Eye

It’s finally time to write a prisoner with some actual brain power. Let’s start by looking at our current prisoners and their unique issues:

Saints always cooperate, making them an easy target for defectors.

Devils always defect, which leads to bad scores for everyone when multiple devils get together.

Madmen act randomly, which means whether or not they make good choices is up to chance.

What we really want is a prisoner that cooperates when cooperating is a good idea and defects in self defense when it suspects its about to be betrayed. A prisoner that can judge whether or not its partner deserves to be trusted.

Trust No One?

But how will our new “judge” type prisoner decide who to trust? The only thing it will know about its partner is their ID. There’s no way to force the other prisoner to reveal their strategy*, or at least there shouldn’t be. After all, the prisoner’s dilemma is meant to model real life and most real humans aren’t mind readers.

So how will our judge decide how to make decisions if he can’t read minds or see the future? The same way we humans do. We trust people who act trustworthy and we are suspicious of people who betray us.

This leads to a strategy called “tit-for-tat”, where you treat your partner the same way they treat you. If they betray you then the next time you meet them you betray them. If they cooperate then the next time you see them you pay them back by cooperating.

To pull this off we’re going to have to give our judges a memory.

Remember that reportResults function that we programed into the original prisoner class? We’ve been ignoring that so far because none of our prisoners needed it. Saints, devils and madmen always act the same way so they don’t really care how their partners act.

But our judge can use the info from reportResults to build a list of people he can and can’t trust. All we have to do is initialize a blank hash when the judge is created. Then every time reportResults is called we update that hash. We then use this hash in cooperate? by taking the provided partner ID, looking up in our hash whether or not they like to cooperate and then matching their behavior.

But wait! What do we do the first time we meet a new player? We can’t copy their behavior if we’ve never seen how they behave!

The answer is to remember the Golden Rule and “Treat others like you want to be treated”. That means the first time a judge meets a new player he will cooperate. Not only is this good manners, it helps make sure that when judges meet each other they will get locked into a loop of infinite cooperation instead of a loop of infinite betrayal.

Designing The Judicial Branch

The two things that make a judge a judge are:

1) It keeps track of how other prisoners treat it

2) If it meets a prisoner more than once it copies their last behavior

So let’s start with feature 1 by giving our prisoner a memory. To do that we’re going to have to create a hash during prisoner initialization and then use the learnResults function to fill it with useful data.

def initialize(id)
   super(id)
   @strategy = "Judge"
   @opponentBehavior = Hash.new()
end

def learnResults(opponentID, opponentCooperated)
   @opponentBehavior[opponentID] = opponentCooperated
end

Now that our opponentBehavior hash is filling up all that’s left is to use it as part of our decision making process. This is a pretty simple two step process. When our judge is asked whether or not it wants to cooperate with a given prisoner it starts by checking whether or not it has an opponentBehvaior entry for that ID. If it does it just mimics back whatever behavior it recorded for that ID, but if it’s empty it instead defaults to cooperation.

def cooperate?(opponentID)
   if( @opponentBehavior.key?(opponentID))
      return @opponentBehavior[opponentID]
   else
      return true
   end
end

Put it all together and what do you get?

class Judge < Prisoner

   def initialize(id)
      super(id)
      @strategy = "Judge"
      @opponentBehavior = Hash.new()
   end

   def cooperate?(opponentID)
      if( @opponentBehavior.key?(opponentID))
         return @opponentBehavior[opponentID]
      else
         return true
      end
   end

   def learnResults(opponentID, opponentCooperated)
      @opponentBehavior[opponentID] = opponentCooperated
   end
end

Featuring Four Different Character Classes!

Now let’s give our code a test by setting up a group with four judges, four saints, four devils and four madmen.

The Group’s Overall Score was -23546 in 1000 rounds with 16 prisoners

ID: 8 Score: -1126 Strategy: Devil

ID: 7 Score: -1170 Strategy: Devil

ID: 6 Score: -1198 Strategy: Devil

ID: 5 Score: -1254 Strategy: Devil

ID: 14 Score: -1391 Strategy: Judge

ID: 16 Score: -1394 Strategy: Judge

ID: 15 Score: -1398 Strategy: Judge

ID: 13 Score: -1425 Strategy: Judge

ID: 12 Score: -1479 Strategy: MadMan

ID: 9 Score: -1483 Strategy: MadMan

ID: 11 Score: -1498 Strategy: MadMan

ID: 10 Score: -1510 Strategy: MadMan

ID: 3 Score: -1766 Strategy: Saint

ID: 2 Score: -1790 Strategy: Saint

ID: 4 Score: -1802 Strategy: Saint

ID: 1 Score: -1862 Strategy: Saint

In our mixed group you can see that the judges outperform both the madmen and the saints and come pretty close to matching the devils.

But that’s not good enough. We didn’t want to just match the devils, we wanted to beat them! It’s absolutely infuriating that our genuinely intelligent judges are somehow being outscored by a bunch of dumb always-defectors.

Grudge Match

OK, time to get to the bottom of this. Why can’t our judges beat the devils? Let’s get rid of all the saints and madmen and isolate our problem.

The Group’s Overall Score was -17859 in 1000 rounds with 10 prisoners

ID: 10 Score: -1558 Strategy: Judge

ID: 7 Score: -1581 Strategy: Judge

ID: 6 Score: -1581 Strategy: Judge

ID: 9 Score: -1592 Strategy: Judge

ID: 8 Score: -1597 Strategy: Judge

ID: 2 Score: -1990 Strategy: Devil

ID: 3 Score: -1990 Strategy: Devil

ID: 4 Score: -1990 Strategy: Devil

ID: 5 Score: -1990 Strategy: Devil

ID: 1 Score: -1990 Strategy: Devil

Well would you look at that. When all you have are devils and judges the judges scream ahead into first place no problem.

It’s not that our judges weren’t smart enough to out-think the devils in the mixed game, it’s that the devils had a bunch of saints sitting around they could exploit for easy points. As soon as we took away the ever-forgiving all cooperators the devils sank like a rock while the judges pulled off a brilliant group victory by cooperating with each other and paying back the deceitful devils with defection after defection.

Impossible To Trust

Let’s finish up with a few more what-if scenarios, like “What if you stuck a bunch of judges with a bunch of saints?”

The Group’s Overall Score was -10000 in 1000 rounds with 10 prisoners

ID: 1 Score: -1000 Strategy: Saint

ID: 2 Score: -1000 Strategy: Saint

ID: 3 Score: -1000 Strategy: Saint

ID: 4 Score: -1000 Strategy: Saint

ID: 5 Score: -1000 Strategy: Saint

ID: 6 Score: -1000 Strategy: Judge

ID: 7 Score: -1000 Strategy: Judge

ID: 8 Score: -1000 Strategy: Judge

ID: 9 Score: -1000 Strategy: Judge

ID: 10 Score: -1000 Strategy: Judge

A ten way tie with a perfect group score, just like when we had a group of pure saints. As long as you’re nice to the judges they’ll be perfectly nice to you.

What if we get rid of the saints and just have judges?

The Group’s Overall Score was -10000 in 1000 rounds with 10 prisoners

ID: 1 Score: -1000 Strategy: Judge

ID: 2 Score: -1000 Strategy: Judge

ID: 3 Score: -1000 Strategy: Judge

ID: 4 Score: -1000 Strategy: Judge

ID: 5 Score: -1000 Strategy: Judge

ID: 6 Score: -1000 Strategy: Judge

ID: 7 Score: -1000 Strategy: Judge

ID: 8 Score: -1000 Strategy: Judge

ID: 9 Score: -1000 Strategy: Judge

ID: 10 Score: -1000 Strategy: Judge

Yet another perfect score! Since judges start off by cooperating they inevitably end up trusting each other and create just as nice a world as a group of pure saints would.

That’s enough happy thoughts for now. Why not try something more sinister? Like a group full of devils with only one judge?

The Group’s Overall Score was -19991 in 1000 rounds with 10 prisoners

ID: 1 Score: -1998 Strategy: Devil

ID: 2 Score: -1998 Strategy: Devil

ID: 3 Score: -1998 Strategy: Devil

ID: 4 Score: -1998 Strategy: Devil

ID: 5 Score: -1998 Strategy: Devil

ID: 9 Score: -1998 Strategy: Devil

ID: 7 Score: -1998 Strategy: Devil

ID: 8 Score: -1998 Strategy: Devil

ID: 6 Score: -1998 Strategy: Devil

ID: 10 Score: -2009 Strategy: Judge

As we’ve seen time and time again there is literally no way to win in a world full of devils. They will always betray you and so you can never get ahead.

But unlike the saint, which was absolutely destroyed by a gang of devils, the judge manages to at least keep his loses pretty low. As you can see from the numbers the judge cooperated exactly once with every single devil and then never trusted them again, thereby preventing them from leeching more points out of him.

* There is actually a variation of the prisoner dilemma where each prisoner is given a complete copy of their partner’s code as an input. We’re not doing that.

Let’s Program A Prisoners Dilemma 5: What Is A Decision Making Process?

The saint and devil prisoners were easy to write but, as we’ve seen, they’re not actually very good at playing the game. The always cooperating saints are wide open to exploitation and the devils will always defect even when cooperating would score them more points in the long run.

We clearly need a prisoner who can actually make choices instead of just doing the same thing again and again.

1d6 Points of SAN Loss

As any game designer can tell you, the easiest way to simulate decision making is with random numbers. Just give the computer a list of decisions and let it pick one at random.

So on that note I give you: The madman.

class MadMan < Prisoner
   def initialize(id)
      super(id)
      @strategy = "MadMan"
   end

   def cooperate?(opponentID)
      choice = rand(2)
      if( choice == 1)
         return true
      else
         return false
      end
   end
end

Ruby weirdness warning: In a lot of languages “0” is the same as false, so you might be tempted to have cooperate? just return the number generated by rand. Don’t do that. In Ruby only “false” and “null” are considered false. Everything else, including the number 0, are considered true. This is useful because it means the number 0 always acts like just a number. On the other hand it messes up a lot of other useful programming shortcuts so all in all it sort of break even.

Anyways, don’t forget to tell our create Prisoners method that there’s a new type of prisoner object for it to work with.

def createPrisoners(saintCount, devilCount, madmanCount)
   prisoners = Array.new
   playerCounter = 0
   saintCount.times{ prisoners.push(Saint.new(playerCounter += 1)) }
   devilCount.times{ prisoners.push(Devil.new(playerCounter += 1)) }
   madmanCount.times{ prisoners.push(MadMan.new(playerCounter += 1)) }
   return prisoners
end

Inmates Are Running The Asylum

Let’s be honest here: Making decisions at random is almost never a good idea. So how does our new class of insane prisoners perform in an actual game?

prisoners = createPrisoners(4, 4, 4)
playPrisonersDilemma(prisoners, 1000)

Ten doesn’t divide evenly into thirds so this time we’ll have four of each type of prisoner. Please not that this changes the perfect score to -12,000.

The Group’s Overall Score was -17905 in 1000 rounds with 12 prisoners

ID: 6 Score: -876 Strategy: Devil

ID: 7 Score: -882 Strategy: Devil

ID: 8 Score: -892 Strategy: Devil

ID: 5 Score: -906 Strategy: Devil

ID: 12 Score: -1499 Strategy: MadMan

ID: 9 Score: -1504 Strategy: MadMan

ID: 10 Score: -1538 Strategy: MadMan

ID: 11 Score: -1564 Strategy: MadMan

ID: 2 Score: -2026 Strategy: Saint

ID: 1 Score: -2060 Strategy: Saint

ID: 4 Score: -2074 Strategy: Saint

ID: 3 Score: -2084 Strategy: Saint

Madmen randomly flip between acting like saints and acting like devils so it makes sense they would wind up scoring squarely in between the two. They don’t just let the devils betray them; sometimes they betray right back. And they alternate between cooperating with saints for small gains and betraying them for big gains.

So all in all it seems like mild insanity is actually a pretty well rounded fit for the cutthroat world of the prisoner’s dilemma.

Also, as promised, the fact that madmen can make actual decisions means that our overall group score now has some variation to it even when running multiple games with the same group.

The Group’s Overall Score was -17995 in 1000 rounds with 12 prisoners

The Group’s Overall Score was -18046 in 1000 rounds with 12 prisoners

The Group’s Overall Score was -17938 in 1000 rounds with 12 prisoners

The Lost And The Damned

So madmen seem to do pretty well in a mixed group… but maybe that’s just because they had some saints to act as backup. What happens when we pair up only devils and madmen?

The Group’s Overall Score was -17523 in 1000 rounds with 10 prisoners

ID: 4 Score: -1440 Strategy: Devil

ID: 1 Score: -1450 Strategy: Devil

ID: 2 Score: -1456 Strategy: Devil

ID: 3 Score: -1456 Strategy: Devil

ID: 5 Score: -1472 Strategy: Devil

ID: 9 Score: -2013 Strategy: MadMan

ID: 7 Score: -2014 Strategy: MadMan

ID: 8 Score: -2068 Strategy: MadMan

ID: 6 Score: -2072 Strategy: MadMan

ID: 10 Score: -2082 Strategy: MadMan

About the same thing as when there were saints, it turns out. The madmen’s habit of cooperating roughly half the time means they still can’t directly compete with the vicious defecting devils, but at least randomly defecting half of the time allows them to sort of defend themselves.

In fact, if you compare this to the time we evenly paired up devils and saints you’ll see that the madmen scored about the same as the saints. But the big difference is that the madmen did much more damage to the devils in the process.

Although it’s up in the air as to whether this is a good thing or not. Taking the devils down a notch is certainly a satisfying feeling but the madmen still lost and the group score is much much worse then when the saints had their match with the devil.

The More Randomness You Have The Less Random It Is

For our final experiment I just want to point out that while I call them “Madmen” the random decision prisoners actually managed to achieve some pretty reliable results, consistently scoring halfway between a saint and a devil.

This is of course because random numbers tend to average out over time. So “cooperates at random” eventually transforms into “consistently cooperates 50% of the time”.

To show this off I’m going to have a bunch of madmen play increasingly long games against each other.

prisoners = createPrisoners(0, 0, 10)
playPrisonersDilemma(prisoners, 10)
playPrisonersDilemma(prisoners, 100)
playPrisonersDilemma(prisoners, 1000)
playPrisonersDilemma(prisoners, 1000000)

The Group’s Overall Score was -141 in 10 rounds with 10 prisoners

ID: 10 Score: -7 Strategy: MadMan

ID: 1 Score: -10 Strategy: MadMan

ID: 6 Score: -12 Strategy: MadMan

ID: 8 Score: -13 Strategy: MadMan

ID: 9 Score: -14 Strategy: MadMan

ID: 5 Score: -15 Strategy: MadMan

ID: 2 Score: -16 Strategy: MadMan

ID: 3 Score: -17 Strategy: MadMan

ID: 7 Score: -18 Strategy: MadMan

ID: 4 Score: -19 Strategy: MadMan

The Group’s Overall Score was -1493 in 100 rounds with 10 prisoners

ID: 8 Score: -132 Strategy: MadMan

ID: 3 Score: -135 Strategy: MadMan

ID: 6 Score: -139 Strategy: MadMan

ID: 5 Score: -144 Strategy: MadMan

ID: 1 Score: -145 Strategy: MadMan

ID: 9 Score: -154 Strategy: MadMan

ID: 7 Score: -156 Strategy: MadMan

ID: 4 Score: -159 Strategy: MadMan

ID: 2 Score: -163 Strategy: MadMan

ID: 10 Score: -166 Strategy: MadMan

The Group’s Overall Score was -14976 in 1000 rounds with 10 prisoners

ID: 3 Score: -1437 Strategy: MadMan

ID: 8 Score: -1457 Strategy: MadMan

ID: 1 Score: -1482 Strategy: MadMan

ID: 4 Score: -1487 Strategy: MadMan

ID: 7 Score: -1491 Strategy: MadMan

ID: 10 Score: -1492 Strategy: MadMan

ID: 5 Score: -1503 Strategy: MadMan

ID: 6 Score: -1514 Strategy: MadMan

ID: 9 Score: -1537 Strategy: MadMan

ID: 2 Score: -1576 Strategy: MadMan

The Group’s Overall Score was -15001829 in 1000000 rounds with 10 prisoners

ID: 3 Score: -1498113 Strategy: MadMan

ID: 10 Score: -1499339 Strategy: MadMan

ID: 1 Score: -1499525 Strategy: MadMan

ID: 9 Score: -1500065 Strategy: MadMan

ID: 7 Score: -1500128 Strategy: MadMan

ID: 2 Score: -1500445 Strategy: MadMan

ID: 4 Score: -1500662 Strategy: MadMan

ID: 5 Score: -1500894 Strategy: MadMan

ID: 8 Score: -1501085 Strategy: MadMan

ID: 6 Score: -1501573 Strategy: MadMan

As you can see, the longer the game lasts the less difference there is in the scores.

After ten rounds the worst score (-19) was almost three times as bad as the best score (-7).

After one hundred rounds the worst score (-166) was only about 25% worse than the best score (-132).

After a thousand rounds there was only a 10% difference between best (-1437) and worst (-1576).

And after a million rounds there was less than a 1% difference between best and worst.

So in the long run cooperating at random is a viable way to take a middle path cooperation and defection.

We Can Be Smarter Than This

Random decision making lead to some interesting outcomes but it failed to come even close to beating the devils. Plus it’s an embarrassingly simple algorithm for AI enthusiasts like ourselves. Surely we can come up with a smarter prisoner. One that actually thinks instead of guessing.

But that’s going to have to wait for next time.

Let’s Program A Prisoners Dilemma Part 4: Heaven or Hell, Let’s Rock!

Last time we wrote enough code to actually play the prisoner’s dilemma. Combined with the Saint and Devil prisoners we wrote we now have enough code to run some interesting experiments.

Heaven

Let’s start by putting together a group of nothing but saints and seeing what happens to them after a thousand rounds.

#A group of all saints
prisoners = createPrisoners(10, 0)
playPrisonersDilemma(prisoners, 1000)

Which results in this:

The Group’s Overall Score was -10000 in 1000 rounds with 10 prisoners

ID: 1 Score: -1000 Strategy: Saint

ID: 2 Score: -1000 Strategy: Saint

ID: 3 Score: -1000 Strategy: Saint

ID: 4 Score: -1000 Strategy: Saint

ID: 5 Score: -1000 Strategy: Saint

ID: 6 Score: -1000 Strategy: Saint

ID: 7 Score: -1000 Strategy: Saint

ID: 8 Score: -1000 Strategy: Saint

ID: 9 Score: -1000 Strategy: Saint

ID: 10 Score: -1000 Strategy: Saint

Because the saints always cooperate they all ended up with the same score. What is much more interesting is their group score, which is actually perfect.

Why is -10,000 perfect? In every round every pair can do one of three things: if both prisoners cooperate the pair loses only 2 points, if one prisoner cooperates but the other defects the pair loses 3 points and if both defect the pair loses 4 points. Ten prisoners means five pairs. Five pairs multiplied by the best case scenario of -2 points gives us -10 points per round. Multiply that by the 1000 rounds we played the game and you find out that the best possible score for a group of this size playing a game this long is in fact -10,000 points.

So, unsurprisingly, when you have a group made up of nothing but saints you get a really really good outcome.

Hell

Now let’s try the inverse and put 10 devil’s together.

#A group of all devils
prisoners = createPrisoners(0, 10)
playPrisonersDilemma(prisoners, 1000)

Which turns out like this:

The Group’s Overall Score was -20000 in 1000 rounds with 10 prisoners

ID: 1 Score: -2000 Strategy: Devil

ID: 2 Score: -2000 Strategy: Devil

ID: 3 Score: -2000 Strategy: Devil

ID: 4 Score: -2000 Strategy: Devil

ID: 5 Score: -2000 Strategy: Devil

ID: 6 Score: -2000 Strategy: Devil

ID: 7 Score: -2000 Strategy: Devil

ID: 8 Score: -2000 Strategy: Devil

ID: 9 Score: -2000 Strategy: Devil

ID: 10 Score: -2000 Strategy: Devil

The devils also wound up with a ten-way tie, but the real story is in that group score: -20,000 points is the absolute worst score a group this size can manage! (-4 points for a double defection * 5 pairs * 1000 rounds)

You probably saw that coming. Of course a big group of devils is going to make an absolute mess of things. I didn’t just draw that name out of a hat.

Wolf Among Sheep

So if you have a group of all saints they will all tie for first place while a group of all devils will tie for last place. That’s good to know but doesn’t do much good for those of us living in a world full of both good and bad people.

So let’s mix things up!

#Saints with one devil among them
prisoners = createPrisoners(9, 1)
playPrisonersDilemma(prisoners, 1000)

The Group’s Overall Score was -11000 in 1000 rounds with 10 prisoners

ID: 10 Score: 0 Strategy: Devil

ID: 5 Score: -1188 Strategy: Saint

ID: 6 Score: -1206 Strategy: Saint

ID: 4 Score: -1210 Strategy: Saint

ID: 2 Score: -1212 Strategy: Saint

ID: 1 Score: -1216 Strategy: Saint

ID: 8 Score: -1220 Strategy: Saint

ID: 9 Score: -1232 Strategy: Saint

ID: 3 Score: -1236 Strategy: Saint

ID: 7 Score: -1280 Strategy: Saint

This test is actually a little interesting because it’s the first one we’ve run where the score will be different every time you play the game. Run the program a few times in a row and see for yourself: The scores will move move up and down a bit. The devil always gets a perfect score but the individual saint scores will go up and down based on how often they had the bad luck of getting stuck with the all-betraying devil.

The fact that the devil always get a perfect score is also pretty interesting, although it isn’t hard to figure out how he pulled it off. The devil never accepts the -1 penalty for cooperating and the saints never penalize their partner with the -2 points of betrayal, so there is literally no way for the devil to lose.

Also, look at the group score. The devil may have messed up the saints ten way tie but at least the group is still within 10% of being perfect.

Diabolic Duo

It doesn’t seem fair to have a single devil running around taking advantage of everything, so let’s give him a taste of his own medicine by introducing a second devil to the mix.

#A group of mostly saints
prisoners = createPrisoners(8, 2)
playPrisonersDilemma(prisoners, 1000)

The Group’s Overall Score was -12000 in 1000 rounds with 10 prisoners

ID: 10 Score: -232 Strategy: Devil

ID: 9 Score: -232 Strategy: Devil

ID: 8 Score: -1408 Strategy: Saint

ID: 7 Score: -1438 Strategy: Saint

ID: 1 Score: -1444 Strategy: Saint

ID: 6 Score: -1446 Strategy: Saint

ID: 2 Score: -1448 Strategy: Saint

ID: 3 Score: -1448 Strategy: Saint

ID: 4 Score: -1452 Strategy: Saint

ID: 5 Score: -1452 Strategy: Saint

Once again the scores will move up and down a bit depending on how often the devils get paired up and betray each other versus how often they get paired up with a defenseless saint, but the overwhelming trend here is for the devils to come out ahead. After all, each devil has an 8 out of 9 chance of getting paired with an easy to exploit saint.

(It is technically possible for the devils to lose if they get paired up a lot… but the odds of that happening a thousand times in a row is really really low. Try running shorter games if you want to see what happens when lightning strikes and devils always get stuck together).

Other interesting things to note include the fact that the devils will always have the same score (since they only lose points when they get paired together and then they always both lose two points). It’s also worth noting that our group score is now 20% worse than perfect. If this was a neighborhood it would still be a nice place to live even if a few of your neighbors smell like sulfur and brimstone and tend to park their cars in front of other people’s driveways.

Wheat and Tears

Let’s level the playing field and mix together an even number of saints and devils.

#An even mix of saints and devils
prisoners = createPrisoners(5, 5)
playPrisonersDilemma(prisoners, 1000)

The Group’s Overall Score was -15000 in 1000 rounds with 10 prisoners

ID: 6 Score: -900 Strategy: Devil

ID: 9 Score: -900 Strategy: Devil

ID: 7 Score: -910 Strategy: Devil

ID: 10 Score: -924 Strategy: Devil

ID: 8 Score: -934 Strategy: Devil

ID: 4 Score: -2030 Strategy: Saint

ID: 2 Score: -2080 Strategy: Saint

ID: 1 Score: -2086 Strategy: Saint

ID: 5 Score: -2116 Strategy: Saint

ID: 3 Score: -2120 Strategy: Saint

About what you would expect. The devils still manage to absolutely destroy the saints. Even worse, our group score is starting to get pretty bad and even after exploiting the saints mercilessly the devils are only a little better off than they would have been if they had decided to cooperate.

I guess the lesson here is that crime only pays when you have a small number of criminals exploiting a large population of good productive citizens. If you have too many criminals there just isn’t enough loot to go around.

A Light in the Darkness

For our last experiment let’s see what happens when you have a group made up mostly of devils and throw in a single saint. Doesn’t take a genius to predict this is going to be pretty bad…

#A single saint in a group of devils
prisoners = createPrisoners(1, 9)
playPrisonersDilemma(prisoners, 1000)

The Group’s Overall Score was -19000 in 1000 rounds with 10 prisoners

ID: 6 Score: -1756 Strategy: Devil

ID: 3 Score: -1766 Strategy: Devil

ID: 10 Score: -1766 Strategy: Devil

ID: 7 Score: -1774 Strategy: Devil

ID: 8 Score: -1774 Strategy: Devil

ID: 4 Score: -1780 Strategy: Devil

ID: 9 Score: -1792 Strategy: Devil

ID: 5 Score: -1794 Strategy: Devil

ID: 2 Score: -1798 Strategy: Devil

ID: 1 Score: -3000 Strategy: Saint

Ouch. Our poor saint wound up with the absolute worst possible individual score this time around. In every single round the saint hit himself with a -1 penalty for cooperating and then got with a -2 penalty from his defecting partner. Doesn’t get any worse. How perfectly terrible.

The group score is also pretty horrible and the devils are now significantly worse off than if they had just cooperated. (Fun experiment, mess with the ratio of saints to devils and find the exact point at which being a devil no longer pays).

One Last Observation

Let’s finish this post off with one last interesting trend you may have noticed: Running an experiment multiple times in a row caused individual scores to bounce around a lot but the group score never changed by even a single point. Why is that?

I’ll give you a few moments to think about it.

The answer is simple enough. Saints always cooperate, which causes the group to lose one point per round. Devils always defect, which causes the group to lose two points per round. Doesn’t matter how you mix up the pairs; the group as a whole will always lose one point per saint and two points per devil.

To get a group score that occasionally changes we would have to have a prisoner that sometimes cooperates (losing the group only one point) and sometimes defects (losing the group two points). Then the group score could go up or down based on how that individual prisoner decided to play.

Which is exactly what we’re going to be doing next time!

Let’s Program A Prisoner’s Dilemma Part 3: Fight In Cell Block D

Last time we programmed a couple prisoners and ran them through their paces. This time we’re going to write the code for the actual prisoner’s dilemma game.

We start by pseudo-coding our algorithm:

  • First: generate a bunch of prisoners objects.
  • Second: decide how many rounds the game should last.
  • During each round:
    •    Randomly pair up prisoners.
    •    Each prisoner is asked whether or not they want to cooperate with their partner
    •    Prisoners lose points based on the decision they and their partner made
    •    Move on to the next round by randomly sorting prisoners into new pairs
  • When all the rounds are done show the stats of every prisoners so we know who won (by losing the least points).
  • Also show the total sum of all the prisoners’ scores so we can compare different mixes of prisoners.

Breaking News: Prisons Filling Up At An Alarming Rate

Let’s start by figuring out how to set up a group of prisoners. We could just hard code an array of prisoner objects but that would leave us in the bad situation of having to rewrite actual code every time we wanted to try a new mix of prisoners. That sounds both boring and error prone so instead let’s come up with a function we can use to create a mix of prisoners on the fly.

def createPrisoners(saintCount, devilCount)
   prisoners = Array.new
   playerCounter = 0
   saintCount.times{ prisoners.push(Saint.new(playerCounter += 1)) }
   devilCount.times{ prisoners.push(Devil.new(playerCounter += 1)) }
   return prisoners
end

We just tell this function how many saints and devils we want and it gives us exactly what we asked for packaged inside a convenient array. It also keeps count of how many prisoners it has created so far and uses that count to make sure every prisoner has a unique ID.

This function also shows off a rather handy Ruby shortcut for writing a classic “for loop”. If you want a loop that runs a certain number of times just take that number, or a variable holding that number, and add .times{ your code here } to the end. Ex: 4.times{ puts “This will print four times” }

Exciting Next-gen Gameplay!

To actually play the prisoners dilemma we need a group of prisoners and an idea of how many rounds we want the game to last, which means our function definition probably needs to look a little like this:

def playPrisonersDilemma(prisoners, rounds)

Then inside the function itself we want to set up a loop that runs once for every round in the game. During those rounds we want to randomly shuffle our prisoners, divide them up into pairs and then ask every prisoner whether they want to cooperate with their temporary partner or not.

At that point we subtract one point from any player who chose to cooperate and subtract two points from any player who was betrayed by their partner (Yes, a player can be hit by both penalties in the same round).

rounds.times{
   pairs = prisoners.shuffle.each_slice(2)
   pairs.each{ |pair|
      firstPlayerCooperates = pair[0].cooperate?(pair[1].id)
      secondPlayerCooperates = pair[1].cooperate?(pair[0].id)

      if(firstPlayerCooperates)
         pair[0].score -= 1
      else
         pair[1].score -= 2
      end

      if(secondPlayerCooperates)
         pair[1].score -= 1
      else
         pair[0].score -= 2
      end

      pair[0].learnResults(pair[1].id, secondPlayerCooperates)
      pair[1].learnResults(pair[0].id, firstPlayerCooperates)
   }
}

Once again we’re using handy Ruby shortcuts to save a bit of typing. We use rounds.times to set up a loop that will play the game the proper number of times. We then use prionser.shuffle to randomly mix up the prisoners and then chain it to each_slice(2) to divide the random mix into random pairs.

Important note: shuffle and slice don’t change the original array. They instead return a transformed copy that has to be assigned to a variable before you can use it. But for us this is actually a good thing because it means we can just shuffle and slice the same prisoners array at the start of each loop without having to worry about it somehow getting messed up between iterations.

Once we’ve got our random pair array we can use pairs.each to write some quick code we want to run once for each piece of data in our array. The each loop starts out by grabbing an item from the array and storing it inside whatever variable we’ve named with the | variable | syntax.

In our case the pairs array is full of tiny two item arrays, so we call our variable pair. From there it’s pretty simple to each half ot he pair whether or not it wants to cooperate with the other half and then we can assign points. Remember our scoring rules: A player who cooperates loses one point. A player who refuses to cooperate forces the other player to lose two points. We also call the learnResults function to let each prisoner know how the round played out.

After the rounds.times loop has finished playing the game all that’s left is to print out the final score.

#Show the stats for the group as a whole as well as for each individual prisoner
groupScore = 0
prisoners.each{ |prisoner| groupScore += prisoner.score }
puts "The Group's Overall Score was #{groupScore} in #{rounds} rounds with #{prisoners.length} prisoners"
prisoners.sort{ |x, y| y.score <=> x.score}.each{ |prisoner| prisoner.report }

Nothing complicated here. We use an each loop to tally up the scores from every player so we can print a group score. Then we sort the prisoners by score and use another each loop to print out the individual stats for every prisoner.

All Together Now

Gluing together all the bits of code we just wrote leaves us with this nice function:

def playPrisonersDilemma(prisoners, rounds)
   if !prisoners.length.even?
      throw "Prisoners Dilemma requires an even number of participants"
   end

   # Make sure each prisoner starts out with a clean slate
   prisoners.each{ |prisoner| prisoner.score = 0}
   
   rounds.times{
      pairs = prisoners.shuffle.each_slice(2)
      pairs.each{ |pair|
         firstPlayerCooperates = pair[0].cooperate?(pair[1].id)
         secondPlayerCooperates = pair[1].cooperate?(pair[0].id)

         if(firstPlayerCooperates)
            pair[0].score -= 1
         else
            pair[1].score -= 2
         end

         if(secondPlayerCooperates)
            pair[1].score -= 1
         else
            pair[0].score -= 2
         end
      }
   }

   #Show the stats for the group as a whole as well as for each individual prisoner
   groupScore = 0
   prisoners.each{ |prisoner| groupScore += prisoner.score }
   puts "The Group's Overall Score was #{groupScore} in #{rounds} rounds with #{prisoners.length} prisoners"
   prisoners.sort{ |x, y| y.score <=> x.score}.each{ |prisoner| prisoner.report }
end

And now we can test it by having a group of prisoners play the game. Let’s try it out with a group of two saints and two devils playing for a thousand rounds:

prisoners = createPrisoners(2, 2)
playPrisonersDilemma(prisoners, 1000)

This should give you output kind of like this:

The Group’s Overall Score was -6000 in 1000 rounds with 4 prisoners

ID: 4 Score: -650 Strategy: Devil

ID: 3 Score: -650 Strategy: Devil

ID: 2 Score: -2350 Strategy: Saint

ID: 1 Score: -2350 Strategy: Saint

Hmm… looks like the Devils had no trouble at all completely crushing the Saints. We’ll look at that match up in more detail next time.

Let’s Program A Prisoner’s Dilemma Part 2: Crime And Punishment

Last time we learned about a game theory thought experiment called the “Iterated Group Prisoner’s Dilemma”. Basically players get randomly paired up and have to decide to either cooperate by accepting a small penalty for themselves or defect by shoving a big penalty onto the other player. The players then get shuffled around and repeat the game with a new partner. After a few hundred rounds (or more) the game comes to a stop and the winner is whoever has lost the fewest points.

This time we’re going to start programing a simulator for that game so we can watch how different strategies play out.

I will be writing this project in Ruby. It’s not really “better” for this program than any of the various other high level languages we could choose from, but I’ve been using it a lot as part of my game programming hobby and figured I might as well stick with it while it was in my head.

So if you want to follow along with my exact code open up a file named prisonerdilemma.rb in your favorite text editor and get ready for some Ruby. Of course, rewriting the project in your favorite language is always an option.

Introducing Prisoner 24601

The most important part of the prisoner’s dilemma is the prisoners. So let’s make a list of what our little digital prisoners need to be able to do in order for this simulation to work.

1) They need to have a unique ID so that other prisoners (and us humans) can recognize them.

2) They need to be able to decide whether to cooperate or defect with other prisoners based on ID.

3) They need to be able to see whether their opponent decided to cooperate or defect.

4) They need to be able to report what strategy they are using so we can keep track of how different strategies perform.

5) They need to be able to keep track of their score so we can calculate who is winning and losing.

Based on all this I’ve thrown together a very simple prisoner class for us to start working with. It can’t really do much except book keeping though. It has variables to keep track of it’s own ID, score and strategy and even exposes the score to outside code so we can update it during the game. It also has a convenient “report” function that will print out all that information in human friendly format.

As for actual game playing code… it does technically have a “cooperate?” function for deciding how to play the game along with a “learnResults” function for keeping track of its opponents. But at the moment both of these functions are empty because this is just skeleton code; it isn’t meant to actually do anything. In fact, if you try to call the “cooperate?” function it will throw an error!

class Prisoner
   attr_reader :id, :strategy, :score
   attr_writer :score

   def initialize(id)
      @id = id;
      @score = 0;
      @strategy = "Prisoner Parent Class"
   end

   def cooperate?(opponentID)
      throw "Prisoner child class must implement a cooperate? method"
   end

   def learnResults(opponentID,oponentCooperated)
   end

   def report
      puts "ID: #{@id} Score: #{@score} Strategy: #{@strategy}"
   end
end

Still, even with two incomplete functions we can still give this a test run. If you were to add some code like this to your project:

testprisoner = Prisoner.new(6)
testprisoner.score = -10
testprisoner.report

You would get some output like this:

ID: 6 Score: -10 Strategy: Prisoner Parent Class

Ye Not Guilty – The “Saint” Strategy

Now that we’ve got a nice generic class up and running we can move on to writing a prisoner with actual dilemma solving code!

For our first attempt we’re going to be writing the simplest possible of all prisoner algorithms: A player who always always chooses to cooperate. We’ll be calling this the “Saint” strategy.

Now the actual code for this will be virtually identical to the generic prisoner. It will just have a different name inside of the “strategy” variable and its “cooperate” function will return true instead of throwing an error.

But we all know that copy pasting code is horrible and hard to maintain, so we’ll be using the miracle of inheritance to avoid any copy pasting.

class Saint < Prisoner
   def initialize(id)
      super(id)
      @strategy = "Saint"
   end

   def cooperate?(opponentID)
      return true
   end
end

That “Saint < Prisoner” tells the compiler that Saint is a type of Prisoner and should get a free copy of all of its code. Then instead of writing an entire new class from scratch we only have to write the few bits that are different.

First up we write a new initialize function so we can set the strategy variable to “Saint” instead of the default from the Prisoner class. But we still want the Prisoner class to handle things like setting our ID and initializing our score so we start out by calling the generic Prisoner initialize function with a call to “super”.

The super call checks whether the parent has a function with the same name as the current function and then calls it. It’s really useful for when you want a child function to do everything the parent did and then a little extra. Just start with super to get the parent’s behavior and then add in the unique child logic afterwards.

Next we write a new cooperate? function, which for our super trusting saint just involves returning “true”. In this case we want to completely override the parent’s version of the function so we leave out the call to super and just write new code.

Father of Lies

That was pretty easy, so let’s keep going by programming a second type of prisoner that perfectly mirrors our first. This time we’ll create a player that always always chooses to defect. Let’s call this the “Devil” strategy.

class Devil < Prisoner
   def initialize(id)
      super(id)
      @strategy = "Devil"
   end

   def cooperate?(opponentID)
      return false
   end
end

Just like with the “Saint” we use inheritance to grab our basic Prionser code and then just add in our new strategy name and cooperate? Algorithm. Couldn’t be easier.

A Not So Epic Clash Between Good And Evil

And now here’s a little code snippet you can use for testing that both of our new classes work:

testSaint = Saint.new(1)
testSaint.report
testDevil = Devil.new(2)
testDevil.report
puts "Saint Cooperates?"
puts testSaint.cooperate?(testDevil.id)
puts "Devil Cooperates?"
puts testDevil.cooperate?(testSaint.id)

Which results in this output:

ID: 1 Score: 0 Strategy: Saint

ID: 2 Score: 0 Strategy: Devil

Saint Cooperates?

true

Devil Cooperates?

False

Let’s Get Ready To Rumble

Now that we have some players assembled our next task will be setting up an actual game for them to play in. But that will have to wait until the next post.

Let’s Program A Prisoner’s Dilemma Part 1: Game Theory Is Less Fun Than It Sounds

A Different Kind Of Gaming

Game Theory, despite the name, is unfortunately not “The study of videogames”. Instead it is the science of mathematically analyzing how decisions are made.

The reason it’s called game theory is because one popular way to analyze real world decision making is to come up with a simplistic “game” that mimics the core dynamic of the real world problem. The game can then be mathematically analyzed and the results applied back to the more complex real world situation.

For instance, a game theorist studying crime might come up with a “game” where people who follow the law get 1 point and people who break the law have a chance of gaining two points but also a chance of losing three points. He can then mathematically analyze how different risk levels influence whether or not crime is a winning strategy.

The Prisoner’s Dilemma

The most famous example of a game theory game is the “Prisoner’s Dilemma”. Two criminal are caught red-handed committing a small crime that will land them in jail for one year. The police also suspect the two criminals are guilty of a larger crime worth two years of jail time, but they don’t have enough evidence to prove it.

The police eventually come up with a plan to interview each criminal separately and offer them this deal: Testify about the other guy’s involvement in the larger crime and we’ll drop the current small charge against you.

And now here’s the “game”: As one of the prisoners do you protect your buddy and keep quiet even though it means going to jail for a year? Or do you tattle in hopes of going free? And how do you know whether or not you can trust the other guy? If he confesses you go to jail for an extra two years.

Other Guy Keeps Quiet Other Guy Confesses
You Keep Quiet You both go to jail for one year You go to jail for three years

Other guy goes free

You Confess You go free

Other guy goes to jail for three years

You both go to jail for two years

Now as mathematicians we want to be able to analyze this choice with actual numbers, so let’s do a little game theorizing. Instead of two prisoners let’s imagine a two player game where each player is allowed to choose to either play nice and cooperate or to betray their partner and defect. If you cooperate you lose 1 point. If you defect the OTHER player loses 2 points.

Player B Cooperates Player B Defects
Player A Cooperates Player A: -1 point

Player B: -1 point

Player A: -3 points

Player B: -0 points

Player A Defects Player A: -0 points

Player B: -3 points

Player A: -2 points

Player B: -2 points

What makes this such an interesting game to study is the fact that the best group strategy and the best individual strategy are complete opposites.

As a group the obvious solution is for both players to cooperate so that between them they only lose 2 points. If one player defects the total loss jumps up to 3 points and if both players defect the total loss is 4 points, as bad as it gets.

But as an individual the equally obvious solution is to defect. After all, if you cooperate you are guaranteed to lose 1 point and might even lose 3 points if the other player defects. But if you yourself defect you might not lose any points and even if the other player defects you only lose 2. There is just no good reason to purposely give up one point.

So the smartest thing to do as an individual is to defect but the smartest thing to do as a group is to cooperate and there doesn’t seem to be any logical way to bridge the gap between these two strategies. Or is there?

Iterated Prisoner’s Dilemma

The core problem in the prisoners dilemma is the fact that it’s a one-time choice. You either cooperate or defect and a few seconds later you find out who won and who lost, end of story. This is why there is no logical reason not to betray the other player. If you defect you’re guaranteed to at least tie so why risk anything else?

But what if you had to play the game multiple times?

This could change things up. Now your decision to cooperate or defect might impact how future games are played. There is now a possibility to build or lose trust.

But sadly you can show that even with multiple games the best logical strategy is still to always defect. The logic goes like this:

1) During the last round you might as well defect since there are no more future rounds in which the other player could punish you.

2) Since a logical opponent will defect during the last round you might as well defect during the second to last round. It’s not like your opponent was going to cooperate in that last round anyways.

3) Since a logical opponent will defect during the second to last round you might as well defect during the third to last round…

4) Continue the pattern and you can see that always defecting is the best strategy.

There are only two ways to break out of this cycle.:

One is to have an infinite number of rounds so that players are always at risk of being punished in the future for bad behavior in the present. We’re not going to do that though for the simple reason that there is no way to run an infinite number of Prisoner’s Dilemmas on a finite computer.

The other way to break the cycle is to introduce more players.

Iterated Group Prisoner’s Dilemma

The iterated group prisoners dilemma works the same as the normal iterated prisoner dilemma except that you have several different players who get randomly paired up before every round.

This drastically changes the dynamics of the game.

In a two-player prisoner’s dilemma double defection is a “stable” strategy because it means neither player will get ahead of the other. You might be losing two points every round but the other guy is too so you aren’t actually losing.

But when there’s a group? Now a players who is constantly losing two points from double defection can end up falling behind other pairs of players who cooperate and thus only lose one point per round. Suddenly it’s not enough to try and figure out whether your current partner is going to cooperate or defect; you also have to worry about whether other players in other matches are cooperating or defecting.

Interesting… But Is It Useful?

The Iterated Group Prisoners Dilemma is not only mathematically interesting, it is highly relevant to real life. Ever day people are faced with scenarios where they can choose to either work together or try to cheat each other. In the long run it’s best for society when people work together but on an individual level exploiting others can provide huge short term benefits.

So by analyzing the prisoners dilemma we can get some interesting insights into why different societies work the way they do along with some ideas on what sort of ethical systems are and aren’t stable.

Of course, real life has all sorts of issues that aren’t fully reflected in a simple tool like the prisoners dilemma so don’t think it somehow has all the answers to all our problems. But it’s still better than nothing and at the very least provides a nice topic for small talk with other math geeks.

What Does Any Of This Have To Do With Programming?

As AI enthusiasts we are obviously very interested in writing programs that can make logical decisions, and the prisoners dilemma provides a great sandbox for practicing that skill. It’s simple enough of a scenario to be implemented in less than a page of code while still being complex enough to produce interesting results.

So here’s the plan for this Let’s Program: We’re going to write a Prisoner’s Dilemma simulation along with a few different types of players each with their own strategy for deciding when to cooperate and when to defect. We’ll then run a bunch of simulations and see if we can find any interesting trends.

Like usual this isn’t exactly an original idea. Game theorists have been writing and testing prisoners dilemma programs for years and if you’re familiar with their research you probably already know everything we’re going to cover in this Let’s Program.

But who cares? The goal here is to practice our programming, not push the cutting edge of computer science. And there’s bound to be at least one reader in my audience who isn’t already a game theory expert.

Actually, quick show of hands: Is there anyone here who has never studied game theory before reading this blog?

There, see? That guy over there is the reason we’re doing this. He’s about to learn something cool.

Let’s Program A JavaScript Game: Index and Code

JavaScript is a powerful programming language that’s built right into most web browsers. Because of this it’s rapidly become the default way to provide user friendly interactive web experiences. The Internet experience as we know it today wouldn’t exist without good old JS.

But who cares about silly little things like that? The real question is: Can we use it to program games? And thanks to the introduction of HTML5 elements like the “canvas” the answer is now a resounding “YES”.

So come along and join me as we program ourselves a simple JavaScript web game.

Index

Let’s Program A JavaScript Game 1: Browser Games With JavaScript

Let’s Program A JavaScript Game 2: The Canvas Of Dreams

Let’s Program A JavaScript Game 3: Designing “Defrag Cycle”

Let’s Program A JavaScript Game 4: V8 600HP Game Engine

Let’s Program A JavaScript Game 5: Press Start To Play

Let’s Program A JavaScript Game 6: When Worlds Collide

Let’s Program A JavaScript Game 7: When Worlds Collide For Real

Let’s Program A JavaScript Game 8: That Sinking Feeling

Let’s Program A JavaScript Game 9: Look Before You Leap

Let’s Program A JavaScript Game 10: Letting The World Pass By

Let’s Program A JavaScript Game 11: This Post Isn’t About Malware, Honest

Let’s Program A JavaScript Game 12: Making A STATEment

Let’s Program A JavaScript Game 13: Some Much Needed Polish

Let’s Program A JavaScript Game 14: A Winner Is You

Let’s Program A JavaScript Game 15: The Computer Is Out To Get You

Let’s Program A JavaScript Game 16: Do You Believe In Magic?

Let’s Program A JavaScript Game 17: Can You Hear Me Now?

Let’s Program A JavaScript Game 18: Moving Pictures

Let’s Program A Javascript 19: Cross Browser Chaos

Let’s Program A JavaScript Game 20: BONUS STAGE!

Code

You can play a complete demo of the game we’re building here: https://scottcornaby.com/games/debugcycle/defrag-cycle-final.html

To run the game locally so you can examine the code and make changes just download all the html, png and ogg files in this folder: https://scottcornaby.com/games/debugcycle/

Let’s Program A JavaScript Game 20: BONUS STAGE!

We’ve successfully built a full browser game from scratch and learned a lot of useful stuff about game design in general and JavaScript in particular.

Now what?

That really depends on what you hope to accomplish.

If You Want More JavaScript Practice

Defrag Cycle is a simple game that could be made infinitely better by the addition of a few key features. So if you want to make sure you really understood this Let’s Program why not grab one of these projects and implement it all on your own?

  • Add a life system so that players can survive a couple enemy collisions before getting a game over.
  • Add a double jump to help make sure the player can always make it over or under the next enemy. Remember that letting the player jump higher will also require modifying the code that decides what the max possible height and distance between new platforms is.
  • Add in some collectible powerups that temporarily stop the clock, give the player bonus defrag points or makes them immune to enemies for a short time.
  • If you’re feeling really ambitious give the player a laser gun so they can shoot down enemies.

If You Want To Make A Professional Quality Browser Game

Knowing how things work at a very basic level is important, but reinventing the wheel every time you want to program a new game is a real waste of time. It’s much more efficient to use existing game code for your engine. Ideally code that other people have already spent a lot of time debugging and testing for cross browser compatibility.

But where to find an existing JavaScript game engine?

You could always just Google “JavasScript game engine”, which is how I found this handy list.

Most of the engines and editing tools on that list are either free or offer free trials, so just grab a few and start messing around until you find one you like. Most of them also offer tools that let you create events and game behavior with a few button clicks instead of having to write actual JavaScript code, which is a real time saver. Just because we know how to do everything the hard way doesn’t mean we always want to do it that way.

So best of luck and be sure to email me a link to any cool games you create.

Let’s Program A Javascript 19: Cross Browser Chaos

Forgive me readers for I have sinned.

I wrote this entire series of articles using nothing but Firefox on Linux to test my code. Only now that we’ve reached the end of the project do I notice that when you try to open it with anything else it crashes.

I should have prevented this by testing every code update on multiple browsers like a good software developer, but late is better than never so let’s get this working in all browsers.

fastSeek Was A Bad Idea

You hopefully remember that we use the Audio class function fastSeek to rewind our background music and sound effects. Game over? Reset the music with fastSeek(0). Did the player stop grazing enemies? Reset the grazing sound effect with fastSeek(0).

But it turns out Firefox is the only browser that actually supports fastSeek. Every other browser will throw an error if you try to use it. So if you open Defrag Cycle in Chrome the whole game will freeze up the first time a sound effect has to reset itself because you either died or stopped grazing.

What other browsers prefer is for you to directly manipulate the currentTime variable inside of each audio object.

So instead of :

bgMusic.fastSeek(0);

We need to do:

bgMusic.currentTime=0;

This also works in Firefox, so just replace all the fastSeek instances in your code with currentTime and you are now cross browser ready.

Sound Format Headaches

Try to open this Defrag Cycle in Internet Explorer and the game won’t even load.

This is because IE doesn’t support the Ogg audio standard and throws an error when you try to ask it to load a .ogg file.

After a bit more research it looks like the only file format that actually has universal browser support is mp3.

So easy fix, right? Just convert our two sound files to MP3.

WRONG!

It turns out there’s a whole bunch of really weird legal issues relating to using MP3. Basically if you make a game with MP3s that more than 5,000 people access you now technically owe $2,500 to the company that owns the patent on MP3 technology. You can read more about that here.

Now as hobby developers we probably don’t want to mess with the legal issues and cost of MP3.

Instead we’re going to stick with Ogg and modify our code to only try to load and play sound files when the user’s browser supports them.

Different Code For Different Browsers

This sound file problem, while frustrating, actually gives us a good opportunity to play with an important JavaScript feature: detecting the what the user’s browser can and can’t do and then running browser specific code.

This is an important technique in non-game related web development too. Not only do different browsers support different audio formats, they support different JavaScript functions and sometimes have conflicting support for things like CSS styling rules. So a lot of professional code starts by figuring out what browser the user has and then making slight changes to the page to make sure everything stays compatible.

For our needs we can rely on the canPlayType function of the Audio class. Give the function the name of an audio standard and it will return whether or not the browser can actually play it.

We can use this to improve our code to only load and play our .ogg files in browsers than support them, like so:

function loadGrazeSFX(){
   grazeSFX = new Audio();
   if(grazeSFX.canPlayType('audio/ogg')){
      grazeSFX.oncanplaythrough = function(){ grazeSFX.oncanplaythrough=0; loadBackgroundMusic();};
      grazeSFX.src = 'wubwub.ogg';
      grazeSFX.loop=true;
   }
   else{
      loadBackgroundMusic();
   }
}
function loadBackgroundMusic(){
   bgMusic = new Audio();
   if(bgMusic.canPlayType('audio/ogg')){
      bgMusic.oncanplaythrough = function(){bgMusic.oncanplaythrough=0;gameLoop();};
      bgMusic.src = 'Ouroboros.ogg';
      bgMusic.loop=true;
   }
   else{
      gameLoop();
   }
}

Pretty simple, right? If the browser supprots Ogg audio we set up the code to load like usual. If it doesn’t we just skip straight to the next function in our loading chain.

We also need similar logic when we try to actually play our music so we don’t accidentally try to reset an audio file that was never actually loaded. So anywhere the code looks like this:

bgMusic.pause();
bgMusic.currentTime=0;
grazeSFX.pause();
grazeSFX.currentTime=0;

We need to expand it to look like this:

if(bgMusic.canPlayType('audio/ogg')){
   bgMusic.pause();
   bgMusic.currentTime=0;
}
if(grazeSFX.canPlayType('audio/ogg')){
   grazeSFX.pause();
   grazeSFX.currentTime=0;
}

Now we only play and load music on browsers that can play and load music. We’re cross browser compatible again!

We Could Have Done That Better

I will be the first to admit that this isn’t particularly elegant code. Nor is it particularly good behavior to just completely drop our music and sound effects on a major browser like IE.

Now this is just a demo project so I’m not too torn up about the code being less than perfect. But if this was a serious project for a paying customer we’d want to be a little bit smarter.

So what could we have done instead?

Well, probably the smartest thing to do (that doesn’t involve buying a MP3 license) is to include multiple versions of each audio file on your server. For instance, IE may not play Ogg but it apparently will play MP4 and WAV.

Then when loading music you use canPlay to decide which file version to load. Ogg for Firefox and Chrome. WAV for IE. Things like that.

This also cleans up the code that plays the audio. Since you know the audio objects will get a file one way or the other you don’t have to worry about wrapping them up in tons of ugly if statements.

Now We’re Really Done… Please?

Looks like Defrag Cycle is running in all my browsers, so time to get back on track and wrap up this Let’s Program with a final post about where we can go from here.

Let’s Program A JavaScript Game 18: Moving Pictures

Game animations are important.

First off, they look cool. Watching a warrior swing his ax is much more entertaining than just watching him sit still while you pretend he’s swinging his ax. (No offense to old school RPGs, of which I own an embarrassing amount.)

Second, they make the game feel more realistic. A character who stands perfectly still while falling hundreds of feet feels fake. It’s much more immersive when the character has billowing clothing or flailing limbs to drive home the fact that he’s falling.

Finally, and maybe most importantly, animations give the player information about what’s happening in the game. They show when and how enemies are attacking and give hints about what the character can and can’t do.

I mean, imagine a game with a two second delay between attacks that you just had to remember. Frustrating! Now imagine a game with a two second animation of your character reloading his giant shotgun after every attack. Now it’s easy to see exactly when you can and can’t attack.

Pseudo Code

Animations are created by showing related images one after another at high speed. Take four pictures of a walking man, cycle through them a couple times a second and the human brain will see one walking man instead of four different pictures.

But in a game it’s (usually) not enough to have one animation. You have to have multiple animations you can switch between as needed. If the player is walking along and suddenly hits the jump button you need to be able to instantly switch from your walking animation to your jumping animation.

But even with that complication the logic of 2D game animation is pretty simple:

  1. Store what animation state your sprite is in (ex: walking, jumping, shooting)
  2. Store how long it’s been in that state (ex: 17 frames)
  3. Use that information to decide what image to show (ex: jump-image-3)

Now in a professional game step 3 can get pretty complicated. A running animation might have logic that looks like:

  • If the sprite has been in “running” state for less than five frames show “warm up 1”
  • If the sprite has been in “running” state for between five and fifteen frames show“warm up 2”
  • If the sprite has been in “running” state for between fifteen and twenty two frames show “warm up 3”
  • If the sprite has been in “running” state for more than twenty two frames switch between “running 1” and “running 2” every three frames.

Fortunately for us our only goal is to make our virus enemies spin while being grazed. That means we only have to worry about two sprite states (grazed and not-grazed) and our step three logic is going to be a pretty simple cycle.

How To Store An Animation

To show an animation you need to quickly switch between multiple related images. And while you could create every one of these images as an independent file we’re going to be using a “sprite sheet” instead.

A “sprite sheet” is one large image made up of multiple, related images glued together. For example, here’s a sprite sheet showing four different stages of a rotating virus enemy:

Load this into your virusImage object instead of the old single virus graphic

Load this into your virusImage object instead of the old single virus graphic

We load the image into the game once and can then create an animation by drawing different parts to the screen at different times. In JavaScript we do this by adding extra arguments to our drawImage calls.

You hopefully remember that by default drawImage accepts three arguments: the image we want to draw and the x and y coordinates of where we want to draw it on the canvas.

But there is also a nine argument version of drawImage. The first argument is still an image. The next four arguments define what part of the image we want to draw by creating a rectangle inside of the image. Then the last four arguments define a rectangle inside of the canvas showing where we want our sub-image to be draw and what size we want it to be stretched to.

So to draw the third frame of our 50×50 rotating virus at canvas coordinate (127, 33) we would do something like this:

context.drawImage(virusImage, 100, 0, 50, 50, 127, 33, 50, 50);

This nine argument function is really just asking for an image and two rectangles.

This nine argument function is really just asking for an image and two rectangles.

In other words: Grab the 50×50 sub image found at point (100, 0) of the virusImage and then draw it inside a 50×50 square at point (127, 33) on the canvas.

Make It So

With that we’re ready to upgrade our viruses so that they spin whenever the player grazes them.

First off, let’s define a new constant at the top of our code to help us keep track of what animation state our viruses are in. Since at the moment they can only be in state “grazed” or “not grazed” we really only need one constant:

var VIRUS_GRAZED = 1; //Used to identify a virus that is being grazed

Now we drop down into the graze logic of updateGame and have the viruses keep track of their current animation state:

//Virus logic
for( i = 0; i < viruses.length; i++){
   //Have all viruses move towards the player
   viruses[i].x-=VIRUS_SPEED;

   //See if the player is grazing this virus
   if(intersectRect(viruses[i], grazeHitbox)){
      grazeCollision = true;
      viruses[i].state = VIRUS_GRAZED;
      viruses[i].stateCounter++;
   }
   else{
      viruses[i].state = 0;
      viruses[i].stateCounter = 0;
   }

   //See if the player has had a lethal collission with this virus
   if(intersectRect(viruses[i], deathHitbox)){
      deathCollision = true;
   }
}

This is mostly the same virus enemy logic as before. But now when a virus is grazed it will set it’s state to VIRUS_GRAZED and increment a counter. And when a virus is not grazed it will erase that state and reset the counter.

Now that we know which viruses are being grazed and how long they’ve been grazed we can update our virus drawing logic down in drawScreen:

//Draw viruses
//Virus has a 200 pixel wide sprite sheet with four different 50x50 rotations all in a row
for(i = 0; i < viruses.length; i++){
   //By default draw the first virus
   virusSheetOffset = 0;

   //If the virus is being grazed make it spin by switching between rotations
   if(viruses[i].state = VIRUS_GRAZED){
      if( viruses[i].stateCounter % 8 < 2){
        virusSheetOffset = 0;
      }
      else if(viruses[i].stateCounter % 8 < 4){
         virusSheetOffset = 50;
      }
      else if(viruses[i].stateCounter % 8 < 6){
         virusSheetOffset = 100;
      }
      else{
         virusSheetOffset = 150;
      }
   }

   context.drawImage(virusImage, virusSheetOffset, 0, 50, 50, viruses[i].x, viruses[i].y, 50, 50);
}

Let’s analyze this by looking at the last line first:

context.drawImage(virusImage, virusSheetOffset, 0, 50, 50, viruses[i].x, viruses[i].y, 50, 50);

We always want to draw a 50×50 virus image at the x and y location of the current virus enemy. That’s why the last four arguments are viruses[i].x, viruses[i].y along with 50 width and 50 height.

We then choose between our four different virus sub-images by changing the “virusSheetOffset”. Since our sprite-sheet is just one big row we always leave the y coordinate of our frame as 0. Larger sprite sheets often have multiple rows and would need to calculate both an x and a y offset.

We choose our virusSheetOffest, and thus our sub-image, by seeing how long the virus has been grazed and switching every two frames. Four different sub-images at two frames each mean we complete a full cycle every eight frames so we calculate our current sub-image by using modulo eight to see whether we are at frame 0-1 of a cycle (image 1), 2-3 (image 2) 4-5 (image 3) or 6-7 (image 4).

Of course, all this only happens if the virus is being grazed. If not we just stick with an offset of 0 to get the default first frame. This means that only grazed viruses spin. The rest just sit there.

An Unrelated Tweak

Before we finish up this Let’s Program there’s one last little game flaw I want to fix.

Currently we start and restart the game by having the player press the up arrow key. Unfortunately if the player happens to already pressing the up key when he dies or wins he will immediately restart the game without any time to see the “game over”, “you win” or “start” screens.

To solve this I created a new global variable

var menuFrames = 0; //Has the player been on the menu long enough to press a button?

I then use this variable to force every menu screen to wait twenty frames before letting the player move on. Here’s an example from the win screen, but the other two functions were modified in the same way:

//Check to see if the user is ready to restart the game
//Slightly delay user input so they don't accidentally skip this screen
function updateWinScreen(){
   menuFrames++;
   if(playerInput.up && menuFrames >= 20){
      gameState = STATE_START_SCREEN;
      menuFrames = 0;
   }
}

Resetting the “menuFrames” counter before switching states is very important. Otherwise only the first menu of the game would have the input delay and then we’d be right back to being able to accidentally skip past important screens.

YOU WIN THE META-GAME!

We now have a complete game that shows off everything from basic animations, collision detection and sounds to real-time world generation and physics simulations. You can play the full version here and I strongly recommend downloading the page source so you can see the complete code.

And with that done all that’s left is a few final thoughts on what to do next.