🚀 go-pugleaf

Jim Babka wrote:

> Actually, I have a theory about this.  I'm not "in the know" about this - I
> don't know the people who put TD-GAMMON together - but I have noticed the
> rather blatant appearance of cheating (program has three unprotected pieces,
> only way I *won't* hit it at least once is if I roll a 1 2, and guess what I
> roll).  My theory is that this is a game that was trained via massive
> reiteration of playing against itself.  During the course of that learning,
> it also "learned" the pattern to the pseudo-random numbers used to generate
> the die rolls.  I understand that they probably change the seed from game to
> game, but depending upon the quality of the random-number generator, there
> can still be a repeating pattern, and that pattern might be built into the
> neural net (or whatever other algorithm is being used) that drives the
> computer play.

There are only three big problems with this theory.

1. The "patterns" in any halfway-decent pseudo-random number generator
   are *way* too subtle to be spotted by a neural network. They just
   aren't the sort of thing you can detect that way.

2. TD-Gammon wasn't trained on a PC running OS-2. It was trained on an
   RS/6000 running (I think) AIX. It's unlikely (though possible) that
   these two systems use the same random number generator.

3. What a neural network like TD-Gammon learns is an estimate of the
   value of the position (I am over-simplifying here). It doesn't get
   told (e.g.) the move number, or the current state of the RNG, or
   anything like that.

In other words: your theory is interesting, but it is a million miles
away from being possible.

-- 
Gareth McCaughan       Dept. of Pure Mathematics & Mathematical Statistics,
gjm11@dpmms.cam.ac.uk  Cambridge University, England.