I've been mucking around with PHP and I've created a noughts and crosses game (shown below if you're reading this after I've got home from work and been able to upload it, or not shown at all if you are reading this earlier than that).
The idea is that you (the player) play noughts and crosses against the program (robot). The program starts off not knowing how to play noughts and crosses, and learns as it loses or wins. It does this by...
- Player makes a move
- Program looks at its game database to see whether the current board state exists.Â
- If it doesn't already exist then then it puts records into the database corresponding to all possible moves in this situation, together with an equal weight value for each possible move.
- The program then picks a weighted random move, based on the weights from the database and moves there.
- It stores this move in its move history list for this game.
- Go back to step one and loop until the game is won or drawn.
- At the end of the game:
- If the program has has won then go through all the moves in the history list and give the weights corresponding to the board state and move randomly picked a boost.
- If the player has won give the weights corresponding to the board state and all other possible moves apart from the one picked a boost.
- If it's a draw do nothing.
But it doesn't seem to learn much. I thought it was because I needed to tweak the weight adjustment stage so that more recent moves in the move history got a bigger impact on their weights than earlier moves, so I did that and nothing much changed. I think I'll have to devise some more cunning weight adjustment criteria.