Today, Chezz pointed me to a really interesting article. Apparently have figured out how to pretty much guarantee a win in “Heads Up Limit Hold ‘Em” Poker. This is the poker equivalent to beating chess masters head to head, like what Deep Blue did in the 90’s and what Watson did more recently on Jeopardy! The difference between these instances though is all players in the game have the same basic information. In chess all the information to inform any move and future moves are available with a glance at the board. In Jeopardy! it’s a little different because it’s knowledge based, but to create the question to the answer, it’s what you know, but the answer is there for everyone at the same time.
In poker, it’s different because you, initially, know only 2 cards out of the 52 in the deck, as the play continues you know more. So you’re dealing with imperfect information about what action to take. This is important, because that’s what you need to do as a player is address that uncertainty. In this program the researchers developed a great learning tool that was able to determine the best course of play and with the experience the researchers gave the program they effectively created an unbeatable computer.
However, the game is limited to a 1 vs. 1 situation with a limit to how much the players are able to bet in any given situation. Those limits are based on multiples of the opening bid. These limits, I’m sure, will eventually be generalized to handle any number of players and then any number of betting options, such as no limit.
Once this happens, I think that these learning systems will have or could have a dramatic impact on a great deal of things. First, trading is essentially poker and the companies that will likely leverage this first will be the companies that deal in high frequency trading. This will make the computers act very differently than they are now and with these new learning algorithms built into them, it could dramatically reshape our stock markets (more than they have been to this point). Second, these systems would be used to “help” with negotiations in any number of situations. I’m thinking initially diplomatic situations where there are a great deal of stakes on the table, which most of them are known, but the information is incomplete. In these cases a computer can greatly augment the capabilities of the diplomat that wouldn’t have been possible in the past, which could either increase the likelihood of a war or reduce it depending on what the goals of the computer are. What does “winning” mean in those cases. So setting those clear boundaries will be important, but that’s why having a person there to augment the machine is crucial as they would provide that feedback over the course of the negotiations.
Finally, this one is by far the largest stretch, but it might be more possible to plan or react to a great deal of the actions of economic entities. This means that governments could leverage these applications to help determine the best determine where to invest as well as where to buy to help truly manage the economy. The central bank could change dramatically.
None of these situations are going to happen overnight. Most likely we’re 2-3 years from multiplayer with no limit hold ’em and 5 years for more monetizable uses for this application. Rest assured these algorithms will be used in a business at some point. Watson and Deep Blue have been repurposed to make IBM money. Expect something similar and I think that these are all very realistic applications that these researcher could pursue. What do you think?