Researchers Have “Solved” Poker and What it Could Mean

Today, Chezz pointed me to a really interesting article. Apparently have figured out how to pretty much guarantee a win in “Heads Up Limit Hold ‘Em” Poker. This is the poker equivalent to beating chess masters head to head, like what Deep Blue did in the 90’s and what Watson did more recently on Jeopardy! The difference between these instances though is all players in the game have the same basic information. In chess all the information to inform any move and future moves are available with a glance at the board. In Jeopardy! it’s a little different because it’s knowledge based, but to create the question to the answer, it’s what you know, but the answer is there for everyone at the same time.

In poker, it’s different because you, initially, know only 2 cards out of the 52 in the deck, as the play continues you know more. So you’re dealing with imperfect information about what action to take. This is important, because that’s what you need to do as a player is address that uncertainty. In this program the researchers developed a great learning tool that was able to determine the best course of play and with the experience the researchers gave the program they effectively created an unbeatable computer.

However, the game is limited to a 1 vs. 1 situation with a limit to how much the players are able to bet in any given situation. Those limits are based on multiples of the opening bid. These limits, I’m sure, will eventually be generalized to handle any number of players and then any number of betting options, such as no limit.

Once this happens, I think that these learning systems will have or could have a dramatic impact on a great deal of things. First, trading is essentially poker and the companies that will likely leverage this first will be the companies that deal in high frequency trading. This will make the computers act very differently than they are now and with these new learning algorithms built into them, it could dramatically reshape our stock markets (more than they have been to this point). Second, these systems would be used to “help” with negotiations in any number of situations. I’m thinking initially diplomatic situations where there are a great deal of stakes on the table, which most of them are known, but the information is incomplete. In these cases a computer can greatly augment the capabilities of the diplomat that wouldn’t have been possible in the past, which could either increase the likelihood of a war or reduce it depending on what the goals of the computer are. What does “winning” mean in those cases. So setting those clear boundaries will be important, but that’s why having a person there to augment the machine is crucial as they would provide that feedback over the course of the negotiations.

Finally, this one is by far the largest stretch, but it might be more possible to plan or react to a great deal of the actions of economic entities. This means that governments could leverage these applications to help determine the best determine where to invest as well as where to buy to help truly manage the economy. The central bank could change dramatically.

None of these situations are going to happen overnight. Most likely we’re 2-3 years from multiplayer with no limit hold ’em and 5 years for more monetizable uses for this application. Rest assured these algorithms will be used in a business at some point. Watson and Deep Blue have been repurposed to make IBM money. Expect something similar and I think that these are all very realistic applications that these researcher could pursue. What do you think?

Passions

During Thanksgiving it’s a time for food, family, and watching copious amounts of TV and movies. This year those movies included “Somm” which is a movie about 4 guys trying to take the Master Sommelier test. Which apparently only about 12% pass each year. Not a super low amount, but also not an easy exam in any way shape or form. It got me thinking about if I could become a Somm (as they are called in the business according to the movie). I think that I do have the right kind of mind for the job, remember flavors of wines the history of region of wine and all of that is right up my alley. I know that, because that’s what I used to do with beer. I used to be able to rattle of several types of beers that if you liked one kind or style that might push your boundaries and give some of the reasoning behind it. I was able to explain why a beer tasted the way it did, etc… That was something I loved and was really passionate about. However, wine just doesn’t hold the same level of interest to me. I don’t know if it’s because beer feels much more close to home, my friends drank beer and avoided wine or what it is. Even now that I cannot drink beer I still haven’t really replaced it with a beverage I’m passionate about. I drink both wine and cider, but I don’t feel a deep down passion for them. Likewise I don’t think I could do that with whiskey, even though I really enjoy drinking whiskey, it just hasn’t captured my imagination as a GREAT drink that I want to learn everything about.

More broadly, the movie has had me thinking about what I’m truly passionate about. I know that a great deal of my interests are reflections of what my friends are interested in. If I’m surrounded by people that love watching football, I’ll watch a lot more football, similarly for college basketball or hockey. I enjoy the games when I watch them, but I rarely will seek them out on my own. I think this is something that is driving my wife crazy, I simply don’t have a lot of things that I’m passionate about that I’ll invest a huge amount of time into. It’s frustrating for me too. I think that is probably the hardest thing about me being Gluten Free I’ve really lost a great passion of mine.

I think many people will agree that I’m passionate about certain things in our political system. I’m all about free speech, investing in science and technology to grow and enable our economy. But I’m also not 100% all in. I’ve been thinking about how to get involved and in what way I’d do this. Ideally, I’d work at a think tank, but there aren’t many around Portland and many of them are either left wing or right wing. I think on many topics I’m a moderate, so neither party truly inspires much confidence.

I’m also passionate about making people’s lives better at work, but I’m not really getting much support at my organization and I’m getting beaten down. It’s one of the most frustrating things you can deal with on a daily basis, knowing there’s a better way to do things, walking your leaders to the kool-aid, but seeing them spit it out and start drinking from the mud instead.

So this leaves in an odd position. The things I’m passionate for I can’t really follow through, which makes me ask What do I have passion for, what should I try to be doing to find things I could become passionate about, how should I act on the topics I do have passion for? I know that there’s something more out there that I could or should be doing, but i have no idea how to get there.

Time Travel could it work?

Apparently some folks thinks they figured out how you could go back in time kill your dear old grampappy and everything would work out alright. It’s a highly convoluted thing and I’m not really sure I understand any of it. But that’s ok, because it’s quantum physics. Quantum physics is one of the complicated types of physics we have discovered (discovered because it was always there, but never applied or understood).

The general idea is that because of gravity, something called a “Closed Time-like Curve” can exist. From what I understand these represent a likelihood of something occurring in like and/or particles. These are the likely ways that light might split into multiple particles (only to recombine later in most cases) or be consumed and re-emitted by another particle. Effectively, it creates a probability distribution that says one of these options might happen. In the case about your granddad surviving, you have to have a 50% chance of survival for everything to work as expected. If the likelihood of an event falls below that, then it wouldn’t happen. Essentially, you would need to create a scenario where your grandfather would survive as often as he died. That sounds like you’d have to do some pretty elaborate planning to be sure he might survive or he might die.

Feynman Diagram

This would work because of that recombination effect that I mentioned earlier. When light moves from point A to B it doesn’t have to go directly there. Richard Feynman created his famous (for math people) diagrams that were able to explain how these particles moved and emitted particles.

In some cases the light would move around and eventually recombine, but it would always end up at the point it was expected to based on the other attributes of that light. Because of these features, the physicist was able to do some experiments with light to actually create a “killing your own grand dad” situation. This allowed them to offer empirical evidence not just theoretical.

That being said, it’s really unclear if anything at that size would ever work in actuality at sizes we can actually interact with on a daily basis. We can stop light and we can teleport light too. That doesn’t mean we’ll be having Scotty beaming us up soon though. Likely this discovery will find it’s way into quantum computing or cryptography as mentioned in the article. Unfortunately it’s not really practical and will probably be discredited in a few years like the whole faster than light fiasco from a few years ago.

Philanthropy, Private industry, and science

Apparently I’m not too happy with the NYT magazine and their exposés of late. First there was the long article about millenials and how they don’t want to work for the “old guard” which is ahistoric and ignores a great deal of the similarities between the silicon valley of today and the past silicon valleys and other similar environs.

Now they are rushing about in concern over private scientific research. Apparently, it’s a new big problem. It’s neither new nor a problem. First of all some historical context. Scientific labs as we know them today were truly founded through industrial labs. These labs were initially in the dye industry back in Germany in the late 1800s, sure there were university labs, but they weren’t researching as big of thing as the industrial labs started. These labs had problems that couldn’t be solved in academic settings. The universities were training grounds for scientists, but in many cases the scientists actually did their doctoral research at Bayer or a similar type dye company. These dye companies almost all became pharmaceutical companies over time because of the similarity in chemistries between dyes and pharmaceuticals.

This was in the 1800s and really hasn’t abated. I’ve written about Bell Labs and Xerox in the past which are essentially the Bayer equivalent for telecom, semiconductors, and computers.

Science has always been a combination of public, private, and universities. In fact, research that I conducted through my master’s degree has shown that the INTERACTION between private industries and universities produces the most important work (in terms of citations). Our concern should not be if science is going private or not. Our concern should be if they are sharing with the broader scientific community. That’s the biggest risk. It’s one of the biggest problems with industrial scientific research – it never reaches the light of day even if it becomes a product.

Why doesn’t it? Well, simply because it’s better protection for some processes for the technique not to be patented. In the case where something is relatively easy to copy (an iPhone) it’s best to patent because you’re protected them. In the case where it’s very difficult to copy (a nitride layer on an Intel chip) it’s best to hide that process as deep as possible. In fact, it’s best if any technique that would uncover the underlying process to make that nitride layer from reverse engineering destroys the product. For Intel, this is the best result, for the rest of the world, it’s suboptimal as Global Foundries and TSMC will struggle for years to reverse engineer the layer if they ever can. This slows the innovation process as a whole, but we’re willing to suffer this inefficiency because Intel makes some nice chips.

Beyond this debate, the author is upset that someone would want to push scientific research in one direction that might only help white people or rich people. Unfortunately, this is capitalism. We may not like it in basic research that is going to be used to cure diseases, but we tolerate it with Intel so we need to be realistic and tolerate it in this case. Furthermore, I think that the author doesn’t understand that adjacencies in research in diseases will arise and we’ll learn more about all humans, not just them white folks. Ironically, at this point the author calls out a researcher that is working with an Oracle billionaire – that researcher works at Rockefeller University.

What are seen now as seminal research institutions in many cases started out through the very philanthropy the author is upset about. Carnegie Mellon University was the combination of two institutions in Pittsburgh started by an industrialist and a banker. It is one of the most respected research organizations in the world. These men were driven by the same desire to push scientific research as Bill Gates and the other (mostly) men on the list.

Is this a perfect system? Not by a long shot, however in the current political environment scientists are going to take money from whatever source they can. It’s merely practicality. A professor will typically have anywhere between 1-10 grad students. These students at the PhD level will likely be fully funded by the professor. If that professor does not get funding, those kids don’t get to keep working and either have to find another adviser or quit. Here’s the kicker in the case that professor does get money – a large proportion of that funding is taken and allocated to less profitable portions of the organization. At University of Texas, this meant that the EE department was probably funding part of the Chemistry Department. Some departments are like the Football team, while others are like the Swimming team. The swimming team might be winners, but are in a small market.

If we truly wanted change in the way we fund scientific research we need to increase the amount of public investment across multiple institutions. We need to increase funding across multiple types of research fields, specifically focusing on the intersections between academic fields. Push for collaboration between industry and universities as well as collaboration across national boundaries. All of these improve the citation rate and quality of the research. We can even work to partner public funds with private funds – we just need full disclosure.

The problem isn’t privatization. We’ve had an oscillation between really publicly funded (1960-70’s with NASA) and really privately funded. In all cases science has marched on – we just need to make sure it keeps on marching.

Science, evidence, and paradigms

Last night was a big debate between Bill Nye the Science Guy and Creationist Ken Ham. This was to help inform people that the science supporting evolution and how that refutes the “science” behind creationism. One of the key questions during the debate was around what would be required to convince Bill Nye that creationism was true and evolution was false. He said “Evidence” essentially. While, this is the ideal answer for a scientist, I find it unlikely. This, of course, isn’t a popular oppinion. It’s not that Bill Nye doesn’t believe that he would change his mind or that he would change his mind quickly, but it’s unlikely. People aren’t purely rational, in a purely rational world, yes that’s exactly what would happen. Even scientists have a serious problem with this. Scientists still suffer from the same sort of denial that global warming denialist, however, this impact is the largest inside of their field rather than outside.

Why do we know that this is true? According to Karl Popper whenever theories are incommensurate it’s unlikely that a leading theoriest in that field will switch to the new theory or paradigm. What does this mean? Well, if we think about scientific theories in terms of technology it will become easier to understand. Let’s look at jets and propellors for airplanes. It was clear in the early 50’s that jet engines were the way to go, but not all companies decided to pursue that type of engine. Instead these companies decided to continually tweak the capabilities of props instead. A similar reaction happened with sail technology and steam engines in this case sail techology was still more effective than steam, it took years before steam would catch up let alone surpass sail.

This similarly happens with scientific theories. What happens is that flaws start to appear that the theory cannot easily explain. For example, in the Geocentric theory planets would seem to track backwards over time and then begin to move forward. Theories about how these planets had small circles that would regularly appear through the course of their normal revolution around earth. The mathematics for this theory became increasingly complex and seemingly less realistic. The heliocentric approach reduced the complexity and eliminated the small circles and allowed for the eventual creation of Newtonian physics. However, whenever this started to break down and Einstein proposed relativity, it was largely ignored for decades. Essentially, it took until that generation retired for relativity to finally get accepted by the broader scientific community. This happens to scientific theories on a regular basis.

In fact, there are some pretty serious debates going on about the full mechanics of evolution. The original basis of the theory are still true, heredity, competition/pressure, and variety, however the nuances are being debated. For instance Richard Dawkin’s theories have started to fall a bit out of favor, while we’re learning that there are some things that we do in our lives that impact our genes. Those changed genes could be inhereted, which could change the next generation – this was Lamarcain to the core. However, Dawkins will likely not accept a different theory than the one he’s devoted to his life to. So, while to some extent it’s true that scientists will and do change their mind, it’s more likely that Science will change while individual scientist will take significantly longer if they ever do.

Science as Diplomacy, nothing new

Healthcare is a big deal these days. In the US costs are soaring, arguably Obama’s legacy rests on his controversial law the Affordable Care Act, and Republicans have risked shutting down the government over the law. So, it’s no surprise when it’s in the news for other reasons. According to The American Association for the Advancement of Science (AAAS), the group that publishes the prestigious journal Science, healthcare is now starting to impact diplomacy. Researchers from several different countries are creating novel ways to introduce addicts in both China and the US to the other’s style of medicine. In the US traditional Chinese methods are being experimented within clinics. While in China “western” medicine is being introduced in a very specific manner to address the same issues.

There are several organizations beyond the one I linked to above that deal with cross-cultural issues using either science, engineering, or healthcare. Organizations like Doctor’s Without Boarders work in many different parts of the world to bring care to those in need. These are volunteers that are doing good deeds to bring care to those that otherwise wouldn’t be able to afford it.

A similar organization is Engineer’s Without Boarders, which improves local conditions of a village or community through technology. For example, one of my friends I met Austin (now teaching at Harvard) would regularly visit a community in Mexico to install different solar panel arrays. This allowed them to have clean drinking water, power for mobile devices, cooking, and heating.

All of these activities help develop good will from one country to another. In fact, foreign exchange students that fully engage with their classmates are also great diplomats. I know that while I was abroad in Europe my friends decided that they would want to visit the US because I introduced them to what people in the US could be like rather than what they’d heard on the TV.

Interestingly, science and engineering have been used for diplomatic reasons for a very long time. I find the most interesting to be the period after World War II. Science played a huge part in rebuilding nearly all of western Europe as well as Japan. Why did we do that? Because we felt having Germany and Japan as allies against the Soviet Union was more important than kicking an enemy while they were down.

The German’s had invented the V2 rocket, which the US wanted to use for our own rockets as well as for nobler purposes. To gain the technical competence to build the rockets we decided to recruit as many Germans scientists as possible and bring them to the US. These were Nazi’s and some of them were likely war criminals, however because we needed them for the Cold War we decided to take advantage of them.

The trade wasn’t simply one way though. The US government encouraged companies to open their doors to European companies. This allowed our allies to rebuild their economy. In fact, the US business leaders like Deming to Japan, which eventually enabled Toyota to best the US automakers with techniques they turned down. These interactions dramatically changed how Europe and Japan evolved over the next few decades.

Diplomacy using science isn’t anything new. It was influential through the Marshall Plan after WWII and will continue to influence the rest of the world. It could be argued that our patent system enables the US prescription drug consumer to subsidize all other economies for their prescription drug use. This is an accidental type of diplomacy that is make the lives better for billions around the world.

I hope that in the future we will continue to influence the world through science and technology support. However, it is best if we help develop the technologies with locals rather than handing them a finished product. As the book Shock of the Old argues in many contexts the newest technology isn’t the best, the best technology is what does the job well enough that the people using the technology can understand and keep running.

Bitcoin, what’s the point?

Digital currency is the biggest new thing on the market. It’s not a new idea at all, however the fact that someone was able to get it up and going is something new. The first time I’d heard of a cyrptocurrency and the potential impact it could have on the economy was in the Cyrptonomicon a 1999 book written by Neal Stephenson, but the idea is clearly much older than that. In fact, Paypal likely was something of a precursor itself to cryptocurrency. It filled a whole for money transferring on the internet, which was difficult in many cases. Paypal and eBay co-evolved as Paypal became the default transaction tool on the site – otherwise every seller would have needed to be able to accept credit cards. Paypal offered a safe way to transfer money directly from one account to another without messing with banks or credit card companies.

After the crash in 2007 and Paypal and other services refusing to transfer funds to Wikileaks people felt it was important to have another option. Fortunately, for those folks Bitcoin was already on the market, having been created in 2009. Bitcoin is essentially a digital version of gold in that users of the currency mine the coins and there is a set limit of 21 million Bitcoins. This has lead to a massive explosion in both awareness and usage of the currency. Currently, many locations accept Bitcoin as a currency so you can buy stuff with it.

Yesterday, Paul Krugman, wrote an op-ed arguing that Bitcoin, like gold, is a foolish standard to adhere too. Essentially, we mine gold and then bury it in a safe and never use it. According to Krugman, Keynes argued that the government should bury printed money and then let private enterprise mine the money – as it’s more desirable to allow private enterprise to spend money rather than government. However, in times where demand is low there’s no incentive for private enterprise to spend that money, unless they can spend it to pull it out of the ground. He then goes to argue that we’re essentially doing the same thing with a Bitcoin mine in Iceland.

This got me thinking a little bit last night. I’m not really sold on Bitcoin, I think that if anything it really points out that the only value in any thing is what people assign to it. Either paper currency, digital, or “precious” metals. Diamonds are artificially high in price because of a cartel. For instance, you can buy a set of lab created diamond earrings weighing in total 2.5 carats for $482 (12/24/2013) but if you were to buy two 1.25 carat diamonds to make those earrings it would put you back over $30,000. There’s a lot of diamonds out there and it’s impossible to tell the difference between lab created and those pulled out of the ground. Yet we put more “value” on the “real” diamonds from the ground. There is nothing intrinsically valuable about diamonds or gold. What both of them are though, are standards for trade, similar to the dollar and Bitcoin.

There is one cyrptocurrency that does have a more altruistic motive though. That one is called Primecoin and it’s encryption algorithm is based upon prime numbers. As the number of Primecoins increase in the market the number of prime numbers eventually discovered will increase. However, this was is mined differently from the other cryptocurrencies so it’s not as easy to switch to this cyrptocurrency. This coin does have an underlying value as every new prime number discovered typically has some sort of prize associated with it as it provides value to the mathematical and scientific communities. What they do with it from there is anyone’s guess. However, it provides value completely disconnected to it’s “value” in the market place.

Could we develop a cryptocurrency that instead of requiring increasingly complex solutions to an encryption key, but one that solves difficult problems at the same time? There are a large number of places where this would be useful such as SETI, Fold.it (folding proteins for science), and researching molecules for big Pharma. I’m sure there are many more options. However, then we take Bitcoin a digital gold and convert it into something more useful to everyone than Primecoin and potentially solve problems as well as mine usable moneys.

Edit: Originally said 40 Million Bitcoins, actually 21 Million