Deepstack

This is an updated version of our previous post on AI Object Detection for Blue Iris that can be found here. You should be able to continue to use the older method referenced in that post, or try out this new one.

Disclosure: This post may contain affiliate links, and as an Amazon Associate I earn from qualifying purchases when you click the links, at no additional cost to you. If you click a link and buy a qualifying item, it is a small, easy way to support my blog, and I greatly appreciate it!

Single Camera Sub-Stream Support

If you are looking to setup AI Object Detection for Blue Iris, you came to the right place. In this article, we are going to go over a new single-camera method for this setup. You may have seen my previous post which highlighted how to do this with a dual-camera setup (two cameras within Blue Iris for a single camera), but we now can do this with just one!

Problem: AI tool does not reach Deepstack 1. Confirm that Deepstack is. Installed; started with the Detection API selected; on the correct port and not not the same port Blue Iris or any other service on this pc uses; activated; can be reached by typing in the Deepstack URL from AI Tool settings into a web browser. Explore life-changing ideas and skills. Get smarter and upgrade yourself, reading just 5 minutes every day. Join 1M+ others and start improving.

I was wondering around the ipcamtalk forums the other day and noticed something very interesting in the “What’s New” section for Blue Iris. Just a few months back Blue Iris Released version 5.2.7.0. This newly released version allowed for a single camera within Blue Iris to contain both the high-resolution mainstream and the lower resolution sub-stream.

What is a sub-stream?

In most mid to high-end IP cameras, you are given two streams: a mainstream and a sub-stream. The main stream is your high resolution stream that is usually 1080p, 1440p, or even 4k. The sub-stream is a much lower resolution stream, usually around 640×480. The sub-stream is used when viewing livestream previews or for viewing the live stream in low bandwidth conditions.

Here is how Blue Iris decides what stream to use:

The main stream is used when:

• Viewing a single camera in the web or app interface
• Performing direct to disc recording (any time you record video)
• Listening to the live audio from the camera

The sub-stream is used when:

• Viewing multiple cameras at once within the web or app interface
• Detecting motion
• Taking alert snapshots
• Doing any operation not listed in the main stream’s usage above

This has many advantages, one of the main ones being CPU utilization. Utilizing the sub-stream in a single camera within Blue Iris really brings down CPU utilization. Some people have seen savings of up to 50% with this setup. This also means our previous two-camera setup can be condensed down to a single camera setup. This means our Blue Iris “canvas” can really be cleaned up. Before with two cameras in Blue Iris per camera, we had to hide the sub-streams or just deal with two of each camera.

AI Object Detection Requirements

The requirements to setup AI object detection for Blue Iris are as follows:

• Blue Iris updated to at-least 5.2.7.0.
• A camera that supports a mainstream and sub-stream.
• A Deepstack.cc account to download our AI Server.
• The most recent AITool from VorlonCD.
• A computer capable of running the Deepstack server and AITool (You should be able to run this on the same computer as Blue Iris)

Blue Iris Setup

Folder Structure

The first thing we need to do is go into the Blue Iris settings and set up our folder structure under Clips and Archiving. Once there we need to set up a folder to save our recorded footage. In my example below, this is just called “new” and I have it set to save up to 225 GB worth of video and to delete anything older than 7 days. Set this to your liking or based on how much room you have to store video.

The next thing we need to do is to set up a folder that will hold our snapshots that we will send to our AI server to process. In my example below, I created the folder called “aiinput” and set it to hold on to up to 1 GB worth of snapshots and only hold on to 12 hours worth of snapshots. The thought here is that anything that does get detected will be recorded so I don’t need to hold on to these for too long.

Web Server

We need to next click on the Web Server tab. In here we need to make note and set up a couple of settings. First let’s write down what port our web server is sitting on. Mine sits on port 83.

Next check off “Use UI3 for non-IE Browsers”.

Next, check what your local LAN address is and write it down. This is important if your AITool and Deepstack install will not reside on the same computer. If you have this all setup on one computer/server, we won’t need it.

Finally, click advanced and make sure that Authentication is set to Non-LAN only and uncheck “Use secure session keys and login page.”

AI User Setup

We need a user that will trigger the alerts. This user is essentially a service account we can use to trigger alerts from the AITool. Click on the Users tab and then click “Add”.

In the user setup we need to set a username and password. Set this to something you can remember for later. The important part here is that you check Administrator and LAN only. Nothing else should really need to be checked off here. There are some things checked in my example below but they are inconsequential to what we are doing.

Camera Setup

In order to setup your camera correctly you will need to setup the main stream and sub-stream options under video configuration. You can see below in the picture that my mainstream is something like:

rtsp://<IP address>:554/cam/realmonitor?channel=1&subtype=0

The sub-stream URL is very similar. We just change the subtype option from 0 to 1. In the sub-stream box you don’t have to put the rest of the URL, just what is after the port number. For example:

/cam/realmonitor?channel=1&subtype=1

Now, this URL is specific to the camera I am setting up here (Amcrest Video Doorbell). Every camera’s URL structure will look different and you will have to do some Googling to figure out what you will need for both of your URLs.

Once everything is setup like the above picture, we should be ready to configure the rest of the tabs.

Trigger Tab

The next thing we need to do is to configure the Trigger tab for the camera. Once we’ve clicked on the trigger tab you’ll want to click the check box for Motion Sensor, Capture alert list image, and set the break time. The break time will say that this trigger is over in that many seconds unless motion is still happening. This is a setting you might have to play a little with.

Deepstack For Windows

Next we need to click into the Configure button for the Motion Sensor. In here you’ll need to adjust the dials to set the sensitivity of what motion to sense. Again, this is a setting you’ll have to play with. If you are getting too many detections then you will need to make it less sensitive. I tend to start super sensitive and dial back until I find a sweet spot.

Also set your make time to 1 second. This is what was suggested to me and seems to work. This is how long motion needs to be detected to sense an event. Once you’ve got these settings similar to what mine are in the gallery below, we can move on.

Record Tab

Deepstack Extravaganza

In the record tab we have a few things to setup. Reference the picture at the bottom of this section for a quick view.

First tick the video option and make sure it’s set to “When triggered”. Make sure the folder option is set to whatever folder you want your video files to go to. From earlier, we are putting these in our “New” folder.

Next, we will tick the “JPEG snapshot each (mm:ss)” option and set the time option to 2 seconds. Tick “Only when triggered”. This will take a snapshot every two seconds when motion is detected and save this to our folder which the AITool will monitor.

Finally, tick the “Pre-trigger video buffer” option and set it to something you find appropriate. This option basically says “When I get triggered to record, I will run the footage back X seconds and start recording from there.” Sometimes, if we have a lot of inputs to the AI server, it can take up to a second or more from motion start to the AITool sending the command to record. This allows us to see just before the motion started in our recorded video.

Deepstack

The AI server we’ll be using is one called Deepstack. You can go to this URL to download the Windows version. Once it’s downloaded, double click on it to open the installer and work through the install wizard to install it.

Now we cant start the server. Open your start menu and type “Deepstack” to locate the executable. Click on it to open and once it opens click “Start Server”. You’ll get a couple of options here. Set the priority to medium, make sure only object detection is ticked, and make sure the port is set to something different than what your Blue Iris web server is set to (we checked what it was early on in this tutorial). I set the Deepstack server to 81 since my Blue Iris server is on port 83.

Once you’ve started the server, we can check to confirm it’s started. Open up a browser and go to the address bar and type in the IP Address and port in the form ipaddress:port. If you are on the same computer as the server you should be able to use 127.0.0.1:port. Here I go to 127.0.0.1:81 and I am presented with a landing page saying my Deepstack server is up and running.

AITool

Download and Install

We’re nearly there. We need to grab the tool that will be the intermediary between Blue Iris and Deepstack. Remember from above, AITool watches for snapshots from Blue Iris, sends them to Deepstack, receives information from Deepstack and then tells Blue Iris to record on that camera.

We’re going to grab VorlonCD’s version of AITool which can be found here. This is a fork of GentlePumpkin’s original tool that has quite a few enhancements made to it. Once you have it downloaded, unzip it somewhere convenient. Go to that folder, right-click on AITool.exe and click run as administrator.

Settings and Camera Setup

First we need to hit up the settings tab. In here we need to setup the Deepstack server address. The format is ipaddress:port. If the Deepstack server is running on the same machine as AITool you should be able to put in 127.0.0.1:port or localhost:port. For example, mine says localhost:81. This is the same port you put in the Deepstack server earlier.

Next we’ll click the cameras tab and click “Add” at the bottom to add a camera. Once the dialog box pops up, give it a name and hit ok. The next option is setting the prefix we’ll be looking for in the picture directory. This should be the same as the short name you setup for the camera we are using. In my example, the short name is “frontdoor”. When Blue Iris takes pictures from that camera, it puts them in our aiinput directory, and the name will start with “frontdoor”.

Next, we need to select the directory to look in. This is the directory you setup in Blue Iris earlier. Mine was called aiinput. Click browse, locate the folder, and click ok.

What to detect

Here’s a bit of a fun part. We get to select what to watch for. You could keep it simple and select just person. You can see in mine that I selected almost everything. This is mostly out of fun; you never know what might walk by! Mostly I want to catch people and vehicles going by. Right under this is your confidence ratings to watch for. To start, leave this at 0% and 100%. As you get data you can decide to up that low end to keep false positives at bay.

The last part is to click the settings button for this camera. The most important part here is the trigger URLs at the bottom. You need to copy and paste the two following URLs in the trigger URL box.

The things you need to replace in here are the IP and port, the username, the password, and the memo portion. The IP and port are for the Blue Iris server. The username and password will be the AI user we setup in Blue Iris, and the memo field on the second URL can be a variable that AI Tool can send to Blue Iris to mark the clip. In mine I have [Detection] which will give the clip a mark of “Person (90.5%)” or something like “Car (84%)”. You can also set the cool down at the top if you don’t want it detecting multiple pictures in a row.

What to Detect – UPDATE

I’ve seen some people that have been having issues with this setup always recording on motion, even when AITool/DeepStack don’t detect anything. This is due to the fact we are grabbing a picture when triggered and grabbing video on triggers. There’s no way to tell it to only record on external triggers so this is something that I had been dealing with for a while.

That is until I found out about the Cancel URL. From my research, the Cancel URL was built for the Sentry AI program and allows you to cancel an alert if the AI comes back with nothing. Well, almost. It cancels them but then stores them under the “Cancelled Alerts” folder. See below for how I have the Cancel URL in AITool setup.

The basically the flagalert=0 means it’s going to cancel alerts/motion triggers that the AI didn’t detect anything for. Again, it still records, it just hides them away in a “Cancelled Alerts” folder. Here is the syntax:

Since I’ve done this, only the alerts that were true hits from AITool/Deepstack show up in my alert lists. The upside being is that ones that didn’t are still available if I think I need them under the “Cancelled Alerts” drop down.

Testing

We should be fully setup. If you go walk in front of this camera (and had Person as one of your detections), you should see the picture show up in the history tab for that camera, and it should show you the annotation of what it found, and the percent confidence. Watch the logs in AITool to confirm your trigger URLs worked. You should be able to head to Blue Iris, list out the alerts (I usually filter by “Flagged” since our trigger URL flags AI Detections) and see where you walked in front of the camera.

If AITool shows the picture, and Blue Iris has the video, you should be good to go. Repeat this as needed for your cameras. If it’s not working, a good place to check is if the camera is taking pictures. If it is, make sure AITool is looking at the right directory, or that the Deepstack server is running and AITool is pointed to the right address for it.

Final Thoughts

There are plenty of other things you can set up with AITool that are super helpful. I use the Telegram integration to send me the snapshots from my front door camera when a person is there, or anytime there’s motion in my back yard. You can also set up MQTT and send topics via MQTT. I use this to send status to sensors in my Home Assistant installation. It basically allows me to turn my cameras into motion sensors for Home Assistant, which allows me to create some pretty powerful automations.

Let me know if this tutorial for AI Object detection for Blue Iris has been helpful to you. If you’re stuck, leave me a comment below and I can try to point you in the right direction. Keep an eye out later this week because I plan to post a video tutorial on my YouTube Channel for this same setup!

Below I’ve posted a list of the cameras I use around the house if you are interested in my favorite cameras!

Thank you to Michael Bowling, Michael Johanson, and Marc Lanctot for contributions to this guide.

Additionally, this would not have been possible without the generous support ofProf. Joan Bruna and his class at NYU, The Mathematics of Deep Learning.Special thanks to him, as well as Martin Arjovsky, my colleague in leading thisrecitation, and my fellow students Ojas Deshpande, Anant Gupta, Xintian Han,Sanyam Kapoor, Chen Li, Yixiang Luo, Chirag Maheshwari, Zsolt Pajor-Gyulai,Roberta Raileanu, Ryan Saxe, and Liang Zhuo.

Along with Libratus, DeepStack is one of two approaches to solving No-Limit Texas Hold-em that debuted coincidentally. This game was notoriously difficultto solve as it has just as large a branching factoras Go, but additionally is a game of imperfect information.

The main idea behind both DeepStack and Libratus is to use Counterfactual Regret Minimization (CFR) to find a mixed strategy that approximates a Nash Equilibrium strategy. CFR’s convergence properties guarantee that we will yield such a strategyand the closer we are to it, the better our outcome will be. They differ intheir implementation. In particular, DeepStack uses deep neural networksto approximate the counterfactual value of each hand at specific points in thegame. While still being mathematically tight, this lets it cut short the necessary computation to reach convergence.

In this curriculum, you will explore the study of games with a tour through game theory and counterfactual regret minimization while building up the requisite understanding to tackle DeepStack. Along the way, you will learnall of the necessary topics, including what is the branching factor, all aboutNash Equilibria, and CFR.

1. MAS: Multi Agent Systems.
2. LT: Marc Lanctot’s Thesis.
3. ICRM: Introduction to Counterfactual Regret Minimization.
4. PLG: Prediction, Learning, and Games.

Motivation: Most of Game Theory, as well as the particular techniques used in DeepStack and Libratus, is built on the framework of Normal Form Games. These are game descriptions and are familiarly represented as a matrix, a famous example being the Prisoner’s Dilemma. In this section, we cover the basics of Normal Form Games. In addition, we go over the rules of Poker and why it had proved so difficult to solve.

Required Reading:

1. MAS: Sections 3.1 & 3.2.
2. LT: Pages 5-7.
3. The Game of Poker: Supplementary #1 on pages 16-17.

Optional Reading:

1. The State of Solving Large Incomplete-Information Games, and Application to Poker (2010)
2. Why Poker is Difficult Very good video by Noam Brown, the main author of Libratus. The first eighteen minutes are the most relevant.

Questions:

1. LT: Prove that in a zero-sum game, the Nash Equilibrium strategies are interchangeable.
   Hint

   Use the definition of a Nash Equilibrium along with the fact that (mu_{i}(sigma_{i}, sigma_{-i}) + mu_{-i}(sigma_{i}, sigma_{-i}) = c).

2. LT: Prove that in a zero-sum game, the expected payoff to each player is the same for every equilibrium.
   Solution

   We will solve both this problem and the one above here. We have that if(mu_{i}(sigma) = mu(sigma_{i}, sigma_{-i})) and(mu_{i}(sigma') = mu(sigma_{i}', sigma_{-i}')) are both Nash Equilibria, then:

   (begin{align}mu_{i}(sigma_{i}, sigma_{-i}) &geq mu_{i}(sigma_{i}', sigma_{-i}) &= c - mu_{-i}(sigma_{i}', sigma_{-i}) &geq c - mu_{-i}(sigma_{i}', sigma_{-i}') &= mu_{i}(sigma_{i}', sigma_{-i}')end{align})

   In a similar fashion, we can show that (mu(sigma_{i}', sigma_{-i}') geq mu(sigma_{i}, sigma_{-i})).

   Consequently, (mu(sigma_{i}', sigma_{-i}') = mu(sigma_{i}, sigma_{-i})),which also implies that the strategies are interchangeable, i.e.(mu(sigma_{i}', sigma_{-i}') = mu(sigma_{i}', sigma_{-i})).
3. MAS: Prove Lemma 3.1.6.
   (textit{Lemma}): If a preference relation (succeq) satisfies the axiomscompleteness, transitivity, decomposability, and monotonicity, and if (o_1 succ o_2)and (o_2 succ o_1), then there exists probability (p) s.t. (forall p' < p),(o_2 succ [p': o_1; (1 - p'): o_3]) and for all (p' > p),([p': o_1; (1 - p'): o_3] succ o_2.)
4. MAS: Theorem 3.1.8 ensures that rational agents need only maximize the expectation of single-dimensional utility functions. Prove this result as a good test of your understanding.
   (textit{Theorem}): If a preference relation (succeq) satisfies the axioms completeness, transitivity, substitutability, decomposability, monotonicity, and continuity, then there exists a function (u: mathbb{L} mapsto [0, 1]) with the properties that:
   1. (u(o_1) geq u(o_2)) iff (o_1 succeq o_2).
   2. (u([p_1 : o_1, ..., p_k: o_k]) = sum_{i=1}^k p_{i}u(o_i)).

Motivation: How do you reason about games? The best strategies in multi-agent scenarios depend on the choices of others. Game theory deals with this problem by identifying subsets of outcomes called solution concepts. In this section, we discuss the fundamental solution concepts: Nash Equilibrium, Pareto Optimality, and Correlated Equilibrium. For each solution concept, we cover what it implies for a given game and how difficult it is to discover a representative strategy.

Required Reading:

1. MAS: Sections 3.3, 3.4.5, 3.4.7, 4.1, 4.2.4, 4.3, & 4.6.
2. LT: Section 2.1.1.

Optional Reading:

1. MAS: Section 3.4.

Questions:

1. Why must every game have a Pareto optimal strategy?
   Solution

   Say that a game does not have a Pareto optimal outcome. Then, for every outcome (O), there was another (O') that Pareto-dominated (O).Say (O_2 > O_1). Because (O_2) is not Pareto optimal, there is some (O_k > O_2). There cannot be a max in this chain (because that max wouldbe Pareto optimal) and thus there must be some cycle. Consequently, thereexists for some agent a strategy (O_j) s.t. (O_j > O_j), which is a contradiction.

2. Why must there always exist at least one Pareto optimal strategy in which all players adopt pure strategies?
3. Why in common-payoff games do all Pareto optimal strategies have the same payoff?
   Solution

   Say two strategies (S) and (S') are Pareto optimal. Then neitherdominates the other, so either (forall i mu_{i}(S) = mu_{i}(S'))or there are two players (i, j) for which (mu_{i}(S) < mu_{i}(S'))and (mu_{j}(S) > mu_{j}(S')). In the former case, we see that thetwo strategies have the same payoff as desired. In the latter case, we havea contradiction because (mu_{j}(S') = mu_{i}(S') > mu_{i}(S) = mu_{j}(S) > mu_{j}(S')). Thus, all of the Pareto optimal strategies must have the same payoff.

4. MAS: Why does definition 3.3.12 imply that the vertices of a simplex must all receive different labels?
   Solution

   This follows from the definitions of (mathbb{L}(v)) and (chi(v)).At the vertices of the simplex, (chi) will only have singular values inits range defined by the vertice itself. Consequently, (mathbb{L}) mustas well.

5. MAS: Why in definition 3.4.12 does it not matter that the mapping is to pure strategies rather than to mixed strategies?
6. Take your favorite normal-form game, find a Nash Equilibrium, and then find a corresponding Correlated Equilibrium.

Motivation: What happens when players don’t act simultaneously? Extensive Form Games are an answer to this question. While this representation of a game always has a comparable Normal Form, it’s much more natural to reason about sequential games in this format. Examples include familiar ones like Go, but also more exotic games like Magic: The Gathering and Civilization. This section is imperative as Poker is best described as an Extensive Form Game.

Required Reading:

1. MAS: Sections 5.1.1 - 5.1.3.
2. MAS: Sections 5.2.1 - 5.2.3.
3. Accelerating Best Response Calculation in Large Extensive Games: This is important for understanding how to evaluate Poker algorithms.

Optional Reading:

1. LT: Section 2.1.2.

Questions:

1. What is the intuition for why not all normal form games can be transformed into perfect-form extensive games?
   Solution

   The problem is one of modeling simultaneity. Perfect information extensive form games have trouble modeling concurrent moves because theyhave an explicit temporal structure of moves.

2. Why does that change when the transformation is to imperfect extensive games?
3. How are the set of behavioral strategies different from the set of mixed strategies?
   Solution

   The set of mixed strategies are each distributions over pure strategies. The set of behavioral strategies are each vectors of distributions over theactions and assign that distribution independently at each Information Set.

4. Succinctly describe the technique demonstrated in the Accelerating Best Response paper.

Motivation: Counterfactual Regret Minimization (CFR) is only a decade old but has already achieved huge success as the foundation underlying DeepStack and Libratus. In the first of two weeks dedicated to CFR, we learn how the algorithm works practically and get our hands dirty coding up our implementation.

The optional readings are papers introducing CFR-D and CFR+, further iterations upon CFR. These are both used in DeepStack.

Required Reading:

1. ICRM: Sections 2.1-2.4.
2. ICRM: Sections 3.1-3.4.
3. LT: Section 2.2.
4. Regret Minimization in Games with Incomplete Information.

Optional Reading: These two papers are CFR extensions used in DeepStack.

1. Solving Imperfect Information Games Using Decomposition: CFR-D.
2. Solving Large Imperfect Information Games Using CFR+: CFR+.

Questions:

1. What is the difference between external regret, internal regret, swap regret, and counterfactual regret?
   Hint

   The definitions of the three are the following:

   • External Regret: How much the algorithm regrets not taking the bestsingle decision in hindsight. We compare to a policy that performs a singleaction in all timesteps.
   • Internal Regret: How much the algorithm regrets making one choiceover another in all instances. An example is whenever you bought Amazon stock,you instead bought Microsoft stock.
   • Swap Regret: Similar to Internal Regret but instead of one categoricalaction being replaced wholesale with another categorical action, now we allowfor any number of categorical swaps.
   • Counterfactual Regret: Assuming that your actions take you to a node, this is the expectation of that node over your opponents' strategies.The counterfactual component is that we assume you get to that node with aprobability of one.
2. Why is Swap Regret important?
   Hint

   Swap Regret is connected to Correlated Equilibrium. Can you see why?

3. Implement CFR (or CFR+ / CFR-D) in your favorite programming language to play Leduc Poker or Liar’s Dice.
4. How do you know if you’ve implemented CFR correctly?
   Solution

   One way is to test it by implementing Local Best Response. It should perform admirably against that algorithm, which is meant to best it.

Motivation: In the last section, we saw the practical side of CFR and how effective it can be. In this section, we’ll understand the theory underlying it. This will culminate with Blackwell’s Approachability Theorem, a generalization of repeated two-player zero-sum games. This is a challenging session but the payoff will be a much keener understanding of CFR’s strengths.

Deepstacks Poker

Required:

1. PLG: Sections 7.3 - 7.7, 7.9.

Optional:

Deep Stack Extravaganza

1. A Simple Adaptive Procedure Leading to Correlated Equilibrium.
2. Prof. Johari’s 2007 Class - 11.
3. Prof. Johari’s 2007 Class - 13.
4. Prof. Johari’s 2007 Class - 14.
5. Prof. Johari’s 2007 Class - 15.

Questions:

1. PLG: Prove Lemma 7.1.
   (textit{Lemma}): A probability distribution (P) over the set of all (K)-tuples(i = (i_{1}, ..., i_{K})) of actions is a correlated equilibrium iff, for everyplayer (k in {1, ..., K}) and actions (j, j' in {1, ..., N_{k}}), we have

   [sum_{i: i_{k} = j} P(i)big(mathcal{l}(i) - mathcal{l}(i^{-}, j')big) leq 0]

   where ((i^{-}, j') = (i_{1}, ..., i_{k-1}, j', i_{k+1}, ..., i_{K})).

2. It’s brushed over in the proof of Theorem 7.5 in PLG, but prove that if set (S) is approachable, then every halfspace (H) containing (S) is approachable.

   Solution

   Because (S in H) is approachable, we can always find a strategy for player one s.t.the necessary approachability clauses hold (see Johari's Lecture 13). Namely, choosethe strategy in (S) that asserts (S) as being approachable.

Motivation: Let’s read the paper! A summary of what’s going on to help with your understanding:

Deepstack Blue Iris

DeepStack runs counterfactual regret minimization at every decision. However, it uses two separate neural networks, one for after the flop and one for after the turn, to estimate the counterfactual values without having to continue running CFR after those moments. This approach is trained beforehand and helps greatly with cutting short the search space at inference time. Each of the networks take as input the size of the pot and the current Bayesian ranges for each player across all hands. They output the counterfactual values for each hand for each player.

In addition to DeepStack, we also include Libratus as required reading. This paper highlights Game Theory and CFR as the really important concepts in this curriculum; deep learning is not necessary to build a champion Poker bot.

Required Reading:

1. DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker.
2. DeepStack Supplementary Materials.
3. Libratus.
4. Michael Bowling on DeepStack.

Optional Reading:

1. DeepStack Implementation for Leduc Hold’em.
2. Noam Brown on Libratus.
3. Depth-Limited Solving for Imperfect-Information Games: This paper is fascinating because it is achieves a poker-playing bot almost as good as Libratus but using a fraction of the necessary computation and disk space.

Questions:

1. What are the differences between the approaches taken in DeepStack and in Libratus?
   Solution

   Here are some differences:

   • A clear difference is that DeepStack uses a deep neural network to reduce the necessary search space, and Libratus does not.
   • DeepStack does not use any action abstraction and instead melds those considerations into the pot size input. Libratus does use a dense action abstraction but adapts it each game and additionally constructs new sub-games on the fly for actions not in its abstraction.
   • DeepStack uses card abstraction by first clustering the hands into 1000 buckets and then considering probabilities over that range. Libratus does not use any card abstraction preflop or on the flop, but does use it on later rounds such that the game's (10^{61}) decision points are reduced to (10^{12}).
   • DeepStack does not have a way to learn from recent games without further neural network training. On the other hand, Libratus improves via a background process that adds novel opponent actions to its action abstraction.
2. Can you succinctly explain “Continual Re-solving”?
3. Can you succinctly explain AIVAT?