Exploration VS Explotation

If you always order the same beer you enjoy, you might be missing out on discovering an even better one. This dilemma isn’t unique to humans, machine learning models face a similar challenge: deciding when to exploit the best-known option and when to explore new alternatives. This is known as the exploration vs. exploitation problem.

A common strategy to balance this trade-off, especially when outcomes are not deterministic, is to consider the uncertainty of each alternative. If you're feeling adventurous (or if an ML model is in its early learning stage), you might prefer options with high uncertainty to discover potentially better choices. On the other hand, if you want a safe bet, you can rely on the best-known option, ignoring or penalising uncertainty.

Even with beers, our preferences aren’t fixed, what you enjoy one day might vary the next. Therefore, using uncertainty in our decision-making can help us navigate these variations. Based on how much you’ve enjoyed each beer in the past (historical feedback), and depending on how adventurous you feel, the following interactive chart will help you decide which beer to go for.