When people think of risk as a concept, many people think of losing money (which, if you were in the stock market last year, yeah, ouch!). But going to zero is not the worst possible outcome from a risk perspective.
No, the worst possible outcome is unlimited losses. Unlimited losses stem from unbounded downside risk, insufficiently mitigated—the nemesis of every risk manager.
How about some examples?
Sure. Back in 2012, an employee at trading firm Knight Capital Group made an error while rolling out new code for a trade routing system, which resulted in Knight losing more than $400 million on its automated trades in about 45 minutes (at which point the issue was identified and the system shut down; otherwise, they would have kept losing money). The company was ultimately acquired by Getco LLC. Its existence as an independent firm ended.
More generally, a canonical example of unbounded downside risk coupled with bounded upside potential (the worst possible combination) is shorting a stock without hedging the trade. When you short a stock, you are betting that the stock price will decrease. And if the stock price does decrease, you make money. But your upside is limited: if you short 100 shares of a $10 stock and it goes to zero, your maximum possible gain is $1000 (100 * $10). Even worse, if the stock price increases instead of decreasing, you lose money, and stock prices do not have an upper bound. If you short a stock and it goes up, it can keep going up pretty much indefinitely, and you can keep losing money pretty much indefinitely (until you cover). Imagine shorting Amazon back in 2001 at 30 cents per share (split-adjusted)—its share price has increased more than 300x since then.
Another example: one reason why people are so worried about future AI is that the downside risk is potentially unbounded and not sufficiently hedged yet1. If AI someday evolves to become artificial general intelligence (AGI) and is not aligned with human interests, it theoretically could eliminate humans. And there is no defined upper bound on possible damage. A hostile AI theoretically could expand into space and eventually reach other galaxies. Vastly improved controls might eventually serve as a hedge against AI risk, but it’s not clear whether downside risks can be sufficiently mitigated to make them bounded. It’s not as easy as hedging a short trade with a counterbalancing long position.
So, potentially unbounded losses are bad. Let’s talk symmetry and asymmetry.
Many risks are asymmetric, and risk scenarios can be asymmetric in either direction. I’m using the term “asymmetric” to mean either the downside outcome is more likely than the upside outcome (you’re more likely to lose than win), or the upside outcome is more likely than the downside outcome (you’re more likely to win than lose).2 The degree of asymmetry can vary widely on a spectrum, with many possible values, and is a major and complex determinant of whether a risk is worth taking or not.
Considering asymmetry as well as the magnitude of downside risk or upside potential, we can identify several categories of risk scenarios3:
Asymmetric toward an unbounded downside with a bounded upside
In plain English: you’re likely to lose more than you gain, and your losses could be unlimited, while your gains are capped.
Example: Ignoring worsening indigestion for months or years until you end up in the emergency room with a bowel obstruction and a diagnosis of Stage Four colon cancer. The downside of waiting and hoping it would just go away is potentially unlimited, and the bounded upside was living in blissful ignorance for a few months or years.Asymmetric toward a bounded downside with a bounded upside
In plain English: you’re likely to lose more than you gain, but your losses can’t be unlimited.
Example: Playing blackjack at a casino (without card-counting). The odds favor the dealer, and the longer you sit at the table and play, the more likely those odds will manifest as a loss for you, but you can only lose what you bet. Your gain is limited to what you can win before the long-term odds get the better of you.Asymmetric toward a bounded downside with an unbounded upside
In plain English: you’re likely to lose more than you gain, but if you succeed, your gains are potentially unlimited.
Example: Starting a startup. You commit a finite amount of time and capital, and you incur opportunity cost if you earn less than you would have earned at a more traditional job. Many startups fail, and chances are you will, too—but if you succeed, your gains are potentially unbounded.Asymmetric toward a bounded upside with a bounded downside
In plain English: the odds are in your favor, but your potential gains have a cap.
Example: Taking an appealing job with a better salary. Unless the job is wildly different than presented, odds are that since you chose the role, you’ll prefer it to your prior job, and you’ll make more money, but your salary will be capped at whatever you can negotiate, which is likely not infinitely larger than your prior salary. The worst thing that can happen (as long as you don’t take a separate action with asymmetric downside, such as committing egregious wrongdoing) is probably that you’re fired from the new job, but it’s unlikely.Asymmetric toward a bounded upside with an unbounded downside
In plain English: you’re likely to succeed, but if things go off the rails, your losses are potentially unlimited.
Example: Speeding 80+ miles per hour on the highway. This will go fine most of the time, and you’ll reach your destination earlier than you would have otherwise, but if you get in an accident, you could permanently harm or kill yourself and others.Asymmetric toward an unbounded upside with a bounded downside
In plain English: if you’re comfortable with the limited downside, are certain it’s really limited, and your circumstances permit, you should take these risks.
Example: Investing $10,000 in Facebook as an angel investor. Most angel investors have enough assets that they can afford to lose $10,000 without really missing it. Facebook was demonstrably growing very fast in its early days, and if you believed it would continue to grow, the upside vastly surpassed the downside.4 Worst case: you lose $10,000. Best case: you make a few million dollars.Symmetric with bounded downside and bounded upside
In plain English: the odds of either outcome are exactly equal, and your losses or gains are limited.
Example: A coin flip to win or lose $100. You will either win or lose $100, no more and no less, and the odds of each outcome are exactly 50%. True coin-flip scenarios are uncommon in complex, messy real life; most risk scenarios are asymmetric, favoring either the downside or the upside.Uncertain symmetry with unbounded downside and unbounded upside
In plain English: it’s hard to measure infinity in either direction, so symmetry is often uncertain if both sides are unbounded. Here be dragons.
Example: Creating artificial general superintelligence. Lots of people want to work on this because the potential gains seem unlimited: solving climate change, biomedical breakthroughs, eliminating unpleasant tasks from humans’ repertoire, even expanding into space and exploring the broader universe5. But the potential downside is not just the end of humans but the end of many worlds. And the directionality of the risk is uncertain right now. Your position on AGI will depend on whether you believe the risk is asymmetric toward the downside (AGI is more likely to be misaligned when it comes into being) or asymmetric toward the upside (humans are likely to figure out alignment, and human-aligned AGI will shepherd us into a new and wonderful era).
Takeaways: thinking through the tradeoffs
To sum up, your risk decisions will depend on the degree of asymmetry you perceive or calculate, the directionality of that asymmetry (favoring the upside or downside), the boundedness of both upside and downside, and your unique situation, comfort level, and margin for error.
One way to approach this is to ask: what’s the worst thing that could go wrong if I take this action, and what’s the worst thing that could go wrong if I do not take this action? Are those worst-case outcomes unbounded or bounded risks?
If you’re facing a potentially unbounded downside risk, you might consider avoiding that path and looking for a different way to accomplish your goals. If you’re facing an asymmetric risk favoring the downside—even if that risk is bounded—and the upside is limited, you also might choose to look for other options.
On the other hand, if potential for gain is higher than the risk of loss, you may choose to take a risk since upside is favored (again, taking into account your own situation, context, comfort level, and margin for error). And if the gain is potentially unbounded, and the maximum potential loss is low and bounded, it may be a golden opportunity (or, it may not be! You may have overlooked vipers at the bottom of the pit, or something could happen two years from now to change the equation. Do your due diligence! There are no guarantees in risk).
Lastly, if potential gain is unbounded, and potential loss is also unbounded, and there is no consensus on whether the downside or upside is more likely (the direction of asymmetry is uncertain), you are on dangerous ground and should step carefully until the lay of the land becomes clearer. Respect uncertainty, measure what you can, and exercise proactive caution to avoid getting caught off-guard. This is risk management.
In my view, it’s clear that controls need vast improvement before further AI development proceeds, based on the bumpy initial rollout of Bing Chat. The only good thing about that rocky rollout was that Bing Chat is an early AI and not a super-advanced AGI, so we have an opportunity to learn from the experience and do better in the future.
I updated this to make it a little clearer. I welcome feedback on these dimensions and categories, as it’s an idea I likely will explore more in future essays.
This may not be exhaustive, but it covers the most common categories. Could there be scenarios with unbounded downsides and unbounded upsides that are asymmetrical in one direction or another? Yes, there could be. I may write an article digging into this more exhaustively in the future.
This is referring purely to monetary upside from the perspective of an angel investor. Back then, Facebook seemed like a nice way to connect with fellow college students and keep in touch over time, and I believe you could not realistically have foreseen that it would become a misinformation amplifier, algorithmic hider of your friends’ posts, or possible culprit behind increasing anxiety and depression.
It’s debatable whether humans expanding beyond Earth would be a net positive for the universe or not. We haven’t done a great job of taking care of our own planet yet, and I think we should focus on getting that right.
“In plain English” this is a most useful and understandable guide for anyone for deciding. Well done and thanks this is going into my permanent file.