Every day, we make choices in the fog of uncertainty: A doctor weighs treatment options when test results are inconclusive. A business leader plans strategy amid shifting market trends. A farmer decides when to plant, guessing at weather that might change. These are human problems—rooted in incomplete information, competing values, and the need to act even when we can’t know the outcome. Now, as we hand more complex decisions to AI, that fog of uncertainty has become a critical challenge: Can artificial intelligence ever navigate it as thoughtfully as we do?
A new large language model framework, developed by Willie Neiswanger—assistant professor of computer science at the USC Viterbi School of Engineering and the USC School of Advanced Computing—and students in USC Viterbi’s Thomas Lord Department of Computer Science, aims to bridge this gap. By combining classical decision theory and utility theory, the framework significantly enhances AI’s ability to face uncertainty and tackle those messy, human-like decisions.
Neiswanger’s research was spotlighted at this year’s International Conference on Learning Representations. He recently discussed how AI handles uncertainty with USC News.
What are your thoughts on the difference between artificial and human intelligence?
Neiswanger: At present, human intelligence has various strengths relative to machine intelligence. However, machine intelligence also has its own advantages that make it valuable. Large language models (LLMs)—AI systems trained on vast amounts of text to understand and generate human-like responses—can, for example, rapidly absorb and synthesize large volumes of information from reports or data sources. They can also generate at scale: simulating multiple possible futures or proposing a wide range of forecasted outcomes. In our work, we aim to leverage the strengths of LLMs while balancing them with human strengths and judgment.
Why do current AI large language models struggle with uncertainty?
Neiswanger: Uncertainty is a fundamental challenge in real-world decision-making. Current AI systems struggle to properly balance uncertainty, evidence, and the process of making predictions based on the likelihood of different outcomes—let alone aligning with user preferences when faced with unknown variables.
Unlike human experts, who can express degrees of confidence and acknowledge the limits of their knowledge, LLMs typically generate responses with apparent confidence, whether they’re drawing from well-established patterns or making shaky predictions that go beyond available data.
How does your research intersect with uncertainty?
Neiswanger: I focus on developing machine learning methods for decision-making under uncertainty—especially sequential decision-making, where choices unfold over time and each decision shapes future options—often in settings where data is costly to acquire. This includes applications like black-box optimization (finding the best solution when you can’t see a system’s inner workings), experimental design (planning studies to get the most useful information), and decision-making in science and engineering—such as materials or drug discovery, and optimizing computer systems.
I’m also interested in how large foundation models (massive AI systems trained on enormous datasets, serving as a base for many applications)—especially LLMs—can both enhance and benefit from these decision frameworks. On one hand, they can help humans make better decisions in uncertain environments; on the other, mathematical methods for optimal choices can improve LLMs, helping them achieve better results with less training data and higher quality in training and fine-tuning.
How did your research address the problem of uncertainty and AI?
Neiswanger: We focused on improving a machine’s ability to quantify uncertainty—essentially teaching it to measure and express how confident it should be about different predictions. Specifically, we developed an uncertainty quantification approach that enables LLMs to make decisions under incomplete information, while also making predictions with measurable, verifiable confidence levels and choosing actions that align with human preferences.
The process starts by identifying key uncertain variables relevant to a decision. Then, language models assign language-based probability scores to different possibilities—like crop yields, stock prices, uncertain event dates, or projected warehouse shipment volumes—using reports, historical data, and other context. These scores are then converted to numerical probabilities, grounding the AI’s “judgment” in measurable terms.
Are there immediate applications?
Neiswanger: In business, it could improve strategic planning by offering more realistic assessments of market uncertainties and competitive dynamics. In medical settings, it might support diagnostics or treatment planning, helping physicians better account for uncertainty in symptoms and test results. For personal decisions, it could help users get more informed, relevant advice from language models about everyday choices.
The system’s ability to align with human preferences is particularly valuable. In cases where a mathematically “best” solution might overlook key human values or constraints, explicitly modeling stakeholder preferences and incorporating them into assessments of outcome value ensures recommendations are not just technically optimal, but practically acceptable to those who implement them.
What’s next for your research?
Neiswanger: We’re now exploring how to extend this framework to more real-world decision-making tasks under uncertainty—including operations research (using math to solve complex business problems), logistics, and healthcare. A key focus moving forward is improving human auditability: developing interfaces that give users clearer visibility into why an LLM made a specific decision, and why that decision is considered optimal.
Uncertainty isn’t a flaw in decision-making—it’s the reality of being human. As AI steps into more of our complex choices, the goal isn’t to replace our ability to navigate that reality, but to enhance it. Neiswanger’s framework is a step toward that: AI that doesn’t pretend to have all the answers, but instead works with us—quantifying its doubts, aligning with our values, and making uncertainty a shared challenge, not a barrier. In the end, the best decisions under uncertainty may not be human or AI alone—but a collaboration that draws on both.
What are the applications of 4-axis robotic arms in industrial production?
