Many instances of human judgment occur in high uncertainty environments such that multiple hypotheses or beliefs are initially considered candidate explanations for a set of observations. Decision-makers often engage in data acquisition and hypothesis testing in such circumstances to improve the accuracy of their judgments. I propose a memory-theoretic account to explain this behavior, positing a dynamic information valuation process driven by changes in memory activation (belief) about hypotheses—a formalization of a principle called hypothesis-guided search. I demonstrate a decision support system (DSS) based on a model implementation of the theory. I also discuss how one could "optimize" the DSS as an actuarial tool or leverage it to anticipate when human operators require aid. I report two empirical studies. The first evaluated the basic premise of the theory that beliefs underly test selection and information foraging. The second study investigated methods for broadening intelligence analysts' reasoning, illustrating a bidirectional relation between information acquisition and hypothesis generation. I will discuss both the applied and theoretical implications of the notion that information acquisition behavior will be sensitive to variables that affect hypothesis generation (e.g., time pressure, dual-task, framing, working memory capacity, etc.).

Complexity is blamed for many of the challenges that face design organizations, including the acute failures of systems engineering (e.g. cost and schedule overruns). Decomposition is believed to be a dominant strategy for managing complexity, since it enables problems to be broken up into more manageable modules that can be designed in parallel. So far, the system architecture literature implicitly assumed that traditional players will develop systems using traditional practices and predominantly pursued technical objectives. However, the rise of “gig" work, and broader adoption of open innovation tools call the efficacy of traditional architecting approaches into question. There is an opportunity to re-think architecting principles with an awareness of who will solve and how the solvers will engage in the design process.

This seminar will characterize two issues regarding complexity and decomposition; and will introduce a framework for leveraging the external knowledge of the crowd to design complex engineered systems. I will demonstrate that joint consideration of problem formulation, organizational knowledge, and outside expertise could significantly improve design process outcomes; and discuss strategies for incorporating solver-awareness into the architecting process. Finally, I will discuss how mixed-methods research approaches could help to develop a novel sociotechnical theory for architecting complex systems.

New observing strategies for Earth science space missions consider distributed operations across platforms, missions, and agencies. Collaborative system architectures seek mutual benefit for all cooperating actors but also introduce new interdependencies which can lead to degraded performance compared to independent alternatives. The interactive uncertainty of coordination failure between actors is not addressed by existing system design methods that view engineering design as a centralized decision-making process. This presentation introduces how the game theoretic concept of risk dominance can inform architectural decisions during conceptual design activities. A measure of risk dominance applied to tradespace analysis helps to inform robust design of collaborative systems to simultaneously mitigate the downside risk of coordination failure and increase the likelihood of successful collaboration. Applying theoretical insights from risk dominance to stabilize collaboration motivates new design methods such as a New Observing Strategies Testbed (NOS-T) as a computational platform to prototype and mature core technology for future Earth science missions.

As our ability to engineer systems that exhibit high degrees of complexity increases, it is imperative to increase equally our ability to design and manage the organizations that support these complex systems throughout their lifecycles. Achieving balance in the sophistication and robustness in the methods for engaging with complex systems through the organizations that support them throughout their lifecycles, unfortunately, has not been the case. There are several challenges that makes this so, for instance:

1) complex engineered systems possess a high degree of variety in behaviors and processes, anticipated or not, that require even more complex organizations possessing higher degrees of variety of responses to manage a complex system’s variety. This adds to developmental costs, but it is often hard to estimate some key aspects of organizational design and more importantly to justify these costs;

2) complex organizations that support complex engineered systems are typically multidimensional organizations designed to support core enterprise systems and processes, while at the same time requiring enough flexibility and adaptability to be able to support several complex engineered systems, each with significant differences in missions, resource requirements, lifecycles, etc.

3) complex organizations exhibit VUCA characteristics (volatility, uncertainty, complexity, and ambiguity) where physical and socially constructed elements interact.

In this presentation, an overview of the Purposeful Human Activity System’s theoretical foundations will be provided and its potential application will be illustrated via a case study.

The primary goal of Systems Engineering is to develop solutions that satisfy stakeholders’ preferences. In current practices, these are represented using textual stakeholder needs statements. These stakeholder needs are further defined and derived into system requirements, which are then decomposed into subsystem and component level requirements that are flowed down the organizational hierarchy to aid in decision-making. The system of interest is realized only when all the stakeholder needs are satisfied, which in turn depends on the satisfaction of lower-level requirements. Given that the organizations that develop these systems consist of hundreds to thousands of decision-makers spread globally across the organizational hierarchy, it is extremely challenging to facilitate consistent decision-making that enables satisfaction of all stakeholders’ preferences. This is exacerbated by the fact that stakeholders’ preferences are represented textually in current practice, which results in a lack of analysis capabilities including, but not limited to, evaluation of inconsistencies in stakeholders’ preferences, identification of optimal solutions that satisfy stakeholders’ preferences, changes/updates in preferences, decision traceability, etc. Value models and utility functions, which are founded on normative Decision Theory, enable a rank ordering of alternatives, but are difficult to form, requiring significant investment and effort with no framework that guarantees accuracy. This talk will discuss a novel approach to preference representation, communication, and execution that is founded on formal logic.

Advances in artificial intelligence enable computers to support humans in new ways as peers in hybrid teams in many complex problem-solving situations. The application domains of this new collaboration paradigm range from information to healthcare, transportation, energy, or defense systems. To leverage the full potential of human-AI collaboration, new decision-making architectures that account for the feedback from human users are necessary. This talk will present an example for such a decision-making architecture on an application case of the real-time strategy game Starcraft 2. The architecture in this talk integrates methods from sequence learning, model predictive control, and game theory. Computers in this architecture learn objectives and strategies from experimental data to support human players with strategic decisions while operational decisions are made by humans. This talk will discuss the computational results showing under which situations this collaboration could add some value to human decision-makers. Finally, the talk will conclude with some important lessons learned from this application problem and discuss some open research questions yet to be answered.