Discounted values

Figure: Time-course of neural signals related to temporally discounted values (Kim et al., 2012) 

Economic Decision Making. Outcomes expected from our actions often vary in multiple dimensions, such as the magnitudes and probabilities of gains and losses as well as how soon such outcomes become available.  We study how these qualitatively different pieces of information can be estimated from diverse sources, and how they might be combined to compare the desirabilities or values of different actions.  To understand the nature of underlying computations for such complex decision making and their neural substrates, we develop novel behavioral tasks that can be learned by non-human primates, and probe the activity of neurons in multiple brain regions, including the prefrontal cortex and basal ganglia (Kim et al., 2008; Cai et al., 2011). We utilize multi-channel recording techniques and machine learning algorithms to investigate the functions of multiple brain areas involved in decision making.

Movie: Token-based decision making task (Seo and Lee, 2009)

Reinforcement Learning. Reinforcement learning theory provides crisp quantitative methods to test how decision makers learn the relationship between actions and their outcomes from the statistical regularities in their environment.  We study how the brain flexibly implements specific reinforcement learning algorithms according to the uncertainty and stability of the environment. For example, depending on the structure of the environment and the amount of experience, animals might rely more on habits and algorithms similar to stimulus-response mapping, or on goal-directed behaviors that require mental simulation of expected outcomes. Moreover, speed of learning needs to be modulated according to the stability of the environment, such that the rate of learning increases with the volatility of the environment. Our previous studies have identified neural signals important for optimizing the learning rate (Massi et al., 2018) and switching between different learning algorithms (Seo et al., 2014).

Temporal and Numerical Cognition. For many behaviors, accurately estimating the timing of important events in the environment is critical for their success. In addition, evaluating and processing the quantities and magnitudes of behaviorally important variables, such as the amount of food or the number of preys or predators, influences the quality of behavioral outcomes. We study how the brain handles multiple concurrent temporal intervals between behaviorally significant events (Kleinman et al., 2016) as well how simple arithmetic operations are implemented in the brain (Massi et al., in preparation).