Stochastic Systems & Learning Laboratory (S2L2)


Research Statement

The main activities of the research lab are Stochastic Systems, Stochastic Optimization, Reinforcement Learning, Statistical Learning, Queueing Theory, Game Theory and Power System Economics. The application domains currently of interest are: Energy/Power systems, Healthcare operations, Transportation and Communication networks and systems.

My interests in Stochastic Systems span stochastic control theory, approximate dynamic programming and reinforcement learning. My group has developed, and is still developing `Empirical dynamic programming’ (EDP), or dynamic programming by simulation. This seems to be a very useful alternative to reinforcement learning algorithms.

My interests in Stochastic Optimization span primarily Risk-aware Stochastic Optimization. We have developed a mathematical framework and techniques that are remarkably effective in solving what are otherwise seemingly very difficult problems in risk-aware optimization.

My interests in Statistical Learning span PAC learning, online learning in multi-armed bandit models, Reinforcement Learning. I developed a PAC Theory for Markov Decision Processes. Recent work has included online decentralized learning algorithms for multi-player multi-armed bandit models.

My work on Queueing Theory has included developing non-classical models called `transitory queues’ and their theory via fluid and diffusion limits. These are relevant in `transitory’ situations wherein either only a finite population of users arrive, or the queues exist only for finite time.

My work on Game Theory has primarily revolved around Network Market Design, wherein we have developed theory and mechanisms for resource allocation in network and combinatorial settings. These are very relevant for bandwidth and spectrum allocation via auctions.

My recent interests include Power System Economics wherein I have been working on analysis of electricity markets, as well as pricing algorithms for demand response. I have also developed stochastic mechanism design for renewable energy integration.

I also have a burgeoning interest in Healthcare System Operations. This has revolved around Data Analytics for Healthcare, and Data-driven Operations Management and Scheduling.

Group Members and PhD Students

  1. 1.Hiteshi Sharma (PhD student, 5th year)

  2. 2.Mehdi Jafarnia (PhD Student, 4th year)

  3. 3.Nathan Dahlin (PhD Student , 4th year)

  4. 4.Krishna C. Kalagarla (PhD Student, 3rd year)

  5. 5.Mukul Gagrani (PhD Student, 5th year)

Group Alumni

  1. 1.Srinivas Yerramalli (*co-advised PhD, March 2013). Currently: QualComm

  2. 2.Arman Khouzani (Postdoc, 2013). Currently: Queen Mary Univ. of London (Lecturer)

  3. 3.William Haskell (Postdoc, June 2014). Currently: National Univ. of Singapore (TT-assistant prof.)

  4. 4.Harsha Honnappa (PhD, Dec. 2014). Currently: Purdue University (TT-assistant prof., Jan 2015)

  5. 5.Dileep Kalathil (PhD, Oct. 2014). Currently: Texas A&M (TT-assistant prof., Fall 2017)

  6. 6.Abhishek Gupta (Postdoc, 2014-15). Currently: Ohio State Univ. (TT-assistant prof., Fall 2015)

  7. 7.Wenyuan Tang (PhD, August 2015). Currently: North Carolina State (TT-assistant prof., Fall 2017)

  8. 8.Naumaan Nayyar (PhD, Sep 2015). Currently: Amazon. Previous: IBM Research and Vivace Systems.

  9. 9.Yi Ouyang (Postdoc 2016-17). Currently: Preferred Networks (Berkeley).

Research Support and Funded Projects

  1. 1.John H. Zumberge Faculty Research and Innovation Award, 2009-10: “The economics of Quality of Service on the Internet”.

  2. 2.NSF Network Science and Engineering (NetSE) Grant, 2009-13 (PI: Rahul Jain): “NetSE: Small; Cooperation and incentives in communication and social networks”.

  3. 3.NSF CAREER Award, 2010-2015 (PI: Rahul Jain): “CAREER: Network Economics: Theory and architectures for incentive-engineered networks”.

  4. 4.Air Force Office of Scientific Research (AFOSR) Grant, 2010-2013 (PI: Rahul Jain): “Distributed control and information fusion over communication networks”.

  1. 5.IBM Faculty Award, 2010: “Smarter Cities: Distributed optimization and control framework for Smart energy networks”.

  1. 6.Office of Naval Research (ONR) Young Investigator Award (2012-2015): “Stochastic dynamic optimization and games: Simulation and learning methods”.

  1. 7.National Science Foundation (2016-19): “Smarter Markets for a Smarter Grid: Pricing randomness, flexibility and risk”.