#+TITLE: A Collection of Literature on Probabilistic Graphical Models
#+LATEX_COMPILER: pdflatex
#+options: toc:nil
#+MACRO: color @@html:<font color="$1">$2</font>@@
#+OPTIONS: timestamp:nil

*The collection of literature work on Probabilistic Graphical Models (PGMs). Source file can be found at git repository [[https://github.com/FirstHandScientist/pgm_map][pgm-map]].*
# org-md-export-to-markdown

* Book and Monograph on PGMs

** Books / Monograph:
   
- Kingma and Welling, 2019, [[https://arxiv.org/abs/1906.02691][An Introduction to Variational Autoencoders]] 
- D. Barber, 2012, [[http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage][Bayesian Reasoning and Machine Learning]]
- Roger D. Peng, [[https://bookdown.org/rdpeng/advstatcomp/][Advanced Statistical Computing]] (in progress)
- Sutton, 2010, [[https://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf][An Introduction to Conditional Random Fields]]
- Wainwright, 2008, [[file:~/Documents/my_eBooks/mLearning/graphical_models_wainwright.pdf][Graphical Models, Exponential Families, and Variational Inference]]
- Koller, 2009, [[file:~/Documents/my_eBooks/mLearning/probabilistic_graphical_models_principles_techniques.pdf][Probabilistic graphical models: principles and techniques]]
- Mark Rowland, 2018, [[https://www.repository.cam.ac.uk/handle/1810/287479][Structure in Machine Learning: Graphical Models and Monte Carlo Methods]]
- Yingzhen Li, 2018, [[https://www.repository.cam.ac.uk/handle/1810/277549][Approximate Inference: New Visions]]
- Adrian Weller, 2014, [[http://mlg.eng.cam.ac.uk/adrian/phd_FINAL.pdf][Methods for Inference in Graphical Models]]
# Cached Region
- Angelino, et al 2016, [[https://www.nowpublishers.com/article/Details/MAL-052][Patterns of Scalable Bayesian Inference]]
- Komodakis etc, 2016, [[https://www.nowpublishers.com/article/Details/CGV-066][(Hyper)-Graphs Inference through Convex Relaxations and Move Making Algorithms: Contributions and Applications in Artificial Vision]]
- Bogdan Savchynskyy, 2019, [[file:~/Documents/my_eBooks/mLearning/discrete_graphical_models_an_optimization_perspective.pdf][Discrete Graphical Models -- An Optimization Perspective]]
- Angelino, 2016, [[https://www.nowpublishers.com/article/Details/MAL-052][Patterns of Scalable Bayesian Inference]]  
- Nowozin, 2011, [[http://www.nowozin.net/sebastian/papers/nowozin2011structured-tutorial.pdf][Structured Learning and Prediction in Computer Vision]]

- Dieng, Adji Bousso, 2020, [[https://academiccommons.columbia.edu/doi/10.7916/d8-rd60-nw75/download][Deep Probabilistic Graphical Modeling]]

- Lou, Qi, 2018, [[https://escholarship.org/uc/item/7sc0m97f][Anytime Approximate Inference in Graphical Models]]
- Ping, Wei, 2016, [[https://escholarship.org/uc/item/7q90z4b5][Learning and Inference in Latent Variable Graphical Models]]
- Forouzan, Sholeh, 2015, [[https://escholarship.org/uc/item/5n4733cz][Approximate Inference in Graphical Models]]
- Qiang, Liu, 2014, [[https://escholarship.org/uc/item/92p8w3xb][Reasoning and Decisions in Probabilistic Graphical Models - A Unified Framework]]

- Yuan Qi, 2005, [[https://affect.media.mit.edu/pdfs/05.qi-phd.pdf][Extending Expectation Propagation for Graphical Models]]
- Thomas P Minka, 2001, [[https://tminka.github.io/papers/ep/minka-thesis.pdf][A family of algorithms for approximate Bayesian inference]]
# David M. Blei
- Dieng, Adji Bousso, 2020, [[https://academiccommons.columbia.edu/doi/10.7916/d8-gt4e-6m45][Deep Probabilistic Graphical Modeling]]




* Inference and Learning of PGMs Papers

** Inference methods and techniques
*** Classical Inference Methods


- Lee et al, 2019, EMP, [[https://arxiv.org/abs/1907.01127][Convergence rates of smooth message passing with rounding in entropy-regularized MAP inference]]
- Knoll, et al, 2018, [[https://arxiv.org/abs/1605.06451][Fixed Points of Belief Propagation -- An Analysis via Polynomial Homotopy Continuation]]
- Cheng Zhag, et al, 2018, [[https://arxiv.org/abs/1711.05597][Advances in Variational Inference]]
- Peters, Janzing, Scholkopf, 2017, Elements of Causal Inference.
- Fletcher, 2017, [[https://arxiv.org/abs/1602.07795][Expectation Consistent Approximate Inference: Generalizations and Convergence]]
- Donoho, et al 2010, [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5503193][Message Passing Algorithms for Compressed Sensing: I. Motivation and Construction]]
- Donoho, et al 2010, [[https://ieeexplore.ieee.org/document/5503228][Message passing algorithms for compressed sensing: II. analysis and validation]]

- Convergence Analysis, Roosta, 2008, [[https://ieeexplore.ieee.org/document/4599175][Convergence Analysis of Reweighted Sum-Product Algorithms]]
- Generalized BP for marginal distributions, Yedidis, et al, 2005, [[https://www.cs.princeton.edu/courses/archive/spring06/cos598C/papers/YedidaFreemanWeiss2004.pdf][Constructing free energy approximations and Generalized belief propagation algorithms]]
- Tree-structured EP, Minka and Qi, [[https://tminka.github.io/papers/eptree/minka-eptree.pdf][Tree-structured approximations by expectation propagation]]
- Winn & Bishop, 2005, [[http://www.jmlr.org/papers/volume6/winn05a/winn05a.pdf][Variational message passing]]
- Welling, Minka, Teh, 2005, [[https://arxiv.org/abs/1207.1426][Structured Region Graphs: Morphing EP into GBP]]
- Max Welling, 2004, [[https://arxiv.org/pdf/1207.4158.pdf][On the Choice of Regions for Generalized Belief Propagation]]
- Opper, Winther, 2005, [[http://www.jmlr.org/papers/volume6/opper05a/opper05a.pdf][Expectation Consistent Approximate Inference]]
- Wainwright et al, 2003, [[http://ssg.mit.edu/group/willsky/publ_pdfs/166_pub_AISTATS.pdf][tree-reweighted belief propagation algorithms and approximated ML esimation by pseudo-moment matching]]

# MPA
- Wainwright and Willsky, 2003, [[https://papers.nips.cc/paper/2206-exact-map-estimates-by-hypertree-agreement.pdf][Exact MAP estimates by hypertree agreement]]
- Tourani et al, 2018, [[https://hci.iwr.uni-heidelberg.de/vislearn/HTML/people/bogdan/publications/papers/tourani-mplp-plus-plus-eccv2018.pdf][MPLP++: Fast, Parallel Dual Block-Coordinate Ascent for Dense Graphical Models]]
- Haller et al, 2018, [[https://arxiv.org/abs/2004.06370][Exact MAP-Inference by Confining Combinatorial Search with LP Relaxation]]
- Globerson, Jaakkola, 2008, [[https://papers.nips.cc/paper/3200-fixing-max-product-convergent-message-passing-algorithms-for-map-lp-relaxations.pdf][Fixing Max-Product: Convergent Message PassingAlgorithms for MAP LP-Relaxations]]

*** Improvements

- Conditioning, Clamping, Divide

  - Zhou et al, 2020, [[https://arxiv.org/abs/1910.13324][Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support]]
  - Eaton and Ghahramani, 2009, [[http://mlg.eng.cam.ac.uk/pub/pdf/EatGha09.pdf][Choosing a Variable to Clamp]]
  - Geier et al, 2015, [[http://auai.org/uai2015/proceedings/papers/158.pdf][Locally Conditioned Belief Propagation]]
  - Weller and Jebara, 2014, [[https://papers.nips.cc/paper/5529-clamping-variables-and-approximate-inference.pdf][Clamping Variables and Approximate Inference]]
  - Nate Derbinsky, José Bento, Veit Elser, Jonathan S. Yedidia, [[https://arxiv.org/abs/1305.1961][An Improved Three-Weight Message-Passing Algorithm]], [[http://people.csail.mit.edu/andyd/CIOG_slides/yedidia_talk_ciog2011.pdf][slide]]

- Linear Response. Welling and Teh, [[https://www.ics.uci.edu/~welling/publications/papers/LR2.pdf][Linear Response Algorithms for Approximate Inference in Graphical Models]]

- Combining with Particle/Stochastic Methods

  - Liu et al, 2015, [[https://papers.nips.cc/paper/5695-probabilistic-variational-bounds-for-graphical-models][Probabilistic Variational Bounds for Graphical Models]]
  - Noorshams and Wainwright, 2013, [[https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6373728][stochastic belief propagation: a low-complexity alternative to the sum-product algorithm]]
  - Lienart, et al, Expectation Particle Belief Propagation
  - Ihler, McAllester, 2009, [[http://proceedings.mlr.press/v5/ihler09a/ihler09a.pdf][Particle Belief Propagation]]
  - Sudderth, [[http://ssg.mit.edu/nbp/][Nonparametric Belief Propagation]]

- Mixture/multi-modal
  - Baque et al, 2017, [[http://openaccess.thecvf.com/content_cvpr_2017/papers/Baque_Multi-Modal_Mean-Fields_via_CVPR_2017_paper.pdf][Multi-Modal Mean-Fields via Cardinality-Based Clamping]]
  - Hao Xiong et al, 2019, [[http://auai.org/uai2019/proceedings/papers/19.pdf][One-Shot Marginal MAP Inference in Markov Random Fields]]

- Layered messages
  - Jampani et al, 2015, [[http://proceedings.mlr.press/v38/jampani15.pdf][Consensus Message Passing for Layered Graphical Models]]

- Patrick Eschenfeldt, Dan Schmidt, Stark Draper, Jonathan Yedidia, 2016, [[https://arxiv.org/abs/1601.04667][Patrick Eschenfeldt, Dan Schmidt, Stark Draper, Jonathan Yedidia]]

*** Application
- [[https://papers.nips.cc/paper/9532-combining-generative-and-discriminative-models-for-hybrid-inference.pdf][Satorras, 2019, Combining Generative and Discriminative Models for Hybrid Inference]]
- [[https://arxiv.org/pdf/1502.03240.pdf][Zheng, 2019, Conditional Random Fields as Recurrent Neural Networks]]
- [[https://arxiv.org/abs/1210.5644][Krahenbuhl, 2011, Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials]]



*** Variational methods   
    
- NIPS tutorial 2016, [[https://media.nips.cc/Conferences/2016/Slides/6199-Slides.pdf][Variational Inference]]
- Kingma and Welling, 2014, Autoencoder: [[https://arxiv.org/abs/1312.6114][Auto-Encoding Variational Bayes]]  
- Kuleshov and Ermon, 2017, NVIL: [[https://arxiv.org/abs/1711.02679][Neural Variational Inference and Learning in Undirected Graphical Models]]
- Li, etc, 2020, AdVIL: [[https://arxiv.org/abs/1901.08400][To Relieve Your Headache of Training an MRF, Take AdVIL]]
- Lazaro-Gredilla, 2019 (Vicarious AI), [[https://arxiv.org/abs/1912.02893][Learning undirected models via query training]]
- Sobolev and Vetrov, 2019, (Section 3 gives interesting discussion on literature works) [[http://papers.nips.cc/paper/8350-importance-weighted-hierarchical-variational-inference][Importance Weighted Hierarchical Variational Inference]]
- Kingma, et al, 2016, [[https://papers.nips.cc/paper/6581-improved-variational-inference-with-inverse-autoregressive-flow][Improved Variational Inference with Inverse Autoregressive Flow]]  
- Rezende, Mohamed, 2015, [[https://arxiv.org/abs/1505.05770][Variational Inference with Normalizing Flows]]

- Domke, 2019, [[https://arxiv.org/abs/1901.08431][Provable Smoothness Guarantees for Black-Box Variational Inference]]
- Zhang, et al, 2018, [[https://arxiv.org/pdf/1711.05597.pdf][Advances in Variational Inference]]
- Blei, 2017, [[https://amstat.tandfonline.com/doi/pdf/10.1080/01621459.2017.1285773?needAccess=true][Variational Inference: A Review for Statisticians]]
- Regier et al, 2017, [[https://papers.nips.cc/paper/6834-fast-black-box-variational-inference-through-stochastic-trust-region-optimization.pdf][Fast Black-box Variational Inferencethrough Stochastic Trust-Region Optimization]]
- Kucukelbir et al, 2016, [[https://arxiv.org/pdf/1603.00788.pdf][Automatic differentiation variational inference]]
- Black-box alpha, 2016, [[http://proceedings.mlr.press/v48/hernandez-lobatob16.pdf][Black-box alpha-divergence minimization]]
- Ranganath et al, 2014, [[http://proceedings.mlr.press/v33/ranganath14.pdf][Black box variational inference]]

** Neural network based methods
*** Deep learning based methods

- Stoller et al, 2020, [[https://arxiv.org/pdf/1905.12660.pdf][Training Generative Adversarial Networks from Incomplete Observations using Factorised Discriminators]]
- Karaletsos, 2016, [[https://arxiv.org/abs/1612.05048][Adversarial Message Passing For Graphical Models]]
- Yiming Yan et al, 2019, [[https://arxiv.org/abs/1906.02428][Amortized Inference of Variational Bounds for Learning Noisy-OR]]

Learning messages

- Heess et al, [[https://papers.nips.cc/paper/5070-learning-to-pass-expectation-propagation-messages.pdf][Learning to Pass Expectation Propagation Messages]], half-automated message passing, message-level automation
- Kuck et al 2020, [[https://arxiv.org/pdf/2007.00295.pdf][Belief Propagation Neural Networks]]
- Victor Garcia Satorras, Max Welling, 2020 [[https://arxiv.org/abs/2003.01998][Neural Enhanced Belief Propagation on Factor Graphs]]
- Yoon et al, 2018, [[https://arxiv.org/abs/1803.07710][Inference in Probabilistic Graphical Models by Graph Neural Networks]]    
- Lin, 2015, [[http://papers.nips.cc/paper/5791-deeply-learning-the-messages-in-message-passing-inference.pdf][Deeply Learning the Messages in Message Passing Inference]]

Graphical Neural Networks

- [[https://arxiv.org/abs/1905.06214][GMNN: Graph Markov Neural Networks]], semi-supervised learning, EM is used for training.
- More generalized computation power: [[https://github.com/deepmind/graph_nets][Graph Net Library]], A graph network takes a graph as input and returns a graph as output.
- Related, [[https://github.com/dmlc/dgl][Deep Graph Library]], for deep learning on graphs
- Scarselli et al, 2009, [[https://persagen.com/files/misc/scarselli2009graph.pdf][The graph neural network model]]
- Satorras and Welling, 2020, [[https://arxiv.org/abs/2003.01998][Neural Enhanced Belief Propagation on Factor Graphs]]    


*** Neural density function estimation
- Chen et al, 2018, ODE: [[https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations][Neural Ordinary Differential Equations]]
- Kingma, Dhariwal, 2018, [[https://arxiv.org/abs/1807.03039][Glow: Generative Flow with Invertible 1x1 Convolutions]]  
- Dinh, Sohl-Dickstein, Bengio, 2017, [[https://arxiv.org/pdf/1605.08803.pdf][Density Estimation using Real NVP]]
- Dinh, Krueger, Bengio, 2014, [[https://arxiv.org/abs/1410.8516][NICE: Non-linear independent component estimation]]  
- Tran, 2019, [[http://papers.nips.cc/paper/9612-discrete-flows-invertible-generative-models-of-discrete-data.pdf][Discrete flows: Invertible generative models of discrete data]]
- Inverse autoregreeeive flow as in previous subsection.
    

** Learning of Graphical Models

*** Parameter Learning

Alternative objective
- Note, [[http://people.csail.mit.edu/dsontag/courses/pgm12/slides/pseudolikelihood_notes.pdf][Maximum Pseudolikelihood Learning]]
- Domke, 2013, [[https://ieeexplore.ieee.org/abstract/document/6420841][Learning Graphical Model Parameters with Approximate Marginal Inference]]

Learning graphical model parameters by approximate inference

- Tang, 2015, [[https://arxiv.org/abs/1503.01228][Bethe Learning of Conditional Random Fields via MAP Decoding]]
- You Lu, 2019, [[https://www.aaai.org/ojs/index.php/AAAI/article/view/4357][Block Belief Propagation for Parameter Learning in Markov Random Fields]]
- Hazan, 2016, [[http://www.jmlr.org/papers/v17/13-260.html][Blending Learning and Inference in Conditional Random Fields]]
- Tang, etc, 2016, [[http://proceedings.mlr.press/v51/tang16a.pdf][Bethe Learning of Graphical Models via MAP Decoding]]
- Ping and Ihler, 2017, [[http://proceedings.mlr.press/v54/ping17a/ping17a.pdf][Belief Propagation in Conditional RBMs for Structured Prediction]]
- Ping, et al, 2014, [[http://proceedings.mlr.press/v32/ping14.pdf][Marginal Structured SVM with Hidden Variables]]

Learning of MRF with neural networks

- Wiseman and Kim, 2019, [[https://papers.nips.cc/paper/9687-amortized-bethe-free-energy-minimization-for-learning-mrfs.pdf][Amortized Bethe Free Energy Minimization for Learning MRFs]]
- Kuleshov and Ermon, 2017, [[https://arxiv.org/abs/1711.02679][Neural Variational Inference and Learning in Undirected Graphical Models]]
- Lazaro-Gredilla et al, 2020, [[https://arxiv.org/abs/2006.06803][Query Training: Learning and inference for directed and undirected graphical models]]

Learning of Directed Graphs

- Chongxuan Li, 2020, [[https://arxiv.org/abs/1901.08400][To Relieve Your Headache of Training an MRF, Take AdVIL]]
- Mnih and Gregor, 2014, [[https://arxiv.org/abs/1402.0030][Neural Variational Inference and Learning in Belief Networks]]
- NIPS tutorial 2016, [[https://media.nips.cc/Conferences/2016/Slides/6199-Slides.pdf][Variational Inference]]

* course materials on pgm
- [[http://www.cs.columbia.edu/~blei/fogm/2020F/index.html][Foundations of Graphical Models]]
- [[https://sailinglab.github.io/pgm-spring-2019/][Probabilistic Graphical Models]]

* PGM, Logic & Decision-making in Dynamic Systems
** Dynamics
+ Kim, Ahn, Bengio, 2019, [[https://arxiv.org/pdf/1910.00775.pdf][Variational Temporal Abstraction]]
+ Yulia Rubanova et al 2019, [[https://arxiv.org/abs/1907.03907][Latent ODEs for Irregularly-Sampled Time Series]]
+ Linderman et al, 2017, [[http://proceedings.mlr.press/v54/linderman17a/linderman17a.pdf][Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems]]
+ Niall Twomey, Michal Kozlowski, Raul Santos-Rodriguez, 2020, [[http://ecai2020.eu/papers/736_paper.pdf][Neural ODEs with stochastic vector field mixtures]]
+ Broderick, T. 2014, [[https://escholarship.org/content/qt9s76h6kh/qt9s76h6kh_noSplash_ae487ff77e18b03b243557a35e50f4a5.pdf][Clusters and features from combinatorial stochastic processes]]
+ VAswani, et al, 2014, [[https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf][Attention Is All You Need]]
+ Bahdanau, et al, 2014, [[https://arxiv.org/abs/1409.0473][Neural Machine Translation by Jointly Learning to Align and Translate]]  
** Logic 
- [[https://dtai.cs.kuleuven.be/problog/index.html][ProbLog]]
  + D. Fierens, G. Van den Broeck, 2015. Inference and learning in probabilistic logic programs using weighted Boolean formulas.   
  + L. De Raedt, A. Kimmig and H. Toivonen, 2017. ProbLog: A probabilistic Prolog and its application in link discovery.

- [[http://starai.cs.ucla.edu/slides/CS201.pdf][Probabilistic Circuit]]
  + Yitao Liang, Guy Van den Broeck, [[https://arxiv.org/abs/1902.10798][Learning Logistic Circuits]]


** Decision-making
+ Sutton, Barto, 2018, [[https://github.com/FirstHandScientist/Reinforcement-Learning-2nd-Edition-by-Sutton-Exercise-Solutions][Reinforcement learning (2ed Edition)]]

+ Martin L. Puterman, 2014, Markov Decision Processes: Discrete Stochastic Dynamic Programming

+ Francois-Lavet, et al 2018, [[https://arxiv.org/abs/1811.12560][An Introduction to Deep Reinforcement Learning]] 

+ Bubeck, Cesa-Bianchi, 2012, [[https://www.microsoft.com/en-us/research/wp-content/uploads/2017/01/SurveyBCB12.pdf][Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems]] 

+ Ziebart, 2010, [[https://www.cs.cmu.edu/~bziebart/publications/thesis-bziebart.pdf][Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy]]

+ Levin, 2018, [[https://arxiv.org/abs/1805.00909][Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review]]

+ Haarnoja, et al 2017, [[https://arxiv.org/pdf/1702.08165.pdf][Reinforcement Learning with Deep Energy-Based Policies]]
  
+ Szepesvari, 2009, [[https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs-lecture.pdf][Algorithms for Reinforcement Learning]]


** Courses

- [[https://www.davidsilver.uk/teaching/][Reinforcement Learning (UCL)]]
- [[http://rail.eecs.berkeley.edu/deeprlcourse/][Deep Reinforcement Learning (CS285)]]
- [[https://www.youtube.com/playlist?list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs][Advanced Deep Learning & Reinforcement Learning]]

** Platform     
   + [[http://deepdive.stanford.edu/#documentation][DeepDive]]

* In Connecting with Others
** Causality
   - Judea Pearl, Causality: Models, Reasoning and Inference
   - [[https://github.com/DataForScience/Causality][Causality Tutorial Notebooks]]

** [[https://github.com/arranger1044/awesome-spn][Awesome Sum-Product Networks]]

** [[http://starai.cs.ucla.edu/code/][StarAI coll.]]

** Repos on Variational Inference
- Repos: [[https://github.com/otokonoko8/implicit-variational-inference][Advanced-variational-inference-paper]]
- Repos: [[https://github.com/otokonoko8/deep-Bayesian-nonparametrics-papers][Deep-Bayesian-nonparametrics-papers]] 


  
** GANs

+ Literature collection: [[https://github.com/hindupuravinash/the-gan-zoo][GAN-zoo]]
+ Repos: [[https://github.com/znxlwm/pytorch-generative-model-collections][Generative adversarial networks]]


# ** Discrete GAN or RBM or Autoencoder

** Optimal Transport (likelihood-free learning)

- Matthed Thorpe, 2018, [[http://www.math.cmu.edu/~mthorpe/OTNotes][Introduction to Optimal Transport]]
- Peyre, Cuturi, 2018, Computational Optimal Transport, [[https://optimaltransport.github.io/resources/][Codes and slides for OT]]