Chapter 12 Probability Minimum

12.1 probability

12.1.1 Conditional Independence

Below are commonly used rules of conditional independence.

Symmetry:
\[ X \perp \! \! \! \! \perp Y \implies Y \perp \! \! \! \! \perp X. \]

Decomposition: \[ X \perp \! \! \! \! \perp A,B \implies X \perp \! \! \! \! \perp A \text{ and } X \perp \! \! \! \! \perp B \]

Weak union: \[ X \perp \! \! \! \! \perp A,B \implies X \perp \! \! \! \! \perp A | B \text{ and } X \perp \! \! \! \! \perp B | A \]

Contraction: \[ X \perp \! \! \! \! \perp A|B \text{ and } X \perp \! \! \! \! \perp B \iff X \perp \! \! \! \! \perp A, B \]

Intersection: \[ X \perp \! \! \! \! \perp A|C, B \text{ and } X \perp \! \! \! \! \perp B|C, A \implies X \perp \! \! \! \! \perp A, B | C \]

Agresti, Alan, and Brent A Coull. 1998. “Approximate Is Better Than ‘Exact’ for Interval Estimation of Binomial Proportions.” The American Statistician 52 (2): 119–26.

Asmussen, Soren, and Peter Glynn. 2008. Stochastic Simulation. Springer-Verlag.

Athey, Susan, Guido W Imbens, and Stefan Wager. 2018. “Approximate Residual Balancing: Debiased Inference of Average Treatment Effects in High Dimensions.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 80 (4): 597–623.

Bakshy, Eytan, Dean Eckles, and Michael S Bernstein. 2014. “Designing and Deploying Online Field Experiments.” In Proceedings of the 23rd International Conference on World Wide Web, edited by acm, 283–92. ACM.

Barber, David. 2012. Bayesian Reasoning and Machine Learning. Cambridge University Press.

Barber, Rina Foygel, and Emmanuel J Candes. 2016. “A Knockoff Filter for High-Dimensional Selective Inference.” arXiv Preprint arXiv:1602.03574.

Bartroff, Jay, Tze Leung Lai, and Mei-Chiung Shih. 2012. Sequential Experimentation in Clinical Trials: Design and Analysis. Vol. 298. Springer Science & Business Media.

Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2014. “Fitting Linear Mixed-Effects Models Using Lme4.” arXiv Preprint arXiv:1406.5823.

Benjamin, Daniel J, James O Berger, Magnus Johannesson, Brian A Nosek, E-J Wagenmakers, Richard Berk, Kenneth A Bollen, et al. 2018. “Redefine Statistical Significance.” Nature Human Behaviour 2 (1): 6.

Benjamini, Yoav, and Yosef Hochberg. 1995. “Controlling the false discovery rate: a practical and powerful approach to multiple testing.” J. R. Stat. Soc. Ser. B, 289–300.

Boos, Dennis D, and Jacqueline M Hughes-Oliver. 2000. “How Large Does n Have to Be for z and t Intervals?” The American Statistician 54 (2): 121–28.

Brown, Morton B, and Robert A Wolfe. 1983. “Estimation of the Variance of Percentile Estimates.” Computational Statistics & Data Analysis 1: 167–74.

Carpenter, Bob, Andrew Gelman, Matt Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Michael A Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. 2016. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 20: 1–37.

Casella, George, and Roger L Berger. 2002. Statistical Inference, Second Edition. Duxbury Press: Pacific Grove, CA.

Chamandy, Nicholas, Omkar Muralidharan, and Stefan Wager. 2015. “Teaching Statistics at Google-Scale.” The American Statistician 69 (4): 283–91.

Coey, Dominic, and Michael Bailey. 2016. “People and Cookies: Imperfect Treatment Assignment in Online Experiments.” In Proceedings of the 25th International Conference on World Wide Web, 1103–11. International World Wide Web Conferences Steering Committee.

Davidian, M, Anastasios A. Tsiatis, and S Leon. 2005. “Semiparametric Estimation of Treatment Effect in a Pretest-Posttest Study with Missing Data.” Stat. Sci. 20 (3): 295–301.

Deng, A., and X. Shi. 2016. “Data-Driven Metric Development for Online Controlled Experiments: Seven Lessons Learned.” In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.

Deng, Alex, Ulf Knoblich, and Jiannan Lu. 2018. “Applying the Delta Method in Metric Analytics: A Practical Guide with Novel Ideas.” In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 233–42. KDD ’18. New York, NY, USA: ACM. https://doi.org/10.1145/3219819.3219919.

Deng, Alex, Jiannan Lu, and Shouyuan Chen. 2016. “Continuous Monitoring of a/b Tests Without Pain: Optional Stopping in Bayesian Testing.” In Data Science and Advanced Analytics (DSAA), 2016 IEEE International Conference on, 243–52. IEEE.

Deng, Alex, Jiannan Lu, and Jonthan Litz. 2017. “Trustworthy Analysis of Online a/b Tests: Pitfalls, Challenges and Solutions.” In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, 641–49. WSDM ’17. New York, NY, USA: ACM. https://doi.org/10.1145/3018661.3018677.

Deng, Alex, Ya Xu, Ron Kohavi, and Toby Walker. 2013. “Improving the sensitivity of online controlled experiments by utilizing pre-experiment data.” In Proc. 6th ACM Int. Conf. Web Search Data Min., edited by acm, 123–32. ACM.

Deng, Shaojie, Roger Longbotham, Toby Walker, and Ya Xu. 2011. “Choice of the Randomization Unit in Online Controlled Experiment.” JSM Proc.

Donner, Allan. 1987. “Statistical Methodology for Paired Cluster Designs.” American Journal of Epidemiology 126 (5): 972–79.

Efron, Bradley. 2011. “Tweedie’s formula and selection bias.” J. Am. Stat. Assoc. 106 (496): 1602–14.

Fisher, Ronald. 1958. “Cigarettes, Cancer, and Statistics.” The Centennial Review of Arts & Science 2: 151–66.

Fisher, Sir Ronald A, and EA Cornish. 1960. “The Percentile Points of Distributions Having Known Cumulants.” Technometrics 2 (2): 209–25.

Fithian, William, Dennis Sun, and Jonathan Taylor. 2014. “Optimal Inference After Model Selection.” arXiv Preprint arXiv:1410.2597.

Freedman, David A. 2008. “On regression adjustments to experimental data.” Adv. Appl. Math. 40 (2): 180–93.

Gelman, Andrew, and John Carlin. 2014. “Beyond Power Calculations: Assessing Type s (Sign) and Type m (Magnitude) Errors.” Perspectives on Psychological Science 9 (6): 641–51.

Gelman, Andrew, and Jennifer Hill. 2006. Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.

Gomez-Uribe, Carlos A, and Neil Hunt. 2016. “The Netflix Recommender System: Algorithms, Business Value, and Innovation.” ACM Transactions on Management Information Systems (TMIS) 6 (4): 13.

Goodman, Steven. 2008. “A Dirty Dozen: Twelve p-Value Misconceptions.” In Seminars in Hematology, 45:135–40. 3. Elsevier.

Hainmueller, Jens. 2012. “Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies.” Political Analysis 20 (1): 25–46.

Hall, Peter. 2013. The Bootstrap and Edgeworth Expansion. Springer Science & Business Media.

Hoenig, John M, and Dennis M Heisey. 2001. “The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis.” The American Statistician 55 (1): 19–24.

Hohnhold, Henning, Deirdre O’Brien, and Diane Tang. 2015. “Focus on the Long-Term: It’s Better for Users and Business.” In Proceedings 21st Conference on Knowledge Discovery and Data Mining. Sydney, Australia. http://dl.acm.org/citation.cfm?doid=2783258.2788583.

Imai, Kosuke, and Marc Ratkovic. 2014. “Covariate Balancing Propensity Score.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 76 (1): 243–63.

Imbens, Guido W, and Donald B Rubin. 2015. Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press.

Johari, Ramesh, Pete Koomen, Leonid Pekelis, and David Walsh. 2017. “Peeking at a/b Tests: Why It Matters, and What to Do about It.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1517–25. ACM.

Jordan, Michael I. 2004. “Graphical Models.” Statistical Science, 140–55.

Kang, Joseph DY, and Joseph L Schafer. 2007. “Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data.” Statistical Science, 523–39.

Klar, Neil, and Allan Donner. 2001. “Current and Future Challenges in the Design and Analysis of Cluster Randomization Trials.” Statistics in Medicine 20 (24): 3729–40.

Kleiner, Ariel, Ameet Talwalkar, Purnamrita Sarkar, and Michael I Jordan. 2014. “A scalable bootstrap for massive data.” J. R. Stat. Soc. Ser. B (Statistical Methodol.

Kohavi, Ron, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, and Ya Xu. 2012. “Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained.” Proc. 18th Conf. Knowl. Discov. Data Min.

Kohavi, Ron, Alex Deng, Brian Frasca, Ya Xu, Toby Walker, and Nils Pohlmann. 2013. “Online Controlled Experiments at Large Scale.” Proc.19th Conf. Knowl. Discov. Data Min.

Kohavi, Ron, Alex Deng, Roger Longbotham, and Ya Xu. 2014. “Seven rules of thumb for web site experimenters.” In Proc. 20th Conf. Knowl. Discov. Data Min., edited by acm, 1857–66. KDD ’14. New York, USA.

Kohavi, Ron, and Roger Longbotham. 2015. “Online Controlled Experiments and a/b Tests.” Encyclopedia of Machine Learning and Data Mining.

Kohavi, Ron, Roger Longbotham, Dan Sommerfield, and Randal M Henne. 2009. “Controlled Experiments on the Web: survey and practical guide.” Data Min. Knowl. Discov. 18: 140–81.

Kohavi, Ron, Roger Longbotham, and Toby Walker. 2010. “Online experiments: Practical lessons.” Computer (Long. Beach. Calif). 43 (9): 82–85.

Kohavi, Ron, David Messner, Seth Eliot, Juan Lavista Ferres, Randy Henne, Vignesh Kannappan, and Justin Wang. 2010. “Tracking Users’ Clicks and Submits: Tradeoffs Between User Experience and Data Loss.” Redmond: Sn.

Krewski, Daniel. 1976. “Distribution-Free Confidence Intervals for Quantile Intervals.” Journal of the American Statistical Association 71 (354): 420–22.

Lauritzen, Steffen L. 1996. Graphical Models. Vol. 17. Clarendon Press.

Li, Fan, Kari Lock Morgan, and Alan M Zaslavsky. 2018. “Balancing Covariates via Propensity Score Weighting.” Journal of the American Statistical Association 113 (521): 390–400.

Lin, Winston. 2013. “Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique.” Ann. Appl. Stat. 7 (1): 295–318.

Lu, Jiannan, and Alex Deng. 2016. “Demystifying the Bias from Selective Inference: A Revisit to Dawid’s Treatment Selection Problem.” ArXiv Pre-Print. http://arxiv.org/abs/1601.05835.

Lu, Jiannan, Yixuan Qiu, and Alex Deng. 2018. “A Note on Type s/m Errors in Hypothesis Testing.” British Journal of Mathematical and Statistical Psychology.

Meyer, John S. 1987. “Outer and Inner Confidence Intervals for Finite Population Quantile Intervals.” Journal of the American Statistical Association 82 (397): 201–4.

Miratrix, Luke W, Jasjeet S Sekhon, and Bin Yu. 2013. “Adjusting treatment effect estimates by post-stratification in randomized experiments.” J. R. Stat. Soc. Ser. B (Statistical Methodol. 75 (2): 369–96.

Morgan, Stephen L, and Christopher Winship. 2014. Counterfactuals and Causal Inference. Cambridge University Press.

Murphy, Kevin P. 2012. Machine learning: a probabilistic perspective. MIT press.

Pearl, Judea. 1995. “Causal Diagrams for Empirical Research.” Biometrika 82 (4): 669–88.

———. 2009. “Causal inference in statistics: An overview.” Stat. Surv. 3: 96–146.

Regulation, General Data Protection. 2016. “Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46.” Official Journal of the European Union (OJ) 59 (1-88): 294.

Ries, Eric. 2011. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Books.

Robins, James M, and Andrea Rotnitzky. 1995. “Semiparametric Efficiency in Multivariate Regression Models with Missing Data.” Journal of the American Statistical Association 90 (429): 122–29.

Senn, S. 2008. “A note concerning a selection "paradox" of Dawid’s.” Am. Stat. Assoc. 62 (3): 206–10. https://doi.org/10.1198/000313008X331530.

Shpitser, Ilya, and Judea Pearl. 2012. “Identification of Conditional Interventional Distributions.” arXiv Preprint arXiv:1206.6876.

Tang, Diane, Ashish Agarwal, Deirdre O’Brien, and Mike Meyer. 2010. “Overlapping Experiment Infrastructure: More, Better, Faster Experimentation.” Proc. 16th Conf. Knowl. Discov. Data Min.

Taylor, Jonathan, and Robert Tibshirani. 2018. “Post-Selection Inference for-Penalized Likelihood Models.” Canadian Journal of Statistics 46 (1): 41–61.

Taylor, Jonathan, and Robert J Tibshirani. 2015. “Statistical Learning and Selective Inference.” Proceedings of the National Academy of Sciences 112 (25): 7629–34.

Tsiatis, Anastasios A. 2006. Semiparametric Theory and Missing Data. Springer-Verlag.

Tsiatis, Anastasios A., Marie Davidian, Min Zhang, and Xiaomin Lu. 2008. “Covariate adjustment for two-sample treatment comparisons in randomized clinical trials: A principled yet flexible approach.” Stat. Med. 27.

Van der Vaart, Aad W. 2000. Asymptotic statistics. Vol. 3. Cambridge university press.

Wager, Stefan, and Susan Athey. 2017. “Estimation and Inference of Heterogeneous Treatment Effects Using Random Forests.” Journal of the American Statistical Association, no. just-accepted.

Wasserman, Larry. 2003. All of Statistics: A Concise Course in Statistical Inference. Springer.

Xu, Ya, Nanyu Chen, Addrian Fernandez, Omar Sinno, and Anmol Bhasin. 2015. “From Infrastructure to Culture: A/b Testing Challenges in Large Scale Social Networks.” In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2227–36. KDD ’15. New York, NY, USA: ACM. https://doi.org/10.1145/2783258.2788602.

Yang, Li, and Anastasios A. Tsiatis. 2001. “Efficiency Study of Estimators for a Treatment Effect in a Pretest-Posttest Trial.” Am. Stat. 55 (4): 314–21.