Conclusions and Resources

This article demonstrated the use of Bayesian methods to solve parameter estimation problems. I discussed several important concepts relevant to using Bayesian methods to solve parameter estimation problems, including maximum likelihood estimators, binomial random variables, the Bernoulli process, the beta distribution, and conjugate priors. I hope the discussion provided you with a better general understanding of Bayesian inference techniques and also helped you to understand some concepts that play important roles in statistics and probability.

I highlighted the important role played by probability distributions in representing the likelihood and prior terms in Bayes theorem. Recall that in the first article of this series, I computed the posterior distribution without resorting to the use of theoretical probability distributions. You can conclude from this that full mastery of Bayes methods involves learning a more theoretically-oriented probability distribution approach to Bayesian inference along with a more empirically-oriented joint frequency approach.

In my next article, I will move from analyzing simple binary surveys to the concepts and techniques that are useful for analyzing simple binary classification surveys and multivariate classification surveys. In doing this, I will examine another classic inference problem that Bayes methods are particularly good at solving - - classification problems.

Resources

• Download the source code used in this article. Updates to article code will be made available at PHPMath.com.
  
  
• Go deeper into Bayes parameter estimation in these lecture notes on Bayes Networks by James Cussens.
  
  
• Learn probability concepts by exploring the Virtual Laboratories in Probability and Statistics.
  
  
• Visit Radford Neal's site for research on Bayesian neural networks and essays on the philosophy of Bayesian inference.
  
  
• Check out the Jsci Project, which aims is to encapsulate scientific methods/principles in the most natural way possible using Java. It provided the probability distributions code used in this article.
  
  
• Try The BUGS Project (Bayesian inference Using Gibbs Sampling), a piece of computer software for the Bayesian analysis of complex statistical models using Markov chain Monte Carlo (MCMC) methods.
  
  
• Read the other articles in the author's series on Bayesian inference:
  • "Implement Bayesian inference using PHP, Part 1" implements the underlying conditional probability calculations using PHP.
  • "Implement Bayesian inference using PHP, Part 3" solves classification problems in medical diagnostic testing and Web survey analysis as it applies Bayesian and conditional probability concepts to both building classifier systems and analyzing the accuracy of their output.
  
  
• Learn to craft Web data-gathering applications in "Apply probability models to Web data using PHP".
  
  
• Explore a Bayesian method for detecting structural changes in a long-range dependent process in "Bayesian Methods for Change-point Detection in Long-range Dependent Processes" (IBM Research, April 2001).
  
  
• For the PHP developer, find out how to write more efficient code "Writing Efficient PHP" (developerWorks, July 2002).
  
  
• Learn how to construct a user-modeling platform with PHP "Web site user modeling with PHP" (developerWorks, December 2003).
  
  
• Read Statistics: Probability, Inference, and Decision, 2nd ed. (International Thomson Publishing; 1975), by Winkler and Hayes, a source that the author relied on for this article.
  
  
• Check out Artificial Intelligence: A Modern Approach (Prentice Hall; 2003), by Russell and Norvig, for an excellent discussion of Bayes inference methods.
  
  
• Browse the developerWorks bookstore for titles on this and other related subjects.
  
  
• Visit developerWorks Web Architecture zone for a range of articles on the topic of Web architecture and usability.