Category Archives: Probability Theory

Interview question: what is the expected number of occurrences of a “HH” pattern in n tosses of an unbiased coin?

Problem: You toss a fair coin times. What is the expected number of occurrences of HH pattern? Math solution. Let to denote the expectation we are looking for. Let to denote the conditional expectation given that the first toss is … Continue reading

Posted in Interview, Math, Probability Theory | Tagged , , | Leave a comment

Meaning of entropy, Kullback–Leibler distance and mutual information

Recently I came across a nice read about entropy by Cover and Thomas. I want to summarize what entropy ‘physically’ means from information coding perspective. The definition of entropy for a discretely valued random variable is: Entropy does not depend … Continue reading

Posted in Machine Learning, Math, Probability Theory | Tagged , , , , | Leave a comment