Category Archives: Machine Learning

cvxpy, cvxopt and convex optimization

cvxpy module is a nice wrapper around cvxopt that follows paradigm of a disciplined convex programming. Its a bit slower, but has a very similar syntax as cvx toolbox for Matlab (cvx in Matlab is great!). Here are some notes: … Continue reading

Posted in convex optimization, Machine Learning | Leave a comment

Installing Scipy on Linux Red Hat

Tried to install scipy using easy_install (also pip). Installation failed with error: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Instead: sudo yum install scipy works seamlessly

Posted in EC2, Machine Learning, Programming, Python | Leave a comment

Meaning of entropy, Kullback–Leibler distance and mutual information

Recently I came across a nice read about entropy by Cover and Thomas. I want to summarize what entropy ‘physically’ means from information coding perspective. The definition of entropy for a discretely valued random variable is: Entropy does not depend … Continue reading

Posted in Machine Learning, Math, Probability Theory | Tagged , , , , | Leave a comment

A good pipeline engineering in machine learning

The following is taken from Engineering Practices in Data Science. Machine learning project must have the following well defined (and followed!) phases: data ingestion -> feature extraction -> training -> ensembling -> validation “This allows disciplined data scientists to try … Continue reading

Posted in Machine Learning | Leave a comment