Here's a list of books I've found very useful in data analysis. One definite omission in this list is a good book on Markov Chain Monte Carlo algorithms, if anyone has any suggestions please post. My ratings for each in terms of introductory, mid, & advanced are mostly on the basis of the mathematical background required to read.
All of statistics - Larry Wasserman. The basis for machine learning & data analysis is statistics. Computers have helped us scale to data sets that were previously inaccessible, but the foundation of statistics remains. This book does a very good bridging statistics and machine learning. It covers enough statistics for you to be able to better analyse your data sets and a variety of machine learning for you to implement. Introductory level.
Understanding Machine Learning - Shalev-Shwartz & Ben-David. This is a very good book on the field, it's much shorter than elements of statistical learning, but does a better job covering the intuition and underlying ideas. Additionally it covers some very important topics like convex optimization and stochastic gradient descent that ignored in many other books that only list machine learning models. It's probably halfway between theory and application. It's definitely the best source for anyone interested in a deeper understanding of Machine Learning. Introductory / Mid Level
The Elements of Statistical Learning - Hastie, Tibshirani, & Friedman. For a long time this was the book in the field. At 700 pages it covers almost all the algorithms in the field. It is mainly concerned with implementation and doesn't approach learning theory, but is a great resource and reference. Introductory level.
http://statweb.stanford.edu/~tibs/ElemStatLearn/Brain & Computer Interfacing - Rajesh Rao This book is predominately concerned with analysis of neural data from MRIs and EEGs. It has a nice introduction to machine learning and then proceeds into domain specific considerations. It's quite focused to analysing brain data, but gives a good idea of the sort of tweaks that have to be made in practice for machine learning. (Also there's a huge amount of publicly available data from FMRIs on nuitblanche.blogspot so it should have practical advice if anyone wants to use that data set. Introductory Level.
Introduction to nonparametric estimation - Tsybakov This short monograph covers nonparametric estimation which is overlooked by most other sources. Mathematically focused, but not very difficult. Mid level.
A Probabilistic Theory of Pattern Recognition - Devroye, Gyorfi, & Lugosi. - A very deep look into the mathematics and theory behind machine learning. Probably only of interest to people doing research. Advanced Level
Gaussian Estimation, sequences, and wavelets - Iain Johnstone
http://statweb.stanford.edu/~imj/GE06-11-13.pdf This covers the sequence model for time series / signal analysis as well as
covers wavelet & fourier basis and their applications. Mid Level.
Deep Learning -
http://www.iro.umontreal.ca/~bengioy/dlbook/version-07-08-2015/dlbook.html, these draft notes are pretty much the only comprehensive / quality textbook in deep learning at the moment. Other texts to my knowledge have essentially been written extremely quickly by people with little background in an effort to cash in on the recent surge of interest. It's still a work in progress so many sections aren't complete, but it is still a valuable source. You can also look into the monograph Learning Deep Architectures for AI by Yoshua Bengio, which can basically be seen as a preliminary version of the textbook I've linked. Introductory Level
Home 