Information Theory: Crowdsourced sources

A roundup of introductory Information Theory sources

From colah

Andrew Little asked a useful question on Twitter: “Anyone know a good source for explaining the basics of information theory (Shannon entropy, etc.) to dense social scientists?”

I’m a dense social scientist who’s been meaning to dig more into information theory, so I collected the highlights of the responses here:

1. Visual Information Theory by colah (Christopher Olah)
2. Information Theory: A Tutorial Introduction by James Stone
3. A Brief Introduction to: Information Theory, Excess Entropy and Computational Mechanics (pdf) by David Feldman
4. Information Theory for Intelligent People by Simon DeDeo (who is currently encouraging you to delete your social media accounts)
5. Elements of Information Theory by Thomas Cover and Joy Thomas (this is the big one in the room)
6. Introduction to Information Theory by Seth Lloyd (this one’s a MOOC)

I’d seen all of these linked around before, but it’s nice to see them all in one place. Bookmarking this so I can finally dig beyond IT 101 when I have a little more reading time.