I have compiled a collection of resources that I have found helpful over the years. While some of the content may be outdated, I hope it is still useful.
Novelty in Science - A guide to reviewers by Michael Black [Blog]
How to Avoid the Pitfalls of Machine Learning: A Guide for Academic Researchers by Michael Lones [PDF]
Some of the Best Advice You'll Ever Receive by Alemi [Blog]
"What one fool can do, so can another, and the fact that some other fool beat you to it shouldn't disturb you: you should get a kick out of having discovered something." - Richard Feynman
On why science writing can’t be separated from science doing by Michael Black (03-Dec 2022) [Tweet-thread]
On Rebuttals
On Paper Writing
On Presentations
Mutual Information-based objectives in unsupervised learning (Aug 2019) [Blog]
The Bandwagon by Claude E. Shannon (IRE Transactions - Information Theory 1959): [Blog]
Semi-Supervised Learning (July 2020): [Blog]
Excerpts from Mathematical Writing by Knuth, Larrabee, Roberts (1989): [PDF]
Unpredictable Black Boxes are Terrible Interfaces by Maneesh Agrawala [Blog]
The Bitter Lesson by Rich Sutton (March 13, 2019) [Blog]
- The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
- We want AI agents that can discover like we can, not which contain what we have discovered.
- The “bitter lesson” of machine learning research argues that general methods that can leverage additional computation ultimately win out against methods that rely on human expertise from Sec 3.6 Scaling [PDF]
Diffusion language models by Sander Dieleman (Jan 09, 2023) [Blog]
Deep Learning Tuning Playbook [Blog]
This document is for engineers and researchers (both individuals and teams) interested in maximizing the performance of deep learning models.
UvA Deep Learning Tutorials with Pytorch Lightning [Website] [Repo] [Youtube Playlist]
Some Courses (with videos) that I found useful.
6.008 Introduction to Inference MIT Tutorial Videos [Link]
Topics: Graphical Models; HMM; Junction tree; MLE; Sampling (Metropolics & gibbs)
STA 4273H: Large Scale Machine Learning Slides and Videos [Link]
Prof. Russ Salakhutdinov
Winter 2015
Topics: Covers Ch.1,3,4,8,9,10,11,12,13 of CM Bishop Book
Missing: Ch.5,6,7 (Neural nets, Kernel methods, Sparse Kernel methods)
Machine Learning for Intelligent Systems - CS4780 - [Webpage] [Youtube]
Prof. Kilian Weinberger Cornell
Fall semester 2018
Topics: SVM, Kernels, Bias-Variance decomposition, Gaussian Process, Bagging, Boosting
Learning Theory - 580.691/491 - [Webpage] [Youtube]
Prof. Reza Shadmehr JHU
Spring semester 2014/2017
Computer Vision (2021) [Webpage]
Prof. Dr. Andreas Geiger
Multiview 3D Geometry in Computer Vision [Website]
Prof. Hyun Soo Park UMN
Spring semester 2018
Good Coverage & Good Homeworks
CMSC 828W: Foundations of Deep Learning [Webpage] [Youtube] [Colab programming modules]
Prof. Soheil Feizi University of Maryland, College Park
Fall 2022
Good intro to theoretical aspects of deep learning. Excellent videos.
This is water by David Foster Wallace (May 2005) [Blog] [SoundCloud]
Providing decent living with minimum energy: A global scenario (May 2005): [PDF]
The Illusion of Progress (Marcellus Blog Post) [Link]
The Age of Average by Alex Murrell [Link]