Adding and Using WordPress Custom Image Sizes: A Guide to the Best Thing Ever

Update April 4, 2017: Added a section detailing how the responsive image system in WordPress 4.4+ works, and how custom image sizes interact with this system.

I really love being able to set custom image sizes in WordPress. The power and flexibility this system opens up for building beautiful, easy-to-administer, performant websites is hard to overstate.