Which PHP mode? Apache vs CGI vs FastCGI

There are multiple ways to execute PHP scripts on a web server. Here is the difference between the modes.