What is an API key?

I see this word in almost every cross service application these days.

What exactly is an API key and what are its uses?

Also, what is the difference between public and private API keys.