How do I get Slim Framework to work without having to put /index.php in the URL?

I am implementing Slim Framework for PHP and everything appears to be working just fine. However, the only way I can get it to work is by putting /index.php in the URL like this:

http://www.example.com/index.php/members/1

I want it to work like this:

http://www.example.com/members/1

I have a feeling it has something to do with my .htaccess but I'm not sure. Here is my .htaccess:

RewriteCond %{REQUEST_FILENAME} !-f
# RewriteRule ^ ./index.php [QSA,L]
RewriteRule ^ . [QSA,L]