Escape double quote in grep

I wanted to do grep for keywords with double quotes inside. To give a simple example:

echo "member":"time" | grep -e "member\""

That does not match. How can I fix it?