I'm running a Centos 7 desktop with gnome, although similar issues regularly come up with Arch and other distros.
When I checked my cpu usage via top, pulseaudio was using nearly 20% cpu.
I hunted around for solutions and there were a lot of incorrect ones, so I'm posting here for karma points to help others.