Laravel Query Relationship on One Model Instance

I am aware that I can use count() to query for Eloquent relationships in Laravel, like so:

if(count($question->answers()))

Where answers() is a hasMany relationship:

 public function answers()
    {
        return $this->hasMany('App\Models\Answer', 'question_id');
    }

My question is, how do I do this when $question is not an entire collection but one Model instance?

$question = Question::where('id',$key)->first(); 

How do I query the above question, and only that question, for a potential relationship using count()?

I always am getting a count() of greater than zero, even when the selected question has no associated answers, which means my if block always runs and returns unwarranted null values:

if(count($question->answers()))
   {
   //returns nulls
   }