Divide an unsigned long for a size_t and assign the result to a double

I have to divide an unsigned long int for a size_t (returned from a dimension of a array with size() ) like this:

vector<string> mapped_samples;
vector<double> mean;
vector<unsigned long> feature_sum;
/* elaboration here */
mean.at(index) = feature_sum.at(index) /mapped_samples.size();

but in this way an integer division takes place (I lose the decimal part. That's no good)

Therefore, I can do:

 mean.at(index) = feature_sum.at(index) / double(mapped_samples.size());

But in this way feature_sum.at(index) is automatically converted (Temporary copy) to double and I could lose precision. How can I tackle the question? I have to use some library?

It could be precision loss when you convert the unsigned long in double (because the unsigned long value could be larger than maximum double) The unsigned long value is the sum of the features (positives values). The samples of feature can be 1000000 or more and the sum of values of the features can be enourmus. The max value of a feature is 2000 thus: 2000*1000000 or more

(I'm using C++11)