Initialize a non-generic object with a generic object

Sometimes I do not get those Ts of C# Generics right. I have a generic struct

public struct ValueWithUnit<T>
{
    public ValueWithUnit(T _value, Unit _unit)
    {
        Unit = _unit;
        Value = _value;
    }
    public Unit Unit { get; }
    public T Value { get; }
}

(Unitis an enum, T should be numeric, but there is no constraint available for that purpose).

For WCF I need a non-generic version of that, with T being double. So I thought of:

public struct DoubleValueWithUnit 
{
    public DoubleValueWithUnit(double _value, Unit _unit)
    {
        Unit = _unit;
        Value = _value;
    }
    public DoubleValueWithUnit(ValueWithUnit<T> _valueWithUnit)
    {
        Unit = _valueWithUnit.Unit;
        Value = Convert.ToDouble(_valueWithUnit.Value);
    }
    public Unit Unit { get; set; }
    public double Value { get; set; }
}

But the second constructor does not compile: error CS0246: The type or namespace name 'T' could not be found ... and Convert.ToDouble complains with Cannot resolve method 'ToDouble(T)' Candidates are...

I know I can add a conversion method to the generic class:

    public DoubleValueWithUnit ToDoubleValueWithUnit()
    {
        return new DoubleValueWithUnit(Convert.ToDouble(Value), Unit);
    }

That works. But is there any possibility to add a constructor with a generic parameter to a non-generic class/struct?