How to infer generic type argument of union?

I want to infer generic arguments of a union like in the attached code. What does InferValueTypes should be?

interface WithValue<T> {}

interface ValuedNumber extends WithValue<number> {}
interface ValuedString extends WithValue<string> {}

type Valued = ValuedNumber | ValuedString;

type InverValueType<T extends WithValue<any>> = T extends WithValue<infer V>  ? V : never;
type AType = InferValueType<ValuedNumber>; // Will infer `number`

type InferValueTypes<T> = ???;
type ValueTypes = InferValueTypes<Valued>; // Expect to infer `string | number`