Conditional types can't be used to index type

Say I want to write a utility type like Merge

type Merge<A, B> = {
  [P in (keyof A | keyof B)]: P extends keyof A ? A[P] : B[P]  
}

However, the above code would get

Type 'P' cannot be used to index type 'B'.

Though I can write it like

type Merge<A, B> = {
  [P in (keyof A | keyof B)]: P extends keyof A ? A[P] : P extends keyof B ? B[P] : never
}

But isn't it too verbose?

I have limited the type of P in the P in (keyof A | keyof B), so why does the P extends keyof B still needed?