sed regular expression error \1 not defined in the RE

This might work for you (GNU sed):

sed 's/\<test[0-9]\{3\}\.abc\.com/eh6.test.abc.com/' file

Replace testnnn.abc.com with eh6.test.abc.com where test begins on a word boundary and nnn is a 3 digit number.

The problem is that the version of sed that macOS uses has different syntax for the -i option than the GNU version, which is causing a chain reaction of misinterpretation.

The -i option tells sed to modify the file "in-place", and takes an optional argument that's used as a suffix to store the original file as a backup. If you don't want a backup, you pass the empty string as the backup suffix. The problem is how you pass that empty string. With GNU sed, the suffix must be directly attached to the -i option (like sed -i.bkp ...), so just using -i with nothing after it works. But the version macOS uses allows the suffix to be passed as a separate "word" (like sed -i .bkp ...), so to specify no backup you need to pass an explicit empty string (like sed -i '' ...).

As a result, when you use sed -i -E ... on macOS, it interprets -E as a suffix to use on a backup file, doesn't interpret it as meaning "use extended regex syntax", and therefore misparses the regular expression as containing no capture groups.

Solution: either pass an explicit null suffix (sed -i '' -E ...) or, if you want it to be compatible with GNU sed, pass an actual backup suffix (and directly attach it so GNU doesn't get confused: sed -i.bkp -E ...). There is no way to portably use sed -i without a backup (see this question).

Also, as anubhava pointed out in the comments, \d isn't supported here, so use [0-9] instead.

Since you're dealing with CSV data, I would suggest better tool awk for this.

awk 'BEGIN{FS=OFS=","} $2 ~ /^test[0-9]{3}\.abc\.com$/ {$2 = "eh6." $2} 1' file

scheme,host,Username,Password
https,eh6.test003.abc.com,test003_0021,abc1!
https,eh6.test046.abc.com,test046_015,abc1!```