How to exclude a pattern that starts with a phrase, but the pattern may contain substring of the phrase?

As your consuming pattern starts with an optional pattern, you need to account for these two scenarios and implement two separate lookbehinds:

(?<!\bwhether or )(?<!\bwhether or not )(?:\bnot )?\bapple\b

See the regex demo. Details:

• (?<!\bwhether or ) - no whether or allowed immediately on the left
• (?<!\bwhether or not ) - no whether or not allowed immediately on the left
• (?:\bnot )? - an optional sequence of not word and a space
• \bapple\b - a whole word apple.

Another approach is to match the string you want to skip and skip it:

\bwhether\s+or\s+not\s+apple\b(*SKIP)(*F)|(?:\bnot\s+)?\bapple\b

See this regex demo. Here, \bwhether\s+or\s+not\s+apple\b matches the whether or not apple and (*SKIP)(*F) drop the match, skips to the position of the failure and restarts a new match search.

The pattern you are looking for is ^(?:not\s)?apple

Python

import re

re.match(r"^(?:not\s)?apple", "apple")  # matches
re.match(r"^(?:not\s)?apple", "not apple")  # matches
re.match(r"^(?:not\s)?apple", "whether or not apple")  # does not match

JavaScript

/^(?:not\s)?apple/.test("apple"); // matches
/^(?:not\s)?apple/.test("not apple"); // matches
/^(?:not\s)?apple/.test("whether or not apple"); // does not match