We are using websearch_to_tsquery() in our app. I've noticed that when passed a "quoted" string to find an exact match, it will interpret a colon character ":" as a word, while to_tsvector does not. This prevents a tsquery from matching a tsvector created from the same string. The behavior exists in at least Postgres 12 and 13. Is this expected?
With a colon:
SELECT websearch_to_tsquery('english', '"Design Patterns in C++: Behavioral - Observer to Visitor"')
'design' <-> 'pattern' <2> 'c' <2> 'behavior' <-> 'observ' <2> 'visitor'
Now with the colon removed:
SELECT websearch_to_tsquery('english', '"Design Patterns in C++ Behavioral - Observer to Visitor"')
'design' <-> 'pattern' <2> 'c' <-> 'behavior' <-> 'observ' <2> 'visitor'
Note the 'c' <2> 'behavior' vs 'c' <-> 'behavior'. A tsvector created from the same string with a colon does not include a colon as a word. That is what I would expect, but this means the tsquery will not match a tsvector created from the same string.
SELECT to_tsvector('english', '"Design Patterns in C++: Behavioral - Observer to Visitor"')
'behavior':5 'c':4 'design':1 'observ':6 'pattern':2 'visitor':8
Documentation for websearch_to_tsquery:
Converts text to a tsquery, normalizing words according to the specified or default configuration. Quoted word sequences are converted to phrase tests. The word “or” is understood as producing an OR operator, and a dash produces a NOT operator; other punctuation is ignored. This approximates the behavior of some common web search tools.
Should I be stripping colons from strings before creating a tsquery? Are there other characters that characters that need to be removed?