ShingleFilter factory in solr

Viewed 63

Enviornment ==> solr - solr-8.9.0, java version "11.0.12" 2021-07-20 LTS

I have indexed following .csv file in solr

ProductName
Washing machine
WashingMachine

Field 'ProductName' is configured in managed schema as follows

<field name="ProductName" type="text_general" multiValued="false" indexed="true" stored="true"/>

Analyzer for field-type:'text_general' is configured in managed-schema as follows

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100" multiValued="true">
    <analyzer type="index">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.LowerCaseFilterFactory"/>
      <filter class="solr.ShingleFilterFactory"/>
    </analyzer>
    <analyzer type="query">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.SynonymGraphFilterFactory" expand="true" ignoreCase="true" synonyms="synonyms.txt"/>
      <filter class="solr.LowerCaseFilterFactory"/>
      <filter class="solr.ShingleFilterFactory"/>
    </analyzer>
  </fieldType>

I have executed following query to solr using curl

curl -G http://localhost:8983/solr/solrCore/select --data-urlencode "q=(ProductName:'WashingMachine~')"

Above query is giving me both rows(1.Washing machine 2.WashingMachine) which i had indexed in solr.

But following query is giving only single document(1.Washing machine)

curl -G http://localhost:8983/solr/solrCore/select --data-urlencode "q=(ProductName:'Washing Machine~')"

Above query is not giving 2nd document(WashingMachine)

I understand that Shingle filter constructs shingles, from the token stream. It combines runs of tokens into a single token.

What should i changed to query or filters/tokenizers or configurations so that second query results both documents? Could someone help me find the missing piece? Any help is appreciated.

0 Answers
Related