Google Dialogflow is detecting wrong intent

user_3137388 0
Viewed 580

We are implementing our customized chatbot using google dialog flow. I created some intents in dialog flow as below.

Intent : DVR
Training Phrases : 
    Unable to play DVR
    Unable to play Recording
    Not able to list Recordings

Intent : listings
Training Phrases : 
    Where are my TV listings

Intent : movies
Training Phrases : 
    Unable to play Movies
    Unable to list movies

Intent : new movies
Training Phrases : 
    New movies are not getting listed
    Very few new movies are getting listed
    How can I buy new movies on mobile app?
    New Movies not displayed
    Seeing blank entries on New Movies
    Seeing blank titles on New Movies

Order of intents in the dialog flow are as above -> DVR, listings, movies, new movies

Created entities as below.

DVR -> DVR, Recordings, Record
Listings -> listings
Movies -> movies
New_Movies -> new movies

When I try below texts, they are all working fine.

unable to play recording -> detected intent : DVR, Entity : DVR
unable to play DVR -> detected intent : DVR, Entity : DVR
Unable to play movies -> detected intent : movies, Entity : Movies

First issue : When I try 'unable to play new movies', detected intent is new movies (with "intentDetectionConfidence": 0.81950855) (entity : New_Movies). Not sure how this is working even though I didn't put any training phrase like unable to play under 'new movies'.

Unable to play New Movies -> detected intent : new movies, Entity : New_Movies, Confidence : 0.81950855

Second issue : if I try 'unable to play DVR New Movies', it is not detecting 'DVR' intent and 'DVR' entity. It is detecting intent as 'new movies' and entity as 'New_Movies' as below.

Unable to play DVR New Movies -> detected intent : new movies, Entity : New_Movies, Confidence : 0.7595941
Unable to play DVR 'New Movies' -> detected intent : new movies, Entity : New_Movies, Confidence : 0.7595941

I am not sure why dialog flow is behaving this way. Can any one please let me know how to resolve these issues (why DVR intent is not detected, why DVR Entity is not detected).

One more issue I found just now. There is one more intent I wrote in google dialogflow as below.

Intent : Remove Device
Training Phrases : 
    Having issues with device removal
    Unable to remove the device from registration
    Unable to remove the device
    Issue with Registration removal

Now I tried with text "unable to play". It should detect intent 'DVR' but it detected intent 'Remove Device' which doesn't have any training phrases related to play.

dialogflow-es
1 Answers

There are two types of matching algorithms used to assign the phrases to specific intents (1). I believe none of these algorithms can work as desired in your specific case:

-The grammar of the training phrases given for different intents is quite similar. This can lead the model to confusion:

“Unable to play DVR, unable to play Movies, unable to remove the device”

To solve this issue, you should provide a wide range of different structures such as “being unable to, can’t play, not allowed to, is not available”.

-As explained in (1), accurate ML matching needs a large number of training phrase examples. Otherwise the model will not be fine-tuned for your specific use cases and it will confuse phrases that, even though for a human may look simple, are somewhat similar:

“list recordings, TV listings, list movies, new movies are getting listed”

Computers understand the language in a different way that humans do. Normally, models have been trained for general objectives and, consequently, the more training examples you provide, the better they will adjust to specific use cases.

Finally, there is another concern regarding the Dialogflow agent you described. Given that the intents are rather similar, some intents might be getting an almost identical confidence score for some phrases. When this happens, the highest score will prevail (for example 0.76 over 0.74). This can be partially solved by configuring the intent priority (2).

Regarding the three issues you described above:

-1st: Even if you did not put “unable to” in this intent, it is the only intent which has the entity “new movies”. Nevertheless, bear in mind that the entity “new movies” contains the entity “movies” and these two intents will get confused in many cases.

-2nd: That phrase contains three entities for different intents “new movies”, “movies” and “DVR”. Probably all these intents will get a score over the threshold but only the highest one prevails. So it is not that the DVR intent is not detected, it is just that the “new movies” intent probably gets a slightly higher score.

-3rd: “Unable to play” does not give the agent enough information to determine which intent to trigger. It might as well even have been detected as “movies” given that you have the training phrase “unable to play movies”. No entity is present in “unable to play” and more than one intent matches the grammar structure “unable to”.

All in all, your intents are too similar to each other and have too few training phrases. Bear in mind that some of these examples would be difficult to classify even for a human (“unable to play”).

Have a nice day!

user_145652950
Related