Configure speech recognition

Infobip Speech Recognition captures end-user's speech when they are communicating using Interactive Voice Response (IVR). The Infobip platform can save and match the spoken input with a predefined word or phrase. Then, depending on your pre-configuration, execute further IVR actions. This gives your end users more flexibility considering they are not limited to only having to tap digits on a keypad.

When creating an IVR scenario to capture end-user's spoken input (using the Capture IVR action) make sure to follow these steps:

1. One of the most important steps when setting up Speech Recognition using the Capture action is to choose the language you expect your end user to be speaking.
2. After that, you can set up some key phrases that need to be matched, as well as additional available parameters such as timeout, silence timeout, and more.
3. You can also choose not to match a key phrase but rather capture everything your end user will say and send that input as text to your platform via API using an additional IVR action.

For the full list of supported languages, see Check supported speech languages.

To learn more on how to preconfigure an IVR scenario and use Speech Recognition, make sure to check out the API documentation.

Best practices

Here are some tips and tricks on how to better use Speech Recognition.

Tip 1 - Know your use case

To use Speech Recognition efficiently, you need to take a few factors into consideration:

1. Always make an effort to think about your specific use case and tailor it to your end-users' needs.
2. Use the most specific variant of the language that is supported in IVR.
3. Assume there might be differences in phone and call quality. Have a plan for how to reach end users in certain hard-to-reach areas.
4. Expect background noise. Anticipate end users might be on their phones while walking in the street, a crowded area, or during rush hour.
5. Prepare short and concise questions. What kind of replies do you expect to receive? Do not use complex terms. Use simple language so your end users can understand you.
6. Be mindful of the speech recognition duration period. Is it reasonable to expect a reply in 5, 10, or 20 seconds?

Tip 2 - Tweak timeout and maxSilence carefully

One of the most important things - consider the time end users need to reply to your question.

Consider the following use case for example:

In this example, IVR option is set to 10 seconds. You expect to capture the following phrases ("info" or "update) in those 10 seconds. Now you test and tweak. Is 10 seconds enough, too long? This depends on multiple factors such as word length, ends-user's speech pattern, and reaction time.

Obviously, you do not want end users to have to wait too long to move on to the next action, nor to get upset for not having enough time to say the keyword.

What if you use maxSilence (as shown below)?

Here, the end user has 10 seconds with a 3 seconds maxSilence option. This means if IVR detects 3 seconds of silence, it stops capturing.

However, if your end user said the keyword within a second, they will have to wait 2 seconds before moving on to the next action in IVR.

What if the end user stops to think and makes a pause longer than 3 seconds? In this case, IVR will not capture the input.

Obviously, it is hard to offer general advice that works for everyone but the suggestions are:

• When possible, ask simple questions and suggest short and simple answers to your users
• Try different variations of timeout and maxSilence and test speech capturing behavior before you offer it to your end users

Tip 3 - Use simple key phrases

The keyPhrases matches captured speech with the provided text, and it branches the IVR scenario based on that match.

Example:

Here is where you have to be careful. You cannot expect your end users to always say the exact keyphrase.

Say your end user says 'Yes, give more info' instead of the keyphrase you defined - 'Yes, I want more info'. Since the keyphrase is not matched, the IF expression is not triggered.

The longer the keyphrase, the lower the odds the end user matches that specific keyphrase successfully.

Example:

A few tweaks have been made to the example above. Now, the probability to match the keyphrase is higher.

The keyphrase does not limit the end-users's speech. End users can say a lot more than just a keyphrase. However, the keyphrase is the key to correctly match and branch the IVR scenario. For example, if the end user says: "Yes, I want more info" or "Yes, give me more info", in both cases the keyphrase will be matched because myVar is more info.

If you are interested in full captured speech you can try using the myVar_Full variable (note that 'myVar' are just examples, you can name these any way you like).

Finally, both examples can be reduced to even shorter phrases:

Keyphrases are always matched by comparing fully captured speech text and provided keyphrases.

The final example in this tip:

Be mindful of how you select specific words for your keyphrases. Say you set your keyphrase to be "discount".

You have one end user say: "No, I do not want a discount" and another end user says: "Yes, I want a discount". See the problem?

In both cases, the keyphrase will match and both end users will receive more details about the discount. Always use short, clear, and simple keyphrases and think about all possible use cases to prevent confusion.

Tip 4 - Capturing starts after sound signal (beep)

Let your end users know they should start speaking after they hear a sound signal (beep).

Example:

If the end user says "info" before the first audio message is played completely, nothing will be captured. If this happens in your use case, you can suggest the end user to start speaking after the sound signal.

Tip 5 - Provide DTMF failover

Sometimes it is difficult to capture voice input, for example, when end users speak with a heavy accent, have speech difficulties, poor signal strength, or echo. In these cases, use DTMF as a failover option.

This means that they can answer in the form of speech input or by pressing predefined digits on their phone keypad.

The current example shows the end user can tap 1 or 0 instead of speaking.

If there is a DTMF failover, it has priority over Speech Recognition. For example, if the end user says something and taps 1, then myVar will be recorded instead of speech.

However, if the end user says "discount" as a keyphrase and does not tap anything on their keypad, myVar and myVar_Full is captured as usual.

Tip 6 - Use maxInputLength for capturing DTMF

When using DTMF failover by pressing digits on the phone keypad, IVR has to decide when to stop capturing.

Revisit the previous example which is slightly modified to demonstrate the point:

By not having maxInputLength the IVR does not know how many digits are going to be pressed, so it will wait for timeout expiration.

In this example, IVR will wait 5 seconds to capture all digits pressed. By using maxInputLength, IVR can stop capturing and proceed with the scenario as soon as it captures the specified number of digits.

As soon as the first digit is pressed, the IVR can continue the scenario if maxInputLength is 1. timeout is still respected in this case, meaning that end user has 5 seconds to say something or press a digit.

Example: