Natural Language Processing (NLP) is a powerful tool that Chatbot builders can use to add a human element to conversations and help reduce user frustration. NLP is, however, much more than just that and exists as a technology that’s become ingrained in our daily lives while also hiding in plain sight.
What is NLP?
NLP is an area of Artificial Intelligence (AI) that deals with how computers process, comprehend and output human language. It’s the input and output that allows deeper and more powerful AI processes to actually create value.
While NLP’s time in the spotlight has been relatively recent, it’s far from being a new technology. Ever since humans have been using computers, applying their capabilities to process communication has been a core function. In fact, one of the most famous computer challenges – the Turing Test – deals fundamentally with how well a computer can communicate.
To dig deeper into NLP, we can look at two core components of the technology. First, there’s Natural Language Understanding (NLU). NLU is the utility tool that deals with a difficult issue: humans are often not the most consistent of communicators. Our text is filled with grammatical errors, typos and incomplete thoughts. Our speech incorporates slang, accents and mispronunciations. NLU seeks to understand these outliers to further improve the computer’s language processing capabilities.
Next is Natural Language Generation (NLG), which focuses on how the computer itself outputs text and speech. NLG is critical in that it allows us to interact with NLP technology on a daily basis. It is largely what brings value to the average user.
NLP usually relies on a type of engine. This engine is where information that is input is processed and ideally transformed into something that can be analyzed and used. A core challenge for the engine, and for NLP as a whole, is understanding intent. Intent is what we are trying to say when we say something. This is difficult for computers as most sentences can have multiple intents.
Understanding intent is a major challenge for NLP, and something that will be at the forefront of its advancement and further integration into everyday life.
How is NLP used?
The uses for NLP are vast and are becoming more common while also advancing in their sophistication.
The average consumer interacts with NLP multiple times daily and likely does not even realize it. Among the most common examples of this is text input. Quite simply, if you’re typing a word on a phone or computer, it’s likely using NLP to help optimize that communication. Examples include:
- Predictive text: Most commonly used on smartphones, though often built in to search engines, NLP is used to try and anticipate what you’ll type next based on factors including but not limited to what you’ve typed in the past.
- Autocomplete: Quite similar to predictive text, but focuses on completing the word you are typing rather than guessing the next word.
- Autocorrect/spellcheck: Another largely smartphone-based application of NLP, this use focuses on identifying and correcting misspellings and typos.
- Grammar correction: Similar to autocorrect, but focussed on grammatical flaws. It’s often incorporated into word processing programs and is also used in web extensions such as Grammarly.
As you can see, NLP is thoroughly integrated into day-to-day modern life. It would be challenging to go a full day without encountering it. Thankfully, it’s prevalence is a positive given all of the ways it helps to improve our communication.
Does my Chatbot need NLP?
Technically, no. But a car doesn’t technically need airbags and would you choose to drive a car without them? Doubtful.
Plus, if you’re building a chatbot, why not include this element that can take your chatbot to the next level? Ignoring NLP when you have the capability to add it means choosing to supply your users with a sub-optimal product.
Getting deeper into it, one of the most frequent issues with chatbots is the instinct of a human user to go outside of what you’ve designed your chatbot to do. No matter how well you lay out a clear path, or provide descriptive buttons, users will inevitably end up doing something you did not predict. When this happens, users get frustrated and stop using the chatbot. If that happens on their first use, it’s unlikely you’ll see them again.
Think of NLP as the safety net that captures usage that goes outside your chatbot’s predetermined parameters. It’s what catches users when things go off the rails, and brings them closer to what they’re looking for. In the past, the only way to correct for this was to wait for user sessions to fail, and then look for the cause and attempt to solve it. Given that people likely won’t return after a failed session, this approach was not sustainable.
Adding NLP to your chatbot can also elevate the overall user experience by serving as a more complete product. People hear “chatbot” and tend to think of something that can talk back to it. This is often true even if the user has no end goal from throwing a few words at the chatbot. NLP allows your program to do this and rise up to the level that users are growing to expect.
How do I add NLP to my chatbot?
Now that we understand the importance of adding NLP to chatbots, it’s time to examine how you can actually do it.
The process of adding NLP depends on two factors:
- How your chatbot is built.
- How sophisticated you’d like the NLP to be.
First, think about how about how you built your chatbot, or where you are planning on building it. Chatbots are generally built either in a specialized chatbot-creation platform or in a more traditional method using code.
If you built your chatbot in a build platform, it’s likely that the platform has some sort of NLP functionality. The sophistication and ease of use depend entirely on the platform that you chose to build on. Some platforms are built to integrate NLP heavily, while others use it as more of a side skill.
If you coded your chatbot yourself, adding NLP functionality will likely be more difficult. You’ll need to develop a deeper understanding of NLP and how it fits with other AI pillars. Then, you’ll have to learn how to use it to get the result you want. Thankfully, there are a number of open source frameworks that can assist you with this. Once you have all of this, you can then begin coding and testing NLP with your chatbot.
Second, you’ll need to decide how sophisticated you want the NLP to be. This can be done by analyzing the likelihood of how often users will need it, and what they’ll need it for. If you have a very straightforward chatbot that uses mostly buttons and a relatively linear conversation path, basic NLP should do. If you’d like your chatbot to be highly conversational, or if there are many different user paths, more sophisticated NLP will be beneficial.
Overall, NLP is a technology that should be celebrated. For all of the work it does for us in our day-to-day lives, it’s far too easy to ignore. And when it comes to chatbots, it definitely should not be ignored. NLP is a tool that any chatbot builder can use to take their builds to a new level of performance and customer satisfaction.