Insights

When Matthew said “Ask and it shall be given” was he talking about NLP?

Human communication has always been a complex synthesis of verbal and non-verbal clues. Among other factors conveyed, the meaning of the said words also depends upon the experiences of the listener and, the tone and the body language of the speaker. But something that has played such a pivotal role in the evolution of the most evolved species in time ought to be intricate which makes it difficult to replicate too. In short, an AI designed to communicate must be able to work at the level of human intelligence. Previously thought impossible, this elevation now seems quite probable with the help of machine learning and natural language processing (NLP).

A commonplace in the online world, scripted chatbots are no strangers to us. Designed for specific purposes and audience, these chatbots use predefined scripts to perform actions and answer a certain set of questions. But these were just an extension of written FAQs. Enter Intelligent chatbots and things change— Forever. Intelligent chatbots learn with every interaction with their clients or in other words, they work on NLP.

NLP, a fusion of mathematical linguistics and AI, is helping computers understand, evaluate and utilize human language. How? By breaking the language components into atomic parts and teaching the machine to deal with those parts separately. Three important steps channel the NLP process:

Step 1: Understanding natural expressions: Even when devoid of non-verbal components, human language is framed by countless flexible parameters including but not limited to misspellings, abbreviations, jargons, accent, emotions and contextual meanings. NLP solves this problem by breaking the language into a logical structure which is fed to computer algorithms for machine learning.

Step 2: Text Generation: The machine creates a readable text with the help of Natural Language Understanding (NLU) and Natural Language Generation (NLG).

Step 3: Interaction: The computer responds in a similar understandable manner.

In the simplest mathematical definition:

The conversion process of splitting text into manageable blocks is complex and consists of many stages. But, basic processing includes:Parsing: Finding sentences and deriving meaning from them.

Tokenization: Dividing text in tokens i.e. relevant units (words, phrases or sentences).

Lemmatization/stemming: Stemming to get rid of inflictions and Lemmatization to derive basic form from the morphological structure of the word. For example: in the word study, y is the inflection, stud is the stem and lemma will be study.

Language detection: Finding human language in sentence.

Part-of-speech-tagging: The computer uses rules to identify grammar and part of speech in the sentence and tags them accordingly.

With their unprecedented success, Siri, Alexa and Google Assistant have established NLP as an indispensable technology.

Amazon’s Alexa is using NLP technology to understand what its users are saying and then to respond to them accordingly. If Alexa makes a mistake in understanding the user request, then the data is used to improve its machine learning and make the setup smarter. Continuous learning is the key here.

Duolingo, a language learning app, uses games to teach language to its users. The user simply needs to invest 5 to 10 minutes of her daily schedule to learn a new language. The app stands apart from other similar apps in the way it has addressed two crucial bottlenecks:

• Inviting users for conversational skills
• Eliminating the fear of embarrassment by creating a native chatbot within its app. This app gives users a choice to practice conversations at their convenience and use their favorite characters until they gain confidence to interact with other users.

With its self-improvement skills, NLP is the closest our technology has come to human language evolution. At the rate that the machines learn with this technology, it will be no surprise that in times to come its communication supersedes that of humans.