Use of deep learning in NLP techniques

Reading time: 30 minutes

In this post we will discuss what Natural Language Processing (NLP hereafter) is and then discover what the emerging techniques of deep learning has in store for it.

After reading this post , you will have an idea about:

• What is NLP?
• The use of deep learning in NLP
• 5 impressive applications of deep learning for NLP

Let's get started!

Introduction to Natural Language Processing

Natural Language Processing, or NLP for short, is broadly defined as the automatic manipulation of natural language like speech or text , by software.
Simply speaking, NLP refers to the ability of the computers to understand human speech or text as it is spoken or written.

In a more comprehensive way , natural language processing can be defined as a branch of Artificial Intelligence that draws inspiration from both computational linguistics and computer science to bridge the gap that exists between human language and a computer's understanding.

Natural Language Processing in real world

As machine learning practitioners interested in working with text data, we are concerned with the tools, methods and applications that this field offers. Today, natural language processing is primarily used for purposes like:

• machine translation
• automated question answering
• image captioning

In fact, NLP has found its applications in numerous real-world use cases including:

• automatic text summarization
• parts-of-speech tagging
• topic extraction
• sentiment analysis
• named entity recognition
• relationship extraction
• stemming

and much more.

Evolution of different approaches towards NLP and why deep learning is the best fit

There are three basic approaches:

• Classical linguistic method

• Statistical methods

• Deep learning methods

• Classical linguistic methods for natural language processing required experts in language defining rules to cover specific cases. These worked in narrow cases but turned out to be fragile.

• Statistical methods improve upon classical linguistic methods by learning rules and models from data rather than requiring them to be specified in a top-down manner. They result in much better performance, but must still be complemented with hand-crafted augmentations by language experts in order to achieve useful results.

• Deep learning methods are starting to out-compete the statistical methods on some challenging natural language processing problems with singular and simpler models.

The promise of deep learning for NLP

The reason why deep learning methods are getting so popular with NLP is because they are delivering on their promise.

The top 3 promises of deep learning for NLP are:

1. The promise of feature learning - That is, that deep learning methods can learn the features from natural language required by the model, rather than requiring that the features be specified and extracted by an expert.

2. The promise of continued improvement - That is, that the performance of deep learning in natural language processing is based on real results and that the improvements appear to be continuing and perhaps speeding up.

3. The promise of End-to-End Models - That is, that large end-to-end deep learning models can be fit on natural language problems offering a more general and better-performing approach.

Deep learning is undoubtedly a vast field of research and study and therefore it is very easy to get overwhelmed and bogged down by specific models intended to lift performance.

From the standpoint of text and speech data , there are 5 methods from deep learning that deserve the most attention for their application in NLP:

1. Embedding layers
2. Multilayer Perceptrons (MLP)
3. Convolutional Neural Networks (CNNs)
4. Recurrent Neural Networks (RNNs)
5. Recursive Neural Networks (ReNNs)

It is not just the performance of deep learning models on benchmark problems that is most interesting; it is the fact that a single model can learn word meaning and perform language tasks, obviating the need for a pipeline of specialized and hand-crafted methods.

Use cases of deep learning to end-user NLP problems

1. Speech Recognition

Speech Recognition is the problem of understanding what was said.
Given an utterance of text as audio data , the model must produce human readable text which means that using a language model , we want to create text output conditioned on audio data.

Speech Recognition has endless applications ranging from assisting differently-abled people to providing guidance and navigation in connected driverless cars.

How deep learning comes into play?

Have a look at this research paper to explore more

2. Machine Translation

It is the problem of converting a source text in one language to another language
Here , a language model is used to output the destination text in the second language, conditioned on the source text.
Given that deep neural networks are used, the field is referred to as neural machine translation.

Some examples include:

• Translating a text document from English to French.
• Translating Spanish audio to German text.
• Translating English text to Italian audio.
• Translating English audio to Spanish audio.

Imagine a situation where you are able to communicate with people having a different native language than yours, in your voice without even knowing how to speak their language.

Check out this video to follow up on such upcoming and latest advances in the field of machine translation and speech recognition.

3. Text Classification

Given an example of text, predict a predefined class label.
A popular classification example is sentiment analysis where class labels represent the emotional tone such as "positive" or "negative".

Below are a few more examples:

• Spam filtering, classifying email text as spam or not.
• Language identification, classifying the language of the source text.
• Genre classification, classifying the genre of a fictional story.

Link to a research paper for you to explore this field more.

4. Caption Generation

Caption generation is the problem of describing the contents of an image.
Given a digital image, such as a photo, generate a textual description of the contents of the image.
A language model is used to create the caption that is conditioned on the image.

Some examples include:

• Describing the contents of a scene.
• Creating a caption for a photograph.
• Describing a video.

Role of deep learning into this?

Give a read to this research paper.

5. Automatic Question Answering

It is the problem where given a subject, such as a document of text, answer a specific question about the subject.

An example of this could be - Question Answering on wikipedia

A research paper for further exploration.

Summary

As a learner in this field , I have tried to sum up the following in this post:

• What NLP is all about
• Why deep learning stands out as the best option to tackle NLP problems
• Some interesting applications of deep learning for NLP

If you liked the content , please do share it with others !