Yekun's Note

Attention mechanism has been widely applied in natural langugage processing and computer vision tasks.

Bash scripts are powerful in data processing. This post records some tips that I encountered during the data processing, and will keep updated.

Text classification is one of the most important fundamental NLP tasks. Its goal is to assign labels to texts, including sentiment analysis, spam detection, topic labeling, Twitter hashtag prediction, domain detection, etc.

Pretraining on ImageNet followed by domain-specific fine-tuning has illustrated compelling improvements in computer vision research. Similarly, Natual Language Processing (NLP) tasks could borrow ideas from this.

Employing pretrained word representations or even langugage models to introduce linguistic prior knowledge has been common sense in amounts of NLP tasks with deep learning.

Why is NER hard in the industry?

This blog dicusses several frequently occurred problems and possible solutions.

Active learning (called query learning or optimal experimental design in statistics). The key hypothesis is, if the learning algorithm is allowed to choose the data from what it learns, it will perform better with less training.

Name Entity Recognition (NER) labels sequences of words in a text which are the names of things, such as person and company names, or gene and protein names.

Neural nets require large scale dataset during training process. However, it is quite expensive to have the access to enough data size. One approach to deal with this issue is Data augmentation, which means increasing the number of data points.

Here we briefly introduce some common evaluation metrics in NER tasks, considering both extracted boundary and entities.

A note for NLP Interview.