Understanding and Implementing POS Tagging with spaCy | CodeSignal Learn

Understanding and Implementing POS Tagging with spaCy | CodeSignal Learn

Introduction to the Lesson

Hello and welcome to this lesson on Understanding and Implementing Part-of-speech (POS) Tagging with spaCy! In this lesson, we'll discuss what POS tagging means, its importance in Natural Language Processing (NLP), and how we can effortlessly perform it using spaCy . By the end of this lesson, you should be able to process a text and tag each token (word) with its corresponding POS using spaCy .

Introduction to POS Tagging

POS tagging is the process of assigning a part-of-speech label (noun, verb, adjective, etc.) to each token (word) in a given text. For example, in the sentence "Sam eats quickly.", "Sam" is a noun, "eats" is a verb, and "quickly" is an adverb. This is important because the meaning of a sentence can significantly be determined by the POS of the words in the sentence. When we perform POS tagging, it not only identifies the POS of a word, but also its grammatical use within the sentence. For instance, "book" can be a noun ("Sam reads a book.") or a verb ("Book a ticket for me."), and POS tagging helps in distinguishing between these uses. In NLP tasks like parsing, text-to-speech conversion, machine translation, extraction of relationships and entities, POS tagging plays a crucial role. For example, in information extraction, if you want to extract all named entities that are 'organizations' from some text, knowing that a word is a proper noun (NNP in the detailed Penn Treebank POS tags set) may not be enough; you would need its context among other words in the text.

Understanding POS Tagging Implementation with spaCy

Implementing POS tagging in spaCy is pretty straightforward. However, it's important to note that POS tagging in spaCy is statistical, meaning it is based on statistical models that consider the context of the words in the text. When we process a text with the nlp object, spaCy tokenizes the text to create a Doc object. This Doc object carries all the computed attributes and properties that we can delve into. For POS tagging, we focus on two token attributes: pos_: This is the simple part-of-speech tag, using the Universal POS tag set. It provides a general POS tag, like 'NOUN', 'VERB', 'ADV' etc. tag_: This is the detailed part-of-speech tag using the Penn Treebank POS tag set. It provides detailed POS information, like 'VBZ' (verb, 3rd person singular present), 'RB' (adverb), etc.

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Performing POS Tagging on a Sample Text Using spaCy

The power of any learning lies in the doing. Let's roll up our sleeves and dive into some code. For POS tagging with spaCy, we need to process the text and loop through the token properties of the processed Doc object. Let's look at how to do that. First, we import the spaCy library and load the English language model. import spacy nlp = spacy.load("en_core_web_sm") Define a sentence of English text that we want to perform POS tagging on: text = "I am learning NLP and using spaCy for POS tagging." Process this text using the nlp function to create a Doc object: doc = nlp(text) Perform POS tagging on each token in the Doc object using a for loop: for token in doc: print(f"{token.text:{10}} {token.lemma_:{10}} {token.pos_:{10}} {token.tag_:{10}}") This small piece of code will give you the POS tagging information for each word in the text, providing the word (token.text), its base form (token.lemma_), simple POS (token.pos_), and detailed POS tag (token.tag_).

Understanding the Output and Next Steps

The output of the above code will be: I I PRON PRP am be AUX VBP learning learn VERB VBG NLP NLP PROPN NNP and and CCONJ CC using use VERB VBG spaCy spacy NOUN NN for for ADP IN POS POS PROPN NNP tagging tagging NOUN NN . . PUNCT . This output illustrates the tokenization and POS tagging of a text. Each row represents a token (word or punctuation) from the original text. The columns include the token itself, its lemma (base form), its simple POS tag, and its detailed POS tag. This tagging helps in understanding not just the role of each word in the sentence, but also its base form, which is crucial for many NLP tasks.

text

I          I          PRON       PRP       
am         be         AUX        VBP       
learning   learn      VERB       VBG       
NLP        NLP        PROPN      NNP       
and        and        CCONJ      CC        
using      use        VERB       VBG       
spaCy      spacy      NOUN       NN        
for        for        ADP        IN        
POS        POS        PROPN      NNP       
tagging    tagging    NOUN       NN        
.          .          PUNCT      .