nltk.sentiment.sentiment_analyzer module

A SentimentAnalyzer is a tool to implement and facilitate Sentiment Analysis tasks using NLTK features and classifiers, especially for teaching and demonstrative purposes.

class nltk.sentiment.sentiment_analyzer.SentimentAnalyzer

Bases: object

A Sentiment Analysis tool based on machine learning approaches.

__init__(classifier=None)

add_feat_extractor(function, **kwargs)

Add a new function to extract features from a document. This function will be used in extract_features(). Important: in this step our kwargs are only representing additional parameters, and NOT the document we have to parse. The document will always be the first parameter in the parameter list, and it will be added in the extract_features() function.

Parameters

• function – the extractor function to add to the list of feature extractors.

• kwargs – additional parameters required by the function function.

all_words(documents, labeled=None)

Return all words/tokens from the documents (with duplicates).

Parameters

• documents – a list of (words, label) tuples.

• labeled – if True, assume that each document is represented by a (words, label) tuple: (list(str), str). If False, each document is considered as being a simple list of strings: list(str).

Return type

list(str)

Returns

A list of all words/tokens in documents.

apply_features(documents, labeled=None)

Apply all feature extractor functions to the documents. This is a wrapper around nltk.classify.util.apply_features.

If labeled=False, return featuresets as:

[feature_func(doc) for doc in documents]

If labeled=True, return featuresets as:

[(feature_func(tok), label) for (tok, label) in toks]

Parameters

documents – a list of documents. If labeled=True, the method expects a list of (words, label) tuples.

Return type

LazyMap

bigram_collocation_feats(documents, top_n=None, min_freq=3, assoc_measure=<bound method NgramAssocMeasures.pmi of <class 'nltk.metrics.association.BigramAssocMeasures'>>)

Return top_n bigram features (using assoc_measure). Note that this method is based on bigram collocations measures, and not on simple bigram frequency.

Parameters

• documents – a list (or iterable) of tokens.

• top_n – number of best words/tokens to use, sorted by association measure.

• assoc_measure – bigram association measure to use as score function.

• min_freq – the minimum number of occurrencies of bigrams to take into consideration.

Returns

top_n ngrams scored by the given association measure.

classify(instance)

Classify a single instance applying the features that have already been stored in the SentimentAnalyzer.

Parameters

instance – a list (or iterable) of tokens.

Returns

the classification result given by applying the classifier.

evaluate(test_set, classifier=None, accuracy=True, f_measure=True, precision=True, recall=True, verbose=False)

Evaluate and print classifier performance on the test set.

Parameters

• test_set – A list of (tokens, label) tuples to use as gold set.

• classifier – a classifier instance (previously trained).

• accuracy – if True, evaluate classifier accuracy.

• f_measure – if True, evaluate classifier f_measure.

• precision – if True, evaluate classifier precision.

• recall – if True, evaluate classifier recall.

Returns

evaluation results.

Return type

dict(str): float

extract_features(document)

Apply extractor functions (and their parameters) to the present document. We pass document as the first parameter of the extractor functions. If we want to use the same extractor function multiple times, we have to add it to the extractors with add_feat_extractor using multiple sets of parameters (one for each call of the extractor function).

Parameters

document – the document that will be passed as argument to the feature extractor functions.

Returns

A dictionary of populated features extracted from the document.

Return type

dict

save_file(content, filename)

Store content in filename. Can be used to store a SentimentAnalyzer.

train(trainer, training_set, save_classifier=None, **kwargs)

Train classifier on the training set, optionally saving the output in the file specified by save_classifier. Additional arguments depend on the specific trainer used. For example, a MaxentClassifier can use max_iter parameter to specify the number of iterations, while a NaiveBayesClassifier cannot.

Parameters

• trainer – train method of a classifier. E.g.: NaiveBayesClassifier.train

• training_set – the training set to be passed as argument to the classifier train method.

• save_classifier – the filename of the file where the classifier will be stored (optional).

• kwargs – additional parameters that will be passed as arguments to the classifier train function.

Returns

A classifier instance trained on the training set.

Return type

unigram_word_feats(words, top_n=None, min_freq=0)

Return most common top_n word features.

Parameters

• words – a list of words/tokens.

• top_n – number of best words/tokens to use, sorted by frequency.

Return type

list(str)

Returns

A list of top_n words/tokens (with no duplicates) sorted by frequency.