nltk.probability module

Classes for representing and processing probabilistic information.

The FreqDist class is used to encode “frequency distributions”, which count the number of times that each outcome of an experiment occurs.

The ProbDistI class defines a standard interface for “probability distributions”, which encode the probability of each outcome for an experiment. There are two types of probability distribution:

• “derived probability distributions” are created from frequency distributions. They attempt to model the probability distribution that generated the frequency distribution.

• “analytic probability distributions” are created directly from parameters (such as variance).

The ConditionalFreqDist class and ConditionalProbDistI interface are used to encode conditional distributions. Conditional probability distributions can be derived or analytic; but currently the only implementation of the ConditionalProbDistI interface is ConditionalProbDist, a derived distribution.

class nltk.probability.ConditionalFreqDist

Bases: defaultdict

A collection of frequency distributions for a single experiment run under different conditions. Conditional frequency distributions are used to record the number of times each sample occurred, given the condition under which the experiment was run. For example, a conditional frequency distribution could be used to record the frequency of each word (type) in a document, given its length. Formally, a conditional frequency distribution can be defined as a function that maps from each condition to the FreqDist for the experiment under that condition.

Conditional frequency distributions are typically constructed by repeatedly running an experiment under a variety of conditions, and incrementing the sample outcome counts for the appropriate conditions. For example, the following code will produce a conditional frequency distribution that encodes how often each word type occurs, given the length of that word type:

>>> from nltk.probability import ConditionalFreqDist
>>> from nltk.tokenize import word_tokenize
>>> sent = "the the the dog dog some other words that we do not care about"
>>> cfdist = ConditionalFreqDist()
>>> for word in word_tokenize(sent):
...     condition = len(word)
...     cfdist[condition][word] += 1

An equivalent way to do this is with the initializer:

>>> cfdist = ConditionalFreqDist((len(word), word) for word in word_tokenize(sent))

The frequency distribution for each condition is accessed using the indexing operator:

>>> cfdist[3]
FreqDist({'the': 3, 'dog': 2, 'not': 1})
>>> cfdist[3].freq('the')
0.5
>>> cfdist[3]['dog']
2

When the indexing operator is used to access the frequency distribution for a condition that has not been accessed before, ConditionalFreqDist creates a new empty FreqDist for that condition.

N()

Return the total number of sample outcomes that have been recorded by this ConditionalFreqDist.

Return type

int

__init__(cond_samples=None)

Construct a new empty conditional frequency distribution. In particular, the count for every sample, under every condition, is zero.

Parameters

cond_samples (Sequence of (condition, sample) tuples) – The samples to initialize the conditional frequency distribution with

conditions()

Return a list of the conditions that have been accessed for this ConditionalFreqDist. Use the indexing operator to access the frequency distribution for a given condition. Note that the frequency distributions for some conditions may contain zero sample outcomes.

Return type

list

copy() → a shallow copy of D.

deepcopy()

plot(*args, samples=None, title='', cumulative=False, percents=False, conditions=None, show=True, **kwargs)

Plot the given samples from the conditional frequency distribution. For a cumulative plot, specify cumulative=True. Additional *args and **kwargs are passed to matplotlib’s plot function. (Requires Matplotlib to be installed.)

Parameters

• samples (list) – The samples to plot

• title (str) – The title for the graph

• cumulative (bool) – Whether the plot is cumulative. (default = False)

• percents (bool) – Whether the plot uses percents instead of counts. (default = False)

• conditions (list) – The conditions to plot (default is all)

• show (bool) – Whether to show the plot, or only return the ax.

tabulate(*args, **kwargs)

Tabulate the given samples from the conditional frequency distribution.

Parameters

• samples (list) – The samples to plot

• conditions (list) – The conditions to plot (default is all)

• cumulative – A flag to specify whether the freqs are cumulative (default = False)

class nltk.probability.ConditionalProbDist

Bases: ConditionalProbDistI

A conditional probability distribution modeling the experiments that were used to generate a conditional frequency distribution. A ConditionalProbDist is constructed from a ConditionalFreqDist and a ProbDist factory:

• The ConditionalFreqDist specifies the frequency distribution for each condition.

• The ProbDist factory is a function that takes a condition’s frequency distribution, and returns its probability distribution. A ProbDist class’s name (such as MLEProbDist or HeldoutProbDist) can be used to specify that class’s constructor.

The first argument to the ProbDist factory is the frequency distribution that it should model; and the remaining arguments are specified by the factory_args parameter to the ConditionalProbDist constructor. For example, the following code constructs a ConditionalProbDist, where the probability distribution for each condition is an ELEProbDist with 10 bins:

>>> from nltk.corpus import brown
>>> from nltk.probability import ConditionalFreqDist
>>> from nltk.probability import ConditionalProbDist, ELEProbDist
>>> cfdist = ConditionalFreqDist(brown.tagged_words()[:5000])
>>> cpdist = ConditionalProbDist(cfdist, ELEProbDist, 10)
>>> cpdist['passed'].max()
'VBD'
>>> cpdist['passed'].prob('VBD') 
0.423...

__init__(cfdist, probdist_factory, *factory_args, **factory_kw_args)

Construct a new conditional probability distribution, based on the given conditional frequency distribution and ProbDist factory.

Parameters

• cfdist (ConditionalFreqDist) – The ConditionalFreqDist specifying the frequency distribution for each condition.

• probdist_factory (class or function) – The function or class that maps a condition’s frequency distribution to its probability distribution. The function is called with the frequency distribution as its first argument, factory_args as its remaining arguments, and factory_kw_args as keyword arguments.

• factory_args ((any)) – Extra arguments for probdist_factory. These arguments are usually used to specify extra properties for the probability distributions of individual conditions, such as the number of bins they contain.

• factory_kw_args ((any)) – Extra keyword arguments for probdist_factory.

class nltk.probability.ConditionalProbDistI

Bases: dict

A collection of probability distributions for a single experiment run under different conditions. Conditional probability distributions are used to estimate the likelihood of each sample, given the condition under which the experiment was run. For example, a conditional probability distribution could be used to estimate the probability of each word type in a document, given the length of the word type. Formally, a conditional probability distribution can be defined as a function that maps from each condition to the ProbDist for the experiment under that condition.

abstract __init__()

Classes inheriting from ConditionalProbDistI should implement __init__.

conditions()

Return a list of the conditions that are represented by this ConditionalProbDist. Use the indexing operator to access the probability distribution for a given condition.

Return type

list

class nltk.probability.CrossValidationProbDist

Bases: ProbDistI

The cross-validation estimate for the probability distribution of the experiment used to generate a set of frequency distribution. The “cross-validation estimate” for the probability of a sample is found by averaging the held-out estimates for the sample in each pair of frequency distributions.

SUM_TO_ONE = False

True if the probabilities of the samples in this probability distribution will always sum to one.

__init__(freqdists, bins)

Use the cross-validation estimate to create a probability distribution for the experiment used to generate freqdists.

Parameters

• freqdists (list(FreqDist)) – A list of the frequency distributions generated by the experiment.

• bins (int) – The number of sample values that can be generated by the experiment that is described by the probability distribution. This value must be correctly set for the probabilities of the sample values to sum to one. If bins is not specified, it defaults to freqdist.B().

discount()

Return the ratio by which counts are discounted on average: c*/c

Return type

float

freqdists()

Return the list of frequency distributions that this ProbDist is based on.

Return type

list(FreqDist)

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.DictionaryConditionalProbDist

Bases: ConditionalProbDistI

An alternative ConditionalProbDist that simply wraps a dictionary of ProbDists rather than creating these from FreqDists.

__init__(probdist_dict)

Parameters

probdist_dict (dict any -> probdist) – a dictionary containing the probdists indexed by the conditions

class nltk.probability.DictionaryProbDist

Bases: ProbDistI

A probability distribution whose probabilities are directly specified by a given dictionary. The given dictionary maps samples to probabilities.

__init__(prob_dict=None, log=False, normalize=False)

Construct a new probability distribution from the given dictionary, which maps values to probabilities (or to log probabilities, if log is true). If normalize is true, then the probability values are scaled by a constant factor such that they sum to 1.

If called without arguments, the resulting probability distribution assigns zero probability to all values.

logprob(sample)

Return the base 2 logarithm of the probability for a given sample.

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.ELEProbDist

Bases: LidstoneProbDist

The expected likelihood estimate for the probability distribution of the experiment used to generate a frequency distribution. The “expected likelihood estimate” approximates the probability of a sample with count c from an experiment with N outcomes and B bins as (c+0.5)/(N+B/2). This is equivalent to adding 0.5 to the count for each bin, and taking the maximum likelihood estimate of the resulting frequency distribution.

__init__(freqdist, bins=None)

Use the expected likelihood estimate to create a probability distribution for the experiment used to generate freqdist.

Parameters

• freqdist (FreqDist) – The frequency distribution that the probability estimates should be based on.

• bins (int) – The number of sample values that can be generated by the experiment that is described by the probability distribution. This value must be correctly set for the probabilities of the sample values to sum to one. If bins is not specified, it defaults to freqdist.B().

class nltk.probability.FreqDist

Bases: Counter

A frequency distribution for the outcomes of an experiment. A frequency distribution records the number of times each outcome of an experiment has occurred. For example, a frequency distribution could be used to record the frequency of each word type in a document. Formally, a frequency distribution can be defined as a function mapping from each sample to the number of times that sample occurred as an outcome.

Frequency distributions are generally constructed by running a number of experiments, and incrementing the count for a sample every time it is an outcome of an experiment. For example, the following code will produce a frequency distribution that encodes how often each word occurs in a text:

>>> from nltk.tokenize import word_tokenize
>>> from nltk.probability import FreqDist
>>> sent = 'This is an example sentence'
>>> fdist = FreqDist()
>>> for word in word_tokenize(sent):
...    fdist[word.lower()] += 1

An equivalent way to do this is with the initializer:

>>> fdist = FreqDist(word.lower() for word in word_tokenize(sent))

B()

Return the total number of sample values (or “bins”) that have counts greater than zero. For the total number of sample outcomes recorded, use FreqDist.N(). (FreqDist.B() is the same as len(FreqDist).)

Return type

int

N()

Return the total number of sample outcomes that have been recorded by this FreqDist. For the number of unique sample values (or bins) with counts greater than zero, use FreqDist.B().

Return type

int

Nr(r, bins=None)

__init__(samples=None)

Construct a new frequency distribution. If samples is given, then the frequency distribution will be initialized with the count of each object in samples; otherwise, it will be initialized to be empty.

In particular, FreqDist() returns an empty frequency distribution; and FreqDist(samples) first creates an empty frequency distribution, and then calls update with the list samples.

Parameters

samples (Sequence) – The samples to initialize the frequency distribution with.

copy()

Create a copy of this frequency distribution.

Return type

FreqDist

freq(sample)

Return the frequency of a given sample. The frequency of a sample is defined as the count of that sample divided by the total number of sample outcomes that have been recorded by this FreqDist. The count of a sample is defined as the number of times that sample outcome was recorded by this FreqDist. Frequencies are always real numbers in the range [0, 1].

Parameters

sample (any) – the sample whose frequency should be returned.

Return type

float

hapaxes()

Return a list of all samples that occur once (hapax legomena)

Return type

list

max()

Return the sample with the greatest number of outcomes in this frequency distribution. If two or more samples have the same number of outcomes, return one of them; which sample is returned is undefined. If no outcomes have occurred in this frequency distribution, return None.

Returns

The sample with the maximum number of outcomes in this frequency distribution.

Return type

any or None

pformat(maxlen=10)

Return a string representation of this FreqDist.

Parameters

maxlen (int) – The maximum number of items to display

Return type

string

plot(*args, title='', cumulative=False, percents=False, show=True, **kwargs)

Plot samples from the frequency distribution displaying the most frequent sample first. If an integer parameter is supplied, stop after this many samples have been plotted. For a cumulative plot, specify cumulative=True. Additional **kwargs are passed to matplotlib’s plot function. (Requires Matplotlib to be installed.)

Parameters

• title (str) – The title for the graph.

• cumulative (bool) – Whether the plot is cumulative. (default = False)

• percents (bool) – Whether the plot uses percents instead of counts. (default = False)

• show (bool) – Whether to show the plot, or only return the ax.

pprint(maxlen=10, stream=None)

Print a string representation of this FreqDist to ‘stream’

Parameters

• maxlen (int) – The maximum number of items to print

• stream – The stream to print to. stdout by default

r_Nr(bins=None)

Return the dictionary mapping r to Nr, the number of samples with frequency r, where Nr > 0.

Parameters

bins (int) – The number of possible sample outcomes. bins is used to calculate Nr(0). In particular, Nr(0) is bins-self.B(). If bins is not specified, it defaults to self.B() (so Nr(0) will be 0).

Return type

int

setdefault(key, val)

Override Counter.setdefault() to invalidate the cached N

tabulate(*args, **kwargs)

Tabulate the given samples from the frequency distribution (cumulative), displaying the most frequent sample first. If an integer parameter is supplied, stop after this many samples have been plotted.

Parameters

• samples (list) – The samples to plot (default is all samples)

• cumulative – A flag to specify whether the freqs are cumulative (default = False)

update(*args, **kwargs)

Override Counter.update() to invalidate the cached N

class nltk.probability.HeldoutProbDist

Bases: ProbDistI

The heldout estimate for the probability distribution of the experiment used to generate two frequency distributions. These two frequency distributions are called the “heldout frequency distribution” and the “base frequency distribution.” The “heldout estimate” uses uses the “heldout frequency distribution” to predict the probability of each sample, given its frequency in the “base frequency distribution”.

In particular, the heldout estimate approximates the probability for a sample that occurs r times in the base distribution as the average frequency in the heldout distribution of all samples that occur r times in the base distribution.

This average frequency is Tr[r]/(Nr[r].N), where:

• Tr[r] is the total count in the heldout distribution for all samples that occur r times in the base distribution.

• Nr[r] is the number of samples that occur r times in the base distribution.

• N is the number of outcomes recorded by the heldout frequency distribution.

In order to increase the efficiency of the prob member function, Tr[r]/(Nr[r].N) is precomputed for each value of r when the HeldoutProbDist is created.

Variables

• _estimate – A list mapping from r, the number of times that a sample occurs in the base distribution, to the probability estimate for that sample. _estimate[r] is calculated by finding the average frequency in the heldout distribution of all samples that occur r times in the base distribution. In particular, _estimate[r] = Tr[r]/(Nr[r].N).

• _max_r – The maximum number of times that any sample occurs in the base distribution. _max_r is used to decide how large _estimate must be.

SUM_TO_ONE = False

True if the probabilities of the samples in this probability distribution will always sum to one.

__init__(base_fdist, heldout_fdist, bins=None)

Use the heldout estimate to create a probability distribution for the experiment used to generate base_fdist and heldout_fdist.

Parameters

• base_fdist (FreqDist) – The base frequency distribution.

• heldout_fdist (FreqDist) – The heldout frequency distribution.

• bins (int) – The number of sample values that can be generated by the experiment that is described by the probability distribution. This value must be correctly set for the probabilities of the sample values to sum to one. If bins is not specified, it defaults to freqdist.B().

base_fdist()

Return the base frequency distribution that this probability distribution is based on.

Return type

FreqDist

discount()

Return the ratio by which counts are discounted on average: c*/c

Return type

float

heldout_fdist()

Return the heldout frequency distribution that this probability distribution is based on.

Return type

FreqDist

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.ImmutableProbabilisticMixIn

Bases: ProbabilisticMixIn

set_logprob(prob)

Set the log probability associated with this object to logprob. I.e., set the probability associated with this object to 2**(logprob).

Parameters

logprob (float) – The new log probability

set_prob(prob)

Set the probability associated with this object to prob.

Parameters

prob (float) – The new probability

class nltk.probability.KneserNeyProbDist

Bases: ProbDistI

Kneser-Ney estimate of a probability distribution. This is a version of back-off that counts how likely an n-gram is provided the n-1-gram had been seen in training. Extends the ProbDistI interface, requires a trigram FreqDist instance to train on. Optionally, a different from default discount value can be specified. The default discount is set to 0.75.

__init__(freqdist, bins=None, discount=0.75)

Parameters

• freqdist (FreqDist) – The trigram frequency distribution upon which to base the estimation

• bins (int or float) – Included for compatibility with nltk.tag.hmm

• discount (float (preferred, but can be set to int)) – The discount applied when retrieving counts of trigrams

discount()

Return the value by which counts are discounted. By default set to 0.75.

Return type

float

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(trigram)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

set_discount(discount)

Set the value by which counts are discounted to the value of discount.

Parameters

discount (float (preferred, but int possible)) – the new value to discount counts by

Return type

None

class nltk.probability.LaplaceProbDist

Bases: LidstoneProbDist

The Laplace estimate for the probability distribution of the experiment used to generate a frequency distribution. The “Laplace estimate” approximates the probability of a sample with count c from an experiment with N outcomes and B bins as (c+1)/(N+B). This is equivalent to adding one to the count for each bin, and taking the maximum likelihood estimate of the resulting frequency distribution.

__init__(freqdist, bins=None)

Use the Laplace estimate to create a probability distribution for the experiment used to generate freqdist.

Parameters

• freqdist (FreqDist) – The frequency distribution that the probability estimates should be based on.

• bins (int) – The number of sample values that can be generated by the experiment that is described by the probability distribution. This value must be correctly set for the probabilities of the sample values to sum to one. If bins is not specified, it defaults to freqdist.B().

class nltk.probability.LidstoneProbDist

Bases: ProbDistI

The Lidstone estimate for the probability distribution of the experiment used to generate a frequency distribution. The “Lidstone estimate” is parameterized by a real number gamma, which typically ranges from 0 to 1. The Lidstone estimate approximates the probability of a sample with count c from an experiment with N outcomes and B bins as c+gamma)/(N+B*gamma). This is equivalent to adding gamma to the count for each bin, and taking the maximum likelihood estimate of the resulting frequency distribution.

SUM_TO_ONE = False

True if the probabilities of the samples in this probability distribution will always sum to one.

__init__(freqdist, gamma, bins=None)

Use the Lidstone estimate to create a probability distribution for the experiment used to generate freqdist.

Parameters

• freqdist (FreqDist) – The frequency distribution that the probability estimates should be based on.

• gamma (float) – A real number used to parameterize the estimate. The Lidstone estimate is equivalent to adding gamma to the count for each bin, and taking the maximum likelihood estimate of the resulting frequency distribution.

• bins (int) – The number of sample values that can be generated by the experiment that is described by the probability distribution. This value must be correctly set for the probabilities of the sample values to sum to one. If bins is not specified, it defaults to freqdist.B().

discount()

Return the ratio by which counts are discounted on average: c*/c

Return type

float

freqdist()

Return the frequency distribution that this probability distribution is based on.

Return type

FreqDist

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.MLEProbDist

Bases: ProbDistI

The maximum likelihood estimate for the probability distribution of the experiment used to generate a frequency distribution. The “maximum likelihood estimate” approximates the probability of each sample as the frequency of that sample in the frequency distribution.

__init__(freqdist, bins=None)

Use the maximum likelihood estimate to create a probability distribution for the experiment used to generate freqdist.

Parameters

freqdist (FreqDist) – The frequency distribution that the probability estimates should be based on.

freqdist()

Return the frequency distribution that this probability distribution is based on.

Return type

FreqDist

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.MutableProbDist

Bases: ProbDistI

An mutable probdist where the probabilities may be easily modified. This simply copies an existing probdist, storing the probability values in a mutable dictionary and providing an update method.

__init__(prob_dist, samples, store_logs=True)

Creates the mutable probdist based on the given prob_dist and using the list of samples given. These values are stored as log probabilities if the store_logs flag is set.

Parameters

• prob_dist (ProbDist) – the distribution from which to garner the probabilities

• samples (sequence of any) – the complete set of samples

• store_logs (bool) – whether to store the probabilities as logarithms

logprob(sample)

Return the base 2 logarithm of the probability for a given sample.

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

update(sample, prob, log=True)

Update the probability for the given sample. This may cause the object to stop being the valid probability distribution - the user must ensure that they update the sample probabilities such that all samples have probabilities between 0 and 1 and that all probabilities sum to one.

Parameters

• sample (any) – the sample for which to update the probability

• prob (float) – the new probability

• log (bool) – is the probability already logged

class nltk.probability.ProbDistI

Bases: object

A probability distribution for the outcomes of an experiment. A probability distribution specifies how likely it is that an experiment will have any given outcome. For example, a probability distribution could be used to predict the probability that a token in a document will have a given type. Formally, a probability distribution can be defined as a function mapping from samples to nonnegative real numbers, such that the sum of every number in the function’s range is 1.0. A ProbDist is often used to model the probability distribution of the experiment used to generate a frequency distribution.

SUM_TO_ONE = True

True if the probabilities of the samples in this probability distribution will always sum to one.

abstract __init__()

Classes inheriting from ProbDistI should implement __init__.

discount()

Return the ratio by which counts are discounted on average: c*/c

Return type

float

generate()

Return a randomly selected sample from this probability distribution. The probability of returning each sample samp is equal to self.prob(samp).

logprob(sample)

Return the base 2 logarithm of the probability for a given sample.

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

abstract max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

abstract prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

abstract samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.ProbabilisticMixIn

Bases: object

A mix-in class to associate probabilities with other classes (trees, rules, etc.). To use the ProbabilisticMixIn class, define a new class that derives from an existing class and from ProbabilisticMixIn. You will need to define a new constructor for the new class, which explicitly calls the constructors of both its parent classes. For example:

>>> from nltk.probability import ProbabilisticMixIn
>>> class A:
...     def __init__(self, x, y): self.data = (x,y)
...
>>> class ProbabilisticA(A, ProbabilisticMixIn):
...     def __init__(self, x, y, **prob_kwarg):
...         A.__init__(self, x, y)
...         ProbabilisticMixIn.__init__(self, **prob_kwarg)

See the documentation for the ProbabilisticMixIn constructor<__init__> for information about the arguments it expects.

You should generally also redefine the string representation methods, the comparison methods, and the hashing method.

__init__(**kwargs)

Initialize this object’s probability. This initializer should be called by subclass constructors. prob should generally be the first argument for those constructors.

Parameters

• prob (float) – The probability associated with the object.

• logprob (float) – The log of the probability associated with the object.

logprob()

Return log(p), where p is the probability associated with this object.

Return type

float

prob()

Return the probability associated with this object.

Return type

float

set_logprob(logprob)

Set the log probability associated with this object to logprob. I.e., set the probability associated with this object to 2**(logprob).

Parameters

logprob (float) – The new log probability

set_prob(prob)

Set the probability associated with this object to prob.

Parameters

prob (float) – The new probability

class nltk.probability.SimpleGoodTuringProbDist

Bases: ProbDistI

SimpleGoodTuring ProbDist approximates from frequency to frequency of frequency into a linear line under log space by linear regression. Details of Simple Good-Turing algorithm can be found in:

• Good Turing smoothing without tears” (Gale & Sampson 1995), Journal of Quantitative Linguistics, vol. 2 pp. 217-237.

• “Speech and Language Processing (Jurafsky & Martin), 2nd Edition, Chapter 4.5 p103 (log(Nc) = a + b*log(c))

• https://www.grsampson.net/RGoodTur.html

Given a set of pair (xi, yi), where the xi denotes the frequency and yi denotes the frequency of frequency, we want to minimize their square variation. E(x) and E(y) represent the mean of xi and yi.

• slope: b = sigma ((xi-E(x)(yi-E(y))) / sigma ((xi-E(x))(xi-E(x)))

• intercept: a = E(y) - b.E(x)

SUM_TO_ONE = False

True if the probabilities of the samples in this probability distribution will always sum to one.

__init__(freqdist, bins=None)

Parameters

• freqdist (FreqDist) – The frequency counts upon which to base the estimation.

• bins (int) – The number of possible event types. This must be larger than the number of bins in the freqdist. If None, then it’s assumed to be equal to freqdist.B() + 1

check()

discount()

This function returns the total mass of probability transfers from the seen samples to the unseen samples.

find_best_fit(r, nr)

Use simple linear regression to tune parameters self._slope and self._intercept in the log-log space based on count and Nr(count) (Work in log space to avoid floating point underflow.)

freqdist()

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the sample’s probability.

Parameters

sample (str) – sample of the event

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

smoothedNr(r)

Return the number of samples with count r.

Parameters

r (int) – The amount of frequency.

Return type

float

class nltk.probability.UniformProbDist

Bases: ProbDistI

A probability distribution that assigns equal probability to each sample in a given set; and a zero probability to all other samples.

__init__(samples)

Construct a new uniform probability distribution, that assigns equal probability to each sample in samples.

Parameters

samples (list) – The samples that should be given uniform probability.

Raises

ValueError – If samples is empty.

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

class nltk.probability.WittenBellProbDist

Bases: ProbDistI

The Witten-Bell estimate of a probability distribution. This distribution allocates uniform probability mass to as yet unseen events by using the number of events that have only been seen once. The probability mass reserved for unseen events is equal to T / (N + T) where T is the number of observed event types and N is the total number of observed events. This equates to the maximum likelihood estimate of a new type event occurring. The remaining probability mass is discounted such that all probability estimates sum to one, yielding:

• p = T / Z (N + T), if count = 0

• p = c / (N + T), otherwise

__init__(freqdist, bins=None)

Creates a distribution of Witten-Bell probability estimates. This distribution allocates uniform probability mass to as yet unseen events by using the number of events that have only been seen once. The probability mass reserved for unseen events is equal to T / (N + T) where T is the number of observed event types and N is the total number of observed events. This equates to the maximum likelihood estimate of a new type event occurring. The remaining probability mass is discounted such that all probability estimates sum to one, yielding:

• p = T / Z (N + T), if count = 0

• p = c / (N + T), otherwise

The parameters T and N are taken from the freqdist parameter (the B() and N() values). The normalizing factor Z is calculated using these values along with the bins parameter.

Parameters

• freqdist (FreqDist) – The frequency counts upon which to base the estimation.

• bins (int) – The number of possible event types. This must be at least as large as the number of bins in the freqdist. If None, then it’s assumed to be equal to that of the freqdist

discount()

Return the ratio by which counts are discounted on average: c*/c

Return type

float

freqdist()

max()

Return the sample with the greatest probability. If two or more samples have the same probability, return one of them; which sample is returned is undefined.

Return type

any

prob(sample)

Return the probability for a given sample. Probabilities are always real numbers in the range [0, 1].

Parameters

sample (any) – The sample whose probability should be returned.

Return type

float

samples()

Return a list of all samples that have nonzero probabilities. Use prob to find the probability of each sample.

Return type

list

nltk.probability.add_logs(logx, logy)

Given two numbers logx = log(x) and logy = log(y), return log(x+y). Conceptually, this is the same as returning log(2**(logx)+2**(logy)), but the actual implementation avoids overflow errors that could result from direct computation.

nltk.probability.entropy(pdist)

nltk.probability.log_likelihood(test_pdist, actual_pdist)

nltk.probability.sum_logs(logs)