nltk.corpus.reader.nkjp module

class nltk.corpus.reader.nkjp.NKJPCorpusReader

Bases: XMLCorpusReader

HEADER_MODE = 2

RAW_MODE = 3

SENTS_MODE = 1

WORDS_MODE = 0

__init__(root, fileids='.*')

Corpus reader designed to work with National Corpus of Polish. See http://nkjp.pl/ for more details about NKJP. use example: import nltk import nkjp from nkjp import NKJPCorpusReader x = NKJPCorpusReader(root=’/home/USER/nltk_data/corpora/nkjp/’, fileids=’’) # obtain the whole corpus x.header() x.raw() x.words() x.tagged_words(tags=[‘subst’, ‘comp’]) #Link to find more tags: nkjp.pl/poliqarp/help/ense2.html x.sents() x = NKJPCorpusReader(root=’/home/USER/nltk_data/corpora/nkjp/’, fileids=’Wilk*’) # obtain particular file(s) x.header(fileids=[‘WilkDom’, ‘/home/USER/nltk_data/corpora/nkjp/WilkWilczy’]) x.tagged_words(fileids=[‘WilkDom’, ‘/home/USER/nltk_data/corpora/nkjp/WilkWilczy’], tags=[‘subst’, ‘comp’])

add_root(fileid)

Add root if necessary to specified fileid.

fileids()

Returns a list of file identifiers for the fileids that make up this corpus.

get_paths()

header(fileids=None, **kwargs)

Returns header(s) of specified fileids.

raw(fileids=None, **kwargs)

Returns words in specified fileids.

sents(fileids=None, **kwargs)

Returns sentences in specified fileids.

tagged_words(fileids=None, **kwargs)

Call with specified tags as a list, e.g. tags=[‘subst’, ‘comp’]. Returns tagged words in specified fileids.

words(fileids=None, **kwargs)

Returns words in specified fileids.

class nltk.corpus.reader.nkjp.NKJPCorpus_Header_View

Bases: XMLCorpusView

__init__(filename, **kwargs)

HEADER_MODE A stream backed corpus view specialized for use with header.xml files in NKJP corpus.

handle_elt(elt, context)

Convert an element into an appropriate value for inclusion in the view. Unless overridden by a subclass or by the elt_handler constructor argument, this method simply returns elt.

Returns

The view value corresponding to elt.

Parameters

• elt (ElementTree) – The element that should be converted.

• context (str) – A string composed of element tags separated by forward slashes, indicating the XML context of the given element. For example, the string 'foo/bar/baz' indicates that the element is a baz element whose parent is a bar element and whose grandparent is a top-level foo element.

handle_query()

class nltk.corpus.reader.nkjp.NKJPCorpus_Morph_View

Bases: XMLCorpusView

A stream backed corpus view specialized for use with ann_morphosyntax.xml files in NKJP corpus.

__init__(filename, **kwargs)

Create a new corpus view based on a specified XML file.

Note that the XMLCorpusView constructor does not take an encoding argument, because the unicode encoding is specified by the XML files themselves.

Parameters

• tagspec (str) – A tag specification, indicating what XML elements should be included in the view. Each non-nested element that matches this specification corresponds to one item in the view.

• elt_handler –

  A function used to transform each element to a value for the view. If no handler is specified, then self.handle_elt() is called, which returns the element as an ElementTree object. The signature of elt_handler is:

  elt_handler(elt, tagspec) -> value
  

handle_elt(elt, context)

Convert an element into an appropriate value for inclusion in the view. Unless overridden by a subclass or by the elt_handler constructor argument, this method simply returns elt.

Returns

The view value corresponding to elt.

Parameters

• elt (ElementTree) – The element that should be converted.

• context (str) – A string composed of element tags separated by forward slashes, indicating the XML context of the given element. For example, the string 'foo/bar/baz' indicates that the element is a baz element whose parent is a bar element and whose grandparent is a top-level foo element.

handle_query()

class nltk.corpus.reader.nkjp.NKJPCorpus_Segmentation_View

Bases: XMLCorpusView

A stream backed corpus view specialized for use with ann_segmentation.xml files in NKJP corpus.

__init__(filename, **kwargs)

Create a new corpus view based on a specified XML file.

Note that the XMLCorpusView constructor does not take an encoding argument, because the unicode encoding is specified by the XML files themselves.

Parameters

• tagspec (str) – A tag specification, indicating what XML elements should be included in the view. Each non-nested element that matches this specification corresponds to one item in the view.

• elt_handler –

  A function used to transform each element to a value for the view. If no handler is specified, then self.handle_elt() is called, which returns the element as an ElementTree object. The signature of elt_handler is:

  elt_handler(elt, tagspec) -> value
  

get_segm_id(example_word)

get_sent_beg(beg_word)

get_sent_end(end_word)

get_sentences(sent_segm)

handle_elt(elt, context)

Convert an element into an appropriate value for inclusion in the view. Unless overridden by a subclass or by the elt_handler constructor argument, this method simply returns elt.

Returns

The view value corresponding to elt.

Parameters

• elt (ElementTree) – The element that should be converted.

• context (str) – A string composed of element tags separated by forward slashes, indicating the XML context of the given element. For example, the string 'foo/bar/baz' indicates that the element is a baz element whose parent is a bar element and whose grandparent is a top-level foo element.

handle_query()

remove_choice(segm)

class nltk.corpus.reader.nkjp.NKJPCorpus_Text_View

Bases: XMLCorpusView

A stream backed corpus view specialized for use with text.xml files in NKJP corpus.

RAW_MODE = 1

SENTS_MODE = 0

__init__(filename, **kwargs)

Create a new corpus view based on a specified XML file.

Note that the XMLCorpusView constructor does not take an encoding argument, because the unicode encoding is specified by the XML files themselves.

Parameters

• tagspec (str) – A tag specification, indicating what XML elements should be included in the view. Each non-nested element that matches this specification corresponds to one item in the view.

• elt_handler –

  A function used to transform each element to a value for the view. If no handler is specified, then self.handle_elt() is called, which returns the element as an ElementTree object. The signature of elt_handler is:

  elt_handler(elt, tagspec) -> value
  

get_segm_id(elt)

handle_elt(elt, context)

Convert an element into an appropriate value for inclusion in the view. Unless overridden by a subclass or by the elt_handler constructor argument, this method simply returns elt.

Returns

The view value corresponding to elt.

Parameters

• elt (ElementTree) – The element that should be converted.

• context (str) – A string composed of element tags separated by forward slashes, indicating the XML context of the given element. For example, the string 'foo/bar/baz' indicates that the element is a baz element whose parent is a bar element and whose grandparent is a top-level foo element.

handle_query()

read_block(stream, tagspec=None, elt_handler=None)

Returns text as a list of sentences.

class nltk.corpus.reader.nkjp.XML_Tool

Bases: object

Helper class creating xml file to one without references to nkjp: namespace. That’s needed because the XMLCorpusView assumes that one can find short substrings of XML that are valid XML, which is not true if a namespace is declared at top level

__init__(root, filename)

build_preprocessed_file()

remove_preprocessed_file()