DKPro Core™ Dataset Reference

73,853 tokens in 28 Arabic Wikipedia articles hand-annotated for named entities.

(This description has been partially copied from the corpus website).

This is a subset of the Ancora corpus (see http://clic.ub.edu/ancora) which was used in the CoNLL-2009 shared task on extracting syntactic and semantic Dependencies in multiple languages.

496,672 lexical tokens; training: 390,302; development: 53,015; test: 53,355

(This description has been partially copied from the README file included with the corpus).

The Coptic Treebank from the Coptic SCRIPTORIUM corpora (http://copticscriptorium.org/).

Version 1 (the directory "da") was orginally called the Danish Dependency Treebank. It was used in the CoNLL 2006 shared task on dependency parsing, but has since been updated with bug fixes and an improved CoNLL conversion which includes a decomposition of the PAROLE part-of-speech tags into the underlying features for number, gender, etc.).

(This description has been sourced from the corpus README file).

This dataset contains the basic information regarding the German corpus provided for the CoNLL-2009 shared task on "Syntactic and Semantic Dependencies in Multiple Languages" (http://ufal.mff.cuni.cz/conll2009-st/). The data of this distribution is derived from the TIGER Treebank and the SALSA Corpus, converted into the syntactic and semantic dependencies compatible with the CoNLL-2009 shared task.

(This description has been sourced from the README file included with the corpus).

The GermEval 2014 NER Shared Task builds on a new dataset with German Named Entity annotation with the following properties:

(This description has been sourced from the dataset website).

Contains annotated text from the German technical news website www.heise.de.

The Named Entity Model for German, Politics (NEMGP) is a collection of texts from Wikipedia and WikiNews, manually annotated with named entity information.

(This description has been sourced from the dataset website).

The Ancient Greek and Latin Dependency Treebank (AGLDT) is the earliest treebank for Ancient Greek and Latin. The project started at Tufts University in 2006 and is currently under development and maintenance at Leipzig University-Tufts University.

(This description has been sourced from the dataset website).

This version derives directly from

"A Standard Corpus of Present-Day Edited American English, for use with Digital Computers." by W. N. Francis and H. Kucera (1964) Department of Linguistics, Brown University Providence, Rhode Island, USA Revised 1971, Revised and Amplified 1979 http://www.hit.uib.no/icame/brown/bcm.html

as distributed with NLTK (version 0.9.2)

(This description has been taken from the README file included with the corpus).

This is the data from the CoNLL-2000 shared task on text chunking. The data consists of the same partitions of the Wall Street Journal corpus (WSJ) as the widely used data for noun phrase chunking: sections 15-18 as training data (211727 tokens) and section 20 as test data (47377 tokens). The annotation of the data has been derived from the WSJ corpus by a program written by Sabine Buchholz from Tilburg University, The Netherlands. Instead of using the part-of-speech tags of the WSJ corpus, the data set used tags generated by the Brill tagger.

(This description has been partially copied from the corpus website).

English Frame and Role Annotations.

(This description has been sourced from the README file included with the corpus).

English Frame and Role Annotations.

(This description has been sourced from the README file included with the corpus).

German Frame and Role Annotations.

(This description has been sourced from the README file included with the corpus).

GUM is an open source multilayer corpus of richly annotated web texts from eight text types. The corpus is collected and expanded by students as part of the curriculum in LING-367 Computational Corpus Linguistics at Georgetown University. The selection of text types is meant to represent different communicative purposes, while coming from sources that are readily and openly available (mostly Creative Commons licenses), so that new texts can be annotated and published with ease.

(This description has been sourced from the dataset website).

This dataset contains release versions of the Georgetown University Multilayer Corpus (GUM), a corpus of English texts from four text types (interviews, news, travel guides, instructional texts). The corpus is created as part of the course LING-367 (Computational Corpus Linguistics) at Georgetown University. For more details see: http://corpling.uis.georgetown.edu/gum.

The dep layer gives a dependency syntax analysis according to the Stanford Dependencies manual. This layer was intially produced using the Stanford Parser and then manually corrected using the Arborator collaborative syntax annotation software. For the annotation project we used non-collapsed dependencies, and dependencies for punctuation tokens have been removed.

(This description has been sourced from the dataset website).

This dataset contains release versions of the Georgetown University Multilayer Corpus (GUM), a corpus of English texts from four text types (interviews, news, travel guides, instructional texts). The corpus is created as part of the course LING-367 (Computational Corpus Linguistics) at Georgetown University. For more details see: http://corpling.uis.georgetown.edu/gum.

The dep layer gives a dependency syntax analysis according to the Stanford Dependencies manual. This layer was intially produced using the Stanford Parser and then manually corrected using the Arborator collaborative syntax annotation software. For the annotation project we used non-collapsed dependencies, and dependencies for punctuation tokens have been removed.

(This description has been sourced from the dataset website).

The CPOS column of the files contains an extended POS tagset as it is used by the English TreeTagger models. The POS column contains the regular PTB tagset.

This dataset contains release versions of the Georgetown University Multilayer Corpus (GUM), a corpus of English texts from four text types (interviews, news, travel guides, instructional texts). The corpus is created as part of the course LING-367 (Computational Corpus Linguistics) at Georgetown University. For more details see: http://corpling.uis.georgetown.edu/gum.

The dep layer gives a dependency syntax analysis according to the Stanford Dependencies manual. This layer was intially produced using the Stanford Parser and then manually corrected using the Arborator collaborative syntax annotation software. For the annotation project we used non-collapsed dependencies, and dependencies for punctuation tokens have been removed.

(This description has been sourced from the dataset website).

This dataset contains release versions of the Georgetown University Multilayer Corpus (GUM), a corpus of English texts from four text types (interviews, news, travel guides, instructional texts). The corpus is created as part of the course LING-367 (Computational Corpus Linguistics) at Georgetown University. For more details see: http://corpling.uis.georgetown.edu/gum.

The dep layer gives a dependency syntax analysis according to the Stanford Dependencies manual. This layer was intially produced using the Stanford Parser and then manually corrected using the Arborator collaborative syntax annotation software. For the annotation project we used non-collapsed dependencies, and dependencies for punctuation tokens have been removed.

(This description has been sourced from the dataset website).

The CPOS column of the files contains an extended POS tagset as it is used by the English TreeTagger models. The POS column contains the regular PTB tagset.

Note that this dataset does not include the Reddit data as it can only be obtained by running a Python script which comes with GUM.

GloVe is an unsupervised learning algorithm for obtaining vector representations for words. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase interesting linear substructures of the word vector space.

(This description has been sourced from the dataset website).

The Manually Annotated Sub-Corpus (MASC) consists of approximately 500,000 words of contemporary American English written and spoken data drawn from the Open American National Corpus (OANC).

A 40K subset of MASC1 data with annotations for Penn Treebank syntactic dependencies and semantic dependencies from NomBank and PropBank in CONLL IOB format. This data set was used in the CoNLL 2008 shared task on Joint Parsing of Syntactic and Semantic Dependencies.

(This description has been sourced from the dataset website).

The NAIST-NTT Ted Talk Treebank is a manually annotated treebank of TED talks that was created through a joint research project of NAIST and the NTT CS Lab. All treebank annotation follows the Penn Treebank standard.

(This description has been sourced from the corpus website/README file in the corpus).

Distributional similarity clusters that can be used e.g. with the Stanford POS tagger.

These clusters are a feature extracted from larger, untagged text which clusters the words into similar classes.

(This description has been sourced from the dataset website).

Universal Dependencies is a project that seeks to develop cross-linguistically consistent treebank annotation for many languages, with the goal of facilitating multilingual parser development, cross-lingual learning, and parsing research from a language typology perspective. The annotation scheme is based on (universal) Stanford dependencies (de Marneffe et al., 2006, 2008, 2014), Google universal part-of-speech tags (Petrov et al., 2012), and the Interset interlingua for morphosyntactic tagsets (Zeman, 2008).

(This description has been sourced from the dataset website).

This is the data from the CoNLL-2002 shared task on language independent named entity recognition. The Spanish data is a collection of news wire articles made available by the Spanish EFE News Agency. The articles are from May 2000. The annotation was carried out by the TALP Research Center (http://www.talp.upc.es/) of the Technical University of Catalonia (UPC) and the Center of Language and Computation (CLiC, http://clic.fil.ub.es/) of the University of Barcelona (UB), and funded by the European Commission through the NAMIC project (IST-1999-12392).

(This description has been sourced from the README file included with the corpus).

This is a subset of the Ancora corpus (see http://clic.ub.edu/ancora) which was used in the CoNLL-2009 shared task on extracting syntactic and semantic Dependencies in multiple languages.

528,440 lexical tokens; training: 427,442; development: 50,368; test: 50,630

(This description has been partially copied from the README file included with the corpus).

IULA Spanish LSP Treebank is an Spanish treebank containing the syntactic annotation of 42,000 sentences (almost 590,000 tokens). It has been developed within the frame of Metanet4U project (Enhancing the European Linguistic Infrastructure, GA 270893).

The sentences in IULA Spanish LSP Treebank are extracted from the Corpus Tècnic de l’IULA, a collection of written texts from the fields of Law, Economy, Genomics, Medicine, and Environment, as well as a contrastive corpus from the press.

(This description has been sourced from the corpus website).

Uppsala Persian Dependency Treebank (UPDT) (Seraji, 2015, Chapter 5, pp. 97-146) is a dependency-based syntactically annotated corpus.

(This description has been sourced from the dataset website).

Deep-sequoia is a corpus of French sentences annotated with both surface and deep syntactic dependency structures.

(This description has been sourced from the dataset website).

The corpus is based on the Croatian part of the SETimes parallel corpus.

(This description has been sourced from the corpus website).

The treebank is a result of an effort in providing free-culture language resources for Croatian by the NLP group at FF Zagreb.

(This description has been sourced from the corpus website).

TUT is a morpho-syntactically annotated collection of Italian sentences, which includes texts from different text genres and domains, released in several annotation formats.

(This description has been sourced from the corpus website).

This file contains the basic information regarding the Japanese corpus provided for the CoNLL-2009 shared task on "Syntactic and Semantic Dependencies in Multiple Languages". The current version corresponds to the release of the training data sets.

The data of this distribution uses portions of the Kyoto University Text Corpus 4.0. The Kyoto University Text Corpus is freely available at http://nlp.kuee.kyoto-u.ac.jp/nl-resource/corpus-e.html.

(This description has been sourced from the README file included with the corpus).

The Ancient Greek and Latin Dependency Treebank (AGLDT) is the earliest treebank for Ancient Greek and Latin. The project started at Tufts University in 2006 and is currently under development and maintenance at Leipzig University-Tufts University.

(This description has been sourced from the dataset website).

The Norwegian Dependency Treebank (NDT) consists of text which is manually annotated with morphological features, syntactic functions and hierarchical structure. The formalism used for the syntactic annotation is dependency grammar. With a few exceptions, the syntactic analysis follows Norsk referensegrammatikk ‘Norwegian Reference Grammar'.

(This description has been sourced from the dataset website).

The FinnTreeBank project is creating a treebank and a parsebank for Finnish. This work is licensed under a Creative Commons Attribution 3.0.

The first and second version of the treebank is annotated by hand and based on 17.000 model senctences in the Large Grammar of Finnish VISK - Iso Suomen Kielioppi. Brief samples of text from other sources, e.g. news items and literature, are also available in the second version. A parsebank for Finnish based on the Europarl and the JRC-Aquis will be available in June 2012.

(This description has been sourced from the dataset website).

Training and test datasets for Dutch in retagged CoNLL format. The data was converted from Alpino XML into CoNLL format based on an adapted version of Erwin Marsi’s conversion software, but PoS tags were replaced by automatically assigned Alpino tags.

(This description has been sourced from the corpus website).

This is the data from the CoNLL-2002 shared task on language independent named entity recognition. The Dutch data consist of four editions of the Belgian newspaper "De Morgen" of 2000 (June 2, July 1, August 1 and September 1). The data was annotated as a part of the Atranos project (http://atranos.esat.kuleuven.ac.be/) at the University of Antwerp.

(This description has been sourced from the README file included with the corpus).

The Norwegian Dependency Treebank (NDT) consists of text which is manually annotated with morphological features, syntactic functions and hierarchical structure. The formalism used for the syntactic annotation is dependency grammar. With a few exceptions, the syntactic analysis follows Norsk referensegrammatikk ‘Norwegian Reference Grammar'.

(This description has been sourced from the dataset website).

The Polish constituency treebank (Składnica frazowa), version 0.5. Trees in the Tiger XML format containing only parse trees selected by dendrologists (one interpretation per sentence).

(This description has been sourced from the corpus website).

The dependency treebank (Składnica zależnościowa), version 0.5, is a result of an automatic conversion of manually disambiguated constituency trees into dependency structures.

(This description has been sourced from the corpus website).

This is the Portuguese part of the CONLL-X Shared Task. The was derived from the Floresta Sintá(c)tica Bosque 7.3 by Sabine Buchholz.

(This description has been partially sourced from the README file included with the corpus).

The jos100k corpus contains 100,000 words of sampled paragraphs from the FidaPLUS corpus. It is meant to serve as a reference annotated corpus of Slovene: its manually-validated annotations cover three level of linguistic description.

(This description has been sourced from the corpus website).

The Slovene Dependency Treebank project built a small syntactically annotated corpus of Slovene texts. The corpus was annotated with dependency analyses, taking the Prague Dependecy Treebank as the model. The Slovene Dependency Treebank is annotated with Analytic Tree Structures and contains a part of the morphosyntactically annotated Slovene component of the parallel MULTEXT-East corpus, i.e. the first third of the Slovene translation of the novel "1984" by G. Orwell, containing 30,000 words.

(This description has been sourced from the corpus website).

This is the preliminary release of the Slovene Dependency Treebank, SDT V0.4 which contains the Prague Dependency Treebank-like annotation of the first part of Slovene translation of Orwell’s "1984", taken from the MULTEXT-East parallel corpus, V3.0, c.f. http://ufal.mff.cuni.cz/pdt/ http://nl.ijs.si/ME/V3/ http://nl.ijs.si/ME/V3/doc/index.html#mtev3-doc-div2-id2305296

(This description has been sourced from the corpus website).

Talbanken05 is a modernized version of Talbanken76, a Swedish treebank of roughly 300,000 words, constructed at Lund University in the 1970s. The treebank comes with no guarantee but is freely available for research and educational purposes as long as proper credit is given for the work done to produce the material (both in Lund and in Växjö).

Dep: Dependency structure annotation (CoNLL-X shared task format in UTF-8).

(This description has been sourced from the corpus website).

Talbanken05 is a modernized version of Talbanken76, a Swedish treebank of roughly 300,000 words, constructed at Lund University in the 1970s. The treebank comes with no guarantee but is freely available for research and educational purposes as long as proper credit is given for the work done to produce the material (both in Lund and in Växjö).

DPS: Deepened phrase structure annotation (TIGER-XML encoding in ISO-8859-1)

(This description has been sourced from the corpus website).

Talbanken05 is a modernized version of Talbanken76, a Swedish treebank of roughly 300,000 words, constructed at Lund University in the 1970s. The treebank comes with no guarantee but is freely available for research and educational purposes as long as proper credit is given for the work done to produce the material (both in Lund and in Växjö).

FPS: Flat phrase structure annotation (TIGER-XML encoding in ISO-8859-1)

(This description has been sourced from the corpus website).