de.tudarmstadt.ukp.dkpro.core.io.jwpl

Class WikipediaPageReader

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.resource.ConfigurableResource_ImplBase

org.apache.uima.collection.CollectionReader_ImplBase

org.apache.uima.fit.component.JCasCollectionReader_ImplBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaStandardReaderBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaPageReader

All Implemented Interfaces:

org.apache.uima.collection.base_cpm.BaseCollectionReader, org.apache.uima.collection.CollectionReader, org.apache.uima.resource.ConfigurableResource, org.apache.uima.resource.Resource

Direct Known Subclasses:

WikipediaArticleReader

public class WikipediaPageReader
extends WikipediaStandardReaderBase

Reads all Wikipedia pages in the database (articles, discussions, etc). A parameter controls whether the full article or only the first paragraph is set as the document text. No Redirects or disambiguation pages are regarded, however.

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_ONLY_FIRST_PARAGRAPH If set to true, only the first paragraph instead of the whole article is used.

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaStandardReaderBase

currentArticleIndex, nrOfArticles, outputPlainText, pageBuffer, pageIdFile, pageIdParamArray, pageIter, pageNameFile, pageNameParamArray, PARAM_OUTPUT_PLAIN_TEXT, PARAM_PAGE_BUFFER, PARAM_PAGE_ID_LIST, PARAM_PAGE_TITLE_LIST, PARAM_PATH_TO_PAGE_ID_LIST, PARAM_PATH_TO_PAGE_TITLE_LIST, parser

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

dbconfig, PARAM_CREATE_DATABASE_CONFIG_ANNOTATION, PARAM_DB, PARAM_HOST, PARAM_LANGUAGE, PARAM_PASSWORD, PARAM_USER, wiki

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
WikipediaPageReader()

Method Summary

Modifier and Type	Method and Description
protected String	getPlainDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)
void	initialize(org.apache.uima.UimaContext context)
protected boolean	isValidPage(de.tudarmstadt.ukp.wikipedia.api.Page page)

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaStandardReaderBase

getDocumentText, getNext, getPage, getProgress, hasNext

Methods inherited from class org.apache.uima.fit.component.JCasCollectionReader_ImplBase

close, getLogger, getNext, initialize

Methods inherited from class org.apache.uima.collection.CollectionReader_ImplBase

destroy, getCasInitializer, getProcessingResourceMetaData, initialize, isConsuming, reconfigure, setCasInitializer, typeSystemInit

Methods inherited from class org.apache.uima.resource.ConfigurableResource_ImplBase

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.uima.resource.ConfigurableResource

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from interface org.apache.uima.resource.Resource

getMetaData, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger

Field Detail

PARAM_ONLY_FIRST_PARAGRAPH

public static final String PARAM_ONLY_FIRST_PARAGRAPH

If set to true, only the first paragraph instead of the whole article is used.

See Also:

Constant Field Values

Constructor Detail

WikipediaPageReader

public WikipediaPageReader()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext context)
                throws org.apache.uima.resource.ResourceInitializationException

Overrides:

initialize in class WikipediaStandardReaderBase

Throws:

org.apache.uima.resource.ResourceInitializationException

getPlainDocumentText

protected String getPlainDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)

Specified by:

getPlainDocumentText in class WikipediaStandardReaderBase

isValidPage

protected boolean isValidPage(de.tudarmstadt.ukp.wikipedia.api.Page page)
                       throws de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException

Specified by:

isValidPage in class WikipediaStandardReaderBase

Throws:

de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException