de.tudarmstadt.ukp.dkpro.core.io.jwpl

Class WikipediaStandardReaderBase

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.resource.ConfigurableResource_ImplBase

org.apache.uima.collection.CollectionReader_ImplBase

org.apache.uima.fit.component.JCasCollectionReader_ImplBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaStandardReaderBase

All Implemented Interfaces:

org.apache.uima.collection.base_cpm.BaseCollectionReader, org.apache.uima.collection.CollectionReader, org.apache.uima.resource.ConfigurableResource, org.apache.uima.resource.Resource

Direct Known Subclasses:

WikipediaDiscussionReader, WikipediaLinkReader, WikipediaPageReader

public abstract class WikipediaStandardReaderBase
extends WikipediaReaderBase

Abstract base class for standard Wikipedia readers reading single articles instead of revision pairs.

Field Summary

Fields

Modifier and Type	Field and Description
protected long	currentArticleIndex
protected long	nrOfArticles
protected boolean	outputPlainText
protected int	pageBuffer
protected String	pageIdFile
protected String[]	pageIdParamArray
protected Iterator<de.tudarmstadt.ukp.wikipedia.api.Page>	pageIter
protected String	pageNameFile
protected String[]	pageNameParamArray
static String	PARAM_OUTPUT_PLAIN_TEXT Whether the reader outputs plain text or wiki markup.
static String	PARAM_PAGE_BUFFER The page buffer size (#pages) of the page iterator.
static String	PARAM_PAGE_ID_LIST Defines an array of page ids of the pages that should be retrieved.
static String	PARAM_PAGE_TITLE_LIST Defines an array of page titles of the pages that should be retrieved.
static String	PARAM_PATH_TO_PAGE_ID_LIST Defines the path to a file containing a line-separated list of page ids of the pages that should be retrieved.
static String	PARAM_PATH_TO_PAGE_TITLE_LIST Defines the path to a file containing a line-separated list of page titles of the pages that should be retrieved.
protected de.tudarmstadt.ukp.wikipedia.parser.mediawiki.MediaWikiParser	parser

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

dbconfig, PARAM_CREATE_DATABASE_CONFIG_ANNOTATION, PARAM_DB, PARAM_HOST, PARAM_LANGUAGE, PARAM_PASSWORD, PARAM_USER, wiki

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
WikipediaStandardReaderBase()

Method Summary

Modifier and Type	Method and Description
protected String	getDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)
void	getNext(org.apache.uima.jcas.JCas jcas)
de.tudarmstadt.ukp.wikipedia.api.Page	getPage()
protected abstract String	getPlainDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)
org.apache.uima.util.Progress[]	getProgress()
boolean	hasNext()
void	initialize(org.apache.uima.UimaContext context)
protected abstract boolean	isValidPage(de.tudarmstadt.ukp.wikipedia.api.Page page)

Methods inherited from class org.apache.uima.fit.component.JCasCollectionReader_ImplBase

close, getLogger, getNext, initialize

Methods inherited from class org.apache.uima.collection.CollectionReader_ImplBase

destroy, getCasInitializer, getProcessingResourceMetaData, initialize, isConsuming, reconfigure, setCasInitializer, typeSystemInit

Methods inherited from class org.apache.uima.resource.ConfigurableResource_ImplBase

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.uima.resource.ConfigurableResource

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from interface org.apache.uima.resource.Resource

getMetaData, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger

Field Detail

PARAM_OUTPUT_PLAIN_TEXT

public static final String PARAM_OUTPUT_PLAIN_TEXT

Whether the reader outputs plain text or wiki markup.

See Also:

Constant Field Values

outputPlainText

protected boolean outputPlainText

PARAM_PAGE_BUFFER

public static final String PARAM_PAGE_BUFFER

The page buffer size (#pages) of the page iterator.

See Also:

Constant Field Values

pageBuffer

protected int pageBuffer

PARAM_PATH_TO_PAGE_ID_LIST

public static final String PARAM_PATH_TO_PAGE_ID_LIST

Defines the path to a file containing a line-separated list of page ids of the pages that should be retrieved. (Optional)

See Also:

Constant Field Values

pageIdFile

protected String pageIdFile

PARAM_PATH_TO_PAGE_TITLE_LIST

public static final String PARAM_PATH_TO_PAGE_TITLE_LIST

Defines the path to a file containing a line-separated list of page titles of the pages that should be retrieved. (Optional)

See Also:

Constant Field Values

pageNameFile

protected String pageNameFile

PARAM_PAGE_ID_LIST

public static final String PARAM_PAGE_ID_LIST

Defines an array of page ids of the pages that should be retrieved. (Optional)

See Also:

Constant Field Values

pageIdParamArray

protected String[] pageIdParamArray

PARAM_PAGE_TITLE_LIST

public static final String PARAM_PAGE_TITLE_LIST

Defines an array of page titles of the pages that should be retrieved. (Optional)

See Also:

Constant Field Values

pageNameParamArray

protected String[] pageNameParamArray

currentArticleIndex

protected long currentArticleIndex

nrOfArticles

protected long nrOfArticles

pageIter

protected Iterator<de.tudarmstadt.ukp.wikipedia.api.Page> pageIter

parser

protected de.tudarmstadt.ukp.wikipedia.parser.mediawiki.MediaWikiParser parser

Constructor Detail

WikipediaStandardReaderBase

public WikipediaStandardReaderBase()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext context)
                throws org.apache.uima.resource.ResourceInitializationException

Overrides:

initialize in class WikipediaReaderBase

Throws:

org.apache.uima.resource.ResourceInitializationException

hasNext

public boolean hasNext()
                throws IOException,
                       org.apache.uima.collection.CollectionException

Throws:

IOException

org.apache.uima.collection.CollectionException

getNext

public void getNext(org.apache.uima.jcas.JCas jcas)
             throws IOException,
                    org.apache.uima.collection.CollectionException

Overrides:

getNext in class WikipediaReaderBase

Throws:

IOException

org.apache.uima.collection.CollectionException

isValidPage

protected abstract boolean isValidPage(de.tudarmstadt.ukp.wikipedia.api.Page page)
                                throws de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException

Throws:

de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException

getProgress

public org.apache.uima.util.Progress[] getProgress()

Specified by:

getProgress in interface org.apache.uima.collection.base_cpm.BaseCollectionReader

Specified by:

getProgress in class WikipediaReaderBase

getDocumentText

protected String getDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)

getPlainDocumentText

protected abstract String getPlainDocumentText(de.tudarmstadt.ukp.wikipedia.api.Page page)

getPage

public de.tudarmstadt.ukp.wikipedia.api.Page getPage()