de.tudarmstadt.ukp.dkpro.core.io.jwpl

Class WikipediaRevisionReaderBase

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.resource.ConfigurableResource_ImplBase

org.apache.uima.collection.CollectionReader_ImplBase

org.apache.uima.fit.component.JCasCollectionReader_ImplBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaRevisionReaderBase

All Implemented Interfaces:

org.apache.uima.collection.base_cpm.BaseCollectionReader, org.apache.uima.collection.CollectionReader, org.apache.uima.resource.ConfigurableResource, org.apache.uima.resource.Resource

Direct Known Subclasses:

WikipediaRevisionPairReader, WikipediaRevisionReader

public abstract class WikipediaRevisionReaderBase
extends WikipediaReaderBase

Abstract base class for all readers based on revisions.

Field Summary

Fields

Modifier and Type	Field and Description
protected de.tudarmstadt.ukp.wikipedia.api.Page	currentArticle
protected long	currentArticleIndex
protected long	currentRevisionIndex
protected long	nrOfArticles
protected boolean	outputPlainText
protected int	pageBuffer
protected Iterator<de.tudarmstadt.ukp.wikipedia.api.Page>	pageIter
static String	PARAM_OUTPUT_PLAIN_TEXT Whether the reader outputs plain text or wiki markup.
static String	PARAM_PAGE_BUFFER The page buffer size (#pages) of the page iterator.
static String	PARAM_PATH_TO_REVISION_ID_LIST Defines the path to a file containing a line-separated list of revision ids of the revisions that should be retrieved.
static String	PARAM_REVISION_ID_LIST Defines an array of revision ids of the revisions that should be retrieved.
protected de.tudarmstadt.ukp.wikipedia.parser.mediawiki.MediaWikiParser	parser
protected Iterator<String>	revIdIterator
protected de.tudarmstadt.ukp.wikipedia.revisionmachine.api.RevisionApi	revisionApi
protected String	revisionIdFile
protected String[]	revisionIdParamArray
protected Set<String>	revisionIds
protected Iterator<Timestamp>	timestampIter

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

dbconfig, PARAM_CREATE_DATABASE_CONFIG_ANNOTATION, PARAM_DB, PARAM_HOST, PARAM_LANGUAGE, PARAM_PASSWORD, PARAM_USER, wiki

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
WikipediaRevisionReaderBase()

Method Summary

Modifier and Type	Method and Description
protected void	addDocumentMetaData(org.apache.uima.jcas.JCas jcas, int pageId, int revisionId)
protected void	addRevisionAnnotation(org.apache.uima.jcas.JCas jcas, de.tudarmstadt.ukp.wikipedia.revisionmachine.api.Revision revision)
org.apache.uima.util.Progress[]	getProgress()
protected Iterator<Timestamp>	getTimestampIter(int pageId)
boolean	hasNext()
void	initialize(org.apache.uima.UimaContext context)

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.io.jwpl.WikipediaReaderBase

getNext

Methods inherited from class org.apache.uima.fit.component.JCasCollectionReader_ImplBase

close, getLogger, getNext, initialize

Methods inherited from class org.apache.uima.collection.CollectionReader_ImplBase

destroy, getCasInitializer, getProcessingResourceMetaData, initialize, isConsuming, reconfigure, setCasInitializer, typeSystemInit

Methods inherited from class org.apache.uima.resource.ConfigurableResource_ImplBase

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.uima.resource.ConfigurableResource

getConfigParameterValue, getConfigParameterValue, setConfigParameterValue, setConfigParameterValue

Methods inherited from interface org.apache.uima.resource.Resource

getMetaData, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger

Field Detail

PARAM_OUTPUT_PLAIN_TEXT

public static final String PARAM_OUTPUT_PLAIN_TEXT

Whether the reader outputs plain text or wiki markup.

See Also:

Constant Field Values

outputPlainText

protected boolean outputPlainText

PARAM_PAGE_BUFFER

public static final String PARAM_PAGE_BUFFER

The page buffer size (#pages) of the page iterator.

See Also:

Constant Field Values

pageBuffer

protected int pageBuffer

PARAM_PATH_TO_REVISION_ID_LIST

public static final String PARAM_PATH_TO_REVISION_ID_LIST

Defines the path to a file containing a line-separated list of revision ids of the revisions that should be retrieved. (Optional)

See Also:

Constant Field Values

revisionIdFile

protected String revisionIdFile

PARAM_REVISION_ID_LIST

public static final String PARAM_REVISION_ID_LIST

Defines an array of revision ids of the revisions that should be retrieved. (Optional)

See Also:

Constant Field Values

revisionIdParamArray

protected String[] revisionIdParamArray

currentArticle

protected de.tudarmstadt.ukp.wikipedia.api.Page currentArticle

revisionApi

protected de.tudarmstadt.ukp.wikipedia.revisionmachine.api.RevisionApi revisionApi

pageIter

protected Iterator<de.tudarmstadt.ukp.wikipedia.api.Page> pageIter

timestampIter

protected Iterator<Timestamp> timestampIter

revIdIterator

protected Iterator<String> revIdIterator

currentArticleIndex

protected long currentArticleIndex

currentRevisionIndex

protected long currentRevisionIndex

nrOfArticles

protected long nrOfArticles

parser

protected de.tudarmstadt.ukp.wikipedia.parser.mediawiki.MediaWikiParser parser

revisionIds

protected Set<String> revisionIds

Constructor Detail

WikipediaRevisionReaderBase

public WikipediaRevisionReaderBase()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext context)
                throws org.apache.uima.resource.ResourceInitializationException

Overrides:

initialize in class WikipediaReaderBase

Throws:

org.apache.uima.resource.ResourceInitializationException

hasNext

public boolean hasNext()
                throws IOException,
                       org.apache.uima.collection.CollectionException

Throws:

IOException

org.apache.uima.collection.CollectionException

getProgress

public org.apache.uima.util.Progress[] getProgress()

Specified by:

getProgress in interface org.apache.uima.collection.base_cpm.BaseCollectionReader

Specified by:

getProgress in class WikipediaReaderBase

getTimestampIter

protected Iterator<Timestamp> getTimestampIter(int pageId)
                                        throws IOException

Throws:

IOException

addRevisionAnnotation

protected void addRevisionAnnotation(org.apache.uima.jcas.JCas jcas,
                                     de.tudarmstadt.ukp.wikipedia.revisionmachine.api.Revision revision)

addDocumentMetaData

protected void addDocumentMetaData(org.apache.uima.jcas.JCas jcas,
                                   int pageId,
                                   int revisionId)
                            throws de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException,
                                   de.tudarmstadt.ukp.wikipedia.api.exception.WikiApiException

Throws:

de.tudarmstadt.ukp.wikipedia.api.exception.WikiTitleParsingException

de.tudarmstadt.ukp.wikipedia.api.exception.WikiApiException