Skip to content

Support pagination in QueryWithContinuation (dual continuation) #69

@CXuesong

Description

@CXuesong

The current implementation of QueryWithContinuation, or the related WikiPagePropertyProvider<T> parameter classes, are not design to support pagination inside page properties (&prop= values, such as &props=revision).

Though there is some basic logic in RefreshPagesAsync to merge prop list when there are some props need pagination, we have not introduced proper public API to let consumer adjust the pagination size of props itself. IWikiPagePropertyProvider<T>.GetMaxPaginationSize is used to restrict the max allowed title count (e.g. count of title= values), instead of props item count (e.g. x in rvlimit=x).

The justification for not breaking the wall between WikiPagePropertyList<T> (calls QueryWithContinuation) and WikiPagePropertyProvider before is the potential complexity to deal with dual continuation (#68 (comment)). But perhaps we may break the wall, and make it possible for consumer to plug IWikiPagePropertyProvider into WikiPagePropertyList instead of deriving from the latter one.

Before this feature has been implemented, for example, you may consider the following approach to get the complete page property list when the pages are taken from another wikilist/generator (#68 (comment))

  • Use CategoryMembersGenerator.ListItemAsync to get a sequence of WikiPageStubs,
  • Convert WikiPageStubs into WikiPages,
  • Refresh WikiPage sequence by batches (e.g., pywikibot batches pages by 30 items)

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions