Page MenuHomePhabricator

Refactor scripts fetching text and other metadata
Open, LowPublic

Description

Scripts like - https://github.com/wiki-ai/wikiclass/blob/master/wikiclass/utilities/fetch_text.py are spread acorss wiki-ai for fetching text and other info specific to the use case. It would do good to move the common functionality to revscoring - https://github.com/wiki-ai/revscoring/blob/master/revscoring/utilities/fetch_text.py and have specific use cases handled through command line arguments.

Event Timeline

A common use case of fetch_text is augmenting the dataset with X info from Y api. This will address:

  • fetching edits - currently supported by revscoring
  • fetching text - currently required by wikiclass, drafttopic and draftquality for getting article text
  • fetch_item_info - currently required by wikiclass for fetching item info from Wikidata.

@Sumit: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Do you still plan to work on this, or do you need any help? Thanks!

@Sumit: I am resetting the assignee of this task because there has not been progress lately (please correct me if I am wrong!). Resetting the assignee avoids the impression that somebody is already working on this task. It also allows others to potentially work towards fixing this task. Please claim this task again when you plan to work on it (via Add Action...Assign / Claim in the dropdown menu) - it would be welcome. Thanks for your understanding!