Jonathan Golder
389c48605e
redfam: Make get_disc_link() able to return wikilink
Issue #64 (https://git.golderweb.de/wiki/jogobot-red/issues/64 )
6 years ago
Jonathan Golder
dfffe97200
redfam: Add method to check disc section
Sometimes disc sections are disapering since the heading is changed
and the famhash changes, so we get a new redfam. Mark those as absent
Issue #64 (https://git.golderweb.de/wiki/jogobot-red/issues/64 )
6 years ago
Jonathan Golder
246e94c228
redfam: Add generator for open redfams to Worker
Issue #64 (https://git.golderweb.de/wiki/jogobot-red/issues/64 )
6 years ago
Jonathan Golder
14db996a43
redfam: Check if moved page exists
To prevent creation of orphaned diskpages in case of special movement
constructs
Issue #71 (https://git.golderweb.de/wiki/jogobot-red/issues/71 )
6 years ago
Jonathan Golder
a466ab4e74
Prepare release-1.1
6 years ago
Jonathan Golder
2c105336b0
RedFamWorker: Exclude users and user talkpages
Users can't be part of valid redundances
Issue #68 (https://git.golderweb.de/wiki/jogobot-red/issues/68 )
6 years ago
Jonathan Golder
6e119ea98f
RedFamWorker: Improve talkpagetoggling
Do not toggle to main page if we have already a talkpage and vice versa
Issue #69 (https://git.golderweb.de/wiki/jogobot-red/issues/69 )
6 years ago
Jonathan Golder
fa13e2a5cf
Follow moved pages
Keep notice together with content
https://de.wikipedia.org/w/index.php?title=Benutzer_Diskussion:Jogo.obb&oldid=176464377#Redundanzhinweis_zu_zwischenzeitlich_verschobenen_Artikeln
Issue #70 (https://git.golderweb.de/wiki/jogobot-red/issues/70 )
6 years ago
Jonathan Golder
93447d8dc6
Prepare release v1.0
Update Copyright Notices
Version information
7 years ago
Jonathan Golder
20103d589d
redfam: article_generator add redfam info to page
Add reference to redfam object and article title from db to Page object
since Page.title() may differe (short Namespaces, anchors, special chars)
Related Task: [FS#161](https://fs.golderweb.de/index.php?do=details&task_id=161 )
7 years ago
Jonathan Golder
e18aa96a84
redfam: article_generator can return talkpage
To make pywikibot.pagegenerators.PageWithTalkPageGenerators unneccessary
so we can manipulate talkpage object directly
Related Task: [FS#161](https://fs.golderweb.de/index.php?do=details&task_id=161 )
7 years ago
Jonathan Golder
5f4640d5ff
Replace urlencoded chars with unicode equivalent
Otherwise we get value errors while marking since pwb replaces those
Related Task: [FS#160](https://fs.golderweb.de/index.php?do=details&task_id=160 )
7 years ago
Jonathan Golder
84802cf521
Remove leading or trailing spaces in articles
Some articles contain spaces between title and anchor part which will
be stripped now
Related Task: [FS#159](https://fs.golderweb.de/index.php?do=details&task_id=159 )
7 years ago
Jonathan Golder
02e53475f1
Prevent lowercase article titles in Parser
Since real lowercase article titles are not allowed, make sure to
convert all first letters of article titles to uppercase. This is
neccessary since pywikibot will return article titles like this.
Related Task: [FS#157](https://fs.golderweb.de/index.php?do=details&task_id=157 )
7 years ago
Jonathan Golder
d9b4fcc0bd
Strip spaces before adding articles to redfam
Some article links have surounding spaces in their linktext. Remove them
before adding article to RedFam to have a cannonical title
Related Task: [FS#155](https://fs.golderweb.de/index.php?do=details&task_id=155 )
7 years ago
Jonathan Golder
3b2cb95f36
Do not fetch marked redfams from db
Exclude marked Redfams from DB-Query to prevent marking them again
Related Task: [FS#138](https://fs.golderweb.de/index.php?do=details&task_id=138 )
7 years ago
Jonathan Golder
14ec71dd09
Rewrite get_disc_link to handle special cases
Use methods of pywikibot site-object and mwparser to get rid of any
special elements like templates or links in headings for construction
of our disc link.
Replace by hand as it otherwise will occur as normal space and
wont work
Related Task: [FS#147](https://fs.golderweb.de/index.php?do=details&task_id=147 )
7 years ago
Jonathan Golder
cd87d1c2bb
Fix already marked articles was reshown bug
Since we search for matching states for articles to include or exclude
in a loop, we could not control the outer loop via default break/
continue. Python docs recommend using Exceptions and try/except
structures to realise that most conveniently.
https://docs.python.org/3/faq/design.html#why-is-there-no-goto
Related Task: [FS#138](https://fs.golderweb.de/index.php?do=details&task_id=138 )
7 years ago
Jonathan Golder
0f930082b4
Also canonicalise anchor parts of articles
Replace spaces in anchors with underscores as spaces are not correct
there
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=114 FS#114]
7 years ago
Jonathan Golder
80c94ccf4f
Replace underscores in article titles
Remove underscores in article titles and replace with spaces to have
canonical state for all articles
Therefore we need to split title and posible anchors in heading parser
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=114 FS#114]
7 years ago
Jonathan Golder
3e69a1c77e
Remove problem indicating stati when set marked
Remove states which are indicating problems in previous runs if
successfully marked article and also whole RedFam
[https://fs.golderweb.de/index.php?do=details&task_id=112 FS#112]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=110 FS#110]
7 years ago
Jonathan Golder
56f326b568
Fix error all current redfams marked when quit
Restructure update_status to make sure, marked is only set when all
articles are marked or gone (means deleted or redirect)
[https://fs.golderweb.de/index.php?do=details&task_id=111 FS#111]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=110 FS#110]
7 years ago
Jonathan Golder
4aaacf1443
Add redfams to redpage-obj after parsing
To have redfams available for updates immediately after parsing. Double
redfams then will be seen as Update.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=108 FS#108]
7 years ago
Jonathan Golder
e16925197c
Fix pep8.. compliance
To be concordant with the coding styles fix pep8 compliance
7 years ago
Jonathan Golder
9ba7d2e517
Change redfam generator filters
Change and clear up the filters in redfam generator to keep track of
article status and use positive conditionals
7 years ago
Jonathan Golder
844fee52ae
Make markpages using new DB/Class structure
Update markpages and RedFamWorker-Code to use the new sqlalchemy based
DB ORM Interface
7 years ago
Jonathan Golder
43e31c108a
Working RedFamWorker query
Modify RedfamWorker class to work with new DB API
7 years ago
Jonathan Golder
89b50e3312
Remove old status API
Now we use the methods of status object directly
7 years ago
Jonathan Golder
467f829af2
Some cleanups
Remove old commented out code from manual mysql solution
7 years ago
Jonathan Golder
6e973369cd
sqlalchemy working for parser
Needs some testing, presumably contains some bugs
8 years ago
Jonathan Golder
65fb2ecb28
Generate Fam status based on article status
Some article states should be reflected in the RedFam status
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=89 FS#89]
8 years ago
Jonathan Golder
870ed4bf25
Update redfam.article_generator use article status
To be able to filter articles by status of that article
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=89 FS#89]
8 years ago
Jonathan Golder
e13320820c
Add API to manage status per article
To be able to track changes to articles to update redfam status
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=89 FS#89]
8 years ago
Jonathan Golder
6149dcdb8b
Apply changes to data structure
See related ticket
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=94 FS#94]
8 years ago
Jonathan Golder
8c56125a7b
Update talkpage notice template
Exact date is not necessary and end could be ommited if of the same
month
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=93 FS#93]
8 years ago
Jonathan Golder
c0b18f88e5
Add filter options to redfam.article_generator
To give the posibility to filter not existing pages or redirect pages or
vice versa.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=87 FS#87]
8 years ago
Jonathan Golder
e5989305a4
Add a generator to redfam yielding article pages
To work on articles of a redfam a generator which yields belonging
articles is necessary
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=87 FS#87]
8 years ago
Jonathan Golder
6717fa4fba
Add method to generate notice for article discpage
We need a method to generate the template to add to article discpages
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=29 FS#29]
8 years ago
Jonathan Golder
3723aba578
Add a method to get link to related reddisc
To generate notices or other stuff it is necessary to add links to the
related reddisc.
This method returns a wikilink to text the redfam's reddisc.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=81 FS#81]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=81 FS#81]
8 years ago
Jonathan Golder
b36dc250d2
Request information about reddisc page for redfams
To generate links to related reddisc it is necessary to get at least the
Title of the related reddisc page. As saving the same data in db is
worse, we retreive it via join from red_pages table
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
b271a0b0b1
Add generator wrapper to fetch RedFams by status and ending
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
ad088126e7
Define method to update Status after Working with RedFam
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
151c22a735
Add fetched mysql_data to _mysql-Object of parent class for using change-method to update db
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
a97d8c722e
Move handling of mysql-Connection from RedFamParser and RedFamWorker to RedFam-Class and make it protected instead of private
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
78eda10562
Remove deprecated methods
Deprecated functions which are not used anymore can be removed to make
code more clearer and improve maintainability
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=80 FS#80]
8 years ago
Jonathan Golder
e28acf88d1
Introduce new directory structure
To clarify which is a bot and which are helper scripts
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
8 years ago
Jonathan Golder
ac54aea698
Use callback to detect redfam.section
Detecting redfam-Sections via RegExp caused some false positives due to
wrong formated things in wikisyntax. See Task
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=78 FS#78]
8 years ago
Jonathan Golder
1e4c8646bf
Reparse redfam-heading with mwparser
See related ticked for detailed failure explanation
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=77 FS#77]
8 years ago
Jonathan Golder
ab430e0085
Use month of reddisc as beginning if missing
Construct a fictive but sensfull beginning if we cant detect one
Needed since beginning is mandatory
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
8 years ago
Jonathan Golder
95be313859
Pass reddisc pywikibot.page object to redfam
To access page information like page title (eg. to get dates from it)
of the reddisc page
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
8 years ago
