e5989305a4
Add a generator to redfam yielding article pages
...
To work on articles of a redfam a generator which yields belonging
articles is necessary
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=87 FS#87]
2016-08-28 13:20:13 +02:00
8ce6f03641
Merge branch 'fs#29-generate-articledisc-notice' into fs#25-mark-done
2016-08-27 19:50:18 +02:00
6717fa4fba
Add method to generate notice for article discpage
...
We need a method to generate the template to add to article discpages
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=29 FS#29]
2016-08-27 19:49:50 +02:00
8acba7d0f9
Merge branch 'fs#81-get-reddisc-link' into fs#25-mark-done
2016-08-27 19:48:24 +02:00
3723aba578
Add a method to get link to related reddisc
...
To generate notices or other stuff it is necessary to add links to the
related reddisc.
This method returns a wikilink to text the redfam's reddisc.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=81 FS#81]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=81 FS#81]
2016-08-27 19:48:03 +02:00
9d3bc74c80
Merge branch 'fs#26-done-redfam-gen' into fs#25-mark-done
2016-08-27 19:46:41 +02:00
b36dc250d2
Request information about reddisc page for redfams
...
To generate links to related reddisc it is necessary to get at least the
Title of the related reddisc page. As saving the same data in db is
worse, we retreive it via join from red_pages table
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
4055dc52d8
Make it possible to get a RedPage-Object by pageid
...
When working on redfams it is necessary to have information about redpage
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
594130c8a6
Restore changes from 45df35431
...
Documented to prevent deleting again
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
b271a0b0b1
Add generator wrapper to fetch RedFams by status and ending
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
ad088126e7
Define method to update Status after Working with RedFam
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
151c22a735
Add fetched mysql_data to _mysql-Object of parent class for using change-method to update db
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
a97d8c722e
Move handling of mysql-Connection from RedFamParser and RedFamWorker to RedFam-Class and make it protected instead of private
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
58dfd8c86a
For RedFamilies not fetched individually we need to provide the fam hash as index
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
9481116777
Add new generator-method to fetch RedFams by Status and Ending
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
eaa7596a8f
Merge branch 'fs#70-refactoring' into test-v3
2016-08-27 19:40:00 +02:00
449d83d7b5
Merge branch 'fs#82-subtask-wrapper' into fs#70-refactoring
2016-08-27 19:19:40 +02:00
4ac9b305f5
Merge branch 'fs#85-move-start-api-to-jogobot' into fs#82-subtask-wrapper
2016-08-27 19:17:10 +02:00
604b7bd8b7
Now use Bot-Start API from jogobot framework
...
API was moved to jogobot to share with other tasks
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
2016-08-27 19:04:13 +02:00
d0fa15d0ed
Update jogobot module to get standart Start-API
...
[FS#84]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
2016-08-27 18:50:36 +02:00
71e41bfed3
Merge branch 'fs#83-wrapper-compatibility' into fs#82-subtask-wrapper
2016-08-27 18:24:14 +02:00
2be0a8903d
Adjust constructor for wrapper-script
...
The new wrapper-script calls a standardized API
We need to be conform with that
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=83 FS#83]
2016-08-27 17:02:51 +02:00
0ceb2e6e83
Add methods to build gen to DiscussionParser
...
With the new wrapper script the Bot gets a GenFactory and has to build
a generator out of it by its own
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=83 FS#83]
2016-08-27 16:58:20 +02:00
3540cc2a7d
Move functional sections to functions in main()
...
To make main() function less complicated functional sections are moved
to dedicated functions
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
460d2db183
Add Bot run with exception handling
...
Errors, especially caused by missing run-method, need to be catched to
provide information in Logfile.
And also to get information wether bot run was successfull
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
156f117b18
Add Bot initiation with exception handling
...
Bot initiation needs to catch errors by Bot to enforce at least a basic
logging.
And also to be sure Init was successfull before starting bot.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
1679e2ad6a
Prepare environment for starting subtasks
...
Before init and run bot we need to provide a environment for it,
like parsed args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
b88efb6bdd
Reflect stucture changes in Code
...
Since bot class is moved to separate dir/file we need to do some changes
to rebuild functionality
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
177a8f920f
Prepare new structure to use subtasks
...
To have only one entry point for the bot we want to have a single file
(red.py) which is calling the specfic task class from bots dir with a
standardized call
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
0549cbd2c2
Merge branch 'fs#80-remove-deprecated-methods' into fs#70-refactoring
2016-08-25 22:43:57 +02:00
78eda10562
Remove deprecated methods
...
Deprecated functions which are not used anymore can be removed to make
code more clearer and improve maintainability
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=80 FS#80]
2016-08-25 22:41:13 +02:00
510771509b
Merge branch 'fs#79-mysql-table-prefix' into fs#70-refactoring
2016-08-25 13:15:23 +02:00
71b99b5f58
Delay definition of db_table_prefix
...
db_table_prefix should be defined at init of MysqlRed and not at import
to have cmdline args already parsed
Otherwise it uses default family
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
2016-08-25 13:06:32 +02:00
77d1de4473
Add a tablename prefix depending on Site
...
To be able to run the bot on different wikis the db tables should be
named pywikibot.Site dependend and changed automatically
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
2016-08-24 23:53:10 +02:00
cac04f344f
Merge branch 'fs#74-helpermodules-lib' into fs#70-refactoring
2016-08-24 23:09:33 +02:00
e28acf88d1
Introduce new directory structure
...
To clarify which is a bot and which are helper scripts
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
2016-08-24 22:41:41 +02:00
af48888535
Merge branch 'fs#78-redfam-section-false-positives' into fs#70-refactoring
2016-08-24 20:05:04 +02:00
ac54aea698
Use callback to detect redfam.section
...
Detecting redfam-Sections via RegExp caused some false positives due to
wrong formated things in wikisyntax. See Task
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=78 FS#78]
2016-08-24 20:02:48 +02:00
2deb02fe47
Merge branch 'fs#77-errors-on-old-archives' into fs#70-refactoring
2016-08-24 19:59:53 +02:00
1e4c8646bf
Reparse redfam-heading with mwparser
...
See related ticked for detailed failure explanation
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=77 FS#77]
2016-08-24 19:57:25 +02:00
fe2810f07c
Merge branch 'fs#76-redfam-without-dates' into fs#70-refactoring
2016-08-24 17:01:00 +02:00
ab430e0085
Use month of reddisc as beginning if missing
...
Construct a fictive but sensfull beginning if we cant detect one
Needed since beginning is mandatory
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:56:54 +02:00
95be313859
Pass reddisc pywikibot.page object to redfam
...
To access page information like page title (eg. to get dates from it)
of the reddisc page
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:53:45 +02:00
0bb0b2d957
Make sure var beginning is always defined
...
To prevent unbound Errors caused by using undeclared variable beginning
if the redfam-section does not contain any timestamp
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:51:23 +02:00
db32c9e8f6
Merge branch 'fs#75-mysql-flush-error-false-reddisc' into fs#70-refactoring
2016-08-24 16:49:37 +02:00
bd2d221c48
Prevent flush from creating cursor without con
...
MysqlRed.flush() tried to create a cursor in any case. If there was no
connection (because the subclasses haven't been instantiated an oursql
Error occured.
Instead, check before if there is a connection and otherwise raise an Error
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 16:46:51 +02:00
ee8ebbc8bc
Make sure only flush db if there are redfams
...
To prevent from doing unnecessary stuff and trying to use not existing
db connection
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 15:47:43 +02:00
dcc4851513
Check reddisc page titles against regex
...
To prevent parsing Pages which have been categorized in configured cats
wrong or are given via cmd params
Parsing them results in unexpected behaviour
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 15:27:42 +02:00
0ea1b0039d
Merge branch 'fs#72-rewrite-reddiscparser' into fs#70-refactoring
2016-08-24 11:23:38 +02:00
2f878ee901
Correct filename in header
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
2016-08-24 11:20:28 +02:00
