b271a0b0b1
Add generator wrapper to fetch RedFams by status and ending
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
ad088126e7
Define method to update Status after Working with RedFam
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
151c22a735
Add fetched mysql_data to _mysql-Object of parent class for using change-method to update db
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
a97d8c722e
Move handling of mysql-Connection from RedFamParser and RedFamWorker to RedFam-Class and make it protected instead of private
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
58dfd8c86a
For RedFamilies not fetched individually we need to provide the fam hash as index
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
9481116777
Add new generator-method to fetch RedFams by Status and Ending
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
2016-08-27 19:44:33 +02:00
eaa7596a8f
Merge branch 'fs#70-refactoring' into test-v3
2016-08-27 19:40:00 +02:00
449d83d7b5
Merge branch 'fs#82-subtask-wrapper' into fs#70-refactoring
2016-08-27 19:19:40 +02:00
4ac9b305f5
Merge branch 'fs#85-move-start-api-to-jogobot' into fs#82-subtask-wrapper
2016-08-27 19:17:10 +02:00
604b7bd8b7
Now use Bot-Start API from jogobot framework
...
API was moved to jogobot to share with other tasks
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
2016-08-27 19:04:13 +02:00
d0fa15d0ed
Update jogobot module to get standart Start-API
...
[FS#84]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=85 FS#85]
2016-08-27 18:50:36 +02:00
71e41bfed3
Merge branch 'fs#83-wrapper-compatibility' into fs#82-subtask-wrapper
2016-08-27 18:24:14 +02:00
2be0a8903d
Adjust constructor for wrapper-script
...
The new wrapper-script calls a standardized API
We need to be conform with that
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=83 FS#83]
2016-08-27 17:02:51 +02:00
0ceb2e6e83
Add methods to build gen to DiscussionParser
...
With the new wrapper script the Bot gets a GenFactory and has to build
a generator out of it by its own
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=83 FS#83]
2016-08-27 16:58:20 +02:00
3540cc2a7d
Move functional sections to functions in main()
...
To make main() function less complicated functional sections are moved
to dedicated functions
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
460d2db183
Add Bot run with exception handling
...
Errors, especially caused by missing run-method, need to be catched to
provide information in Logfile.
And also to get information wether bot run was successfull
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
156f117b18
Add Bot initiation with exception handling
...
Bot initiation needs to catch errors by Bot to enforce at least a basic
logging.
And also to be sure Init was successfull before starting bot.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
1679e2ad6a
Prepare environment for starting subtasks
...
Before init and run bot we need to provide a environment for it,
like parsed args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
b88efb6bdd
Reflect stucture changes in Code
...
Since bot class is moved to separate dir/file we need to do some changes
to rebuild functionality
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
177a8f920f
Prepare new structure to use subtasks
...
To have only one entry point for the bot we want to have a single file
(red.py) which is calling the specfic task class from bots dir with a
standardized call
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
2016-08-27 15:40:09 +02:00
0549cbd2c2
Merge branch 'fs#80-remove-deprecated-methods' into fs#70-refactoring
2016-08-25 22:43:57 +02:00
78eda10562
Remove deprecated methods
...
Deprecated functions which are not used anymore can be removed to make
code more clearer and improve maintainability
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=80 FS#80]
2016-08-25 22:41:13 +02:00
510771509b
Merge branch 'fs#79-mysql-table-prefix' into fs#70-refactoring
2016-08-25 13:15:23 +02:00
71b99b5f58
Delay definition of db_table_prefix
...
db_table_prefix should be defined at init of MysqlRed and not at import
to have cmdline args already parsed
Otherwise it uses default family
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
2016-08-25 13:06:32 +02:00
77d1de4473
Add a tablename prefix depending on Site
...
To be able to run the bot on different wikis the db tables should be
named pywikibot.Site dependend and changed automatically
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
2016-08-24 23:53:10 +02:00
cac04f344f
Merge branch 'fs#74-helpermodules-lib' into fs#70-refactoring
2016-08-24 23:09:33 +02:00
e28acf88d1
Introduce new directory structure
...
To clarify which is a bot and which are helper scripts
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
2016-08-24 22:41:41 +02:00
af48888535
Merge branch 'fs#78-redfam-section-false-positives' into fs#70-refactoring
2016-08-24 20:05:04 +02:00
ac54aea698
Use callback to detect redfam.section
...
Detecting redfam-Sections via RegExp caused some false positives due to
wrong formated things in wikisyntax. See Task
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=78 FS#78]
2016-08-24 20:02:48 +02:00
2deb02fe47
Merge branch 'fs#77-errors-on-old-archives' into fs#70-refactoring
2016-08-24 19:59:53 +02:00
1e4c8646bf
Reparse redfam-heading with mwparser
...
See related ticked for detailed failure explanation
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=77 FS#77]
2016-08-24 19:57:25 +02:00
fe2810f07c
Merge branch 'fs#76-redfam-without-dates' into fs#70-refactoring
2016-08-24 17:01:00 +02:00
ab430e0085
Use month of reddisc as beginning if missing
...
Construct a fictive but sensfull beginning if we cant detect one
Needed since beginning is mandatory
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:56:54 +02:00
95be313859
Pass reddisc pywikibot.page object to redfam
...
To access page information like page title (eg. to get dates from it)
of the reddisc page
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:53:45 +02:00
0bb0b2d957
Make sure var beginning is always defined
...
To prevent unbound Errors caused by using undeclared variable beginning
if the redfam-section does not contain any timestamp
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
2016-08-24 16:51:23 +02:00
db32c9e8f6
Merge branch 'fs#75-mysql-flush-error-false-reddisc' into fs#70-refactoring
2016-08-24 16:49:37 +02:00
bd2d221c48
Prevent flush from creating cursor without con
...
MysqlRed.flush() tried to create a cursor in any case. If there was no
connection (because the subclasses haven't been instantiated an oursql
Error occured.
Instead, check before if there is a connection and otherwise raise an Error
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 16:46:51 +02:00
ee8ebbc8bc
Make sure only flush db if there are redfams
...
To prevent from doing unnecessary stuff and trying to use not existing
db connection
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 15:47:43 +02:00
dcc4851513
Check reddisc page titles against regex
...
To prevent parsing Pages which have been categorized in configured cats
wrong or are given via cmd params
Parsing them results in unexpected behaviour
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
2016-08-24 15:27:42 +02:00
0ea1b0039d
Merge branch 'fs#72-rewrite-reddiscparser' into fs#70-refactoring
2016-08-24 11:23:38 +02:00
2f878ee901
Correct filename in header
...
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
2016-08-24 11:20:28 +02:00
17bfb32ded
Building generators of config cats in sep Function
...
Since the main()-Function was too complex the logic to build generators
out of categories provided in jogobot.conf was moved in a separate
function
[https://fs.golderweb.de/index.php?do=details&task_id=73 FS#73]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
2016-08-24 11:17:00 +02:00
6cb92c1da7
Rewrite parse control using pywikibot.bot classes
...
To use the default pywikibot.classes making life easier at some point
Beeing standardconform with pywikibot in handling args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
2016-08-23 21:53:44 +02:00
a8605bcee6
Mv pages-parser.py to reddiscparser.py
...
New, more meaningfull naming conventions, from redpage to reddisc (page)
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
2016-08-23 21:50:22 +02:00
5d31bdd7eb
Jogobot submodule updated
2016-08-23 21:28:13 +02:00
7f8ab1897e
Merge branch 'fs#69-deprecated-decorators-param-str' into test-v3
2016-08-23 21:26:47 +02:00
79dbde2413
Provide Replacement to @deprecated() as str
...
Since use of pywikibot-master (or Python3.5 @see ticket below)
the @deprecator requires a str as param and no callable object like
done before
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=69 FS#69]
2016-08-23 21:23:24 +02:00
36a480a042
Merge branch 'fs#68-mysql-db-port' into test-v3
2016-08-23 21:16:09 +02:00
bd9dbdfa17
Make use of declared db_host_port
...
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
2016-08-23 21:12:07 +02:00
944bea488a
Merge branch 'restucture-parsers' into test-v3
2016-03-05 15:02:31 +01:00
