Jonathan Golder
0ceb2e6e83
Add methods to build gen to DiscussionParser
With the new wrapper script the Bot gets a GenFactory and has to build
a generator out of it by its own
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=83 FS#83]
8 years ago
Jonathan Golder
3540cc2a7d
Move functional sections to functions in main()
To make main() function less complicated functional sections are moved
to dedicated functions
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
460d2db183
Add Bot run with exception handling
Errors, especially caused by missing run-method, need to be catched to
provide information in Logfile.
And also to get information wether bot run was successfull
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
156f117b18
Add Bot initiation with exception handling
Bot initiation needs to catch errors by Bot to enforce at least a basic
logging.
And also to be sure Init was successfull before starting bot.
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
1679e2ad6a
Prepare environment for starting subtasks
Before init and run bot we need to provide a environment for it,
like parsed args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
b88efb6bdd
Reflect stucture changes in Code
Since bot class is moved to separate dir/file we need to do some changes
to rebuild functionality
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
177a8f920f
Prepare new structure to use subtasks
To have only one entry point for the bot we want to have a single file
(red.py) which is calling the specfic task class from bots dir with a
standardized call
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=82 FS#82]
8 years ago
Jonathan Golder
0549cbd2c2
Merge branch 'fs#80-remove-deprecated-methods' into fs#70-refactoring
8 years ago
Jonathan Golder
78eda10562
Remove deprecated methods
Deprecated functions which are not used anymore can be removed to make
code more clearer and improve maintainability
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=80 FS#80]
8 years ago
Jonathan Golder
510771509b
Merge branch 'fs#79-mysql-table-prefix' into fs#70-refactoring
8 years ago
Jonathan Golder
71b99b5f58
Delay definition of db_table_prefix
db_table_prefix should be defined at init of MysqlRed and not at import
to have cmdline args already parsed
Otherwise it uses default family
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
8 years ago
Jonathan Golder
77d1de4473
Add a tablename prefix depending on Site
To be able to run the bot on different wikis the db tables should be
named pywikibot.Site dependend and changed automatically
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
8 years ago
Jonathan Golder
cac04f344f
Merge branch 'fs#74-helpermodules-lib' into fs#70-refactoring
8 years ago
Jonathan Golder
e28acf88d1
Introduce new directory structure
To clarify which is a bot and which are helper scripts
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
8 years ago
Jonathan Golder
af48888535
Merge branch 'fs#78-redfam-section-false-positives' into fs#70-refactoring
8 years ago
Jonathan Golder
ac54aea698
Use callback to detect redfam.section
Detecting redfam-Sections via RegExp caused some false positives due to
wrong formated things in wikisyntax. See Task
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=78 FS#78]
8 years ago
Jonathan Golder
2deb02fe47
Merge branch 'fs#77-errors-on-old-archives' into fs#70-refactoring
8 years ago
Jonathan Golder
1e4c8646bf
Reparse redfam-heading with mwparser
See related ticked for detailed failure explanation
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=77 FS#77]
8 years ago
Jonathan Golder
fe2810f07c
Merge branch 'fs#76-redfam-without-dates' into fs#70-refactoring
8 years ago
Jonathan Golder
ab430e0085
Use month of reddisc as beginning if missing
Construct a fictive but sensfull beginning if we cant detect one
Needed since beginning is mandatory
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
8 years ago
Jonathan Golder
95be313859
Pass reddisc pywikibot.page object to redfam
To access page information like page title (eg. to get dates from it)
of the reddisc page
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
8 years ago
Jonathan Golder
0bb0b2d957
Make sure var beginning is always defined
To prevent unbound Errors caused by using undeclared variable beginning
if the redfam-section does not contain any timestamp
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
8 years ago
Jonathan Golder
db32c9e8f6
Merge branch 'fs#75-mysql-flush-error-false-reddisc' into fs#70-refactoring
8 years ago
Jonathan Golder
bd2d221c48
Prevent flush from creating cursor without con
MysqlRed.flush() tried to create a cursor in any case. If there was no
connection (because the subclasses haven't been instantiated an oursql
Error occured.
Instead, check before if there is a connection and otherwise raise an Error
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
ee8ebbc8bc
Make sure only flush db if there are redfams
To prevent from doing unnecessary stuff and trying to use not existing
db connection
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
dcc4851513
Check reddisc page titles against regex
To prevent parsing Pages which have been categorized in configured cats
wrong or are given via cmd params
Parsing them results in unexpected behaviour
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
0ea1b0039d
Merge branch 'fs#72-rewrite-reddiscparser' into fs#70-refactoring
8 years ago
Jonathan Golder
2f878ee901
Correct filename in header
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
17bfb32ded
Building generators of config cats in sep Function
Since the main()-Function was too complex the logic to build generators
out of categories provided in jogobot.conf was moved in a separate
function
[https://fs.golderweb.de/index.php?do=details&task_id=73 FS#73]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
6cb92c1da7
Rewrite parse control using pywikibot.bot classes
To use the default pywikibot.classes making life easier at some point
Beeing standardconform with pywikibot in handling args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
a8605bcee6
Mv pages-parser.py to reddiscparser.py
New, more meaningfull naming conventions, from redpage to reddisc (page)
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
5d31bdd7eb
Jogobot submodule updated
8 years ago
Jonathan Golder
7f8ab1897e
Merge branch 'fs#69-deprecated-decorators-param-str' into test-v3
8 years ago
Jonathan Golder
79dbde2413
Provide Replacement to @deprecated() as str
Since use of pywikibot-master (or Python3.5 @see ticket below)
the @deprecator requires a str as param and no callable object like
done before
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=69 FS#69]
8 years ago
Jonathan Golder
36a480a042
Merge branch 'fs#68-mysql-db-port' into test-v3
8 years ago
Jonathan Golder
bd9dbdfa17
Make use of declared db_host_port
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
8 years ago
Jonathan Golder
944bea488a
Merge branch 'restucture-parsers' into test-v3
8 years ago
Jonathan Golder
7cac294181
Merge branch 'parser-script' into restucture-parsers
8 years ago
Jonathan Golder
a24f208449
Add parse-pages.py Script
8 years ago
Jonathan Golder
0af7eb11d6
Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
except for returning a generator of text-sections
8 years ago
Jonathan Golder
7422307985
Rewrite RedPage.parse using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder
b81694c6d3
Rewrite heading_parser using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder
a2dfffc74b
Let old date-extracting methods use dates_extract and mark them as deprecated
8 years ago
Jonathan Golder
163972c924
New method dates_extract which finds begining and ending at once
8 years ago
Jonathan Golder
baf4ae2a07
Merge branch 'new-structure' into test-v3
8 years ago
Jonathan Golder
10f64199ab
Remove relativ imports as we don't are in a package anymore
8 years ago
Jonathan Golder
24f1a7f516
Remove __init__.py as we won't use it as a package
8 years ago
Jonathan Golder
9113a40704
Merge branch 'warning-non-flushed-mysql-cache' into test-v3
8 years ago
Jonathan Golder
f53a5b3745
Output a warning if there are update/insert querys cached when exit programm
8 years ago
Jonathan Golder
673e49c55a
Merge branch 'jogobot' into test-v3
Use new jogobot package
8 years ago
