Jonathan Golder
ee8ebbc8bc
Make sure only flush db if there are redfams
To prevent from doing unnecessary stuff and trying to use not existing
db connection
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
dcc4851513
Check reddisc page titles against regex
To prevent parsing Pages which have been categorized in configured cats
wrong or are given via cmd params
Parsing them results in unexpected behaviour
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
0ea1b0039d
Merge branch 'fs#72-rewrite-reddiscparser' into fs#70-refactoring
8 years ago
Jonathan Golder
2f878ee901
Correct filename in header
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
17bfb32ded
Building generators of config cats in sep Function
Since the main()-Function was too complex the logic to build generators
out of categories provided in jogobot.conf was moved in a separate
function
[https://fs.golderweb.de/index.php?do=details&task_id=73 FS#73]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
6cb92c1da7
Rewrite parse control using pywikibot.bot classes
To use the default pywikibot.classes making life easier at some point
Beeing standardconform with pywikibot in handling args
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
a8605bcee6
Mv pages-parser.py to reddiscparser.py
New, more meaningfull naming conventions, from redpage to reddisc (page)
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
8 years ago
Jonathan Golder
5d31bdd7eb
Jogobot submodule updated
8 years ago
Jonathan Golder
7f8ab1897e
Merge branch 'fs#69-deprecated-decorators-param-str' into test-v3
8 years ago
Jonathan Golder
79dbde2413
Provide Replacement to @deprecated() as str
Since use of pywikibot-master (or Python3.5 @see ticket below)
the @deprecator requires a str as param and no callable object like
done before
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=69 FS#69]
8 years ago
Jonathan Golder
36a480a042
Merge branch 'fs#68-mysql-db-port' into test-v3
8 years ago
Jonathan Golder
bd9dbdfa17
Make use of declared db_host_port
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
8 years ago
Jonathan Golder
944bea488a
Merge branch 'restucture-parsers' into test-v3
8 years ago
Jonathan Golder
7cac294181
Merge branch 'parser-script' into restucture-parsers
8 years ago
Jonathan Golder
a24f208449
Add parse-pages.py Script
8 years ago
Jonathan Golder
0af7eb11d6
Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
except for returning a generator of text-sections
8 years ago
Jonathan Golder
7422307985
Rewrite RedPage.parse using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder
b81694c6d3
Rewrite heading_parser using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder
a2dfffc74b
Let old date-extracting methods use dates_extract and mark them as deprecated
8 years ago
Jonathan Golder
163972c924
New method dates_extract which finds begining and ending at once
8 years ago
Jonathan Golder
baf4ae2a07
Merge branch 'new-structure' into test-v3
8 years ago
Jonathan Golder
10f64199ab
Remove relativ imports as we don't are in a package anymore
8 years ago
Jonathan Golder
24f1a7f516
Remove __init__.py as we won't use it as a package
8 years ago
Jonathan Golder
9113a40704
Merge branch 'warning-non-flushed-mysql-cache' into test-v3
8 years ago
Jonathan Golder
f53a5b3745
Output a warning if there are update/insert querys cached when exit programm
8 years ago
Jonathan Golder
673e49c55a
Merge branch 'jogobot' into test-v3
Use new jogobot package
8 years ago
Jonathan Golder
24adafeee7
Changes for new jogobot-module
8 years ago
Jonathan Golder
b26f04db8c
Use updated version of jogobot with ast.literal_eval parsed config entrys
8 years ago
Jonathan Golder
f29dfd5003
Use new jogobot module
8 years ago
Jonathan Golder
ef9c13324a
Improve documentation of MysqlRed.flush()
9 years ago
Jonathan Golder
e186f2f22b
Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
9 years ago
Jonathan Golder
7d6cd8bb30
Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it
9 years ago
Jonathan Golder
4e21b6696a
Remove unnecessary whitespace from error messages
9 years ago
Jonathan Golder
6992f82f02
Start Implementing of RedFamWorker
9 years ago
Jonathan Golder
dbcfe8f106
Add a generator to MysqlRedFam to retrieve redfams from db by status
9 years ago
Jonathan Golder
8059bb9992
Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
9 years ago
Jonathan Golder
b5ca69077c
Remove double appearence of heading parameter in repression of RedFam
9 years ago
Jonathan Golder
523d029fdc
Fix bug causing db table cells containing empty strings
9 years ago
Jonathan Golder
4518efc504
Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
Reformat MySQL querys to remove whitespace generated by indetation
9 years ago
Jonathan Golder
b1b37f9b9e
Implement functions for flushing db query caches
9 years ago
Jonathan Golder
8dc7fe678d
Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles
9 years ago
Jonathan Golder
53f53ddb8b
Implement cached querys in MysqlRedFam
9 years ago
Jonathan Golder
26f5912f88
Collect writing db querys for running once in MysqlRedPage
Add classmethod to MysqlRed for executing collected querys
9 years ago
Jonathan Golder
1dea5d7e84
NOT WORKING Cache SQL querys to reduce amount of querys
9 years ago
Jonathan Golder
b514eb5c42
Move configuration to jogobot module
Use custom Error classes
9 years ago
Jonathan Golder
db5bb7401e
Update RedFam class to rebuild the whole structure of RedFamPaser generated object
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
9 years ago
Jonathan Golder
be0041804a
Clean up
9 years ago
Jonathan Golder
af6d483280
Use a additional module to store custom config ...
9 years ago
Jonathan Golder
7c9061d37f
Use redundances as python package
9 years ago
Jonathan Golder
74b2dc727c
Clean up PEP8 styleguide compatibility with flake8
9 years ago