wiki/jogobot-red
1
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity
110 Commits 2 Branches 14 Tags
71b99b5f5837e43af1cc57f1890bfbf6d4d0e382
Commit Graph
110 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
golderweb 71b99b5f58 Delay definition of db_table_prefix 
db_table_prefix should be defined at init of MysqlRed and not at import
to have cmdline args already parsed
Otherwise it uses default family

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
 2016-08-25 13:06:32 +02:00
golderweb 77d1de4473 Add a tablename prefix depending on Site 
To be able to run the bot on different wikis the db tables should be
named pywikibot.Site dependend and changed automatically

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
 2016-08-24 23:53:10 +02:00
golderweb cac04f344f Merge branch 'fs#74-helpermodules-lib' into fs#70-refactoring 2016-08-24 23:09:33 +02:00
golderweb e28acf88d1 Introduce new directory structure 
To clarify which is a bot and which are helper scripts

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
 2016-08-24 22:41:41 +02:00
golderweb af48888535 Merge branch 'fs#78-redfam-section-false-positives' into fs#70-refactoring 2016-08-24 20:05:04 +02:00
golderweb ac54aea698 Use callback to detect redfam.section 
Detecting redfam-Sections via RegExp caused some false positives due to
wrong formated things in wikisyntax. See Task

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=78 FS#78]
 2016-08-24 20:02:48 +02:00
golderweb 2deb02fe47 Merge branch 'fs#77-errors-on-old-archives' into fs#70-refactoring 2016-08-24 19:59:53 +02:00
golderweb 1e4c8646bf Reparse redfam-heading with mwparser 
See related ticked for detailed failure explanation

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=77 FS#77]
 2016-08-24 19:57:25 +02:00
golderweb fe2810f07c Merge branch 'fs#76-redfam-without-dates' into fs#70-refactoring 2016-08-24 17:01:00 +02:00
golderweb ab430e0085 Use month of reddisc as beginning if missing 
Construct a fictive but sensfull beginning if we cant detect one
Needed since beginning is mandatory

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
 2016-08-24 16:56:54 +02:00
golderweb 95be313859 Pass reddisc pywikibot.page object to redfam 
To access page information like page title (eg. to get dates from it)
of the reddisc page

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
 2016-08-24 16:53:45 +02:00
golderweb 0bb0b2d957 Make sure var beginning is always defined 
To prevent unbound Errors caused by using undeclared variable beginning
if the redfam-section does not contain any timestamp

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=76 FS#76]
 2016-08-24 16:51:23 +02:00
golderweb db32c9e8f6 Merge branch 'fs#75-mysql-flush-error-false-reddisc' into fs#70-refactoring 2016-08-24 16:49:37 +02:00
golderweb bd2d221c48 Prevent flush from creating cursor without con 
MysqlRed.flush() tried to create a cursor in any case. If there was no
connection (because the subclasses haven't been instantiated an oursql
Error occured.
Instead, check before if there is a connection and otherwise raise an Error

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
 2016-08-24 16:46:51 +02:00
golderweb ee8ebbc8bc Make sure only flush db if there are redfams 
To prevent from doing unnecessary stuff and trying to use not existing
db connection

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
 2016-08-24 15:47:43 +02:00
golderweb dcc4851513 Check reddisc page titles against regex 
To prevent parsing Pages which have been categorized in configured cats
wrong or are given via cmd params
Parsing them results in unexpected behaviour

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
 2016-08-24 15:27:42 +02:00
golderweb 0ea1b0039d Merge branch 'fs#72-rewrite-reddiscparser' into fs#70-refactoring 2016-08-24 11:23:38 +02:00
golderweb 2f878ee901 Correct filename in header 
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
 2016-08-24 11:20:28 +02:00
golderweb 17bfb32ded Building generators of config cats in sep Function 
Since the main()-Function was too complex the logic to build generators
out of categories provided in jogobot.conf was moved in a separate
function

[https://fs.golderweb.de/index.php?do=details&task_id=73 FS#73]

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
 2016-08-24 11:17:00 +02:00
golderweb 6cb92c1da7 Rewrite parse control using pywikibot.bot classes 
To use the default pywikibot.classes making life easier at some point
Beeing standardconform with pywikibot in handling args

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
 2016-08-23 21:53:44 +02:00
golderweb a8605bcee6 Mv pages-parser.py to reddiscparser.py 
New, more meaningfull naming conventions, from redpage to reddisc (page)

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=72 FS#72]
 2016-08-23 21:50:22 +02:00
golderweb 5d31bdd7eb Jogobot submodule updated 2016-08-23 21:28:13 +02:00
golderweb 7f8ab1897e Merge branch 'fs#69-deprecated-decorators-param-str' into test-v3 2016-08-23 21:26:47 +02:00
golderweb 79dbde2413 Provide Replacement to @deprecated() as str 
Since use of pywikibot-master (or Python3.5 @see ticket below)
the @deprecator requires a str as param and no callable object like
done before

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=69 FS#69]
 2016-08-23 21:23:24 +02:00
golderweb 36a480a042 Merge branch 'fs#68-mysql-db-port' into test-v3 2016-08-23 21:16:09 +02:00
golderweb bd9dbdfa17 Make use of declared db_host_port 
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports

Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
 2016-08-23 21:12:07 +02:00
golderweb 944bea488a Merge branch 'restucture-parsers' into test-v3 2016-03-05 15:02:31 +01:00
golderweb 7cac294181 Merge branch 'parser-script' into restucture-parsers 2016-03-05 15:01:09 +01:00
golderweb a24f208449 Add parse-pages.py Script 2016-03-04 20:39:41 +01:00
golderweb 0af7eb11d6 Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams 
except for returning a generator of text-sections
 2016-03-03 20:41:14 +01:00
golderweb 7422307985 Rewrite RedPage.parse using mwparserfromhell to make it simpler 2016-03-03 17:37:46 +01:00
golderweb b81694c6d3 Rewrite heading_parser using mwparserfromhell to make it simpler 2016-03-03 17:30:39 +01:00
golderweb a2dfffc74b Let old date-extracting methods use dates_extract and mark them as deprecated 2016-03-03 17:23:44 +01:00
golderweb 163972c924 New method dates_extract which finds begining and ending at once 2016-03-03 17:20:57 +01:00
golderweb baf4ae2a07 Merge branch 'new-structure' into test-v3 2016-03-02 17:22:04 +01:00
golderweb 10f64199ab Remove relativ imports as we don't are in a package anymore 2016-03-02 17:19:11 +01:00
golderweb 24f1a7f516 Remove __init__.py as we won't use it as a package 2016-03-02 17:14:30 +01:00
golderweb 9113a40704 Merge branch 'warning-non-flushed-mysql-cache' into test-v3 2016-03-02 17:13:08 +01:00
golderweb f53a5b3745 Output a warning if there are update/insert querys cached when exit programm 2016-03-02 17:10:08 +01:00
golderweb 673e49c55a Merge branch 'jogobot' into test-v3 
Use new jogobot package
 2016-02-29 11:37:40 +01:00
golderweb 24adafeee7 Changes for new jogobot-module 2016-02-29 11:35:48 +01:00
golderweb b26f04db8c Use updated version of jogobot with ast.literal_eval parsed config entrys 2016-02-29 11:13:14 +01:00
golderweb f29dfd5003 Use new jogobot module 2016-02-28 18:00:46 +01:00
golderweb ef9c13324a Improve documentation of MysqlRed.flush() test-v2 2015-09-20 18:17:59 +02:00
golderweb e186f2f22b Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys 2015-09-20 17:45:07 +02:00
golderweb 7d6cd8bb30 Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it 2015-09-19 22:44:43 +02:00
golderweb 4e21b6696a Remove unnecessary whitespace from error messages 2015-09-19 20:51:52 +02:00
golderweb 6992f82f02 Start Implementing of RedFamWorker 2015-09-19 20:51:21 +02:00
golderweb dbcfe8f106 Add a generator to MysqlRedFam to retrieve redfams from db by status 2015-09-19 19:50:38 +02:00
golderweb 8059bb9992 Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash 2015-09-19 19:49:20 +02:00