bd9dbdfa17
Make use of declared db_host_port
...
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
2016-08-23 21:12:07 +02:00
944bea488a
Merge branch 'restucture-parsers' into test-v3
2016-03-05 15:02:31 +01:00
7cac294181
Merge branch 'parser-script' into restucture-parsers
2016-03-05 15:01:09 +01:00
a24f208449
Add parse-pages.py Script
2016-03-04 20:39:41 +01:00
0af7eb11d6
Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
...
except for returning a generator of text-sections
2016-03-03 20:41:14 +01:00
7422307985
Rewrite RedPage.parse using mwparserfromhell to make it simpler
2016-03-03 17:37:46 +01:00
b81694c6d3
Rewrite heading_parser using mwparserfromhell to make it simpler
2016-03-03 17:30:39 +01:00
a2dfffc74b
Let old date-extracting methods use dates_extract and mark them as deprecated
2016-03-03 17:23:44 +01:00
163972c924
New method dates_extract which finds begining and ending at once
2016-03-03 17:20:57 +01:00
baf4ae2a07
Merge branch 'new-structure' into test-v3
2016-03-02 17:22:04 +01:00
10f64199ab
Remove relativ imports as we don't are in a package anymore
2016-03-02 17:19:11 +01:00
24f1a7f516
Remove __init__.py as we won't use it as a package
2016-03-02 17:14:30 +01:00
9113a40704
Merge branch 'warning-non-flushed-mysql-cache' into test-v3
2016-03-02 17:13:08 +01:00
f53a5b3745
Output a warning if there are update/insert querys cached when exit programm
2016-03-02 17:10:08 +01:00
673e49c55a
Merge branch 'jogobot' into test-v3
...
Use new jogobot package
2016-02-29 11:37:40 +01:00
24adafeee7
Changes for new jogobot-module
2016-02-29 11:35:48 +01:00
b26f04db8c
Use updated version of jogobot with ast.literal_eval parsed config entrys
2016-02-29 11:13:14 +01:00
f29dfd5003
Use new jogobot module
2016-02-28 18:00:46 +01:00
ef9c13324a
Improve documentation of MysqlRed.flush()
2015-09-20 18:17:59 +02:00
e186f2f22b
Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
2015-09-20 17:45:07 +02:00
7d6cd8bb30
Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it
2015-09-19 22:44:43 +02:00
4e21b6696a
Remove unnecessary whitespace from error messages
2015-09-19 20:51:52 +02:00
6992f82f02
Start Implementing of RedFamWorker
2015-09-19 20:51:21 +02:00
dbcfe8f106
Add a generator to MysqlRedFam to retrieve redfams from db by status
2015-09-19 19:50:38 +02:00
8059bb9992
Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
2015-09-19 19:49:20 +02:00
b5ca69077c
Remove double appearence of heading parameter in repression of RedFam
2015-09-19 19:47:09 +02:00
523d029fdc
Fix bug causing db table cells containing empty strings
2015-09-19 19:45:34 +02:00
4518efc504
Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
...
Reformat MySQL querys to remove whitespace generated by indetation
2015-09-18 18:08:13 +02:00
b1b37f9b9e
Implement functions for flushing db query caches
2015-09-17 20:00:13 +02:00
8dc7fe678d
Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles
2015-09-17 19:57:53 +02:00
53f53ddb8b
Implement cached querys in MysqlRedFam
2015-09-17 19:56:39 +02:00
26f5912f88
Collect writing db querys for running once in MysqlRedPage
...
Add classmethod to MysqlRed for executing collected querys
2015-09-16 21:02:02 +02:00
1dea5d7e84
NOT WORKING Cache SQL querys to reduce amount of querys
2015-09-16 18:31:54 +02:00
b514eb5c42
Move configuration to jogobot module
...
Use custom Error classes
2015-09-15 21:21:05 +02:00
db5bb7401e
Update RedFam class to rebuild the whole structure of RedFamPaser generated object
...
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
2015-09-15 21:19:07 +02:00
be0041804a
Clean up
2015-09-13 13:19:22 +02:00
af6d483280
Use a additional module to store custom config ...
2015-09-13 13:14:37 +02:00
7c9061d37f
Use redundances as python package
2015-09-13 13:13:28 +02:00
74b2dc727c
Clean up PEP8 styleguide compatibility with flake8
2015-09-13 11:53:47 +02:00
cafe08dd7f
Added license GPLv3 + Reworked code
...
Better compatibility with https://www.mediawiki.org/wiki/Manual:Pywikibot/Development_guidelines
2015-09-13 01:22:27 +02:00
ad100e96eb
Remove timezone from timestamp patterns since it is not stored and causes errors on labs
2015-09-12 22:35:12 +02:00
c78480a75d
Use mysql config from pywikibot user-config.py
2015-09-12 22:33:44 +02:00
6e0d3200c2
Change visibility of attributes changed and parsed in RED_PAGE to protected
2015-09-12 14:24:45 +02:00
f50f4b664f
Add additional possibilities to detect wether a red_page is an archive or not
2015-09-12 13:48:01 +02:00
e70655f679
Catch sections with more then 8 articles
2015-09-12 12:42:23 +02:00
14e865c6e6
We need to save the red_fam section heading in database for generating section links since format is not consistent
2015-09-12 00:50:04 +02:00
a97e2cea5a
Since some timestamps are broken we need to reconstruct them by regex match groups
...
Prevents ValueErrors of datetime.strptime in most cases
2015-09-11 23:47:35 +02:00
6e64d8448e
Prevent RED_PAGE from resetting status if parser is not fired
2015-09-11 22:29:16 +02:00
deaa1d855b
Fix bug which prevents detecting of red_fam sectionheading when there is whitespace after closing heading tags (===)
2015-09-11 22:27:25 +02:00
90da231712
Move the code for instanciating of RED_FAM_PARSER to the end of loop
2015-09-11 17:05:18 +02:00
