|
|
7cac294181
|
Merge branch 'parser-script' into restucture-parsers
|
2016-03-05 15:01:09 +01:00 |
|
|
|
a24f208449
|
Add parse-pages.py Script
|
2016-03-04 20:39:41 +01:00 |
|
|
|
0af7eb11d6
|
Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
except for returning a generator of text-sections
|
2016-03-03 20:41:14 +01:00 |
|
|
|
7422307985
|
Rewrite RedPage.parse using mwparserfromhell to make it simpler
|
2016-03-03 17:37:46 +01:00 |
|
|
|
b81694c6d3
|
Rewrite heading_parser using mwparserfromhell to make it simpler
|
2016-03-03 17:30:39 +01:00 |
|
|
|
a2dfffc74b
|
Let old date-extracting methods use dates_extract and mark them as deprecated
|
2016-03-03 17:23:44 +01:00 |
|
|
|
163972c924
|
New method dates_extract which finds begining and ending at once
|
2016-03-03 17:20:57 +01:00 |
|
|
|
baf4ae2a07
|
Merge branch 'new-structure' into test-v3
|
2016-03-02 17:22:04 +01:00 |
|
|
|
10f64199ab
|
Remove relativ imports as we don't are in a package anymore
|
2016-03-02 17:19:11 +01:00 |
|
|
|
24f1a7f516
|
Remove __init__.py as we won't use it as a package
|
2016-03-02 17:14:30 +01:00 |
|
|
|
9113a40704
|
Merge branch 'warning-non-flushed-mysql-cache' into test-v3
|
2016-03-02 17:13:08 +01:00 |
|
|
|
f53a5b3745
|
Output a warning if there are update/insert querys cached when exit programm
|
2016-03-02 17:10:08 +01:00 |
|
|
|
673e49c55a
|
Merge branch 'jogobot' into test-v3
Use new jogobot package
|
2016-02-29 11:37:40 +01:00 |
|
|
|
24adafeee7
|
Changes for new jogobot-module
|
2016-02-29 11:35:48 +01:00 |
|
|
|
b26f04db8c
|
Use updated version of jogobot with ast.literal_eval parsed config entrys
|
2016-02-29 11:13:14 +01:00 |
|
|
|
f29dfd5003
|
Use new jogobot module
|
2016-02-28 18:00:46 +01:00 |
|
|
|
ef9c13324a
|
Improve documentation of MysqlRed.flush()
test-v2
|
2015-09-20 18:17:59 +02:00 |
|
|
|
e186f2f22b
|
Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
|
2015-09-20 17:45:07 +02:00 |
|
|
|
7d6cd8bb30
|
Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it
|
2015-09-19 22:44:43 +02:00 |
|
|
|
4e21b6696a
|
Remove unnecessary whitespace from error messages
|
2015-09-19 20:51:52 +02:00 |
|
|
|
6992f82f02
|
Start Implementing of RedFamWorker
|
2015-09-19 20:51:21 +02:00 |
|
|
|
dbcfe8f106
|
Add a generator to MysqlRedFam to retrieve redfams from db by status
|
2015-09-19 19:50:38 +02:00 |
|
|
|
8059bb9992
|
Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
|
2015-09-19 19:49:20 +02:00 |
|
|
|
b5ca69077c
|
Remove double appearence of heading parameter in repression of RedFam
|
2015-09-19 19:47:09 +02:00 |
|
|
|
523d029fdc
|
Fix bug causing db table cells containing empty strings
|
2015-09-19 19:45:34 +02:00 |
|
|
|
4518efc504
|
Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
Reformat MySQL querys to remove whitespace generated by indetation
|
2015-09-18 18:08:13 +02:00 |
|
|
|
b1b37f9b9e
|
Implement functions for flushing db query caches
|
2015-09-17 20:00:13 +02:00 |
|
|
|
8dc7fe678d
|
Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles
|
2015-09-17 19:57:53 +02:00 |
|
|
|
53f53ddb8b
|
Implement cached querys in MysqlRedFam
|
2015-09-17 19:56:39 +02:00 |
|
|
|
26f5912f88
|
Collect writing db querys for running once in MysqlRedPage
Add classmethod to MysqlRed for executing collected querys
|
2015-09-16 21:02:02 +02:00 |
|
|
|
1dea5d7e84
|
NOT WORKING Cache SQL querys to reduce amount of querys
|
2015-09-16 18:31:54 +02:00 |
|
|
|
b514eb5c42
|
Move configuration to jogobot module
Use custom Error classes
|
2015-09-15 21:21:05 +02:00 |
|
|
|
db5bb7401e
|
Update RedFam class to rebuild the whole structure of RedFamPaser generated object
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
|
2015-09-15 21:19:07 +02:00 |
|
|
|
be0041804a
|
Clean up
test-v1
|
2015-09-13 13:19:22 +02:00 |
|
|
|
af6d483280
|
Use a additional module to store custom config ...
|
2015-09-13 13:14:37 +02:00 |
|
|
|
7c9061d37f
|
Use redundances as python package
|
2015-09-13 13:13:28 +02:00 |
|
|
|
74b2dc727c
|
Clean up PEP8 styleguide compatibility with flake8
|
2015-09-13 11:53:47 +02:00 |
|
|
|
cafe08dd7f
|
Added license GPLv3 + Reworked code
Better compatibility with https://www.mediawiki.org/wiki/Manual:Pywikibot/Development_guidelines
|
2015-09-13 01:22:27 +02:00 |
|
|
|
ad100e96eb
|
Remove timezone from timestamp patterns since it is not stored and causes errors on labs
|
2015-09-12 22:35:12 +02:00 |
|
|
|
c78480a75d
|
Use mysql config from pywikibot user-config.py
|
2015-09-12 22:33:44 +02:00 |
|
|
|
6e0d3200c2
|
Change visibility of attributes changed and parsed in RED_PAGE to protected
|
2015-09-12 14:24:45 +02:00 |
|
|
|
f50f4b664f
|
Add additional possibilities to detect wether a red_page is an archive or not
|
2015-09-12 13:48:01 +02:00 |
|
|
|
e70655f679
|
Catch sections with more then 8 articles
|
2015-09-12 12:42:23 +02:00 |
|
|
|
14e865c6e6
|
We need to save the red_fam section heading in database for generating section links since format is not consistent
|
2015-09-12 00:50:04 +02:00 |
|
|
|
a97e2cea5a
|
Since some timestamps are broken we need to reconstruct them by regex match groups
Prevents ValueErrors of datetime.strptime in most cases
|
2015-09-11 23:47:35 +02:00 |
|
|
|
6e64d8448e
|
Prevent RED_PAGE from resetting status if parser is not fired
|
2015-09-11 22:29:16 +02:00 |
|
|
|
deaa1d855b
|
Fix bug which prevents detecting of red_fam sectionheading when there is whitespace after closing heading tags (===)
|
2015-09-11 22:27:25 +02:00 |
|
|
|
90da231712
|
Move the code for instanciating of RED_FAM_PARSER to the end of loop
|
2015-09-11 17:05:18 +02:00 |
|
|
|
529ed09702
|
Use pywikibot.output instead of print
|
2015-09-11 16:42:12 +02:00 |
|
|
|
1892c6d015
|
Fix wrong intendet line (RED_FAM_PARSER only was instantiated when ending-datetime workaround fired)
|
2015-09-11 16:41:04 +02:00 |
|
