Jonathan Golder
|
a24f208449
|
Add parse-pages.py Script
|
8 years ago |
Jonathan Golder
|
0af7eb11d6
|
Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
except for returning a generator of text-sections
|
8 years ago |
Jonathan Golder
|
7422307985
|
Rewrite RedPage.parse using mwparserfromhell to make it simpler
|
8 years ago |
Jonathan Golder
|
b81694c6d3
|
Rewrite heading_parser using mwparserfromhell to make it simpler
|
8 years ago |
Jonathan Golder
|
a2dfffc74b
|
Let old date-extracting methods use dates_extract and mark them as deprecated
|
8 years ago |
Jonathan Golder
|
163972c924
|
New method dates_extract which finds begining and ending at once
|
8 years ago |
Jonathan Golder
|
baf4ae2a07
|
Merge branch 'new-structure' into test-v3
|
8 years ago |
Jonathan Golder
|
10f64199ab
|
Remove relativ imports as we don't are in a package anymore
|
8 years ago |
Jonathan Golder
|
24f1a7f516
|
Remove __init__.py as we won't use it as a package
|
8 years ago |
Jonathan Golder
|
9113a40704
|
Merge branch 'warning-non-flushed-mysql-cache' into test-v3
|
8 years ago |
Jonathan Golder
|
f53a5b3745
|
Output a warning if there are update/insert querys cached when exit programm
|
8 years ago |
Jonathan Golder
|
673e49c55a
|
Merge branch 'jogobot' into test-v3
Use new jogobot package
|
8 years ago |
Jonathan Golder
|
24adafeee7
|
Changes for new jogobot-module
|
8 years ago |
Jonathan Golder
|
b26f04db8c
|
Use updated version of jogobot with ast.literal_eval parsed config entrys
|
8 years ago |
Jonathan Golder
|
f29dfd5003
|
Use new jogobot module
|
8 years ago |
Jonathan Golder
|
ef9c13324a
|
Improve documentation of MysqlRed.flush()
|
9 years ago |
Jonathan Golder
|
e186f2f22b
|
Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
|
9 years ago |
Jonathan Golder
|
7d6cd8bb30
|
Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it
|
9 years ago |
Jonathan Golder
|
4e21b6696a
|
Remove unnecessary whitespace from error messages
|
9 years ago |
Jonathan Golder
|
6992f82f02
|
Start Implementing of RedFamWorker
|
9 years ago |
Jonathan Golder
|
dbcfe8f106
|
Add a generator to MysqlRedFam to retrieve redfams from db by status
|
9 years ago |
Jonathan Golder
|
8059bb9992
|
Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
|
9 years ago |
Jonathan Golder
|
b5ca69077c
|
Remove double appearence of heading parameter in repression of RedFam
|
9 years ago |
Jonathan Golder
|
523d029fdc
|
Fix bug causing db table cells containing empty strings
|
9 years ago |
Jonathan Golder
|
4518efc504
|
Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
Reformat MySQL querys to remove whitespace generated by indetation
|
9 years ago |
Jonathan Golder
|
b1b37f9b9e
|
Implement functions for flushing db query caches
|
9 years ago |
Jonathan Golder
|
8dc7fe678d
|
Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles
|
9 years ago |
Jonathan Golder
|
53f53ddb8b
|
Implement cached querys in MysqlRedFam
|
9 years ago |
Jonathan Golder
|
26f5912f88
|
Collect writing db querys for running once in MysqlRedPage
Add classmethod to MysqlRed for executing collected querys
|
9 years ago |
Jonathan Golder
|
1dea5d7e84
|
NOT WORKING Cache SQL querys to reduce amount of querys
|
9 years ago |
Jonathan Golder
|
b514eb5c42
|
Move configuration to jogobot module
Use custom Error classes
|
9 years ago |
Jonathan Golder
|
db5bb7401e
|
Update RedFam class to rebuild the whole structure of RedFamPaser generated object
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
|
9 years ago |
Jonathan Golder
|
be0041804a
|
Clean up
|
9 years ago |
Jonathan Golder
|
af6d483280
|
Use a additional module to store custom config ...
|
9 years ago |
Jonathan Golder
|
7c9061d37f
|
Use redundances as python package
|
9 years ago |
Jonathan Golder
|
74b2dc727c
|
Clean up PEP8 styleguide compatibility with flake8
|
9 years ago |
Jonathan Golder
|
cafe08dd7f
|
Added license GPLv3 + Reworked code
Better compatibility with https://www.mediawiki.org/wiki/Manual:Pywikibot/Development_guidelines
|
9 years ago |
Jonathan Golder
|
ad100e96eb
|
Remove timezone from timestamp patterns since it is not stored and causes errors on labs
|
9 years ago |
Jonathan Golder
|
c78480a75d
|
Use mysql config from pywikibot user-config.py
|
9 years ago |
Jonathan Golder
|
6e0d3200c2
|
Change visibility of attributes changed and parsed in RED_PAGE to protected
|
9 years ago |
Jonathan Golder
|
f50f4b664f
|
Add additional possibilities to detect wether a red_page is an archive or not
|
9 years ago |
Jonathan Golder
|
e70655f679
|
Catch sections with more then 8 articles
|
9 years ago |
Jonathan Golder
|
14e865c6e6
|
We need to save the red_fam section heading in database for generating section links since format is not consistent
|
9 years ago |
Jonathan Golder
|
a97e2cea5a
|
Since some timestamps are broken we need to reconstruct them by regex match groups
Prevents ValueErrors of datetime.strptime in most cases
|
9 years ago |
Jonathan Golder
|
6e64d8448e
|
Prevent RED_PAGE from resetting status if parser is not fired
|
9 years ago |
Jonathan Golder
|
deaa1d855b
|
Fix bug which prevents detecting of red_fam sectionheading when there is whitespace after closing heading tags (===)
|
9 years ago |
Jonathan Golder
|
90da231712
|
Move the code for instanciating of RED_FAM_PARSER to the end of loop
|
9 years ago |
Jonathan Golder
|
529ed09702
|
Use pywikibot.output instead of print
|
9 years ago |
Jonathan Golder
|
1892c6d015
|
Fix wrong intendet line (RED_FAM_PARSER only was instantiated when ending-datetime workaround fired)
|
9 years ago |
Jonathan Golder
|
99f050acd3
|
Add workaround to detect ending datetime if there is no done notice
|
9 years ago |