wiki
/
jogobot-red
1
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity
233 Commits
2 Branches
14 Tags
2.5 MiB
Tree: 0c135ef1bb
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0c135ef1bb'
${ noResults }
Commit Graph

233 Commits (0c135ef1bb4306e5a91a3b61d3cd9c14b50082d6)
 

Author SHA1 Message Date
Jonathan Golder 7cac294181 Merge branch 'parser-script' into restucture-parsers
8 years ago
Jonathan Golder a24f208449 Add parse-pages.py Script
8 years ago
Jonathan Golder 0af7eb11d6 Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams
8 years ago
Jonathan Golder 7422307985 Rewrite RedPage.parse using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder b81694c6d3 Rewrite heading_parser using mwparserfromhell to make it simpler
8 years ago
Jonathan Golder a2dfffc74b Let old date-extracting methods use dates_extract and mark them as deprecated
8 years ago
Jonathan Golder 163972c924 New method dates_extract which finds begining and ending at once
8 years ago
Jonathan Golder baf4ae2a07 Merge branch 'new-structure' into test-v3
8 years ago
Jonathan Golder 10f64199ab Remove relativ imports as we don't are in a package anymore
8 years ago
Jonathan Golder 24f1a7f516 Remove __init__.py as we won't use it as a package
8 years ago
Jonathan Golder 9113a40704 Merge branch 'warning-non-flushed-mysql-cache' into test-v3
8 years ago
Jonathan Golder f53a5b3745 Output a warning if there are update/insert querys cached when exit programm
8 years ago
Jonathan Golder 673e49c55a Merge branch 'jogobot' into test-v3
8 years ago
Jonathan Golder 24adafeee7 Changes for new jogobot-module
8 years ago
Jonathan Golder b26f04db8c Use updated version of jogobot with ast.literal_eval parsed config entrys
8 years ago
Jonathan Golder f29dfd5003 Use new jogobot module
8 years ago
Jonathan Golder ef9c13324a Improve documentation of MysqlRed.flush()
9 years ago
Jonathan Golder e186f2f22b Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
9 years ago
Jonathan Golder 7d6cd8bb30 Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it
9 years ago
Jonathan Golder 4e21b6696a Remove unnecessary whitespace from error messages
9 years ago
Jonathan Golder 6992f82f02 Start Implementing of RedFamWorker
9 years ago
Jonathan Golder dbcfe8f106 Add a generator to MysqlRedFam to retrieve redfams from db by status
9 years ago
Jonathan Golder 8059bb9992 Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
9 years ago
Jonathan Golder b5ca69077c Remove double appearence of heading parameter in repression of RedFam
9 years ago
Jonathan Golder 523d029fdc Fix bug causing db table cells containing empty strings
9 years ago
Jonathan Golder 4518efc504 Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
9 years ago
Jonathan Golder b1b37f9b9e Implement functions for flushing db query caches
9 years ago
Jonathan Golder 8dc7fe678d Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles
9 years ago
Jonathan Golder 53f53ddb8b Implement cached querys in MysqlRedFam
9 years ago
Jonathan Golder 26f5912f88 Collect writing db querys for running once in MysqlRedPage
9 years ago
Jonathan Golder 1dea5d7e84 NOT WORKING Cache SQL querys to reduce amount of querys
9 years ago
Jonathan Golder b514eb5c42 Move configuration to jogobot module
9 years ago
Jonathan Golder db5bb7401e Update RedFam class to rebuild the whole structure of RedFamPaser generated object
9 years ago
Jonathan Golder be0041804a Clean up
9 years ago
Jonathan Golder af6d483280 Use a additional module to store custom config ...
9 years ago
Jonathan Golder 7c9061d37f Use redundances as python package
9 years ago
Jonathan Golder 74b2dc727c Clean up PEP8 styleguide compatibility with flake8
9 years ago
Jonathan Golder cafe08dd7f Added license GPLv3 + Reworked code
9 years ago
Jonathan Golder ad100e96eb Remove timezone from timestamp patterns since it is not stored and causes errors on labs
9 years ago
Jonathan Golder c78480a75d Use mysql config from pywikibot user-config.py
9 years ago
Jonathan Golder 6e0d3200c2 Change visibility of attributes changed and parsed in RED_PAGE to protected
9 years ago
Jonathan Golder f50f4b664f Add additional possibilities to detect wether a red_page is an archive or not
9 years ago
Jonathan Golder e70655f679 Catch sections with more then 8 articles
9 years ago
Jonathan Golder 14e865c6e6 We need to save the red_fam section heading in database for generating section links since format is not consistent
9 years ago
Jonathan Golder a97e2cea5a Since some timestamps are broken we need to reconstruct them by regex match groups
9 years ago
Jonathan Golder 6e64d8448e Prevent RED_PAGE from resetting status if parser is not fired
9 years ago
Jonathan Golder deaa1d855b Fix bug which prevents detecting of red_fam sectionheading when there is whitespace after closing heading tags (===)
9 years ago
Jonathan Golder 90da231712 Move the code for instanciating of RED_FAM_PARSER to the end of loop
9 years ago
Jonathan Golder 529ed09702 Use pywikibot.output instead of print
9 years ago
Jonathan Golder 1892c6d015 Fix wrong intendet line (RED_FAM_PARSER only was instantiated when ending-datetime workaround fired)
9 years ago