wiki/jogobot-red
1
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity
81 Commits 2 Branches 14 Tags
0af7eb11d65261c157e97ec0099927777b7b682a
Commit Graph
81 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
golderweb 0af7eb11d6 Move parsing of redfams from RedPageParser to RedFamParser.parser so RedPageParse won't do anything with redfams 
except for returning a generator of text-sections
 2016-03-03 20:41:14 +01:00
golderweb 7422307985 Rewrite RedPage.parse using mwparserfromhell to make it simpler 2016-03-03 17:37:46 +01:00
golderweb b81694c6d3 Rewrite heading_parser using mwparserfromhell to make it simpler 2016-03-03 17:30:39 +01:00
golderweb a2dfffc74b Let old date-extracting methods use dates_extract and mark them as deprecated 2016-03-03 17:23:44 +01:00
golderweb 163972c924 New method dates_extract which finds begining and ending at once 2016-03-03 17:20:57 +01:00
golderweb baf4ae2a07 Merge branch 'new-structure' into test-v3 2016-03-02 17:22:04 +01:00
golderweb 10f64199ab Remove relativ imports as we don't are in a package anymore 2016-03-02 17:19:11 +01:00
golderweb 24f1a7f516 Remove __init__.py as we won't use it as a package 2016-03-02 17:14:30 +01:00
golderweb 9113a40704 Merge branch 'warning-non-flushed-mysql-cache' into test-v3 2016-03-02 17:13:08 +01:00
golderweb f53a5b3745 Output a warning if there are update/insert querys cached when exit programm 2016-03-02 17:10:08 +01:00
golderweb 673e49c55a Merge branch 'jogobot' into test-v3 
Use new jogobot package
 2016-02-29 11:37:40 +01:00
golderweb 24adafeee7 Changes for new jogobot-module 2016-02-29 11:35:48 +01:00
golderweb b26f04db8c Use updated version of jogobot with ast.literal_eval parsed config entrys 2016-02-29 11:13:14 +01:00
golderweb f29dfd5003 Use new jogobot module 2016-02-28 18:00:46 +01:00
golderweb ef9c13324a Improve documentation of MysqlRed.flush() 2015-09-20 18:17:59 +02:00
golderweb e186f2f22b Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys 2015-09-20 17:45:07 +02:00
golderweb 7d6cd8bb30 Strip leading and trailing whitespace in Links to prevent wrong fam_hashes (when receiving redfam from db) since MySQL drops it 2015-09-19 22:44:43 +02:00
golderweb 4e21b6696a Remove unnecessary whitespace from error messages 2015-09-19 20:51:52 +02:00
golderweb 6992f82f02 Start Implementing of RedFamWorker 2015-09-19 20:51:21 +02:00
golderweb dbcfe8f106 Add a generator to MysqlRedFam to retrieve redfams from db by status 2015-09-19 19:50:38 +02:00
golderweb 8059bb9992 Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash 2015-09-19 19:49:20 +02:00
golderweb b5ca69077c Remove double appearence of heading parameter in repression of RedFam 2015-09-19 19:47:09 +02:00
golderweb 523d029fdc Fix bug causing db table cells containing empty strings 2015-09-19 19:45:34 +02:00
golderweb 4518efc504 Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected 
Reformat MySQL querys to remove whitespace generated by indetation
 2015-09-18 18:08:13 +02:00
golderweb b1b37f9b9e Implement functions for flushing db query caches 2015-09-17 20:00:13 +02:00
golderweb 8dc7fe678d Fix bug caused by adding fam_hash to repr of RedFam class since it was not defined yet while output of warning caused by to many articles 2015-09-17 19:57:53 +02:00
golderweb 53f53ddb8b Implement cached querys in MysqlRedFam 2015-09-17 19:56:39 +02:00
golderweb 26f5912f88 Collect writing db querys for running once in MysqlRedPage 
Add classmethod to MysqlRed for executing collected querys
 2015-09-16 21:02:02 +02:00
golderweb 1dea5d7e84 NOT WORKING Cache SQL querys to reduce amount of querys 2015-09-16 18:31:54 +02:00
golderweb b514eb5c42 Move configuration to jogobot module 
Use custom Error classes
 2015-09-15 21:21:05 +02:00
golderweb db5bb7401e Update RedFam class to rebuild the whole structure of RedFamPaser generated object 
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
 2015-09-15 21:19:07 +02:00
golderweb be0041804a Clean up 2015-09-13 13:19:22 +02:00
golderweb af6d483280 Use a additional module to store custom config ... 2015-09-13 13:14:37 +02:00
golderweb 7c9061d37f Use redundances as python package 2015-09-13 13:13:28 +02:00
golderweb 74b2dc727c Clean up PEP8 styleguide compatibility with flake8 2015-09-13 11:53:47 +02:00
golderweb cafe08dd7f Added license GPLv3 + Reworked code 
Better compatibility with https://www.mediawiki.org/wiki/Manual:Pywikibot/Development_guidelines
 2015-09-13 01:22:27 +02:00
golderweb ad100e96eb Remove timezone from timestamp patterns since it is not stored and causes errors on labs 2015-09-12 22:35:12 +02:00
golderweb c78480a75d Use mysql config from pywikibot user-config.py 2015-09-12 22:33:44 +02:00
golderweb 6e0d3200c2 Change visibility of attributes changed and parsed in RED_PAGE to protected 2015-09-12 14:24:45 +02:00
golderweb f50f4b664f Add additional possibilities to detect wether a red_page is an archive or not 2015-09-12 13:48:01 +02:00
golderweb e70655f679 Catch sections with more then 8 articles 2015-09-12 12:42:23 +02:00
golderweb 14e865c6e6 We need to save the red_fam section heading in database for generating section links since format is not consistent 2015-09-12 00:50:04 +02:00
golderweb a97e2cea5a Since some timestamps are broken we need to reconstruct them by regex match groups 
Prevents ValueErrors of datetime.strptime in most cases
 2015-09-11 23:47:35 +02:00
golderweb 6e64d8448e Prevent RED_PAGE from resetting status if parser is not fired 2015-09-11 22:29:16 +02:00
golderweb deaa1d855b Fix bug which prevents detecting of red_fam sectionheading when there is whitespace after closing heading tags (===) 2015-09-11 22:27:25 +02:00
golderweb 90da231712 Move the code for instanciating of RED_FAM_PARSER to the end of loop 2015-09-11 17:05:18 +02:00
golderweb 529ed09702 Use pywikibot.output instead of print 2015-09-11 16:42:12 +02:00
golderweb 1892c6d015 Fix wrong intendet line (RED_FAM_PARSER only was instantiated when ending-datetime workaround fired) 2015-09-11 16:41:04 +02:00
golderweb 99f050acd3 Add workaround to detect ending datetime if there is no done notice 2015-09-11 16:22:19 +02:00
golderweb 8b7dc5dbf2 Improve sectionheading regex to reduce false positives, require at least two links 2015-09-11 16:18:46 +02:00
golderweb 5e39ea0b06 Correct use of pywikibot page.title() 2015-09-11 14:12:26 +02:00
golderweb f8dacb53e1 Add param to RED_PAGE for predefining page status archived when using cat of archives 2015-09-11 14:10:33 +02:00
golderweb ebf7a8fe0b Add alternative syntax for done_notice 2015-09-11 13:56:16 +02:00
golderweb 47238f0f96 On archived red_fams do not delete possibly existing ending 2015-09-11 13:37:58 +02:00
golderweb 907ccad63a Detect if red_fam is archived but has no detectable ending, set status 2 anyway 2015-09-11 13:12:29 +02:00
golderweb 31ea512c41 Prevent overwriting of ending with None by only calling RED_FAM_PARSER if there was no ending before 2015-09-11 13:01:04 +02:00
golderweb 4f7cfc65cc Catch ValueErrors caused by missing points after month abreviation in RED_PAGE_PARSER.__timestamp 2015-09-10 23:09:35 +02:00
golderweb a26b92082b Implement parsing process in RED_PAGE.parse() 2015-09-10 23:08:51 +02:00
golderweb 984c269aa4 Implement classmethods for detecting sectionheading, beginning and ending in class RED_FAM_PARSER 2015-09-10 23:06:54 +02:00
golderweb 0f4ce7c4c9 Add workaround to get mysql.data dictionary while adding new pages/fams 2015-09-10 23:02:38 +02:00
golderweb 7535172d30 Update mysql db if anything of re_fam changes 2015-09-10 11:57:35 +02:00
golderweb 9e865e1c2f Some clean ups 2015-09-10 11:57:10 +02:00
golderweb 79e232d52b Add gitignore file 2015-09-10 11:46:05 +02:00
golderweb ea92415b37 Add dummy -method for parsing to RED_PAGE 2015-09-10 11:24:25 +02:00
golderweb 30e1672557 Implement methods for parsing red_fams 
Add Interfaces for RED_FAM
 2015-09-10 11:23:15 +02:00
golderweb 6d1ed33699 Add methods for MySQL actions while parsing RED_FAMs 2015-09-10 11:20:20 +02:00
golderweb 45df354315 Prevent descendant classes of MYSQL_RED from deleting connection to db 2015-09-10 11:18:22 +02:00
golderweb 73b5d87e8d Implemment handling of changed page meta data (rev_id, page_title, status) 2015-09-07 22:51:07 +02:00
golderweb cef90e4fac Handle building of MYSQL_RED_PAGE objects 2015-09-07 19:04:16 +02:00
golderweb c800a13f0c Change behavior of MYSQL_RED(_PAGE), handle save requested data in object for simple update handling 2015-09-07 19:00:04 +02:00
golderweb 0beb97d67d Use qmark parametrization style for mysql query 2015-09-07 16:21:57 +02:00
golderweb 773557a591 Use DictCursor ouf oursql to get better readability in MYSQL_RED_PAGE 2015-09-06 16:02:30 +02:00
golderweb d2dfa15ddf Implement a class RED_PAGE for handling redundance pages 2015-09-06 15:43:10 +02:00
golderweb 28d9a26cf1 Always handle beginning and ending definitions via setter functions 
Setter functions for beginning and ending decide wether it is a datetime object or a parseable timestamp string
 2015-09-06 15:04:49 +02:00
golderweb 122cc8b309 Set date attributes to none if nothing given, rather than don't set the attributes 2015-09-06 14:37:27 +02:00
golderweb 72410d17b4 Expand __init__ Method of RED_FAM to catch beginning, end and status 
Add a __repr__ Method to RED_FAM
 2015-09-05 18:36:30 +02:00
golderweb 10cfa79ee0 Implement a class/modul RED_FAM for handling redundance families 2015-09-05 18:06:17 +02:00
golderweb 49def6f0f0 Implement a class MYSQL_RED_FAM as desecendant of MYSQL_RED for handling querys about red_families 2015-09-05 17:13:09 +02:00
golderweb f5ac6a06d3 Implement a class MYSQL_RED_PAGE as desecendant of MYSQL_RED for handling querys about red_pages 2015-09-05 17:10:55 +02:00
golderweb 597df69275 Save mysqldb-connection as class atribute to make it usable for all decandant classes of MYSQL_RED 2015-09-05 17:09:01 +02:00
golderweb 71b9d3301d Initial structure of MySQL-Module 
Use class to use an object for all MySQL stuff
 2015-09-04 22:19:06 +02:00