Jonathan Golder
a35546e53d
Fix removed pywikibot config property db_hostname
https://phabricator.wikimedia.org/rPWBC2d73643f70a3f3289ff83e7ec142727d79d2649c
3 years ago
Jonathan Golder
93447d8dc6
Prepare release v1.0
Update Copyright Notices
Version information
7 years ago
Jonathan Golder
1b6faf9e53
Use own db for red-task
Since we have several tables and sometimes need to create a copy on
replication servers.
7 years ago
Jonathan Golder
b4c193eedc
Disable echoing of SQLAlchemy Egine
We don't need this extensive output for production
7 years ago
Jonathan Golder
ff03ca8f13
Explicitly set charset for PyMySQL-Connection
Since PyMySQL-Connection otherwise uses charset 'latin-1', explicitly
set connection charset to 'utf8'
http://docs.sqlalchemy.org/en/rel_1_0/dialects/mysql.html#charset-selection
http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html?highlight=url#sqlalchemy.engine.url.URL
Related Task: [FS#156](https://fs.golderweb.de/index.php?do=details&task_id=156 )
7 years ago
Jonathan Golder
30de2a2e12
Replace oursql with PyMySQL
Since this is prefered on toolsforge and works out of the box after
installing via pip, replace oursql which caused some problems.
Especially oursql was not able to connect to db via ssh tunnel.
Related Task: [FS#144](https://fs.golderweb.de/index.php?do=details&task_id=144 )
7 years ago
Jonathan Golder
281f1c49a8
mysqlred: Set family via pywikibot
Get family/language part of table names from PyWikiBot Site
7 years ago
Jonathan Golder
3fe47e666f
Fix polymorphism problem with relationships
Since we are using subclasses of the ORM mapped classes, disable
typechecks for ORM relations
7 years ago
Jonathan Golder
e16925197c
Fix pep8.. compliance
To be concordant with the coding styles fix pep8 compliance
7 years ago
Jonathan Golder
43e31c108a
Working RedFamWorker query
Modify RedfamWorker class to work with new DB API
7 years ago
Jonathan Golder
bf8e47f916
Improve new status API
Make sure state changes are only detected as such by sqlalchemy if they
are real changes
7 years ago
Jonathan Golder
467f829af2
Some cleanups
Remove old commented out code from manual mysql solution
7 years ago
Jonathan Golder
6e973369cd
sqlalchemy working for parser
Needs some testing, presumably contains some bugs
8 years ago
Jonathan Golder
0ebf307bb8
Add markpages as subtask
Markpages is a subtask of our Red-Bot
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=89 FS#89]
# The following line will be added automatically
# Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=88 FS#88]
8 years ago
Jonathan Golder
e13320820c
Add API to manage status per article
To be able to track changes to articles to update redfam status
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=89 FS#89]
8 years ago
Jonathan Golder
6149dcdb8b
Apply changes to data structure
See related ticket
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=94 FS#94]
8 years ago
Jonathan Golder
b36dc250d2
Request information about reddisc page for redfams
To generate links to related reddisc it is necessary to get at least the
Title of the related reddisc page. As saving the same data in db is
worse, we retreive it via join from red_pages table
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
594130c8a6
Restore changes from 45df35431
Documented to prevent deleting again
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
58dfd8c86a
For RedFamilies not fetched individually we need to provide the fam hash as index
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
9481116777
Add new generator-method to fetch RedFams by Status and Ending
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=26 FS#26]
8 years ago
Jonathan Golder
71b99b5f58
Delay definition of db_table_prefix
db_table_prefix should be defined at init of MysqlRed and not at import
to have cmdline args already parsed
Otherwise it uses default family
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
8 years ago
Jonathan Golder
77d1de4473
Add a tablename prefix depending on Site
To be able to run the bot on different wikis the db tables should be
named pywikibot.Site dependend and changed automatically
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=79 FS#79]
8 years ago
Jonathan Golder
e28acf88d1
Introduce new directory structure
To clarify which is a bot and which are helper scripts
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=74 FS#74]
8 years ago
Jonathan Golder
bd2d221c48
Prevent flush from creating cursor without con
MysqlRed.flush() tried to create a cursor in any case. If there was no
connection (because the subclasses haven't been instantiated an oursql
Error occured.
Instead, check before if there is a connection and otherwise raise an Error
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=75 FS#75]
8 years ago
Jonathan Golder
bd9dbdfa17
Make use of declared db_host_port
The port to connect to MySQL-Server was previously always assumed as
the default one. So the library was incompatible to db's on nonstandard
ports
Related Task: [https://fs.golderweb.de/index.php?do=details&task_id=68 FS#68]
8 years ago
Jonathan Golder
f53a5b3745
Output a warning if there are update/insert querys cached when exit programm
8 years ago
Jonathan Golder
24adafeee7
Changes for new jogobot-module
8 years ago
Jonathan Golder
ef9c13324a
Improve documentation of MysqlRed.flush()
9 years ago
Jonathan Golder
e186f2f22b
Use dictionary with page_id / fam_hash as key for cached_insert_data to prevent double entrys
9 years ago
Jonathan Golder
dbcfe8f106
Add a generator to MysqlRedFam to retrieve redfams from db by status
9 years ago
Jonathan Golder
8059bb9992
Change behavior of MysqlRedFam to be able to get instance without knowen fam_hash
9 years ago
Jonathan Golder
4518efc504
Fix bug (Cached querys not executed) caused by class attribute protection level --> changed from private to protected
Reformat MySQL querys to remove whitespace generated by indetation
9 years ago
Jonathan Golder
53f53ddb8b
Implement cached querys in MysqlRedFam
9 years ago
Jonathan Golder
26f5912f88
Collect writing db querys for running once in MysqlRedPage
Add classmethod to MysqlRed for executing collected querys
9 years ago
Jonathan Golder
1dea5d7e84
NOT WORKING Cache SQL querys to reduce amount of querys
9 years ago
Jonathan Golder
db5bb7401e
Update RedFam class to rebuild the whole structure of RedFamPaser generated object
Move fam_hash() method from RedFamParser to RedFam
Define custom Error classes
9 years ago
Jonathan Golder
be0041804a
Clean up
9 years ago
Jonathan Golder
af6d483280
Use a additional module to store custom config ...
9 years ago
Jonathan Golder
74b2dc727c
Clean up PEP8 styleguide compatibility with flake8
9 years ago
Jonathan Golder
cafe08dd7f
Added license GPLv3 + Reworked code
Better compatibility with https://www.mediawiki.org/wiki/Manual:Pywikibot/Development_guidelines
9 years ago
