Merge branch 'pull/6'

# Conflicts:
#	.gitignore

Author: qubitstream

.gitignore

@@ -62,5 +62,6 @@ target/
 
 .idea
 .vscode
+*.iml
 
 # Created by .ignore support plugin (hsz.mobi)

README.md

@@ -3,31 +3,36 @@
 
 ## What is it?
 
-A Python 3 script to __automate the download of SQL backups via a [phpMyAdmin](https://www.phpmyadmin.net/) web interface__.
+A Python 3 script to __automate the download of SQL backups via a 
+[phpMyAdmin](https://www.phpmyadmin.net/) web interface__.
 
-This is useful when your web hosting provider does not grant you access the the console (for `mysqldump`) but you want to automate the backup of your database (without having to manually use the browser).
+This is useful when your web hosting provider does not grant you access to a console (for `mysqldump`) but
+you want to automate the backup of your database (without having to manually use the browser).
 
-It has been tested with Python 3.4+ on Linux and Windows and phpMyAdmin 4.3.6, 4.5.4.1 and 4.7.0-dev
+It has been tested with Python 3.4+ on Linux and Windows and the following versions of phpMyAdmin:
+`4.3.x - 4.8.x, 5.0.0` 
 
-_Note_: The web interface of phpMyAdmin may change in the future and break this script. Please file a bug report (including your version of phpMyAdmin) if you encounter this issue.
+_Note_: The web interface of phpMyAdmin may change in the future and break this script. Please file a bug report
+(including your version of phpMyAdmin) if you encounter this issue.
 
 ## Usage
 
     usage: phpmyadmin_sql_backup.py [-h] [-o OUTPUT_DIRECTORY] [-p]
                                     [-e EXCLUDE_DBS]
-                                    [--compression {none,zip,gzip}]
+                                    [--compression {none,zip,gzip,bzip2}]
                                     [--basename BASENAME] [--timeout TIMEOUT]
                                     [--overwrite-existing]
                                     [--prefix-format PREFIX_FORMAT] [--dry-run]
+                                    [--http-auth HTTP_AUTH]
                                     URL USERNAME PASSWORD
-
+    
     Automates the download of SQL dump backups via a phpMyAdmin web interface.
-
+    
     positional arguments:
       URL                   phpMyAdmin login page url
       USERNAME              phpMyAdmin login username
       PASSWORD              phpMyAdmin login password
-
+    
     optional arguments:
       -h, --help            show this help message and exit
       -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
@@ -38,7 +43,7 @@ _Note_: The web interface of phpMyAdmin may change in the future and break this
       -e EXCLUDE_DBS, --exclude-dbs EXCLUDE_DBS
                             comma-separated list of database names to exclude from
                             the dump
-      --compression {none,zip,gzip}
+      --compression {none,zip,gzip,bzip2}
                             compression method for the output file - must be
                             supported by the server (default: none)
       --basename BASENAME   the desired basename (without extension) of the SQL
@@ -54,11 +59,12 @@ _Note_: The web interface of phpMyAdmin may change in the future and break this
                             format. Must be used with --prepend-date to be in
                             effect
       --dry-run             dry run, do not actually download any file
-      --http-auth           Basic http authentication, using format
+      --http-auth HTTP_AUTH
+                            Basic HTTP authentication, using format
                             "username:password"
-
-    Written by Christoph Haunschmidt, version: 2016-03-12.3
-
+    
+    Written by Christoph Haunschmidt et al., version: 2019-05-07.0     
+                                                                                   
 ### Examples
 
     phpmyadmin_sql_backup.py "http://www.example.com/phpmyadmin/" your_user your_password
@@ -69,12 +75,17 @@ Downloads a plain text `.sql` backup of all databases to the current working dir
 
     phpmyadmin_sql_backup.py "http://www.example.com/phpmyadmin/" your_user your_password --exclude-dbs mydb2,mydb4 --prepend-date --basename example_dump --output-directory /tmp --compression zip
 
-Downloads a zipped dump with databases `mydb2` & `mydb4` excluded, the base name `example_dump` and a prepended UTC date / time to the directory `/tmp`, e.g. `/tmp/2016-03-11--15-19-04-UTC_example_dump.zip`.
+Downloads a zipped dump with databases `mydb2` & `mydb4` excluded, the base name `example_dump` and a prepended
+UTC date / time to the directory `/tmp`, e.g. `/tmp/2016-03-11--15-19-04-UTC_example_dump.zip`.
 
 ## Requirements
 
  - A [Python 3.4+](https://www.python.org/) installation on your system
- - [Grab - python web-scraping framework](http://grablib.org/): Install via `pip install -U Grab` or see the [installation instructions](http://docs.grablib.org/en/latest/usage/installation.html) if you run into problems.
+ - [Grab - python web-scraping framework](http://grablib.org/): Install via `pip install -U Grab` or see 
+   the [installation instructions](http://docs.grablib.org/en/latest/usage/installation.html) if you run into problems.
+
+__Note for Windows users__: while it is possible to install the requirements natively, it is often easier to use the
+[Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10) if you are using Windows 10
 
 ## Changelog
 
@@ -84,6 +95,9 @@ Currently, there is no changelog; the best option at the moment is to read the c
 
 [GNU GPL3](https://www.gnu.org/licenses/gpl-3.0.html)
 
-## Author
+## Contributors
 
-- Christoph Haunschmidt
+ - Christoph Haunschmidt (original author)
+ - Jonas Bengtsson (older frame-based phpMyAdmin support)
+ - Benoît Courtine (HTTP Auth)
+ 

phpmyadmin_sql_backup.py

@@ -21,26 +21,35 @@
 # tested on Python 3.4+
 # requires: grab (http://grablib.org/)
 #
-# Christoph Haunschmidt 2016-03
+# Christoph Haunschmidt, started 2016-03
 
 import argparse
+import datetime
 import os
 import re
 import sys
-import datetime
 
 import grab
 
-
-__version__ = '2016-03-12.3'
+__version__ = '2019-05-07.0'
 
 CONTENT_DISPOSITION_FILENAME_RE = re.compile(r'^.*filename="(?P<filename>[^"]+)".*$')
 DEFAULT_PREFIX_FORMAT = r'%Y-%m-%d--%H-%M-%S-UTC_'
 
 
+def is_login_successful(g):
+    return g.doc.text_search("frame_content") or g.doc.text_search("server_export.php")
+
+
+def open_frame_if_phpmyadmin_3(g):
+    frame_url_selector = g.doc.select("id('frame_content')/@src")
+    if frame_url_selector.exists():
+        g.go(frame_url_selector.text())
+
+
 def download_sql_backup(url, user, password, dry_run=False, overwrite_existing=False, prepend_date=True, basename=None,
                         output_directory=os.getcwd(), exclude_dbs=None, compression='none', prefix_format=None,
-                        timeout=60, http_auth=None, **kwargs):
+                        timeout=60, http_auth=None):
     prefix_format = prefix_format or DEFAULT_PREFIX_FORMAT
     exclude_dbs = exclude_dbs.split(',') or []
     encoding = '' if compression == 'gzip' else 'gzip'
@@ -52,12 +61,13 @@ def download_sql_backup(url, user, password, dry_run=False, overwrite_existing=F
 
     g.doc.set_input_by_id('input_username', user)
     g.doc.set_input_by_id('input_password', password)
-    g.doc.submit()
+    g.submit()
+
+    if not is_login_successful(g):
+        raise ValueError('Could not login - did you provide the correct username / password?')
+
+    open_frame_if_phpmyadmin_3(g)
 
-    try:
-        g.doc.text_assert('server_export.php')
-    except Exception as e:
-        raise ValueError('Could not login - did you provide the correct username / password? ({})'.format(e))
     export_url = g.doc.select("id('topmenu')//a[contains(@href,'server_export.php')]/@href").text()
     g.go(export_url)
 
@@ -67,13 +77,13 @@ def download_sql_backup(url, user, password, dry_run=False, overwrite_existing=F
         print('Warning: no databases to dump (databases available: "{}")'.format('", "'.join(dbs_available)),
             file=sys.stderr)
 
-    file_response = g.doc.submit(
+    file_response = g.submit(
         extra_post=[('db_select[]', db_name) for db_name in dbs_to_dump] + [('compression', compression)])
 
-    re_match = CONTENT_DISPOSITION_FILENAME_RE.match(g.response.headers['Content-Disposition'])
+    re_match = CONTENT_DISPOSITION_FILENAME_RE.match(g.doc.headers['Content-Disposition'])
     if not re_match:
         raise ValueError(
-            'Could not determine SQL backup filename from {}'.format(g.response.headers['Content-Disposition']))
+            'Could not determine SQL backup filename from {}'.format(g.doc.headers['Content-Disposition']))
 
     content_filename = re_match.group('filename')
     filename = content_filename if basename is None else basename + os.path.splitext(content_filename)[1]
@@ -102,33 +112,37 @@ def download_sql_backup(url, user, password, dry_run=False, overwrite_existing=F
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(
         description='Automates the download of SQL dump backups via a phpMyAdmin web interface.',
-        epilog='Written by Christoph Haunschmidt, version: {}'.format(__version__))
+        epilog='Written by Christoph Haunschmidt et al., version: {}'.format(__version__))
 
     parser.add_argument('url', metavar='URL', help='phpMyAdmin login page url')
     parser.add_argument('user', metavar='USERNAME', help='phpMyAdmin login username')
     parser.add_argument('password', metavar='PASSWORD', help='phpMyAdmin login password')
     parser.add_argument('-o', '--output-directory', default=os.getcwd(),
-        help='output directory for the SQL dump file (default: the current working directory)')
+                        help='output directory for the SQL dump file (default: the current working directory)')
     parser.add_argument('-p', '--prepend-date', action='store_true', default=False,
-        help='prepend current UTC date & time to the filename; see the --prefix-format option for custom formatting')
+                        help='prepend current UTC date & time to the filename; '
+                             'see the --prefix-format option for custom formatting')
     parser.add_argument('-e', '--exclude-dbs', default='',
-        help='comma-separated list of database names to exclude from the dump')
-    parser.add_argument('--compression', default='none', choices=['none', 'zip', 'gzip'],
-        help='compression method for the output file - must be supported by the server (default: %(default)s)')
+                        help='comma-separated list of database names to exclude from the dump')
+    parser.add_argument('--compression', default='none', choices=['none', 'zip', 'gzip', 'bzip2'],
+                        help='compression method for the output file - must be supported by the '
+                             'server (default: %(default)s)')
     parser.add_argument('--basename', default=None,
-        help='the desired basename (without extension) of the SQL dump file (default: the name given by phpMyAdmin); '
-        'you can also set an empty basename "" in combination with --prepend-date and --prefix-format')
+                        help='the desired basename (without extension) of the SQL dump file (default: the name given '
+                             'by phpMyAdmin); you can also set an empty basename "" in combination with '
+                             '--prepend-date and --prefix-format')
     parser.add_argument('--timeout', type=int, default=60,
-        help='timeout in seconds for the requests (default: %(default)s)')
+                        help='timeout in seconds for the requests (default: %(default)s)')
     parser.add_argument('--overwrite-existing', action='store_true', default=False,
-        help='overwrite existing SQL dump files (instead of appending a number to the name)')
+                        help='overwrite existing SQL dump files (instead of appending a number to the name)')
     parser.add_argument('--prefix-format', default='',
-        help=str('the prefix format for --prepend-date (default: "{}"); in Python\'s strftime format. '
-                 'Must be used with --prepend-date to be in effect'.format(DEFAULT_PREFIX_FORMAT.replace('%', '%%'))))
+                        help=str('the prefix format for --prepend-date (default: "{}"); in Python\'s strftime format. '
+                                 'Must be used with --prepend-date to be in effect'.format(
+                            DEFAULT_PREFIX_FORMAT.replace('%', '%%'))))
     parser.add_argument('--dry-run', action='store_true', default=False,
-        help='dry run, do not actually download any file')
+                        help='dry run, do not actually download any file')
     parser.add_argument('--http-auth', default=None,
-        help='Basic http authentication, using format "username:password"')
+                        help='Basic HTTP authentication, using format "username:password"')
 
     args = parser.parse_args()
 
@@ -143,4 +157,4 @@ def download_sql_backup(url, user, password, dry_run=False, overwrite_existing=F
         sys.exit(1)
 
     print('{} saved SQL dump to: {}'.format(('Would have' if args.dry_run else 'Successfully'), dump_fn),
-        file=sys.stdout)
+          file=sys.stdout)