Initial commit of code and documentation

Author: lindonroberts

.gitignore

@@ -0,0 +1,4 @@
+*.pyc
+*/__pycache__
+*.egg-info
+docs/_build

LICENSE.txt

@@ -0,0 +1,104 @@
+==================================================================================
+Py-BOBYQA: Derivative-Free Solver for Bound-Constraint Minimization |PyPI Version|
+==================================================================================
+Py-BOBYQA is a flexible package for solving bound-constrained general objective minimization, without requiring derivatives of the objective. It is a Python implementation of the BOBYQA algorithm by Powell.
+
+More details about Py-BOBYQA can be found in our paper: C. Cartis, J. Fiala, B. Marteau and L. Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, technical report, University of Oxford, (2018).
+
+The original paper by Powell is: M. J. D. Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, technical report DAMTP 2009/NA06, University of Cambridge (2009), and the original Fortran implementation is available `here <http://mat.uc.pt/~zhang/software.html>`_.
+
+Documentation
+-------------
+See manual.pdf or `here <http://people.maths.ox.ac.uk/robertsl/pybobyqa>`_.
+
+Requirements
+------------
+Py-BOBYQA requires the following software to be installed:
+
+* Python 2.7 or Python 3 (http://www.python.org/)
+
+Additionally, the following python packages should be installed (these will be installed automatically if using *pip*, see `Installation using pip`_):
+
+* NumPy 1.11 or higher (http://www.numpy.org/)
+* SciPy 0.18 or higher (http://www.scipy.org/)
+* Pandas 0.17 or higher (http://pandas.pydata.org/)
+
+Installation using pip
+----------------------
+For easy installation, use *pip* (http://www.pip-installer.org/) as root::
+
+    $ [sudo] pip install pybobyqa
+
+or alternatively *easy_install*::
+
+    $ [sudo] easy_install pybobyqa
+
+If you do not have root privileges or you want to install Py-BOBYQA for your private use, you can use::
+
+    $ pip install --user pybobyqa
+
+which will install Py-BOBYQA in your home directory.
+
+Note that if an older install of Py-BOBYQA is present on your system you can use::
+
+    $ [sudo] pip install --upgrade pybobyqa
+
+to upgrade Py-BOBYQA to the latest version.
+
+Manual installation
+-------------------
+Alternatively, you can download the source code from `Github <https://github.com/numericalalgorithmsgroup/pybobyqa>`_ and unpack as follows:
+
+ .. code-block:: bash
+
+    $ git clone https://github.com/numericalalgorithmsgroup/pybobyqa
+    $ cd pybobyqa
+
+Py-BOBYQA is written in pure Python and requires no compilation. It can be installed using:
+
+ .. code-block:: bash
+
+    $ [sudo] pip install .
+
+If you do not have root privileges or you want to install Py-BOBYQA for your private use, you can use:
+
+ .. code-block:: bash
+
+    $ pip install --user .
+
+instead.
+
+Testing
+-------
+If you installed Py-BOBYQA manually, you can test your installation by running:
+
+ .. code-block:: bash
+
+    $ python setup.py test
+
+Alternatively, the HTML documentation provides some simple examples of how to run Py-BOBYQA.
+
+Examples
+--------
+Examples of how to run Py-BOBYQA are given in the `documentation <http://people.maths.ox.ac.uk/robertsl/pybobyqa>`_, and the `examples <https://github.com/numericalalgorithmsgroup/pybobyqa/tree/master/examples>`_ directory in Github.
+
+Uninstallation
+--------------
+If Py-BOBYQA was installed using *pip* you can uninstall as follows:
+
+ .. code-block:: bash
+
+    $ [sudo] pip uninstall pybobyqa
+
+If Py-BOBYQA was installed manually you have to remove the installed files by hand (located in your python site-packages directory).
+
+Bugs
+----
+Please report any bugs using GitHub's issue tracker.
+
+License
+-------
+This algorithm is released under the GNU GPL license.
+
+.. |PyPI Version| image:: https://img.shields.io/pypi/v/PyBOBYQA.svg
+                  :target: https://pypi.python.org/pypi/PyBOBYQA

README.md

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = python3 -msphinx
+SPHINXPROJ    = DFO-Gen
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

README.rst

@@ -0,0 +1,81 @@
+Advanced Usage
+==============
+This section describes different optional user parameters available in Py-BOBYQA.
+
+In the last section (:doc:`userguide`), we introduced :code:`pybobyqa.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the paper [CFMR2018]_.
+
+The default values, used if no override is given, in some cases vary depending on whether :code:`objfun` has stochastic noise; that is, whether evaluating :code:`objfun(x)` several times at the same :code:`x` gives the same result or not. Whether or not this is the case is determined by the :code:`objfun_has_noise` input to :code:`pybobyqa.solve()` (and not by inspecting :code:`objfun`, for instance).
+
+General Algorithm Parameters
+----------------------------
+* :code:`general.rounding_error_constant` - Internally, all interpolation points are stored with respect to a base point :math:`x_b`; that is, we store :math:`\{y_t-x_b\}`, which reduces the risk of roundoff errors. We shift :math:`x_b` to :math:`x_k` when :math:`\|s_k\| \leq \text{const}\|x_k-x_b\|`, where 'const' is this parameter. Default is 0.1.
+* :code:`general.safety_step_thresh` - Threshold for when to call the safety step, :math:`\|s_k\| \leq \gamma_S \rho_k`. Default is :math:`\gamma_S =0.5`.
+* :code:`general.check_objfun_for_overflow` - Whether to cap the value of :math:`r_i(x)` when they are large enough that an OverflowError will be encountered when trying to evaluate :math:`f(x)`. Default is :code:`True`. 
+
+Logging and Output
+------------------
+* :code:`logging.n_to_print_whole_x_vector` - If printing all function evaluations to screen/log file, the maximum :code:`len(x)` for which the full vector :code:`x` should be printed also. Default is 6.
+* :code:`logging.save_diagnostic_info` - Flag so save diagnostic information at each iteration. Default is :code:`False`.
+* :code:`logging.save_poisedness` - If saving diagnostic information, whether to include the :math:`\Lambda`-poisedness of :math:`Y_k` in the diagnostic information. This is the most computationally expensive piece of diagnostic information. Default is :code:`True`.
+* :code:`logging.save_xk` - If saving diagnostic information, whether to include the full vector :math:`x_k`. Default is :code:`False`.
+
+Initialization of Points
+------------------------
+* :code:`init.random_initial_directions` - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default is :code:`True`.
+* :code:`init.random_directions_make_orthogonal` - If building initial interpolation set with random directions, whether or not these should be orthogonalized. Default is :code:`True`.
+* :code:`init.run_in_parallel` - If using random directions, whether or not to ask for all :code:`objfun` to be evaluated at all points without any intermediate processing. Default is :code:`False`.
+
+Trust Region Management
+-----------------------
+* :code:`tr_radius.eta1` - Threshold for unsuccessful trust region iteration, :math:`\eta_1`. Default is 0.1. 
+* :code:`tr_radius.eta2` - Threshold for very successful trust region iteration, :math:`\eta_2`. Default is 0.7. 
+* :code:`tr_radius.gamma_dec` - Ratio to decrease :math:`\Delta_k` in unsuccessful iteration, :math:`\gamma_{dec}`. Default is 0.5 for smooth problems or 0.98 for noisy problems (i.e. :code:`objfun_has_noise = True`). 
+* :code:`tr_radius.gamma_inc` - Ratio to increase :math:`\Delta_k` in very successful iterations, :math:`\gamma_{inc}`. Default is 2. 
+* :code:`tr_radius.gamma_inc_overline` - Ratio of :math:`\|s_k\|` to increase :math:`\Delta_k` by in very successful iterations, :math:`\overline{\gamma}_{inc}`. Default is 4. 
+* :code:`tr_radius.alpha1` - Ratio to decrease :math:`\rho_k` by when it is reduced, :math:`\alpha_1`. Default is 0.1 for smooth problems or 0.9 for noisy problems (i.e. :code:`objfun_has_noise = True`). 
+* :code:`tr_radius.alpha2` - Ratio of :math:`\rho_k` to decrease :math:`\Delta_k` by when :math:`\rho_k` is reduced, :math:`\alpha_2`. Default is 0.5 for smooth problems or 0.95 for noisy problems (i.e. :code:`objfun_has_noise = True`). 
+
+Termination on Small Objective Value
+------------------------------------
+* :code:`model.abs_tol` - Tolerance on :math:`f(x_k)`; quit if :math:`f(x_k)` is below this value. Default is :math:`-10^{20}`. 
+
+Termination on Slow Progress
+----------------------------
+* :code:`slow.history_for_slow` - History used to determine whether the current iteration is 'slow'. Default is 5. 
+* :code:`slow.thresh_for_slow` - Threshold for objective decrease used to determine whether the current iteration is 'slow'. Default is :math:`10^{-8}`. 
+* :code:`slow.max_slow_iters` - Number of consecutive slow successful iterations before termination (or restart). Default is :code:`20*len(x0)`. 
+
+Stochastic Noise Information
+----------------------------
+* :code:`noise.quit_on_noise_level` - Flag to quit (or restart) if all :math:`f(y_t)` are within noise level of :math:`f(x_k)`. Default is :code:`False` for smooth problems or :code:`True` for noisy problems. 
+* :code:`noise.scale_factor_for_quit` - Factor of noise level to use in termination criterion. Default is 1. 
+* :code:`noise.multiplicative_noise_level` - Multiplicative noise level in :math:`f`. Can only specify one of multiplicative or additive noise levels. Default is :code:`None`. 
+* :code:`noise.additive_noise_level` - Additive noise level in :math:`f`. Can only specify one of multiplicative or additive noise levels. Default is :code:`None`. 
+
+Interpolation Management
+--------------------------------
+* :code:`interpolation.precondition` - whether or not to scale the interpolation linear system to improve conditioning. Default is :code:`True`.
+* :code:`interpolation.minimum_change_hessian` - whether to solve the underdetermined quadratic interpolation problem by minimizing the Frobenius norm of the Hessian, or change in Hessian. Default is :code:`True`.
+
+Multiple Restarts
+-----------------
+* :code:`restarts.use_restarts` - Whether to do restarts when :math:`\rho_k` reaches :math:`\rho_{end}`, or (optionally) when all points are within noise level of :math:`f(x_k)`. Default is :code:`False` for smooth problems or :code:`True` for noisy problems. 
+* :code:`restarts.max_unsuccessful_restarts` - Maximum number of consecutive unsuccessful restarts allowed (i.e.~restarts which did not reduce the objective further). Default is 10. 
+* :code:`restarts.rhoend_scale` - Factor to reduce :math:`\rho_{end}` by with each restart. Default is 1. 
+* :code:`restarts.use_soft_restarts` - Whether to use soft or hard restarts. Default is :code:`True`. 
+* :code:`restarts.soft.num_geom_steps` - For soft restarts, the number of points to move. Default is 3. 
+* :code:`restarts.soft.move_xk` - For soft restarts, whether to preserve :math:`x_k`, or move it to the best new point evaluated. Default is :code:`True`. 
+* :code:`restarts.hard.use_old_fk` - If using hard restarts, whether or not to recycle the objective value at the best iterate found when performing a restart. This saves one objective evaluation. Default is :code:`True`.
+* :code:`restarts.soft.max_fake_successful_steps` - The maximum number of successful steps in a given run where the new (smaller) objective value is larger than the best value found in a previous run. Default is :code:`maxfun`, the input to :code:`pybobyqa.solve()`.
+* :code:`restarts.auto_detect` - Whether or not to automatically determine when to restart. This is an extra condition, and restarts can still be triggered by small trust region radius, etc. Default is :code:`True`.
+* :code:`restarts.auto_detect.history` - How many iterations of data on model changes and trust region radii to store. There are two criteria used: trust region radius decreases (no increases over the history, more decreases than no changes), and change in model Jacobian (consistently increasing trend as measured by slope and correlation coefficient of line of best fit). Default is 30.
+* :code:`restarts.auto_detect.min_chg_model_slope` - Minimum rate of increase of :math:`\log(\|g_k-g_{k-1}\|)` and :math:`\log(\|H_k-H_{k-1}\|_F)` over the past iterations to cause a restart. Default is 0.015.
+* :code:`restarts.auto_detect.min_correl` - Minimum correlation of the data sets :math:`(k, \log(\|g_k-g_{k-1}\|))` and :math:`(k, \log(\|H_k-H_{k-1}\|_F))` required to cause a restart. Default is 0.1.
+
+
+References
+----------
+
+.. [CFMR2018]   
+   C. Cartis, J. Fiala, B. Marteau and L. Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, technical report, University of Oxford, (2018).
+

docs/Makefile

@@ -0,0 +1,183 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+# Py-BOBYQA documentation build configuration file, created by
+# sphinx-quickstart on Wed Nov  8 10:59:20 2017.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, '../pybobyqa/')
+from version import __version__
+
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ['sphinx.ext.mathjax']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = 'Py-BOBYQA'
+copyright = '2017, Lindon Roberts'
+author = 'Lindon Roberts'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = __version__
+# The full version, including alpha/beta/rc tags.
+release = __version__
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = '20 June, 2016'
+# Else, today_fmt is used as the format for a strftime call.
+today_fmt = '%d %B %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This patterns also effect to html_static_path and html_extra_path
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+html_title = 'Py-BOBYQA v%s documentation' % __version__
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# This is required for the alabaster theme
+# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars
+html_sidebars = {
+    '**': [
+        'about.html',
+        'navigation.html',
+        'relations.html',  # needs 'show_related': True theme option to display
+        'searchbox.html',
+        'donate.html',
+    ]
+}
+
+
+# -- Options for HTMLHelp output ------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Py-BOBYQAdoc'
+
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'Py-BOBYQA.tex', 'Py-BOBYQA Documentation',
+     'Lindon Roberts', 'manual'),
+]
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'py-bobyqa', 'Py-BOBYQA Documentation',
+     [author], 1)
+]
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'Py-BOBYQA', 'Py-BOBYQA Documentation',
+     author, 'Py-BOBYQA', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+
+