〽Optimisation〽

• A repository contatining tutorials on the most used optimisation packages implemented in Python programming language, covering:
  • Gradient-based methods
  • Gradient-free methods
  • Single-objective optimisation
  • Multi-objectve optimisation
  • Bayesian optmisation

---

Classification

• Convex optimization, is the problem of minimizing a convex function over convex constraints. It is a class of problems for which there are fast and robust optimization algorithms, both in theory and in practice..
• Linear programming (LP), also called linear optimization, is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships.
• Local optimisation
• Global optimisation
• Mixed-Integer Optimization solves convex optimization problems where some of the decision variables can only have discrete or integer values.
• Parametric optimisation is a type of mathematical optimization, where the optimization problem is solved as a function of one or multiple parameters
• Symbolic optimisation
• Fitting optimisation

---

Open source packages

• Facebook nevergrad A gradient-free optimization platform.
• HumpDay A Package to Help You Choose a Python Global Optimizer.
• Scikit-Optimise Scikit-Optimize, or skopt, is a simple and efficient library to minimize (very) expensive and noisy black-box functions. It implements several methods for sequential model-based optimization. skopt aims to be accessible and easy to use in many contexts.The library is built on top of NumPy, SciPy and Scikit-Learn. We do not perform gradient-based optimization.
• SciPy Optimize SciPy optimize provides functions for minimizing (or maximizing) objective functions, possibly subject to constraints. It includes solvers for nonlinear problems (with support for both local and global optimization algorithms), linear programing, constrained and nonlinear least-squares, root finding, and curve fitting.
• CVXOPT CVXOPT is a free software package for convex optimization based on the Python programming language. It can be used with the interactive Python interpreter, on the command line by executing Python scripts, or integrated in other software via Python extension modules. Its main purpose is to make the development of software for convex optimization applications straightforward by building on Python’s extensive standard library and on the strengths of Python as a high-level programming language
• CVXPY is an open source Python-embedded modeling language for convex optimization problems. It lets you express your problem in a natural way that follows the math, rather than in the restrictive standard form required by solvers. At the end of the day CVXPY is a wrapper that tries to make things easier.

---

Proprietary packages

• Gurobi
• MOSEK is a software package for the solution of linear, mixed-integer linear, quadratic, mixed-integer quadratic, quadratically constraint, conic and convex nonlinear mathematical optimization problems.
• FICO Xpress optimizer is a commercial optimization solver for linear programming, mixed integer linear programming, convex quadratic programming, convex quadratically constrained quadratic programming, second-order cone programming and their mixed integer counterparts.

---

New entries

• Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

---