EPP_Project_Gunnemann_Wardenbach.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EPP Project: Adding scalar function benchmark problems to estimagic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "by Alyssa Gunnemann and Luis Wardenbach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from multiprocessing import cpu_count\n",
    "from estimagic import get_benchmark_problems\n",
    "from estimagic import run_benchmark\n",
    "from estimagic import profile_plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We introduce a benchmark set for estimagic consisting of 283 problems for 78 functions. The functions are based on [a collection of optimization problem functions by Axel Thevenot](https://tinyurl.com/2p8d8c5p). We verified function maths, aswell as the minima using [online resources on optimization test problems](https://www.sfu.ca/~ssurjano/optimization.html) by the Simon Fraser University and the 2013 paper [\"A Literature Survey of Benchmark Functions for Global Optimization Problems\"](https://arxiv.org/pdf/1308.4008.pdf) by Jamil & Yang . We wrote parameterized unit tests, similarly to those for the \"more_wild\" problem set, to cover our newly added functions and problems. Our work is collected in [this](https://github.com/OpenSourceEconomics/estimagic/pull/271) pull request.\n",
    "\n",
    "Due to our project focusing on adding scalar benchmark functions to estimagic, we decided to summarize and describe our project in a Jupyter Notebook instead of using the template discussed in class. The template is helpful when documenting research projects, but since we added \"scalar_functions\" to an existing project in a pull request, our project has a different structure and it does not fit the structure of the template."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The \"scalar_functions\" benchmark set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We expanded estimagic's get_benchmark_function to be able to select a subset of problems by function input dimension. For example, the following code will fetch all problems where the function input dimension is equal to 4.\n",
    "```\n",
    "problems = get_benchmark_problems(\"scalar_functions\", lower_input_dim=4 ,upper_input_dim=4)\n",
    "\n",
    "```\n",
    "The problem set can be subdivided into four categories of function input dimension: Problems of dimension 2, with dimension between 3-5, with dimension between 10-20 and with dimension greater than 100. The following code constructs an overview of the size of the problem subsets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tiny      109\n",
       "small      58\n",
       "medium     56\n",
       "large      54\n",
       "total     277\n",
       "dtype: int64"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_dimensions = {\n",
    "    \"tiny\": (2, 2),\n",
    "    \"small\": (3, 5),\n",
    "    \"medium\": (10, 20),\n",
    "    \"large\": (100, None),\n",
    "}\n",
    "\n",
    "problems = {\n",
    "    key: get_benchmark_problems(\n",
    "        \"scalar_functions\", lower_input_dim=lo, upper_input_dim=up\n",
    "    )\n",
    "    for key, (lo, up) in input_dimensions.items()\n",
    "}\n",
    "\n",
    "info = pd.Series({key: len(value) for key, value in problems.items()})\n",
    "info[\"total\"] = info.sum()\n",
    "info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizer performance on the new problem sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To show that our problems are well-constructed we will benchmark three of estimagic's algorithms using the 3-5 dimension problem set, as well as the 10-20 dimension problem set. The chosen optimizers are Nelder-Mead, L-BFGS-B and BOBYQA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizers = [\n",
    "    \"scipy_neldermead\",\n",
    "    \"scipy_lbfgsb\",\n",
    "    \"nlopt_bobyqa\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimizer performance on the small dimensional problem set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGoCAYAAABL+58oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABQF0lEQVR4nO3dd3xc9ZX//9fRqFuu2AY3YtNtY9NEMdUGEiAQTAKBEIhD/yahZ0OS/SUsJmVTYDeEhCVrshhSKAECOISSUAwEArhgcAGCcQEXsNxky6qjOb8/7p3RSFYZyRrNaPR+Ph7z8L137r1zZiQxh/Np5u6IiIiI5JK8TAcgIiIi0t2U4IiIiEjOUYIjIiIiOUcJjoiIiOQcJTgiIiKSc/IzHUBnnXrqqf70009nOgwRERHpGdaVi3pdBWfjxo2ZDkFERESyXK9LcEREREQ6ogRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHKOEhwRERHJOUpwREREJOekLcExs7vNbIOZLWnjeTOz281suZm9bWaHpisWERER6VvSWcG5Bzi1nedPA/YNH1cAd6YxFhEREelD0pbguPtLwOZ2TpkO/M4DrwGDzGxEuuIRERGRviOTfXBGAR8l7a8Jj4mIiIjskvxMB5AKM7uCoBmLffbZhwULFmQ4IhEREYkrKChg8uTJmQ6jmUwmOGuBMUn7o8NjO3H3WcAsgPLycj/ssMPSH52IiIj0WplsopoDzAhHUx0FVLr7+gzGIyIiIjkibRUcM7sfmAoMNbM1wE1AAYC7/wZ4EvgssByoBi5OVywiIiLSt6QtwXH38zt43oEr0/X6IiIi0ndpJmMRERHJOUpwREREJOcowREREZGcowRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHKOEhwRERHJOUpwREREJOcowREREZGcowRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHKOEhwRERHJOUpwREREJOcowREREZGcowRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHKOEhwRERHJOUpwREREJOcowREREZGcowRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHKOEhwRERHJOUpwREREJOcowREREZGcowRHREREco4SHBEREck5SnBEREQk5yjBERERkZyjBEdERERyjhIcERERyTlKcERERCTnKMERERGRnKMER0RERHJOWhMcMzvVzN4zs+Vm9t1Wnt/TzF4wszfN7G0z+2w64xEREZG+IW0JjplFgDuA04AJwPlmNqHFad8H/uTuhwBfAv4nXfGIiIhI35HOCs4RwHJ3X+Hu9cADwPQW5zgwINweCKxLYzwiIiLSR+Sn8d6jgI+S9tcAR7Y4ZybwNzO7GugHnJzGeERERKSPSGeCk4rzgXvc/b/MbArwezM70N1jySeZ2RXAFQB77rlnBsJsX2zLKmI7NmY6DBERkYyJDJ+AFZZmOoyEdCY4a4ExSfujw2PJLgVOBXD3f5pZMTAU2JB8krvPAmYBlJeXe7oC7oqG956i9skbMh2GiIhIRpVe+AiRYftnOoyEdPbBmQfsa2bjzKyQoBPxnBbnfAicBGBm44FioCKNMXW7xo/eyHQIIiIi0kLaKjjuHjWzq4BngAhwt7svNbMfAPPdfQ7wb8BdZnY9QYfji9w9qyo0HfGaLQDYgJEUf/qHGY5GREQkM/IGjun4pB6U1j447v4k8GSLY/+RtL0MOCadMaSb12wFIK//CPL3bNmHWkRERDJBMxnvIq/ZDICVDM5wJCIiIhKX6VFUvZbXVdG47s3E6CklOCIiItlDCU4XVT9yKbFPlib2rXRIBqMRERGRZGqi6oJYVUWz5AbLIzJG/W9ERESyhSo4XRDb0JTcFJ08k/xxJ5BXNiyDEYmIiEgyJThd0PjJssR2wb6fwYoHtHO2iIiI9DQ1UXVBY9g8ZYM+peRGREQkC6mC0wlev4Po6leJfbwYgMjuEzMckYiIiLRGCU4n1D71XaIrXkjsK8ERERHJTmqi6oTGT5Y07RSUkr/3iZkLRkRERNqkCk6K3D2x7lT+PidRfMpPsmpZeBEREWmiCk6q6rZDLApAZOQhSm5ERESymBKcFHnt1sS2lmUQERHJbkpwUhTb+mFiWwmOiIhIdlOCk6K6F3+W2FaCIyIikt2U4KTIY7HEdt6wAzIYiYiIiHRECU6q6rYDUHDgOVikIMPBiIiISHuU4KTI64MER0sziIiIZD8lOCnwaB00NgQ7Rf0zG4yIiIh0SAlOCjxsngIwJTgiIiJZTwlOKpTgiIiI9CpKcFKgCo6IiEjvogQnBfEOxgAUlmUuEBEREUmJEpwUeG1yBUejqERERLKdEpwUeH1VYtuKVMERERHJdkpwUhB954nEtio4IiIi2U8JTgdiW1bTuHZ+04GCkswFIyIiIilRgtOB2Pb1ie28PSZjZhmMRkRERFKhBKcj3rTIZtHx38pgICIiIpIqJTgdSUpwzPRxiYiI9Ab6xu5IrLFpOy+SuThEREQkZfmZDiDruTdtq4Ij0i53p7Kqjo8qKlmzYRsfb66iMeYdXygivd7njx/PbgOyZyCOEpwOuCdVcJTgiAAQizkVlTtYs2EbH23YxkcV21izYRtrKrZRVVOf6fBEJANOPmwvJTi9SlIfHCU40tc0RBtZv6mKNWEC81FFkMSsrdhGXUNjh9cX5OdRmK+mXZG+INsGGSvB6YgSHOkDauoaWFOxnTVh01K8IrN+cxWxFJqYykoKGT1sAGOGB4/RwwYwevgAhg3sR15elv1XT0T6BCU4HVETleSQyqpa1lQEzUprKpoSmY2V1Sldv9uAEkaHCcyYMIkZM2wgA8uKNEeUiGSVNhMcMzu0vQvdfWH3h5OFkjoZm0ZRSS/g7mysrE4kMckVmW3VdR1en2fGHkPKmhKZeEVm2ABKiwt64B2IiOy69io4/xX+WwyUA28BBkwG5gNT0htaloipgtOeRe9/zMuLP8RdI2UyraGhkbWbtrO2Yju19dEOzy/Iz0skLsmJzMih/SlQvxkR6eXaTHDcfRqAmf0ZONTdF4f7BwIzeyS6bKBh4m2qqWvgJ3/8R0pfppI5/YoLEn1ikhOZ4YP7EcnT77SI5KZU+uDsH09uANx9iZmNT2NM2UV9cNq08F/rE8nN4P7F+rLMsLw8Y4/B/Rg9fGCzRGZw/2L1jxGRPieVBOdtM/st8Idw/wLg7fSFlF1co6ja9OqSNQAUF+Yz61ufo7BAzRoiIpIdUklwLga+Dlwb7r8E3Jm2iLJNswRHX+Bx9Q2NzH9vHQCHHzBSyY2IiGSVDhMcd681s98AT7r7ez0QU3ZpluCozB+3aPnHieapKRPHZDgaERGR5jpMcMzsTOAWoBAYZ2YHAz9w9zPTHFt2yLHFNj/8pJJHXnqHul3sGPzhJ5UAFBZEOGz/Ed0RmoiISLdJpYnqJuAIYC6Auy8ys3HpDCqrJFVwLAf64Pz60Td478NN3Xa/Q/cdQXGh5osUEZHskso3U4O7V7YYhdF3Jj3JoU7GazduSyQ3QwaUUFZcuEv3Kyst5PyTD+yO0ERERLpVKgnOUjP7MhAxs32Ba4BX0xtWFsmhTsZz31yd2P7hpdMYPWxABqMRERFJn1RKElcDE4E64D6gErgujTFlFc+RTsbuzouLVgGw7+ghSm5ERCSnpVLBOcDdvwd8L93BZKVmE/313grOO6s38smWHQBMPXhsZoMRERFJs1QSnP8ysz2Ah4EH3X1JmmPKKrFP3mnayZIKzkcbKvnj3xdT04mRUJ9srgIgkmccN3nPdIUmIiKSFVKZB2damOCcC/yvmQ0gSHR+lPboskBs66qmnSwZJv6/cxaweMWGLl176H4jGFhW3M0RiYiIZJeUxve6+8fA7Wb2AvBt4D+APpHgUFCa2MyGYeJrNmxLJDe7D+7HoE4kK/1KCvnqqQelKzQREZGskcpEf+OB84CzgU3Ag8C/pTmu7NFQA0D+3idmOJDAM/M+SGzfdPEJjBqqzsIiIiItpVLBuZsgqTnF3delOZ6s49HaYCM/8806dQ1RXli4EoDJew1XciMiItKGVPrgTOmJQLJWWMGxgpIMBwKvLvmI7TX1AJxyxD4ZjkZERCR7tZngmNliWp+x2AB398lpiypLuDu+oyLYyVAFp7Kqlt/MWcCmymo+DkdCDexXxJETRmUkHhERkd6gvQrOGT0WRZaKbVqetLNri1N21d/mf8CrSz5qduzk8r0oyM+OEV0iIiLZqM0Ex90T8/qb2e7A4eHuG+7etTHKvU19VWIzMvLgjISwan2wandxYT4HfGoouw0o4ZwTJmQkFhERkd4ilVFU5wK3EKwmbsCvzOwGd384zbFlnMeaZjG2fsMzEsOaim0AjP/UUGZePDUjMYiIiPQ2qYyi+h5weLxqY2bDgGcJZjbObcnrUGVgkr/GWIy1G4MEZ8zwgT3++iIiIr1VKjPX5bVoktqU4nWY2alm9p6ZLTez77ZxzrlmtszMlprZfanct8ck97vJwCR/G7bsoCEaJFljhmtIuIiISKpSqeA8bWbPAPeH++cBT3Z0kZlFgDuATwNrgHlmNsfdlyWdsy/w78Ax7r7FzDLTDtSWpAqO9XAFZ0dtPVfd9lRiXwmOiEjf5u54fT1eU0ustgavqSFWUxv+W0Ostmnba2qDf2uDc+LbxGIdv1AXDf3m9RSMHJm2+3dWKvPg3GBmXwCODQ/NcvdHU7j3EcByd18BYGYPANOBZUnnXA7c4e5bwtfKrs7LzSo4PZvgvLZ0DdHGpl9ENVGJiPQ+7o7X1TUlIfGEpDYpCQmTk8R2K+fEt9OZoOwqb8jMaOO2pLoW1Z/N7CXgOODDFO89Ckge37wGOLLFOfsBmNkrQASY6e5Pp3j/tPMM9sGpqmlIbH/tzMMoKyns0dcXEemrPBbD4wlHbS2x6haVkJbVkvDc5pWTpn/x1qaUSy8rKSGvuBgrKSavpAQrLiGvpBgi6fsuyyvN/IS4ydqb6O8J4LvuvsTMRgALgfnAXmZ2l7vf1k2vvy8wFRgNvGRmk9x9a4tYrgCuANhzzz274WVTlDSKqqcTnLqkTPjk8r169LVFRHobb2xMrRJSUxsmK20049TU4HV1Pf8GzBJJSV5pPCEpCRKU4hKsNJ6wBMebkpfS8JxirKSUvJJirKgIy8v84tCZ1l4FZ5y7Lwm3Lwb+7u4zzKw/8ApwWwf3XguMSdofHR5LtgZ43d0bgJVm9i+ChGde8knuPguYBVBeXt5zqXBygtPDTVR19UGCE8kzTeonIjnJo9FmyUispjrsX9KiOtKyglK7c/Li9fU9/wYikbA6ElZJSoIqSbBf2rRd2lRBCZKYpO3weisqwsx6/j3ksPYSnIak7ZOAuwDcfbuZpdIIOA/Y18zGESQ2XwK+3OKcx4DzgdlmNpSgyWpFaqH3AE+aB6eHR1HVNQSvXVSQUiuiiEjauTve0NBmdWSnCkptLV7del+SWE0NRDPQZ6OgoFn1o1kzTkkrVZLSICFpSl5KyAurKRQUKCnJYu19e35kZlcTVFkOBZ4GMLMSoKCjG7t71MyuAp4h6F9zt7svNbMfAPPdfU743GfMbBnQCNzg7pt26R11pww2Uf1t3gcAFBWqepNNGisr2fbYHGoWLMAbGzu+QCQXhB1lY7W1kIHfeyssbNGXpKnpJqiOtJGgxBOS+PklJVi+/qexr2jvJ30p8APgZOC8pH4xRwGzU7m5uz9JiyHl7v4fSdsOfDN8ZB/PTBNVYyyWmP+mpLDDXFJ6QGzHDrY98Ve2P/lUZtrnRXqZIOko3rkvSSuVkERfkvg1YfNOPFmxNHaMldzV3lpUG4CvtXL8BeCFdAaVLTxDFZzVH1cSC3vdH7rfiB57XdlZrK6OqqefYducvxDbsSNxvOiAA8jfY/cMRibSs6ywsHmVpLSVviTx54qL1clVMk61uvZkqJPx0pVN0wGdcfS+Pfa60sSjUaqef4Ftjz5K45atieOFe+/NoC+dR/GkAzMXnIiIdEgJTnsyNA/O0lUVAAzuX8weQ8p67HUlmP+i+pVXqXzoYaIbmhLNgtGjGXjeFykpL1enQhGRXkAJTnuSZjLuqaUa3J2lK4MEZ+LY4foy7SHuTs38+VQ++BANa9YkjkeGD2PQOedQeuwxKrmLiPQiHSY4ZvZz4EdADcFIqsnA9e7+hzTHlnHR1a827aRhmPg7qyu47aHX2F7dNH+DO1TXBSP0J44b1u2v2Vc0rFlDxX//gsatlaldEM5cGpc3aBADv/B5yk6cplEXIiK9UCr/5f6Mu3/bzD4PrAK+ALwE5HyC02wtqvzibr/9M298wMebd7T6nBkctI86sXaFu7P5t3cTXbe+09fm9evHgDM/R9mpp5BXVJSG6EREpCekkuDEzzkdeMjdK/tMs0le08djke4frr21KqgYDO5fzLGTmi9BMWHsMEYN1QriXVHzxjzq3n0XgKKJEyncc0wHVwQiu+1G2bSp5PXrl8boRESkJ6SS4DxhZu8SNFF93cyGAbUdXJMbwk7GeXtMTsvtt+0I5lPZa+RgLjvj0LS8Rl/jDQ1s/eN9AOT1L2PY9deSV6aO2iIifU2HHUvc/bvA0UB5uGbUDmB6ugPLCvGJ/tJUsaoME5wBpWoK6S7bn346Mfpp4NlnK7kREemjUulkHAGOBcaaWfL5/522qLJFWMGxNMyB4+6JCs7AfkpwukNjZSWVf34MgPyRIyk7+aTMBiQiIhmTShPVXwiapBYDqSyymTvi8+CkYQTV/PfWUR8NKkT9leB0idfXs+E/f0rde+8lHQxmgB78lQs1+klEpA9L5RtgtLunpxNKtouFCU4a5j+Z84+mL+Uxwwd2+/37gpq33050Jk5WfNBkSg45uOcDEhGRrJFKgvOUmX3G3f+W9miyjKepgrO9uq7ZbMWH7z+yW+/fV9QuXhJsRCIMOPNzQLDAX9mJ0zIYlYiIZINUEpzXgEfNLA9oAIxgIfDcH8OcpgRn3rvraIwFTSnfPHcKeXl9ZNh9N6tdshSAov32Y9B552Y4GhERySapJDj/DUwBFruHHRz6ikSC072djF9d8hEA/UsLmThWsxV3RXTzFqJr1wJQfODEDEcjIiLZJpUE5yNgSZ9LbiApwemeCktjLMb26noWLf8YgCMnjCYS0fpGXVG3ZEliu/hArewtIiLNpZLgrADmmtlTQF38oLv3gWHiwSin7hgmvnbjNr531/Ns2d40R+LRE0fv8n37qnjzlJWUULj3XhmORkREsk0qCc7K8FEYPvqObqzgzH93XbPkZlBZMZP31lpTXeHu1IYVnKLxB2g4uIiI7KTDbwZ3vxnAzErdvTr9IWWRWPf1wamtb1q48+LTDuaw/UdQkN/9Ewj2BdH162ncvBlQ85SIiLSuww4gZjbFzJYB74b7B5nZ/6Q9sqzQffPg1NYHzV35kTzOOu4AzX2zC+LNUwDFk5TgiIjIzlL55r4NOAXYBODubwHHpzGmrOGx7hsmXtcQVHCKClS12VXx5qm8gQMpGK1+TCIisrOUOi+4+0fWvB9KY3rCyTLdOA9OXUPwkRUXqr9IV8Tq6/EdO3CH2qVBBad44kQsTQuhiohI75bSMHEzOxpwMysArgXeSW9YWaIbE5xn568AoKhACU5n1a9cySc3/xCvrW12XM1TIiLSllS+ub8GXAmMAtYCB4f7ua8bJ/orKwkGoH28pWqX79XXVL8xb6fkxgoKKD6oby6RJiIiHUtlFNVG4IIeiCX7hAlOdzSDRBuDe514yNhdvldfE63YCEBe//4MDJdkKNp3H/KHDMlkWCIiksXaTHDM7FdAm7MXu/s1aYkom8TCrkbdUMFpiAb3GlRWvMv36msaK4KFSQtGjqT/ySdlOBoREekN2qvgzO+xKLJW9wwTj8U8sbim5r7pvOjGoIITGTY0w5GIiEhv0WaC4+73Ju+b2YDgsG9Pe1RZwhsbgo1d7GTc0Ng06Cw/X2tPdYZHozRu2gRA/lAlOCIikppUJvorN7PFwNvAEjN7y8wOS39omeXReqgLc7ldTXCiscR2QUQVnM7Y8JOfJbaV4IiISKpSGbN8N/ANd38ZwMyOBWYDOT2EJbZlRWLb63Zt5FM0OcFRBSdl3thI3bvvJvYL998vg9GIiEhvksq3bWM8uQFw938A0XbOzw2xpGalfU/epVslN1EpwUld9JNPIPzsyk49hcIxYzIckYiI9BbtjaI6NNx80cz+F7ifYFTVecDc9IeWYd5UdbG8XZucT01UXdOwdm1iu98xR2cwEhER6W3a++b+rxb7NyVttzl8PGfEkopUuzhMPD5EHFTB6YyGNU0JTsGoURmMREREepv2RlFN68lAsk1ioU3ohlFUTffK1zDxlLg71a+/DkBk8GDySkszHJGIiPQmqYyiGmhm/21m88PHf5nZwJ4ILqM8aT3RvF2t4CQ3UamCk4rNd/2WhlWrAcgfNTLD0YiISG+Tyrft3cB24NzwsY1gFFVuS+pkrCaqnlez8M3EdsnknB6wJyIiaZBK79m93f3spP2bzWxRmuLJHkkVHNvFCk60MXmYuJqoOuL19cS2bgWgeNIk+p9xemYDEhGRXieVckJNOPcNAGZ2DFCTvpCyRKwbm6ga1UTVGdFw5mKA0mOmYLu4VIaIiPQ9qVRwvgb8LqnfzRbgq+kLKUt4cifjXazgJDVR5SvB6VBjuPYUQP7QYRmMREREeqt2ExwziwBfcfeDwrWocPdtPRJZhnnyMPEuVBBiMWf52s3U1kdZsW5L4riaqDpW9/7yxHZEyzOIiEgXtJvguHtjvHmqryQ2Cc0qOJ2f6O/up97kL6/8a6fj6mTcvmhFBZV/eijYMSN/6G6ZDUhERHqlVL653zSzOcBDwI74QXf/c9qiygbN+uB0Pil5Z9XGnY4NH9yPIf1LdiWqnFe7dGliO3/ECCx/12aRFhGRvimVb49iYBNwYtIxB3I7wWk2iqrzX7J1DUET14HjhvHlkycBsPeoIUTUB6ddDWvXJbb3+MmPMxiJiIj0Zh31wRkG3AEsd/etPRJRtmg2D07nk5K6huD6oYP6MXHc8O6KKufFl2coGD2KvKKiDEcjIiK9VZvf3GZ2GbAU+BXwrpmd2WNRZQHfxZmM6+qDCk5xgZpYOiMaLrCZP1JrT4mISNe19+17HTDR3SvMbC/gj8CcHokqGzQ2NG13YZh45Y46AIoKNWoqVfUfrSG6YQMABVqeQUREdkF7bS/17l4B4O4rgD7VXtDwzl+adjrZRLV5W9M8iEUFSnBSEd1Qwcc3fDuxXzB6dAajERGR3q69Cs5oM7u9rX13vyZ9YWVeXuluxAeKW+mQTl27+pOtie3S4sLuCyqHVT33XNNOJELxxImZC0ZERHq99hKcG1rsL0hnINnGo7UA5A2fgHWyghPvYAxw0N67d2tcucijUapemAtAXv8yRv7yNvJKSzMblIiI9GptJjjufm9PBpJ1GoJmJivq3+lLa+ubZkFWH5yOVb8xj9i2YB7JwTNmKLkREZFdpklZ2hCv4JBf3Olr65MqOEUaRdWhePNUXlkZpUcekeFoREQkFyjBaUNsU7AekhV0fubhbeEIKoDiQiU47WlYu5a6pcsA6HfC8Vih+iyJiMiua28enJ+F/36x58LJDt5Q3TRMvAsVnL/N+yCxXajFNdtV9dzzie2yk05s50wREZHUtVfB+ayZGfDvPRVMtoht/7hpJ3nCvxQVFzVVbQo1TLxNsfp6drz0MgBFEydSMFJz34iISPdor/3kaWALUGZm2wAjWIPKAHf3AT0QX2bEmjoJ5+81rdOXV2ypBuC0I/fptpByUc1rrxOrqgKg/6dPynA0IiKSS9qs4Lj7De4+CPiruw9w9/7J//ZciBnQ2JTgEOlcH5qqmnqq64LmreGD+3VnVDln+7Nh5+KBAygpL89wNCIikks6/PZ29+lmtjtweHjo9fgMxzkr1rRMQ2dXEt+4tTqxPWyQEpy21H/4IfX/+hcAZVOnYvnqjC0iIt2nw2+VsJPxrcBcguapX5nZDe7+cJpjyxhPruDkFXTq2tffWZPYHj5I87kANG7bRs38BXhDU+JYs2hRsGGmzsUiItLtUvnf5u8Dh7v7BgAzGwY8C+RsgpPcB6czTVSVVbXc9+ySxL4qOIFN/3MntYveavW54smTyR8+vIcjEhGRXJfKPDh58eQmtCnF63qvpASnM01Uayq2JbaLC/MZ3L/zQ8xzUf3yD1o9boWFDPzCWT0bjIiI9AmpfHs/bWbPAPeH++cBT6ZyczM7FfglEAF+6+4/beO8swkqQoe7+/xU7p1WSX1w6ESCs626PrH9n5efSDDKvm+L1dQkRkoN+PxZ9D/t1MRzecXFmthPRETSIpVOxjeY2ReAY8NDs9z90Y6uM7MIcAfwaWANMM/M5rj7shbn9QeuBV7vbPDp4rGu9cHZtqM2sT2gX1F3htRrRTduTGwXjBpFZEBuD8ATEZHskFJ5wt3/DPy5k/c+Alju7isAzOwBYDqwrMV5PwR+xs6rl2dOs07GneiDk7REw8B+ap4CaKxoSnDyhw3NYCQiItKXpLMvzSjgo6T9NeGxBDM7FBjj7n9NYxyd5nVNfWmsE52MF7y3Hgj632gG40ByBScyVAmOiIj0jIx1FjazPOC/gX9L4dwrzGy+mc2vqEj/FDzRD15o2omk1kekqqaedz8MvszLStSvJC4ar+BEIkQGD85sMCIi0meklOCYWYmZ7d/Je68FxiTtjw6PxfUHDgTmmtkq4ChgjpntNKWtu89y93J3Lx82bFgnw+g8K27qJ2L9R6R0zabKpgn+NHqqSePGICGN7DYEy8vtwXciIpI9OvzGMbPPAYsI1qbCzA42szkp3HsesK+ZjTOzQuBLQOI6d69096HuPtbdxwKvAWdmxyiqoA9O3uBxKY+EqmtoWpTzSycemJaweqN4E1X+0PQnpiIiInGp/C/1TIIOw1sB3H0RMK6ji9w9ClwFPAO8A/zJ3Zea2Q/M7Mwuxtsz4qOoOtHBuLa+qWOy+t80aUwkOOp/IyIiPSeVb/AGd69sUcnwVG7u7k/SYs4cd/+PNs6dmso9e0JiqYZOdDCuT6rgFBdqXSUAb2igcctWACIaQSUiIj0olW/ipWb2ZSBiZvsC1wCvpjesDEtUcFKfA2fz9prEdpEqODRu2cLWBx5M7KuCIyIiPSmVJqqrgYlAHXAfUAlcl8aYMi9McDqzTMO8d9cltouLVMHZ/Nu72fHiS4n9/B7oHC4iIhLX7jdxOBvxX919GvC9ngkpCzSGSzV0oomqtLip2jO8jy+yGduxo2m1cKBo/HiKDujsIDwREZGua/cb3N0bzSxmZgPdvbKngso070In41gs6JY0YreyPr8GVc2ChdAY9Eka/r1/p3jSpAxHJCIifU0q3+BVwGIz+zuwI37Q3a9JW1SZtgsJTkRzvVD9xhsA5JWVUTR+fIajERGRviiVb/CurEPVu4WriVsk9U7GjbEYAHl9vHoTq62l9q23ASgpPwzLV38kERHpeamsJn5vTwSSTWIb3w82OlPB8aCCk5fXNxOcWFUV2556moYPP8IbggSx9IgjMhyViIj0VR1+g4dDw38CTAASaxC4+15pjCtjvH5H005YlUlFUxNV30xwts35C9vm/CWxbyUlFE/SjM4iIpIZqXQYmQ3cCUSBacDvgD+kM6hMSl5JnKLUR0PFE5y+WsGpffe9YCMSITJ4EIO+dB5WkHoTn4iISHdKpQ2mxN2fMzNz99XATDNbALQ6I3Gvl1S1yR91WMqXJfrg9MEExxsbaVi1CoCyk05kyCUXZzYgERHp81JJcOrMLA9438yuIlgRvCy9YWWQJzVLWeojohJNVH2wk3HD2rV4fT0AhXt1uEyZiIhI2qXyDX4tUEqwRMNhwFeAr6YzqIzqYoLTmGii6nvDxOtXrEhsF+6Vk12zRESkl0llFNW8cLMKyP22h2YJTuprSvWVUVR17y+n6vnnExP5AdSHzVNWWEjBqFEZikxERKRJKqOo9gNuAD6VfL67n5jGuDLGmyU4qScrjX2gk3Hjtm1U/PwWYtu3t/p8wdixWEQLjYqISOal0gfnIeA3wF1AYwfn9n6e9BY7U8HpA8PEt9z7u0RyExk6tFkCmFdczMCzv5Cp0ERERJpJJcGJuvudaY8kWyRXcDrRnybXZzKuWbCQ6ldeBaDk8HKGfvP6Pr/mloiIZK82ExwzGxJu/sXMvgE8CtTFn3f3zWmOLTNiXRxF5bm7FlWsuprN/3c3AFZayuCLL1ZyIyIiWa29Cs4CwIH4N9kNSc85kJvDZZIqONaZUVSNudsHZ+t999O4OchnB194AflDBmc4IhERkfa1meC4e9+c0KSLw8TXVAQzIPf2PjiNlZVUPvwIjVu3BgdiMWoWLASgaOIE+k2bmqnQREREUtZeE9XhwEfu/nG4PwM4G1gNzMzZJqoudDKurKptuqSXN91UPfc8VX9/dqfjVljIkMsv7/XvT0RE+ob2mqj+FzgZwMyOB34KXA0cDMwCzkl3cJngYV8aIOUKzrpNVYntPXcf0N0h9ahE5SYvj4LRowGw/Hz6f+4MCvbYPXOBiYiIdEJ7CU4kqUpzHjDL3R8BHjGzRWmPLFOaVXBSS3AqtjatQH74Ab17ojuvC/qRRwYPZsTPf5rhaERERLqmvW/wiJnFE6CTgOeTnktleHnvlFzBSXFE1IakBGf4oNRXIM9GXhesKWVFRRmOREREpOvaS1TuB140s41ADfAygJntA1T2QGyZEUueyzDFCs6WagDKSgopLS5IQ1A9J1YfVHCsqDDDkYiIiHRde6OofmxmzwEjgL95U+eUPIK+OLkpeZh4BxWcrVW1zH5qEW/+az0AwwaVpjW0nlC78E0A8gpVwRERkd6r3aYmd3+tlWP/Sl84WaATi20+/fpy5r65KrE/Yrf+aQqqZ8Rqk0aDqYIjIiK9WO72pemqZp2M2x8SvW5TsC5TYUGEyXvtzrnTJqQzsrSLrluX2C4cOzZzgYiIiOwiJTgteCcqOBVbgs7F+4wawo1fPT6dYfWIhjVrE9ulxx6TwUhERER2Te4tnLSrOjGT8Sfh6KnhOdD3BqBhbZjgmFEwYkRmgxEREdkFSnBaiqW2mni0McaWbUGflWG9fGh4XDzByd9jD6ygd48GExGRvk0JTguxrauadtppovrTC0sTK4gPH5wrCU7QB6dg1MgMRyIiIrJrlOC0ENu0IrFtRWWtntMQbeTB55cm9kf28tFTAN7QQPTjjwEoGNW7Z2MWERFRgtOCFTb1p8kra33tpY82bEts7zGkjInjhqU9rnRr+PjjxCzOSnBERKS3U4LTUvglb6VD2zxl+dqmhdRvnHF8TqywnTyCqmC0EhwREendlOC0FB9F1c4IquVrggSnuDCfkUN7f/MUQHRtU4KTP1J9cEREpHdTgtNSIsFpuyoTr+DsPXIweXm9v3oDTRWcyNCh5BUXZzgaERGRXaOJ/lpwwiW3WklwauoauOX+V/lg3RYA9hk9pCdDS4sdr7zKtsfnqIOxiIjkFCU4LcXXFG2liWreu+tYEC6sCXDAnm330+ktKh96iOjHnyT2C8d+KoPRiIiIdA8lOC0lZjLeOcHZUVOf2D7l8L05ckLvr3bEdgSzMefvsTvFBx9M/9M/m+GIREREdp0SnJYSfXB2fqquoWkhzotOO5hIOzMd9xZeFyRtJeXlDL7wggxHIyIi0j16/zd0urTSRFXXEE1sFxW0vxBnb+CxGF4fJDhWVJThaERERLqPEpyW2hkmXlsfVHDyI3lEIr3/o4snNwB5SnBERCSH9P5v6e4WJjitTd73wpsrgdyo3kDzBMcKCzMYiYiISPdSgtNSfBRVK51w4klPcl+c3sxr6xLbVqwKjoiI5A4lOC21M0y8pq4BgH1zYP4bAK9PSnAKleCIiEjuUIKzk9ZnMm6INlJTF3QyPmTfPXo6qLSI1SUlOEVqohIRkdyhBKelNpqotu1oSgYG9MuNpQziQ8QB8opy4z2JiIiAEpydeBujqCqTEpyB/XKjOcdVwRERkRylBKelNvrgPPaPdxPbuZLgbHvir4lt9cEREZFcogRnJ633wXnvw02J7T2GlPVkQGnh7tSvXJnYjwwelLlgREREupkSnJZa6YPj7mzeXgPAiN3K2G1gaQYC616Nmzfj1dUAFE+eRGTAgAxHJCIi0n2U4LTUShNV5Y466sO5bz571L6ZiKrbJVdvBnz+rMwFIiIikgZKcFrynZuoKrbuSGwPG9SvpyNKi/oPVgQbZhSOHZvRWERERLqbEpyWwgqOJVVwNmxpSnCGD+r9zVPuTv2KIMHJHzmSvJKSDEckIiLSvfIzHUD2CSs4GNHGGDf+9nmWrd6YeHbY4N5dwfGGBj65+YfUL18OQOFe4zIckYiISPdTBaelpD44K9ZtaZ7cDCqlf0nvni+m7l/vJ5IbgOIJEzIYjYiISHqogtNSIsExGqJNi2pO2ms4V3zusFZXGe9NvLY2sV128kn0O/64DEYjIiKSHqrgtJQ0k3G0MZY4fP5JB7Ln7gMzFFT3idU3Lc/Q/9RTsEgkg9GIiIikhxKcFhJLNWA0xjxxPD+SGx+V1zVVcDR7sYiI5Co1Ue2kqYkquYIT6cUJTqymJtH0FqtqGhGm9adERCRXpTXBMbNTgV8CEeC37v7TFs9/E7gMiAIVwCXuvjqdMXUoqZNxY1KCk5/X+xIcj8Wo+Nkt1L71VqvPW5EqOCIikpvS9q1tZhHgDuA0YAJwvpm1HLLzJlDu7pOBh4GfpyuelCX1wWloVsHpfZ2LG9aubTO5iey2G1aoCo6IiOSmdFZwjgCWu/sKADN7AJgOLIuf4O4vJJ3/GnBhGuNJTSLBgcZYUgWnFzZR1b3TtAL6gLO/QF5pOKGf5VFyyMFYL6xKiYiIpCKdCc4o4KOk/TXAke2cfynwVBrj6aQ8GhubOhlHemEyUPfOOwDkDRjAwHPO7vVD3EVERFKVFZ2MzexCoBw4oY3nrwCuANhzzz3TG0xYwbEWw8R7Wydjd6fu3aCCU3TAAUpuRESkT0nnt/ZaYEzS/ujwWDNmdjLwPeBMd69r7UbuPsvdy929fNiwYWkJNvFadVXxwJo1URX0sgQn+vHHNG7ZCkDx+AMyG4yIiEgPS+e39jxgXzMbZ2aFwJeAOcknmNkhwP8SJDcb0hhLSqIfvo5vWxPsmPHiW00DunpbJ+Pk/jdFSnBERKSPSVuC4+5R4CrgGeAd4E/uvtTMfmBmZ4an3QKUAQ+Z2SIzm9PG7XpEdOVLie28weMoS1p3qrgwK1rzUhZvnrJ+pRSku1lPREQky6T1W9vdnwSebHHsP5K2T07n63eaNzVJFR71dXz5awAM7l/c6zoZ14YVnKL999doKRER6XP0zZcsnuAUDcDyixJ9cIYP7pfBoDovunEjjRUVgPrfiIhI36QEJ1liDpygv00sXIuqt1VvmvW/OWB8BiMRERHJjN7VsSTdEkPEgxW244tt5vWCIdbuTvSTTyAapebNN4FgKYbCcWMzG5iIiEgGKMFJ1kYFJy8v+xOczb/5X3a8+FKzY0X77Yfl60csIiJ9T+9qe0m3pIU2oWmphkiWJzjRigp2vPTyTsdLDi/PQDQiIiKZp/+9b6ZpoU2AmPeOCk7V8y8kkrPBF19E3oABRAb0p2i8+t+IiEjfpAQniffCTsYejVL1wlwACvfZh/6nfCazAYmIiGSB7P3mzoQWTVSxXtDJuGbhQmJbtwJQdvKJmQ1GREQkS6iCkywx0V+8D068gpPZBCe6cRONW7a0+tz2p54GwEpLKZ0ypSfDEhERyVpKcJJl4Siq2sVL2PDj/+zwvH7HH0deUVEPRCQiIpL91ESVrGUTVZjwZDLBqVm4sMNzrKCA/p/5dA9EIyIi0juogpMsMdFf9jRRNaxZC0D+iBEMnvGVVs8pGDWS/OHDezIsERGRrKYEJ1lbTVQZ7GTcsDZIcAr3GkfJIQdnLA4REZHeRE1Uybz1eXAyNUw8Vl1N4+bNABSMGpWRGERERHojVXCStUhwtlbVAj3XB8djMerff59YXT0A0Y8/TjxXMFoJjoiISKqU4CRL6mRcXdeQ2O2pJqot99xL1d/+3upzquCIiIikTk1USTypgrNu4/bE8XhTVbrVvLmo1eP5I0aQv8cePRKDiIhILlAFp5mmTsZ19dHE0SkTR6f/levqaNy4EYB+J06j7ITjw2eMgrGfwiKRtMcgIiKSK5TgJEtqoqpraEwcLipM/8cUXbc+8frFBx5I0f77p/01RUREcpWaqJIlNVElV3CKCtJfPWlYuyaxrf42IiIiu0YJTrLEWlRGTVKCU5zmCo43NrL9mb+FL20UjFB/GxERkV2hBCdZ2ERklsdzC1YmDhcWpDfBqXzoYerfXw5A/u7DscLCtL6eiIhIrlOCkyypiWpgv6aFKweXFaf1ZetXNiVTpUcemdbXEhER6QuU4CRLdDK2xNDwUUP7p32iPw8n9osMHcqg87+U1tcSERHpC5TgJPNw5JTlJdahKshPfwfjWF0wY3LBmDFpfy0REZG+QAlOEk9abDOW6I/TA68bVnDyitT3RkREpDsowUni28O1nyySSHB6Yh2q6Lp1wcsWFnVwpoiIiKRCCU7IPYZXfRLsmPXYOlSN27Yltq1YCY6IiEh3UIITV78jsel124nFguaqdCc4Das/TGxHBg5M62uJiIj0FUpwQt5Qm9gumDA90ck43U1U0XD9KYDSY45J62uJiIj0FUpw4qI1iU3LLybWQ01UiQTHjPyhu6X1tURERPoKJTghb2hKcCgoSRpFleY+OBUVAEQGDcLytfapiEg2u+iii3j44Ye7dO1jjz3GsmXL2j1n6tSpzJ8/P+V7zpw5k1tvvbVL8eQ6faOGPKkPjuUX414HdE8TVePWrVTPmw/R6E7P1S0PlmiIDB26y68jIiLZ67HHHuOMM85gwoQJmQ6lT1CCE2pcMbdpp6CEWCzok9MdXXA2/vJ26t55t91z8ocpwRERaem3Tyxk5fot3XrPcSMGc9kZh7Z7zqpVqzjttNM49thjefXVVxk1ahSPP/54s3Oee+45vvWtbxGNRjn88MO58847KSoqYuzYsZx77rk89dRTlJSUcN9997FhwwbmzJnDiy++yI9+9CMeeeQR9t5771Zf+/e//z2XXXYZ0WiUu+++myOOOILNmzdzySWXsGLFCkpLS5k1axaTJ08G4K233mLKlCls3LiRb3/721x++eXMmDGDL3zhC5x11lkAXHDBBZx77rl85jOf4eKLL+att97igAMOYN26ddxxxx2Ul5fz9a9/nXnz5lFTU8M555zDzTffvOsfdgapiSour2nG4sgeBybNg7PrH1H9qtXtPm+FhfRTB2MRkazy/vvvc+WVV7J06VIGDRrEI488kniutraWiy66iAcffJDFixcTjUa58847E88PHDiQxYsXc9VVV3Hddddx9NFHc+aZZ3LLLbewaNGiNpMbgOrqahYtWsT//M//cMkllwBw0003ccghh/D222/zn//5n8yYMSNx/ttvv83zzz/PP//5T37wgx+wbt06Lr30Uu655x4AKisrefXVVzn99NO58847KS0t5Z133uHmm29mwYIFifv8+Mc/Zv78+bz99tu8+OKLvP322931UWaEKjhxsXCZhkhB2Mk4THB2sQ+ONzTgNUH/ngHTz6T/587Y6RwrLCRPK4iLiOyko0pLOo0bN46DDz4YgMMOO4xVq1YlnnvvvfcYN24c++23HwBf/epXueOOO7juuusAOP/88xP/Xn/99Z163fi1xx9/PNu2bWPr1q384x//SCRYJ554Ips2bWJbOI/a9OnTKSkpoaSkhGnTpvHGG29w1lln8Y1vfIOKigoeeeQRzj77bPLz83nppZe45pprAJg8eXKiCgTwpz/9iVmzZhGNRlm/fj3Lli1r9nxvowQn5PF1qPKCjyQ+THxX+xg3bq9KbEeGDiVSVrZrNxQRkR5RVNQ0+WokEqGmpqads5tLHqDS2cEqLc/v6Pq2zp8xYwZ/+MMfeOCBB5g9e3a791i5ciW33nor8+bNY/DgwVx00UXU1ta2e022UxNVXKx5gpOYyXgXO+HEtjfNVBzp33+X7iUiItlh//33Z9WqVSwPB4r8/ve/54QTTkg8/+CDDyb+nTJlCgD9+/dn+/btHd47fu0//vEPBg4cyMCBAznuuOP44x//CMDcuXMZOnQoAwYMAODxxx+ntraWTZs2MXfuXA4//HAgGPF12223ASQ6Nh9//PHcd999ACxZsiTRDLVt2zb69evHwIED+eSTT3jqqae6/uFkCVVw4hILbQY5X2Kiv10s4dSvXJnYzgt/GUVEpHcrLi5m9uzZfPGLX0x0Mv7a176WeH7Lli1MnjyZoqIi7r//fgC+9KUvcfnll3P77bfz8MMPt9kPp7i4mEMOOYSGhgbuvvtuIBgOfskllzB58mRKS0u59957E+dPnjyZadOmsXHjRm688UZGjhwJwO6778748eMTHY0Bvv71r3PxxRczfvx4xo8fz2GHHQbAQQcdxCGHHMIBBxzAmDFjOCYH+oWax0sVvUR5ebl3Zo6AVNU+9wMa3v4TVjqEsv/3Elf+4knWVGzjmAPH8O0vd/0HveaKrxEL20lH3PpzCkaP7q6QRUQkC40dO5b58+czNMPTf1RXVzNp0iQWLlzIwDaWApo6dSq33nor5eXlPRxdp3Sp0qAmqrhYvIITjKbqjtXEPRYjltRmmx9m1SIiIun07LPPMn78eK6++uo2k5tcpyaqkHs4CV+LJqpdmck4umEDNDQAMOTyy7BuGHIuIiLZLXm0VVuuvPJKXnnllWbHrr32Wi6++OJuieHkk09m9er2pyiBoD9PrlKCExev4CQ6Ge96Badh7drEdv4oVW9ERCRwxx13ZDqEnKeSQlx8mHi8gpOYB6frt4yuaUpwCkaN6vqNREREpFNUwYkLh4lbXj5VNfVUbK0G2h9FFauvZ8dzzxPd3Po04rWLg+F3eQMHaoi4iIhID1KCE5c0TPzPL72TOJwfibRxAex4YS5b7v1dh7dW9UZERKRnKcGJi4WdjPMifLK5aWXxqYeMbfOS6McfJ7atoKDVc6y0hP6fPbVbQhQREZHUKMEJebyCkxehriFIdsaNGMT4T7U9j0FsR9CMFdltN0bd8au0xygiItll/vz5/O53v+P222/vlvvNnTuXW2+9lSeeeIKZM2dSVlbGt771rZ3Ou+GGG3jyySf57Gc/yy233NKp1ygrK6OqqqrjE3s5JThxsaZOxnX1QYJTXNj+xxOrCfvplJamNTQRkb6qdu5PiVW82633zBt2AMVTv9st9yovL8/IJHmzZs1i8+bNRNrpRtHXaRRVXNJim3UNwXZRQfu/OPEKjhIcEZHcsmPHDk4//XQOOuggDjzwQB588EHmzZvH0UcfzUEHHcQRRxzB9u3bmTt3LmeccQYQLKfwla98hSlTprDvvvty1113AcGil4899lji3hdccAGPP/54SnG89dZbO93vzDPPpKqqisMOO4wHH3yQDz74gKOOOopJkybx/e9/n7JwUef169dz/PHHc/DBB3PggQfy8ssvJ+57/fXXM3HiRE466SQqKiq64yPLOqrghBpXvwrAjtoo7320CYCijio41UGCY0pwRETSorsqLZ319NNPM3LkSP76178CUFlZySGHHMKDDz7I4YcfzrZt2ygpKdnpurfffpvXXnuNHTt2cMghh3D66adz6aWX8otf/IKzzjqLyspKXn311WZrSbWntfvNmTOHsrIyFi1aBMAZZ5zBtddey/nnn89vfvObxLX33Xcfp5xyCt/73vdobGykOvzO2rFjB+Xl5fziF7/gBz/4ATfffDO//vWvd/ETyz6q4LSwsqI6sV1a1HrH4bhYddAZOa+fEhwRkVwyadIk/v73v/Od73yHl19+mQ8//JARI0YkVuoeMGAA+fk7/0/w9OnTKSkpYejQoUybNo033niDE044gffff5+Kigruv/9+zj777FavbU1r92vpn//8J1/84hcB+PKXv5w4fvjhhzN79mxmzpzJ4sWL6R9OV5KXl8d5550HwIUXXsg//vGPzn04vYQSnJCV7oYPGM3TOw5PHDvliH3avcarg3Wm1EQlIpJb9ttvPxYuXJho9vnzn/+c0nUtl/eJ78+YMYM//OEPzJ49m0suuSTlONq6XyqOP/54XnrpJUaNGsVFF13E737X+rQmu7IkUTZTghMq+38v0nDOn1lYvx8A3zirvN0RVO6eaKJSgiMiklvWrVtHaWkpF154ITfccAOvv/4669evZ968eQBs376daDS603WPP/44tbW1bNq0iblz5yYqPhdddBG33XYbABMmTEg5jrbul+yoo47ikUceAeCBBx5IHF+9ejW77747l19+OZdddhkLFy4EIBaL8fDDDwNBM9axxx6bcjy9ifrgJIk2xhLbkUj7uZ/X1SXWr1ITlYhIblm8eDE33HADeXl5FBQUcOedd+LuXH311dTU1FBSUsKzzz6703WTJ09m2rRpbNy4kRtvvJGRI4N1CHfffXfGjx/PWWed1ak42rpfsttuu40LL7yQH//4x5x66qmJ1cPnzp3LLbfcQkFBAWVlZYkKTr9+/XjjjTf40Y9+xPDhw3nwwQc7+en0DhZfVLK3KC8v9/nz56fl3ms3buMb//0kANefexRTDx7b5rnRzZtZ942rABhy2aWUnXxSWmISEZHeob15a6qrq5k0aRILFy5MJCDdpbq6mpKSEsyMBx54gPvvvz/lUVq9RJfa0FTBSdLY2JTs5eftXMGp+2AFVc8+izc04DU1iePWr1+PxCciIr3Ps88+y6WXXsr111/f7ckNwIIFC7jqqqtwdwYNGsTdd9/d7a/RGynBSdJRE9WWu++m/oMVOx3XQpoiIjJz5sxWj5988smsXr262bFnnnmG73znO82OjRs3jkcffbTTr3vcccfx1ltvdfq6XKcEJ0ljrCnByY/sXBGLbtgAgJWUEBkwAIDCvcZRNP6AnglQRERywimnnMIpp5yS6TBymhKcJNGkJqpIiyYqj8WIVQXz3vQ/9RQGnXduj8YmIiIiqUvrMHEzO9XM3jOz5Wa203SUZlZkZg+Gz79uZmPTGU9HGhuTKzjNP5rY9u0QdsiOV29EREQkO6UtwTGzCHAHcBowATjfzFoO/r8U2OLu+wC/AH6WrnhS0awPTl7zJqrY9u2J7TwlOCIiIlktnU1URwDL3X0FgJk9AEwHliWdMx2YGW4/DPzazMwzMHb9lWv/P7Zur+UKgpcu/f0KKkoKE8/Hm6cAIgPUqVhERCSbpbOJahTwUdL+mvBYq+e4exSoBHZLY0xtGvXJKiZVf8yk6k+YVP0JkWVLqFmwMPGoe++9xLmRIUMyEaKIiGSZ+fPnc80112Q0hqlTp9La/HD33HMPV111VQYi6lh8xfN06hWdjM3sCuAKgD333DMtr1HRrylpKSnKZ7cBrc9OXDzpQPJbmUlSRES635Z7f0f9qtUdn9gJhWM/xeCvzuiWe5WXl1NeXt4t98q0aDSa8iKgvUE638laYEzS/ujwWGvnrDGzfGAgsKnljdx9FjALgpmM0xHs4f+Xe0vFi4hI1+zYsYNzzz2XNWvW0NjYyI033shee+3Ftddey44dOygqKuK5555jwYIF3HrrrTzxxBPMnDmTDz74gOXLl7Nx40a+/e1vc/nllzNjxgy+8IUvJJZpuOCCCzj33HOZPn36Tq97zz33MGfOHKqrq/nggw/4/Oc/z89//nMA/va3v3HTTTdRV1fH3nvvzezZs3eqhMyePZuf/OQnDBo0iIMOOoiioiIAKioq+NrXvsaHH34IBMs7HHPMMYmYV6xYwZ577sn+++/PypUrWbFiBR9++CG/+MUveO2113jqqacYNWoUf/nLXygoKGDBggV885vfpKqqiqFDh3LPPfcwYsQI7rrrLmbNmkV9fT377LMPv//97yktLWXlypV8+ctfpqqqqtX3nRbunpYHQfK0AhgHFAJvARNbnHMl8Jtw+0vAnzq672GHHeYiIiLp9PDDD/tll12W2N+6dauPGzfO33jjDXd3r6ys9IaGBn/hhRf89NNPd3f3m266ySdPnuzV1dVeUVHho0eP9rVr1/rcuXN9+vTpifuMHTvWGxoaWn3d2bNn+7hx43zr1q1eU1Pje+65p3/44YdeUVHhxx13nFdVVbm7+09/+lO/+eab3d39hBNO8Hnz5vm6det8zJgxvmHDBq+rq/Ojjz7ar7zySnd3P//88/3ll192d/fVq1f7AQcckIj50EMP9erq6sT+Mccc4/X19b5o0SIvKSnxJ5980t3dzzrrLH/00Ue9vr7ep0yZ4hs2bHB39wceeMAvvvhid3ffuHFj4r1873vf89tvv93d3T/3uc/5vffe6+7uv/71r71fv36d+XF0KQ9JWwXH3aNmdhXwDBAB7nb3pWb2A2C+u88B/g/4vZktBzaHSY6IiEhGTZo0iX/7t3/jO9/5DmeccQaDBg1ixIgRidW8B7Qxmnb69OmUlJRQUlLCtGnTeOONNzjrrLP4xje+QUVFBY888ghnn312u01BJ510UmJJhwkTJrB69Wq2bt3KsmXLOOaYYwCor69nypQpza57/fXXmTp1KsOGDQPgvPPO41//+hcQLBexbFnTGJ9t27ZRVVUFwJlnnklJSUniudNOO42CggImTZpEY2Mjp556auIzWbVqFe+99x5Llizh05/+NACNjY2MGDECgCVLlvD973+frVu3UlVVlZjM8JVXXkmseP6Vr3xlp1mc0yGtjW3u/iTwZItj/5G0XQt8MZ0xiIiIdNZ+++3HwoULefLJJ/n+97/PiSeemNJ1Ztbq/owZM/jDH/7AAw88wOzZs9u9R7xZCSASiRCNRnF3Pv3pT3P//fd38p0EYrEYr732GsXFxTs916/Feorx14+vpB5/D3l5eYlYJk6cyD//+c+d7nXRRRfx2GOPcdBBB3HPPfcwd+7cxHMtP5t0S+tEfyIiIr3RunXrKC0t5cILL+SGG27g9ddfZ/369cybNw+A7du3E41Gd7ru8ccfp7a2lk2bNjF37txExeeiiy7itttuA4KqTGcdddRRvPLKKyxfvhwI+gjFqzNxRx55JC+++CKbNm2ioaGBhx56KPHcZz7zGX71q18l9hctWtTpGOL2339/KioqEglOQ0MDS5cuBYLPZcSIETQ0NPDHP/4xcc0xxxzDAw88ANDseDrlTndpERGRbrJ48WJuuOGGRBXjzjvvxN25+uqrqampoaSkhGeffXan6yZPnsy0adPYuHEjN954IyPDUbe7774748ePT3Q07qxhw4Zxzz33cP7551NXVwfAj370I/bbb7/EOSNGjGDmzJlMmTKFQYMGcfDBByeeu/3227nyyiuZPHky0WiU448/nt/85jddiqWwsJCHH36Ya665hsrKSqLRKNdddx0TJ07khz/8IUceeSTDhg3jyCOPZHs4Se4vf/lLvvzlL/Ozn/2sxzoZm/f8nHq7pLy83Fsb7y8iIpJJM2fOpKysjG9961s7PVddXc2kSZNYuHBhon+NpKxLbVtqohIREUmjZ599lvHjx3P11VcruelBquCIiIj0sGeeeWankUTjxo3j0UcfzVBEWa1LFRwlOCIiIpLN1EQlIiIiAkpwREREJAcpwREREZGcowRHREREco4SHBEREck5SnBEREQk5/S6YeJmVgGsTtPthwIb03Rv6Zg+/8zS5595+hlklj7/zGrr89/o7qd29ma9LsFJJzOb7+7lmY6jr9Lnn1n6/DNPP4PM0uefWd39+auJSkRERHKOEhwRERHJOUpwmpuV6QD6OH3+maXPP/P0M8gsff6Z1a2fv/rgiIiISM5RBUdERERyjhIcERERyTlKcAAzO9XM3jOz5Wb23UzHk4vMbIyZvWBmy8xsqZldGx4fYmZ/N7P3w38Hh8fNzG4PfyZvm9mhmX0HucHMImb2ppk9Ee6PM7PXw8/5QTMrDI8XhfvLw+fHZjTwHGFmg8zsYTN718zeMbMp+hvoOWZ2ffjfnyVmdr+ZFetvIL3M7G4z22BmS5KOdfp33sy+Gp7/vpl9NZXX7vMJjplFgDuA04AJwPlmNiGzUeWkKPBv7j4BOAq4Mvycvws85+77As+F+xD8PPYNH1cAd/Z8yDnpWuCdpP2fAb9w932ALcCl4fFLgS3h8V+E58mu+yXwtLsfABxE8LPQ30APMLNRwDVAubsfCESAL6G/gXS7B2g5SV+nfufNbAhwE3AkcARwUzwpak+fT3AIPqzl7r7C3euBB4DpGY4p57j7endfGG5vJ/gP+yiCz/re8LR7gbPC7enA7zzwGjDIzEb0bNS5xcxGA6cDvw33DTgReDg8peXnH/+5PAycFJ4vXWRmA4Hjgf8DcPd6d9+K/gZ6Uj5QYmb5QCmwHv0NpJW7vwRsbnG4s7/zpwB/d/fN7r4F+Ds7J007UYITfMl+lLS/JjwmaRKWeg8BXgd2d/f14VMfA7uH2/q5dL/bgG8DsXB/N2Cru0fD/eTPOPH5h89XhudL140DKoDZYTPhb82sH/ob6BHuvha4FfiQILGpBBagv4FM6OzvfJf+FpTgSI8yszLgEeA6d9+W/JwHcxZo3oI0MLMzgA3uviDTsfRh+cChwJ3ufgiwg6bSPKC/gXQKmzSmEySaI4F+pFAFkPRK5++8EhxYC4xJ2h8dHpNuZmYFBMnNH939z+HhT+Jl9/DfDeFx/Vy61zHAmWa2iqAZ9kSC/iCDwnI9NP+ME59/+PxAYFNPBpyD1gBr3P31cP9hgoRHfwM942RgpbtXuHsD8GeCvwv9DfS8zv7Od+lvQQkOzAP2DXvSFxJ0OpuT4ZhyTth2/X/AO+7+30lPzQHiPeK/CjyedHxG2Kv+KKAyqaQpneTu/+7uo919LMHv+PPufgHwAnBOeFrLzz/+czknPF+VhV3g7h8DH5nZ/uGhk4Bl6G+gp3wIHGVmpeF/j+Kfv/4Gel5nf+efAT5jZoPDStxnwmPtc/c+/wA+C/wL+AD4XqbjycUHcCxBGfJtYFH4+CxBm/ZzwPvAs8CQ8HwjGN32AbCYYORDxt9HLjyAqcAT4fZewBvAcuAhoCg8XhzuLw+f3yvTcefCAzgYmB/+HTwGDNbfQI9+/jcD7wJLgN8DRfobSPtnfj9Bn6cGgirmpV35nQcuCX8Wy4GLU3ltLdUgIiIiOUdNVCIiIpJzlOCIiIhIzlGCIyIiIjlHCY6IiIjkHCU4IiIiknOU4Ij0IDNzM/uvpP1vmdnMbrr3PWZ2Tsdn7vLrfDFcCfuFFsfHmlmNmS1KehR24+teZGYjk/Z/2x0L44b3rWgRd6fvG77/JR2f2eb115lZadL+k2Y2qKv3E+nrlOCI9Kw64AtmNjTTgSRLmsk1FZcCl7v7tFae+8DdD0561HdTiAAXEUyxD4C7X+buy7rp3g+2iLu77tsZ1xEsAAmAu3/Wg8U4RaQLlOCI9KwoMAu4vuUTLSswZlYV/jvVzF40s8fNbIWZ/dTMLjCzN8xssZntnXSbk81svpn9K1x/CjOLmNktZjbPzN42s/+XdN+XzWwOwYyuLeM5P7z/EjP7WXjsPwgmbfw/M7sllTccfx/h9jlmdk/S+73dzF4N31fye/9O+Npvhe/3HKAc+GNYYSkxs7lmVt5WrPHXNrMfh/d5zcx2J0Vm9oCZnZ60f08Y/9jwc1sYPo5u5dqLzOzXSftPmNnUcPvO8Ge01MxuDo9dQ5C8vRCvjJnZqngibGbfDN/bEjO7Ljw2Nqyk3RXe629mVhK/n5ktC3/eD6T6nkVySqZnOdRDj770AKqAAcAqgrVtvgXMDJ+7Bzgn+dzw36nAVmAEwcyra4Gbw+euBW5Luv5pgv9x2Zdg1tBi4Arg++E5RQQz6Y4L77sDGNdKnCMJprYfRrBI5PPAWeFzc2llVl1gLFBD00zVdyS/j3D7HOCepHgfCuOdACwPj58GvAqUhvtDWnvd+H4HsTrwuXD75/HPoUXcFxGs8r0o6VECfB64NzynkGA14xKCKktxeHxfYH7S+1+SdM9fJ73GE8DUFu8nEr6HyeH+KmBo0jWrgKHAYQSzuvYDyoClwCHh60WBg8Pz/wRcGG6vo2lG3kGZ/r3XQ49MPFTBEelhHqyi/jvgmk5cNs/d17t7HcE05n8Ljy8m+KKL+5O7x9z9fWAFcADBui0zzGwR8DrBNOn7hue/4e4rW3m9w4G5HixMGAX+CByfQpzJTVRXpnD+Y2G8y4B4deVkYLa7VwO4++YO7tFerPUEyQXAApp/VslaNlHVAE8B08ysiCDpeik8XgDcZWaLCRK0zvbXOdfMFgJvAhNTuP5Y4FF33+HuVQSLRB4XPrfS3Re18v7eJqh2XUiQBIn0OUpwRDLjNoK+LP2SjkUJ/ybNLI+gahBXl7QdS9qPEVQt4lquveIE67tcnfTlPc7d4wnSjl15EylKjqm4xXPJ78vS8NoN7h5//Uaaf1btcvdaggrLKcB5wIPhU9cDnwAHEVSQWutInfhZhooBzGwcQdXuJHefDPyVnT+Tzkj+/JLf3+kEa/ocCszrZB8rkZygBEckA8KqxJ8Ikpy4VQTNEQBnElQKOuuLZpYX9svZC3iPYNXdr5tZAYCZ7Wdm/dq7CcHigieY2VAziwDnAy92IR6AT8xsfJi0fT6F8/8OXBwfUWRmQ8Lj24H+aY61pQeBiwkqJk+HxwYC6909BnyFoKmppVXAweHPYgxwRHh8AEFSWRn2Bzot6Zq23t/LwFkWrILdj+AzfLmtgMPPeYy7vwB8J4y3LIX3KpJTlNWLZM5/AVcl7d8FPG5mbxF8mXaluvIhwRf+AOBr7l5rZr8laLpYaGZG0N/krPZu4u7rzey7wAsElZW/uvvjXYgH4LsEzUQVBP1/2v2ydfenzexgYL6Z1QNPAv8fQZ+d35hZDTClm2M9z8yOTdr/hru/StAU+HvgcW8aEfY/wCNmNoO2f06vACsJOm+/AywMY33LzN4kWNH6o/C8uFnA02a2zpNGqLn7wrBj9hvhod+6+5tmNraN9xIB/mBmAwk+j9tdo7GkD9Jq4iIiIpJz1EQlIiIiOUcJjoiIiOQcJTgiIiKSc5TgiIiISM5RgiMiIiI5RwmOiIiI5BwlOCIiIpJz/n+iZ9kRetnsAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "results_small = run_benchmark(\n",
    "    problems[\"small\"],\n",
    "    optimizers,\n",
    "    logging_directory=\"benchmark_logs_small\",\n",
    "    n_cores=cpu_count(),\n",
    ")\n",
    "fig = profile_plot(\n",
    "    problems=problems[\"small\"],\n",
    "    results=results_small,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimizer performance on the medium dimensional problem set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_small = run_benchmark(\n",
    "    problems[\"medium\"],\n",
    "    optimizers,\n",
    "    logging_directory=\"benchmark_logs_medium\",\n",
    "    n_cores=cpu_count(),\n",
    ")\n",
    "fig = profile_plot(\n",
    "    problems=problems[\"medium\"],\n",
    "    results=results_small,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpretation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can see from these figures, that L-BFGS-B performs the best, both in speed and successful optimizations. An interesting observation here is L-BFGS-B's near invariance to function input scale/dimension, as it solves problems at nearly the same speed over both sets. The Nelder-Mead optimizer however does struggle a lot with the increase in dimension of input problems. However, this mostly affects the speed at which it solves problems, and not it's problem solving rate compared to the other two optimizers. From these figures it is clearly visible that not every optimizer can solve every problem set, but they can each solve a substantial amount. It is thus neither to easy nor to hard to solve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quality of the benchmark set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The figures also show, that the benchmark set is well-constructed, since the problems are of varying difficulty to each optimizer. One can see this is in the difference solution rates, aswell as solving speed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Function tag dictionary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We constructed a dictionary of dictionaries for function attributes based on Thevenot's work. Attributes are \"continuous\", \"convex\", \"seperable\", \"differentiable\", \"randomized_term\", \"parametric\" which are assigned either True or False and \"input_start\" and \"input_end\" defining the possible dimensionality of the function's input domain with either integeres or \"None\". As an example we have inserted a snapshot of the function tag dictionary for the ackley function.\n",
    "```\n",
    "SCALAR_FUNCTIONS_TAGS = {\n",
    "    \"ackley\": {\n",
    "        \"continuous\": True,\n",
    "        \"convex\": False,\n",
    "        \"separable\": True,\n",
    "        \"differentiable\": True,\n",
    "        \"mutimodal\": True,\n",
    "        \"randomized_term\": False,\n",
    "        \"parametric\": True,\n",
    "        \"input_start\": 1,\n",
    "        \"input_end\": None,\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Remarks on difficulties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We collected issues with Thevenot's code and materials in this csv file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "remarks_on_thevenots_code = pd.read_csv(\"remarks_on_thevenots_code.csv\")\n",
    "remarks_on_thevenots_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This includes sources with which we compared his work. Issues included false minimas or false minima locations as well as errors in coded mathematics of functions like missing brackets or exponents.\n",
    "There are a few functions that currently do not work with their associated problems and are thus collected in a seperate dictionary and can be accessed as \"scalar_functions_extra\" via the get_benchmark_problems function. These functions have a variety of issues, rooting among other issues in either random components or one-dimensionality. This set includes 9 problems for 6 functions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ideas on how to scale problems with arbitrary dimensions in the future"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As of now the get_benchmark_problems function filters the \"scalar_functions\" problem set by their input dimension (the length of the vectors in \"start_x\" and \"solution_x\", for an example see the problem for the Ackley function below). In the future one could create problem sets on the fly for a specific dimension. A function which utilizes the dimension properties given in SCALAR_FUNCTION_TAGS can filter all elliglble functions for a problem dimension d. It would then write scalable problems, like that of the Ackley function, according to the given dimension to be equal to this manual notation:\n",
    "```\n",
    "SCALAR_FUNCTIONS_PROBLEMS = {\n",
    "    \"ackley_good_start\": {\n",
    "        \"criterion\": ackley,\n",
    "        \"start_x\": np.full(d, 0.45),\n",
    "        \"solution_x\": np.zeros(d),\n",
    "        \"start_criterion\": ackley(\"start_x\"),\n",
    "        \"solution_criterion\": 0,\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions that easily scale"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a number of functions where the minimum is always the same whatever the dimension of the function input and the minimum location is a unit vector multiplied by a scalar of arbitrary length. For example the Ackley function's global minimum is always 0 and it's global minimum is always located at the zero vector.\n",
    "```\n",
    "def ackley(x):\n",
    "    out = (\n",
    "        -20 * np.exp(-0.2 * np.sqrt(np.mean(x**2)))\n",
    "        - np.exp(np.mean(np.cos(2 * np.pi * x)))\n",
    "        + 20\n",
    "        + np.exp(1)\n",
    "    )\n",
    "    return out\n",
    "\n",
    "SCALAR_FUNCTIONS_PROBLEMS = {\n",
    "    \"ackley_good_start\": {\n",
    "        \"criterion\": ackley,\n",
    "        \"start_x\": np.full(10, 0.45),\n",
    "        \"solution_x\": np.zeros(10),\n",
    "        \"start_criterion\": 4.053325482003931,\n",
    "        \"solution_criterion\": 0,\n",
    "    }\n",
    "}\n",
    "```\n",
    "These functions, like the Ackley function in the example above, work with inputs of arbitrary length. Some functions had to be rewritten from Thevenot's original code to allow for more than two dimensions (e.g. Ackley N.4 and Brown). The above problem (\"ackley_good_start\") is of dimension 10, but could easily be adjusted to an arbitrary dimension by replacing dimension 10 with a variable in \"start_x\" and \"solution_x\".\n",
    "Functions which are multi-dimensional and need no further work except from the arbitrary dimension implementation are: \n",
    "Ackley, Ackley N.4, Alpine N.1, Brown, Exponential,\n",
    "Griewank, Happy Cat, Powell, Rastrigin, Rosenbrock, Rotated Hyper-Ellipsoid, Salomon, Schwefel, Schwefel 2.20, Schwefel 2.21, Schwefel 2.22, Schwefel 2.23, Sphere, Sum Squares, Thevenot, Xin-She Yang N.2, Xin-She Yang N.3, Xin-She Yang N.4 and Zakharov."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions that need specific work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some functions minima or minima locations depend on their dimension and thus need a bit more work to make them usable with arbitrary dimensions. For example the Alpine N.2 function's minimum is located at\n",
    "```\n",
    "f(x_minimum) = -2.808 ** d\n",
    "```\n",
    "where d is the the dimension of the function input x_minimum. Functions with similar problems of which we have implemented certain scalar cases which only need to be generalized are Alpine2, Dixon-Price, Trid, Qing, Styblinski-Tank."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions which cannot be easily changed from the two-dimensional case"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are functions which are multi-dimensional according to literature but rarely used outside of the two-dimensional for differing reasons, such as for example changing global minima. These functions would require a substantial amount of work and research to implement for multi-dimensionality and it is not clear whether this makes sense. These functions are Langermann, both Perm Functions, Michalewicz, Shubert, Shubert N.3 and Shubert N. 4."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}