udemy-ML/13-Support-Vector-Machines/01-SVM-Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "\n",
    "<a href='https://www.udemy.com/user/joseportilla/'><img src='../Pierian_Data_Logo.png'/></a>\n",
    "___\n",
    "<center><em>Copyright by Pierian Data Inc.</em></center>\n",
    "<center><em>For more information, visit us at <a href='http://www.pieriandata.com'>www.pieriandata.com</a></em></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machines\n",
    "## SVM - Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The concrete slump test measures the consistency of fresh concrete before it sets. It is performed to check the workability of freshly made concrete, and therefore the ease with which concrete flows. It can also be used as an indicator of an improperly mixed batch.\n",
    "\n",
    "<img src=\"Types_of_concrete_slump.jpg\">\n",
    "\n",
    "Our data set consists of various cement properties and the resulting slump test metrics in cm. Later on the set concrete is tested for its compressive strength 28 days later.\n",
    "\n",
    "Input variables (7)(component kg in one M^3 concrete):\n",
    "* Cement\n",
    "* Slag\n",
    "* Fly ash\n",
    "* Water\n",
    "* SP\n",
    "* Coarse Aggr.\n",
    "* Fine Aggr.\n",
    "\n",
    "Output variables (3):\n",
    "* SLUMP (cm)\n",
    "* FLOW (cm)\n",
    "* **28-day Compressive Strength (Mpa)**\n",
    "\n",
    "Data Source: https://archive.ics.uci.edu/ml/datasets/Concrete+Slump+Test\n",
    "\n",
    "*Credit: Yeh, I-Cheng, \"Modeling slump flow of concrete using second-order regressions and artificial neural networks,\" Cement and Concrete Composites, Vol.29, No. 6, 474-480, 2007.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('../DATA/cement_slump.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Cement</th>\n",
       "      <th>Slag</th>\n",
       "      <th>Fly ash</th>\n",
       "      <th>Water</th>\n",
       "      <th>SP</th>\n",
       "      <th>Coarse Aggr.</th>\n",
       "      <th>Fine Aggr.</th>\n",
       "      <th>SLUMP(cm)</th>\n",
       "      <th>FLOW(cm)</th>\n",
       "      <th>Compressive Strength (28-day)(Mpa)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>273.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>210.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>904.0</td>\n",
       "      <td>680.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>34.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>163.0</td>\n",
       "      <td>149.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>180.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>843.0</td>\n",
       "      <td>746.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>41.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>162.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>840.0</td>\n",
       "      <td>743.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>41.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>162.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>838.0</td>\n",
       "      <td>741.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>21.5</td>\n",
       "      <td>42.08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>154.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>144.0</td>\n",
       "      <td>220.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>923.0</td>\n",
       "      <td>658.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>26.82</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Cement   Slag  Fly ash  Water    SP  Coarse Aggr.  Fine Aggr.  SLUMP(cm)  \\\n",
       "0   273.0   82.0    105.0  210.0   9.0         904.0       680.0       23.0   \n",
       "1   163.0  149.0    191.0  180.0  12.0         843.0       746.0        0.0   \n",
       "2   162.0  148.0    191.0  179.0  16.0         840.0       743.0        1.0   \n",
       "3   162.0  148.0    190.0  179.0  19.0         838.0       741.0        3.0   \n",
       "4   154.0  112.0    144.0  220.0  10.0         923.0       658.0       20.0   \n",
       "\n",
       "   FLOW(cm)  Compressive Strength (28-day)(Mpa)  \n",
       "0      62.0                               34.99  \n",
       "1      20.0                               41.14  \n",
       "2      20.0                               41.81  \n",
       "3      21.5                               42.08  \n",
       "4      64.0                               26.82  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Cement                                0.445656\n",
       "Slag                                 -0.331522\n",
       "Fly ash                               0.444380\n",
       "Water                                -0.254320\n",
       "SP                                   -0.037909\n",
       "Coarse Aggr.                         -0.160610\n",
       "Fine Aggr.                           -0.154532\n",
       "SLUMP(cm)                            -0.223499\n",
       "FLOW(cm)                             -0.124189\n",
       "Compressive Strength (28-day)(Mpa)    1.000000\n",
       "Name: Compressive Strength (28-day)(Mpa), dtype: float64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr()['Compressive Strength (28-day)(Mpa)']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhQAAAGrCAYAAACYDg7YAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABN7klEQVR4nO3dd7ycZZ3+8c9F6B0REeklgNQAIQKigqKiIqggRSyIiP5c27q6iwurrq4r9l6IiGADEURZ6S2C9ACBECB0kKJIL1JCzvX747kPGYZzknMyz8ycOXO9X695ZZ77eeb53pOcnPnOXWWbiIiIiFYs0u0KRERERO9LQhEREREtS0IRERERLUtCERERES1LQhEREREtS0IRERERLUtCERERMY5IOkrSfZKuHea8JH1P0s2SrpG0dR1xk1BERESML0cDu87n/JuAieVxMPDjOoImoYiIiBhHbJ8PPDifS/YAfuHKJcCKklZrNW4SioiIiP6yOvDXhuO7SllLFm31BhHj1cDfNuzIuvQ7f+CgToQBYOmr7+pYrBsOWbsjcVa+Sh2JA7DizU91LNY9r1qqY7GeWa5joRhYojPbPay44fy+oNfvyjf/T0s/iKP5fTNhtZs+RNVVMWiq7amtxK9DEoqIiIguG2BgxNeW5KGVBOJuYM2G4zVKWUvS5REREdFlcz0w4kcNTgbeW2Z7bAc8YvveVm+aFoqIiIguG6C+riBJxwI7AS+WdBfweWAxANs/AU4F3gzcDPwTeH8dcZNQREREdNkczx3xtQsaXWN7vwWcN/AvIw44QkkoIiIiuqzOFopuSUIRERHRZXPHQUKRQZnREZJeKuk4SbdIukLSqZI27EI9/rPTMSMiFmQAj/gxViWhiLaTJOAkYJrt9W1vA3wWWLUL1UlCERFjzlx7xI+xKglFdMLOwJwyuhgA21fbvkDSZyRdXjao+W8ASetIukHS0ZJulPRrSbtIulDSTZKmlOuWKZvgXCbpKkl7lPIDJP1e0unl+q+V8sOBpSTNkPTrzv81REQMbWAUj7EqYyiiEzYDrmgulPQGqs1ppgACTpb0auBOYAPgncCBwOXAu4Adgd2pWhneBhwKnGv7QEkrApdJOrvcfhKwFfA0MFvS920fIumjtie1521GRCycZ8Zwy8NIpYUiuukN5XEVcCWwMVWCAXCb7Zm2B4BZwDllqtNMYJ2G1x8iaQYwDVgSWKucO8f2I7afAq4DRrQOtKSDJU2XNH3qLx9p8e1FRIxMWigiRmYWsNcQ5QK+YvuI5xVK61C1LAwaaDgeYN7PrYA9bc9uev0rml4/lxH+rDcuadupvTwiIubSuT1p2iUtFNEJ5wJLSHpuMxtJWwCPAgdKWraUrS7pJaO47xnAx8qgTyRtNYLXzJG02ChiRES03YBH/hir0kIRbWfbkt4OfEfSfwBPAbcDnwQeBi4uOcHjwLupWhRG4kvAd4BrJC0C3AbstoDXTC3XX2l7/1G9kYiINhkPLRRJKKIjbN8D7D3Eqe+WR7PNGl57QMPz2wfP2X4S+NAQsY4Gjm443q3h+X8A/zG62kdEtFcSioiIiGjZHPf+CIQkFBEREV02dxwMaUxCERER0WUDTpdHREREtChjKCIiIqJlczOGImL82vkDB3Ukznk/O7IjcQDeuNf7OhZr+Zs68wvy/tc805E4AEvf37klTFaa3bk1EZd4cE7HYk146tkORVqyQ3GKN7f28jlMqKceXZSEIiIiosvGQwtF77+DiIiIHjeARvwYCUm7Spot6WZJhwxxfi1J55Wdmq+R1GIbS1ooIiIiuq7OaaOSJgA/BF4P3AVcLulk29c1XHYYcLztH0vaBDiVeRsvLpQkFBEREV1Wc5fHFOBm27cCSDoO2INq5+VBBpYvz1cA7mk1aBKKiIiILhuodwTC6sBfG47vAl7RdM0XgDMlfQxYBtil1aAZQxEREdFlz3jCiB+SDpY0veFx8IIjvMB+wNG216Cao/LLssniQksLRURERJcNjKLLw/ZUqp2Th3M3sGbD8RqlrNEHgF3L/S6WtCTwYuC+EVekSVoooqdIOlTSrDIqeYakV0iaJmlyt+sWEbGw5rLIiB8jcDkwUdK6khYH9gVObrrmTuB1AJJeTrVwxz9aeQ9poYieIWl7YDdga9tPS3oxsHiXqxUR0bK5Ne7lYftZSR8FzgAmAEfZniXpi8B02ycD/wb8VNK/Ug3QPMC2W4mbhCJ6yWrA/bafBrB9P4A07z+ipB8D2wJLASfY/nwpfzPwLeAJ4EJgPdu7dbT2ERHDqHlQJrZPpZoK2lj2uYbn1wGvrDNmujyil5wJrCnpRkk/kvSaIa451PZkYAvgNZK2KH2DRwBvsr0NsMpwARoHO91756VteRMREc3mepERP8aqsVuziCa2Hwe2AQ6m6uv7raQDmi7bW9KVwFXApsAmwMbArbZvK9ccO58YU21Ptj15tbWaZ1lFRLTHHE8Y8WOsSpdH9BTbc4FpwDRJM4HndruStC7waWBb2w9JOpqO7xAUETF6da6U2S29/w6ib0jaSNLEhqJJwB0Nx8tTjZF4RNKqwJtK+WxgPUnrlON92lzViIhRGbBG/Bir0kIRvWRZ4PuSVgSeBW6m6v44AcD21ZKuAm6gWiXuwlL+pKSPAKdLeoJqSlVExJgxHlooklBEz7B9BbDDEKd2arjmgGFefp7tjVVNCfkhML32CkZELKTRLGw1ViWhiH7xQUnvo1q34iqqWR8REWPCWB5sOVJJKKIv2P428O1u1yMiYihzGbtjI0YqCUVERESXpcsjIiIiWjaWF6waqSQUERERXTaQLo+I8Wvpq+/qSJw37vW+BV9UkzNOOKZjsXb+wEEdibPkg4t1JA7A3a/uWCjWPv2ZjsV6bK3O7bH37NJLdCSO3vJAR+LUJS0UERER0bLM8oiIiIiWjeUVMEcqCUVERESX1b19eTckoYiIiOiyuWmhiIiIiFalyyMiIiJaNh4GZfZ+p01ERESPq3v7ckm7Spot6WZJhwxzzd6SrpM0S9JvWn0PaaGIjpM0F5jZUPQ2YB3g07Z3a0O8dYA/2d6s7ntHRNShzqW3JU2g2lX59cBdwOWSTrZ9XcM1E4HPAq+0/ZCkl7QaNwlFdMOTtic1FpQP/YiIvlTzSplTgJtt3wog6ThgD+C6hms+CPzQ9kMAtu9rNWi6PGJMkbSIpJskrdJwfPPgccN1UyRdLOkqSRdJ2qiUbyrpMkkzJF1TsnCACZJ+Wpr2zpS0VIffWkTEsOZaI36MwOrAXxuO7ypljTYENpR0oaRLJO3a6ntIQhHdsFT5wJ8h6aTGE7YHgF8B+5eiXYCrbf+j6R43AK+yvRXwOeB/S/mHge+WFpDJVP+RACZSZeObAg8Dew5VMUkHS5ouafpfn7i2lfcYETFiA15kxI/G31PlcfBChFyU6vfiTsB+wE8lrdjKe0iXR3TDC7o8mhwF/BH4DnAg8PMhrlkBOKa0QBgY3NDhYuBQSWsAv7d9kySA22zPKNdcQTVm4wVsTwWmArxpjY97xO8oIqIFz45iDEXj76lh3A2s2XC8RilrdBdwqe05wG2SbqRKMC4fcUWapIUixhzbfwX+Lum1VH2Bpw1x2ZeA88pAy7cCS5bX/gbYHXgSOLXcA+DphtfOJcl0RIwhNc/yuByYKGldSYsD+wInN13zB6rWCSS9mKoL5NZW3kN+qcZYdSRV18cvbc8d4vwKzMu4DxgslLQecKvt70laC9iCFv+TRES0W52zPGw/K+mjwBnABOAo27MkfRGYbvvkcu4Nkq6j+pL1GdstbdGahCLGqpOpujqG6u4A+BpVl8dhwCkN5XsD75E0B/gb1diK5dtZ0YiIVtW9UqbtU4FTm8o+1/DcwKfKoxZJKKLjbC87RNk0YFpD0ZZUgzFvGOYeF1M10Q06rJQfDhzedPmDwHNrUNj+xsLUOyKiXWqeNtoVSShizCmruv0/5s30iIgY17KXR0QbDNPKEBExbj070PtzJJJQREREdFlaKCIiIqJlGUMRMY7dcMjaHYmz/E2da+rc+QMHdSzWeT87smOxdvi3D3UkznqfuaQjcQDu+OIOHYu14uTmhWjbZ/uX3NmxWNcdukXHYvHm1l6eFoqIiC7rVDIRvaWjyUQNklBEREREyzIoMyIiIlrmtFBEREREqzIoMyIiIlqWMRQRERHRsnR5RERERMvGQwtF7w8rjXFD0rclfbLh+AxJRzYcf1PSkDvjSTpA0ss6UM2IiNrNHVhkxI+xauzWLPrRhcAOAJIWAV4MbNpwfgfgomFeewAwqoRCUlroImJMsEf+GKuSUMRYchGwfXm+KXAt8JiklSQtAbwceIOkyyVdK2mqKnsBk4FfS5ohaSlJ20j6s6QrSkvHagCSpkn6jqTpwCe68B4jIl5gAI34MVYloYgxw/Y9wLOS1qJqjbgYuJQqyZgMzAR+YHtb25sBSwG72T4BmA7sb3sS8CzwfWAv29sARwFfbgi1uO3Jtr/ZobcWETFftkb8GKuSUMRYcxFVMjGYUFzccHwhsLOkSyXNBF7L87tEBm0EbAacJWkGcBiwRsP53w4XXNLBkqZLmv7YhZ3btyEi+tuANeLHWJU+5BhrBsdRbE7V5fFX4N+AR4GfAz8FJtv+q6QvAEsOcQ8Bs2xvP8Q5gCeGC257KjAVYN0ffHMM91ZGxHgylsdGjFRaKGKsuQjYDXjQ9lzbDwIrUnV7DA7IvF/SssBeDa97DFiuPJ8NrCJpewBJi0kaqiUjImJMGBhYZMSPkZC0q6TZkm6WdMh8rttTkiVNbvU9JKGIsWYm1eyOS5rKHrF9P1ULxbXAGcDlDdccDfykdHFMoEo2virpamAGZfZIRMRYVGeXh6QJwA+BNwGbAPtJ2mSI65ajGpx+aR3vIV0eMabYngss31R2QMPzw6jGRDS/7kTgxIaiGcCrh7hup3pqGhFRn5q7PKYAN9u+FUDSccAewHVN130J+CrwmTqCpoUiIiKiy0Yzy6Nx8Hh5HNx0u9Wpxp8NuquUPUfS1sCatk+p6z2khSIiIqLLRjMdtHHw+MIoCwd+i2pBwNokoYiIiOiymqeD3g2s2XC8RikbtBzV1PppkgBeCpwsaXfb0xc2aBKKiIiIbqt3DMXlwERJ61IlEvsC73oulP0I1eB3oFpBGPh0K8kEJKGIiIjoujpXwLT9rKSPUs2GmwAcZXuWpC8C022fXFuwBkkoIoax8lWdWZHu/tc805E4AEs+uFjHYu3wbx/qSJyLvnlER+IATFnywx2LtehTHQvFEke8qGOxZiy+ckfiPDGxt+Yc1L2wle1TgVObyj43zLU71REzCUVERESXjeU9OkYqCUVERES3JaGIiIiIVnmg2zVoXRKKiIiILkuXR0RERLRuHOw2moQiIiKiy9JCEREREa0bBy0UvTVRN2IEJB0qaZakayTNkPQKSdMkzZZ0taQLJW3U7XpGRMyjUTzGprRQxLgiaXtgN2Br209LejGweDm9v+3Bnfm+DuzerXpGRDzPOJjlkRaKGG9WA+63/TSA7ftt39N0zfnABh2vWUTEcKyRP8aoJBQx3pwJrCnpRkk/kvSaIa55KzCzw/WKiBiWPfLHWJWEIsYV248D2wAHA/8AfivpgHL615JmAK8EPj3U6yUdLGm6pOn/uOHiDtQ4IoJqUOZIH2NUxlDEuGN7LjANmCZpJvC+cmr/BW3Pa3sqMBVg8ge+NYb/60bEuDKGuzJGKi0UMa5I2kjSxIaiScAdXapORMSIaGDkj7EqLRQx3iwLfF/SisCzwM1U3R8ndLNSERHzNQ5aKJJQxLhi+wpghyFO7dThqkREjNw46GBNQhEREdFtSSgiIiKiZeMgocigzIiIiG6reWErSbuW7QZulnTIEOc/Jem6skXBOZLWbvUtJKGIiIjosjpneUiaAPwQeBOwCbCfpE2aLrsKmGx7C6pB619r9T0koYiIiBhfpgA3277V9jPAccAejRfYPs/2P8vhJcAarQbNGIqIYax481MdibP0/Yt1JA7A3a/uWCjW+8wlHYkzZckPdyQOwGVf/knHYr1l8q4di/Xo9i23do/YMnd35v+VJyzZkTh10SjGUJQNDg9uKJpaFuUbtDrw14bju4BXzOeWHwBOG3kNhpaEIiIiottGsQ5F44q+rZL0bmAyMNS+R6OShCIiIqLb6p3lcTewZsPxGqXseSTtAhwKvGZwh+ZWZAxFREREt9W7OdjlwERJ60paHNgXOLnxAklbAUcAu9u+r463kBaKiIiILqtzjw7bz0r6KHAGMAE4yvYsSV8Epts+Gfg61VYFv5MEcKft3VuJm4QiIiKi22pe2Mr2qcCpTWWfa3i+S70Rk1BERER03WhmeYxVSSgiIiK6bRzsNppBmYGkl0o6TtItkq6QdKqkDbtdLwBJf5DUmQUNIiK6pd5BmV2RhKLPqRqNcxIwzfb6trcBPgus2up9JbX08yVpRWAbYAVJ67Vyr6b7TqjrXhERdahz6e1uSUIROwNzbD+3BKDtq21fUJKCr0u6VtJMSfsASFq2bCZzZSnfo5SvUzaj+QVwLbCmpKMbXv+v5br1JZ1eWkMukLTxMHV7B/B/VMvG7jtYWF5/Sbnn/0h6vJQvIulHkm6QdFZpadmrnLtd0lclXQm8s+6/xIiIVsgjf4xVSShiM+CKYc69A5gEbAnsAnxd0mrAU8DbbW9NlZB8s7R0AEwEfmR7U+DFwOq2N7O9OfDzcs1U4GOlNeTTwI+Gib8fcGx57NdQ/l3gu+WedzXVdx2qzXDeA2zfdL8HbG9t+7hh4iHpYEnTJU2/+97LhrssIqJe6fKIcW5H4Fjbc23/HfgzsC0g4H8lXQOcTbVu/GAXyR22B8c83AqsJ+n7knYFHpW0LLAD1dznGVQLq6zWHFjSqlTJyV9s3wjMkbRZOb098Lvy/DdN9f2d7QHbfwPOa7rtbxf0hm1PtT3Z9uTVV5uyoMsjIuqRhCLGgVlU4xRGY39gFWAb25OAvwODO/E8MXiR7YeoWjemAR8GjqT6mXvY9qSGx8uHiLE3sBJwm6TbqVoe9hviutF4YsGXRER0Xro8Yjw4F1ii7F4HgKQtJL0KuADYR9IESasArwYuA1YA7rM9R9LOwJBbFUp6MbCI7ROBw4CtbT9KlSS8s1wjSVsO8fL9gF1tr2N7HaqkZ3AcxSXAnuX5vg2vuRDYs4ylWBXYabR/GRERsXCSUPQ52wbeDuxSpo3OAr4C/I1q9sc1wNVUice/l66EXwOTJc0E3gvcMMztVwemla6NX1HNHoGqheMDkq6maiHZo/FFktahSlKemy5q+zbgEUmvAD4JfKp0uWwAPFIuO5FqTMV1Jd6VDeca7797WYI2ImJMGA+zPLKwVWD7HqouhqF8pjwar7+fFw54HLRZw3VXA1sPEe82YNf51Od2qmSkuXxrAElLA9vZtqR9gY3K+QFJn7b9uKSVqVpTZpZz6zTc52SaNsqJiOiqMdyVMVJJKKIXbQP8oMwseRg4sOHcn8r6FYsDXyotKhERY1sSiojOs30B1WDPoc7t1NnaRES0biwPthypJBQRERHdloQiIiIiWjWWB1uOVBKKiIiIbksLRcT4dc+rlupInJVmd+6rydqnP9OxWHd8cYeOxFn0qY6EAeAtk4ednFS7U6af3rFYGxz/oY7FYqAz/69ecnlvfUJnDEVERES0bhwkFFnYKiIiottq3stD0q5l9+ebJR0yxPklJP22nL+0LCjYkiQUERERXVbnXh6SJgA/BN5EtfvyfpI2abrsA8BDtjcAvg18tdX3kIQiIiKiy2peensKcLPtW20/AxxH0xYH5fiY8vwE4HVlscCFloQiIiKi20bR5SHpYEnTGx4HN91tdeCvDcd38cLtDJ67xvazVPserdzKW8igzIiIiG4bxaBM21OBqW2ry0JKC0VERESXaRSPEbgbWLPheI1SNuQ1khYFVgAeWMjqA0koogWS5kqa0fBYR9JFNceYIem4Ou8ZETHm1DvL43JgoqR1JS0O7MsLd1g+GXhfeb4XcK7tliavpssjWvGk7UlNZbWtZiTp5cAE4FWSlrH9RE33nWB7bh33ioioQ50LW9l+VtJHgTOofoceZXuWpC8C022fDPwM+KWkm4EHqZKOliShiFpJetz2spJ2Ar4A3A9sBlwBvNu2JW0DfAtYtpw/wPa9Q9xuP+CXwMupRiT/psTYluo/wwBwFvAm25tJWho4usSbDbwM+Bfb0yU9DhwB7AL8C/CX2t98RMTCqnnBXNunAqc2lX2u4flTwDvrjJkuj2jFUg3dHScNcX4r4JNU86DXA14paTHg+8BetrcBjgK+PMz996Ga7nQsVXIx6OfAh0rrSGNLw0eo5lVvAvwXsE3DuWWAS21vaXvYZKJx9PRDV1483GUREbWqcx2KbkkLRbRiqC6PRpfZvguqsRDAOsDDVC0IZ5UpzxOAF7ROSJoM3G/7Tkl3A0dJehFVHr+c7cFP+98Au5XnOwLfBbB9raRrGm45FzhxQW+ocfT0Jod9ewz/142IcWUc/LZJQhHt9HTD87lUP28CZtnefgGv3Q/YWNLt5Xh5YE/gdwtZl6cybiIixqqx3PIwUunyiE6bDawiaXsASYtJ2rTxAkmLAHsDm9tex/Y6VGMo9rP9MPCYpFeUyxsHEl1YXkdZZnbzdr6RiIja1LyXRzekhSI6yvYzkvYCvidpBaqfwe8AsxouexVwt+17GsrOBzaRtBrVGvQ/lTQA/JlqhTeAHwHHSLoOuKHc8xGaSPpwqctP6nxvERELa4RLao9pSShiodledrgy29OAaQ3lH214PgN49Xzu+2dgu6ayucBLASQ9ZnuL8vwQYHq57CmqmSRPSVofOBu4o7muSSQiYswZwy0PI5WEInrRWyR9lurn9w7ggFK+NHBemUki4CNlY5yIiDFtPIyhSEIRPcf2b4HfDlH+GDC58zWKiGhREoqIiIholVpb9XpMSEIRERHRbb2fTyShiBjOM8t1Js4SD87pTCDgsbUW71isFSf/o2OxljjiRR2J8+j2a3ckDsAGx3+oY7Fu3vuIjsW65dnHOxLnA+f9a0fi1CWzPCIiuqxTyUREO2VQZkRERLQuCUVERES0Ki0UERER0bokFBEREdGqtFBEREREyzTQ+xlFEoqIiIhu6/18ItuXR0REdJsGRv5oKY70IklnSbqp/LnSENdMknSxpFmSrpG0z0junYQiFkjSoQ0/WDMkvULSNEmTm667XdKLG453kvSn8vwASZa0S8P5t5WyvcrxNEmzJV0t6UJJGzVce4Kk9Wp4L8dJmtjqfSIiauVRPFpzCHCO7YnAOeW42T+B99reFNgV+I6kFRd04yQUMV+Stgd2A7YuW4bvAvx1IW83E9i34Xg/4Oqma/a3vSVwDPD1UodNgQm2b13IuI1+DPx7DfeJiKiNPPJHi/ag+v1K+fNtzRfYvtH2TeX5PcB9wCoLunESiliQ1YD7bT8NYPv+8gO2MC4ApkhaTNKywAbAjGGuPb+cB9gf+OPgCUm7SrqytGScU8q+IOkYSRdIukPSOyR9TdJMSaeXLc0H67CLpIwfiogxQwMe8aNFq9q+tzz/G7DqfOslTQEWB25Z0I2TUMSCnAmsKelGST+S9JoW7mXgbOCNVFnyyfO59q1ULRoArwSuAJC0CvBTYM/SkvHOhtesD7wW2B34FXCe7c2BJ4G3ANgeAG4GthwqqKSDJU2XNP3RSy9emPcYETF6o+jyaPw9VR4HN95K0tmSrh3iscfzQtrz7USRtBrwS+D95XfnfOVbWsyX7cclbQO8CtgZ+K2kofrcYOgfzOay44CPAysA/wb8Z9P5X0t6Ergd+FgpWw0Y3GlqO+B827eV+j3Y8NrTbM+RNBOYAJxeymcC6zRcdx/wMkqS8rzK2lOBqQAbfPXb42DcdUT0gtF0ZTT+nhrm/C7DnZP0d0mr2b63JAz3DXPd8sApwKG2LxlJvZJQxALZngtMA6aVD+v3DXPpA8BKwP3l+EUNzwfvdZmkzYF/2r5RUvM99rc9vansSWDJEVR1sFtmQNKckn0DDPD8n/Ulyz0jIsYGd+z7y8lUv8MPL3/+sfkCSYsDJwG/sH3CSG+cLo+YL0kbNc2KmATcMczl04D3lNdNAN4NnDfEdYfwwpaJ+bmeeeMpLgFeLWndEmdhtprcELh2IV4XEdEWHRyUeTjwekk3UQ2yPxxA0mRJR5Zr9gZeDRxQZvbNkDRpQTdOC0UsyLLA98uUoWepxh8cDJwAnCJpTrnuYuAg4MeSrgZE1eXwq+Yb2j5tlHU4BdgJONv2P0p/4e8lLULVXPf6kd5I0qrAk7b/Nso6RES0T4caKGw/ALxuiPLpVL/Dsf0rhvjdvSBJKGK+bF8B7DDEqZ2Gecm7hrnP0cDRQ5Qf0PB8uHueAJwn6fO255aE5HlJie0vNB0vO8y5dwFHDBMnIqIrNLf3h2ylyyPGPNtPAp8HVq/hdg8zbw52RMTY0LmFrdomLRTRE2yfUdN9fl7HfSIi6pTdRiMiIqJ1nZvl0TZJKCIiIrosLRQR49jAEp35Hz7hqWc7Egfg2aWX6Fis7V9yZ0fizFh85Y7EAVjm7qc6FouBpToW6pZnH+9YrPUXXXbBF9XgqZUmdCRObZJQRERERKvGwyyPJBQRERFdpoyhiIiIiJb1fj6RhCIiIqLr0kIRERERrcosj4iIiGhZBmVGRERE69LlERERES3r/Xwim4PF6EmaK2lGw2MdSTtJ+tMQ124q6VxJsyXdJOm/NM/9klYq160myZJ2bHjtPyStXJ5/UtJ7a6j7RyUd2Op9IiLqJHvEj7EqCUUsjCdtT2p43D7URZKWAk4GDre9EbAl1VboH7Ft4BJg+3L5DsBV5U8kbQQ8YPsBSYsCBwK/qaHuRwEfq+E+ERH1sUf+GKOSUEQ7vQu40PaZALb/CXwUOKScv4iSQJQ/v83zE4wLy/PXAlfafhZA0gaSzpZ0taQrJa1fWkj+LOmPkm6VdLik/SVdJmmmpPUb6nC7pCltfu8RESM3MIrHGJWEIhbGUg3dHSfN57pNgSsaC2zfAiwraXmqhGEwoZgCnASsWY53oEo4AF7ZdJ9fAz+0PdjicW8p3xL4MPBy4D3AhranAEfy/FaJ6cCrhqqwpIMlTZc0/dGLLp7PW4uIqI8GBkb8GKuSUMTCaOzyeHsL97kc2ErSMsBith8HbpW0Ac9voVgN+AeApOWA1W2fBGD7qdLqAHC57XttPw3cApxZymcC6zTEvQ942VAVsj3V9mTbk5ffYfuhLomIqF+HujwkvUjSWWVM21mD49iGuXZ5SXdJ+sFI7p2EItrpOmCbxgJJ6wGP2360JAI3UY2PuLJccgnwZuAlwOxS9iSw5AjiPd3wfKDheIDnz2hastwzImJs6FyXxyHAObYnAucwrwt6KF8Czh/pjZNQRDv9GthR0i7w3CDN7wFfa7jmIuCTwGD/wsXAJ4BLysBNgOuBDQBsPwbcJelt5Z5LSFp6lPXaELh2tG8mIqJdOjjLYw/gmPL8GOBtQ9ZH2gZYlXktvQuUhCLq9LrSPHaXpLuASVQ/vIdJmk3V9XA50Nh8diGwHvMSiiuBNZg3fgLgNODVDcfvAT4u6Zpy3UtHWc9XAmeN8jUREe3TuVkeq9oeHHf2N6qk4XkkLQJ8E/j0aG6cha1i1GwvO0TZNGCpYV6y03zu9TtADcdPA0s0XXOHpAckTbR9k+2bqGZ+NLoVmNbwmp0ank8bPCdpK2CW7QeGq1NERMeNYrClpIOBgxuKptqe2nD+bIb+onVo44FtS0PuIvIR4FTbd0ka4vTQklBErziEanDmTS3e58XAf7VenYiIGo1ibERJHqbO5/wuw52T9HdJq9m+V9JqVIPUm20PvErSR4BlgcUlPW57fuMtklBEb7A9m3mDNFu5T7o6ImLM6eAKmCcD7wMOL3/+sfkC2/s/Vy/pAGDygpIJyBiKiIiI7uvcGIrDgddLugnYpRwjabKkI1u5cVooIiIium2gMy0UZfzY64Yonw4cNET50cDRI7l3EoqIYay44YMdijSSJTbqobd0bizqdYdu0ZE4T0zsXEOrJ3Tu3+oll3duz4YPnPevHYv11EoTOhLnksN/0pE483yqtZeP4T06RioJRURERLeN4SW1RyoJRURERLd1qMujnZJQREREdJvTQhERERGtyhiKiIiIaFm6PCIiIqJlaaGIiIiIlo2DWR4LnMAt6aWSjpN0i6QrJJ0qacNOVK4uko6UtEkN99lN0lWSrpZ0naQPlfK31XH/EdbhPxueryNpRNtwS/qkpPeW51+XdIOkaySdJGnFUr6YpGMkzZR0vaTPjvDet0t68UK8l8UlnS9p0Yay1ST9qTzfSZIlHdRwflIpG9UueA2v31zS0Qvz2oiIthkYGPljjJpvQqFqm7GTgGm217e9DfBZhtjutB0aP2haYfsg29e1WJfFqDZjeavtLYGtmLe75duAIROKut5Dg/9c8CVD1uFA4Del6CxgM9tbADdS/ZsCvBNYwvbmwDbAhySt03KNh2H7GeAcYJ+G4k8BP204vhbYu+F4P+DqFmLOBNaQtNbC3iMionadW3q7bRbUQrEzMMf2c0uO2b7a9gWqfF3SteUb7T7w3LfKP0v6o6RbJR0uaX9Jl5Xr1i/XHS3pJ5KmS7pR0m6l/ABJJ0s6FzhH0jKSjiqvv0rSHuW6TUvZjPJNe2K59pTSgnBtQ52mlXXKPyzp64PvpcT6QXn+7ob7HSGpeTm35ai6iB4ofw9P254taQdgd+Dr5bXrl3jfkTQd+ISkbcrfyRWSzig7vA3W66sl7o2SXlXKl5Z0fGkFOUnSpaX+hwNLlTi/LvWaIOmnkmZJOlPSUFuIvxa40vazpe5nDj4HLgHWGPznBZYpCchSwDPAo803k7RyiTVL1drvajj3h/I+Z6naYhdJB0r6TsM1H5T07XL4B2D/htvvCZzecHwHsKSkVUuCuytwWsO9pkn6bvk7uVbSlFI+RdLF5WfmIkkbNdzz/4B9h/h7iojojj5IKDYDrhjm3DuAScCWVBuMfH3wg7KUfRh4OfAeYEPbU4AjgY813GMdYArwFuAnkgbXtd0a2Mv2a6j2bz+3vH7nEmeZcv/v2p4ETAbuovqwucf2lrY34/kfTAAnAm9vON4HOE7Sy8vzV5b7zeX5H3LYfpBql7Y7JB1bkqRFbF9Uyj9je5LtW8pLFrc9Gfge8P3yfrYBjgK+3HDrRct7+yTw+VL2EeAh25tQbbW9TanDIcCTJc5g/SYCP7S9KfAw1Qdys1cy/L/jgcz7gD4BeAK4F7gT+EZ5380+D/ylxDwJaPy2f2B5n5OBj0taGTgeeGtp5QF4f/l7gKoFYlsASeuW9/10U7wTqFpPdgCuBJrPL13+3T7ScN8bgFfZ3gr4HPC/DddPB1411F+GpINLkjv9/tOuHOqSiIj6DXjkjzGqleb4HYFjbc8F/i7pz1QfDI8Cl9u+F0DSLcCZ5TUzqZKCQcfbHgBuknQrsHEpP6vhg+wNwO6a12e+JNUH2MXAoZLWAH5v+yZJM4FvSvoq8CfbFzRW2PY/SqvJdsBNJd6FwL9QfWhfXn0JZimG2CPe9kGSNqdKoD4NvB44YJi/n9+WPzeiSszOKveeQPWBPej35c8rqBIsqP5uv1tiXivpmmFiANxme8YQ92i0GnB9c6GkQ4FngcHWjilUydTLgJWACySdbfvWppe+miqhxPYpkh5qOPdxSYNJ25rARNuXlBan3SRdDyxWuh6wPVfSM5KWK/X8xxD1P57q73Nj4FiqxKLRseVe50taXtWYkOWAYyRNpGp5Wazh+vvKe3wB21OpurbY+tTDxu7/3IgYVzx3brer0LIFJRSzgL0W4r6N3yAHGo4HmmI2/8IePH6ioUzAnrZnN117vaRLqVo3TpX0IdvnStoaeDPwP5LOsf3FptcdR9UnfwNwkm2XpvRjbC9wEGL5IJwp6ZfAbQyfUAy+BwGzbG8/zHWDfzdzWbgEr/Hvei5VMtTsSZp2oFK1x/1uwOvs59rQ3gWcbnsOcJ+kC4HJkt4EfLBc8+bhKiJpJ6pka3vb/5Q0rSHukVTjP24Aft700iWAp4aqJ4Dtv0maQ5XAfYIXJhRD/Rx9CTjP9ttVjQOZ1nB+yRIrImJsGMNdGSO1oC6Pc4ElBvvCASRtUfr6LwD2kTRB0ipU31ovG2X8d0paRNW4ivWA5qQB4AzgY+VDH0lblT/XA261/T3gj8AWkl4G/NP2r4CvU3WdNDsJ2INqcN9xpewcYC9JLyn3fpGktRtfJGnZ8oE5aBJV/z7AY1TfiIcyG1hF0vblPotJ2nSYawddSBmIqGr2yOYN5+Y0dB2M1PXABoMHknYF/h3Y3fY/G667k2q8BaVbaTvgBts/LN0sk2zfA5xPlXxQko2VyutXoOqy+KekjcvrAbB9KVWLxbsoLQrl9SsD95ck5kaGbmGBqtviP0qLWLPBsTI7Ao/YfqTU5e5y/oCm6zek6mqJiBgbxvssj/LN9e3ALqqmjc4CvgL8jeqD+RqqEffnAv9u+2+jjH8nVRJyGvBh208Ncc2XqJqrrynxv1TK9waulTSDqkvhF1QfvJeVss8D/zPEe3qI6gN2bduXlbLrgMOAM0v3wllUze+NBPy7pNnl/v/NvA+q44DPlAGA6zfFe4aqleerkq4GZvDCb9jNfkSVhFxX3sMs4JFybmr5u/j1cC8ewmlUCd+gH1AlQGeVwYyDg25/CCxb/p4vB35ue6julv8GXl2uewfVvyNUY1YWLd0ah1MN+Gx0PHBh+TcYtDNwCoDtJ4BbJG3Q9DpsX2T7D8O8v6ckXQX8BPhAKfsa8JVS3tzy81zMiIgxYRwMypS7VDlVawH8yfYJXanAGKZqhslitp8qCcrZwEYlOVnYe55ElfTdVFc9F6IOfwK+bfuchrLfA4fYvrEcvx3YxvZhI7znNODTtqeP8PolgD8DOzbMdBlSp8ZQrPzlF/TytM0Dhw6Vs7fH8t9fviNxHpo42ga7hbf0/Z37dmgt+Jq6LP54597XUys1T6Brj0sO/8mCL6rRIi+9saV/sTcu+74R/7454/FjOvjTMXJZKXNsWho4r3RtCPhIK8lEcQhVq0vHE4oySPIy4OqmZGJx4A+DyQSA7ZNKN0i7rEWVwMw3mYiI6Kgx3PIwUl1LKGwf0K3YY53tx6imXdZ5z9kMPUal7Ww/TDVuobn8GaququbyI0dx751GWZeb6EJSFRExXx2a5SHpRVSz5tYBbgf2buqGHrxuLarB9GtSDXR/s+3b53fvBS69HREREe3lAY/40aJDgHNsT6SakHDIMNf9Avi67ZdTLSnwgqUUmiWhiIiI6DYPjPzRmj2AY8rzY6i2jnieMrtwUdtnAdh+vGlG4JCSUERERHRZB1soVh1ceJJqxuZQe3NtCDws6fdl9uLX9cLtKF6ga7M8IsYjSQeX1TYTa4zHGo/vabzGGo/vqRVlbaiDG4qmNtZZ0tnAS4d46aFUiziu2HDtQ7ZXarxI0l7Az6g2wbyTaszFqbZ/Nt96JaGIqI+k6WUPl8Qa47HG43sar7HG43vqFkmzgZ1s36tq/61ptjdqumY74Kuu9tNC0nuA7Wz/y/zunS6PiIiI/nEy8L7y/H1UK003uxxYsayCDdUKytct6MZJKCIiIvrH4cDrJd1EtffS4QCSJks6EqpNG6k2wDxH1aabAn66oBtnYauIenWy7zWxeiNOYvVOnE7H6jjbDwCvG6J8OnBQw/FZwBajuXfGUERERETL0uURERERLUtCERERES1LQhEREREty6DMiHiBss37nlQbCD33e8L2F2uOMwH4uO1v13nfsUjSF2x/oUOxxvziTGOBpDWAfYFXAS8DngSuBU4BTrNbX+e6nyShiGiRpF/afs+CymqK9X9UO/81egSYDhxh+6maQv2x3PcK4Oma7vkCtudK2g9oa0IhaSLwDWB9YCbwadt3tynWIsBeto9vOnVFm2JtZ/ui5lN1x2qKuSUNH8C2F7hx1ELGmswLP+zPGmp3zIW498+B1YE/AV+l2vxqSaplp3cFDpV0iO3zW43VLzLLI6JFkq60vXXD8QRgpu1N2hDru8AqwLGlaB/gUaokY/m6khhJ19rerI57jSDWt4HFqJb3fWKw3PaVNca4gGr3xPOB3YHtbb+jrvsPEa+TKzteZXurDsRZH/gPqrULbgL+wbwP4H8CR1At69zyt3pJ7wc+BtxGlYg1fti/kiqx+C/bd7YQYzPb187n/OLAWrZvXtgY/SYJRcRCkvRZ4D+Bpah+oUL1zfAZqrX1P9uGmJfb3naoMkmzbG9aU5ypwPdtz6zjfguIdd4Qxbb92hpjzLA9qeH4eUlg3SQdDtzPC5OkB9sQ6xvAxcDv3cZf6JKOBX4MXNAcR9JLgHcBD9k+ZqjXjzLWvwBH2X5ymPOTgJVtn9NqrKhPEoqIFkn6SjuSh2FiXQ+8cfCbmaS1gDNsv7yOb6plVTxTdYdOBG6l6vIQ1Yf8qBa6GSsk3QDsx7yugF8D+w+er7M1pMS7bYhi216vzjgl1mPAMsCzwFPM+7davu5Y41HpDvsKsAlVKwgA7fi3Gu+SUETUQNLqwNo8fwBj7X2vkt4M/AS4heqDY13gI8A04IO2v9Pi/dee33nbd7Ry/2Firgr8L/Ay22+StAlVl8R8dzYcZYzzqBKlxrEFz/3yq7M1ZLwrXXpv4YUDdr/VhljrUnV9NMfavcYYfwE+TzWO563A+4FFbH+urhj9IglFRItK8/a+VJvnzC3FrvOXXlO8JYCNy+HsGgdiNsZYH7jL9tOSdqJagvcXth9uQ6zTgJ8Dh9reUtKiwFW2N68xxhTgr7bvLcfvo5rFcjvwhbq7IiQNNT7jEaqxNbUOYJQ0VNfNI8Adtp+tM1aJdypVS8hM4LnxErb/uw2xrqbaRrs51p9rjHGF7W0kzRz8mRssqytGv0hCEdGish3wFrbbNhuiKd5mvLB59hc1x5gBTKb6Zngq1ayPTW2/uc44JdbgGJDnumyaxzzUEONKYBfbD0p6NXAc1TffScDLbe9VV6wS7xRge2BwfMhOVIML1wW+aPuXNca6BNia6kMXYHOqQYsrAP/P9pl1xSrxrulU15ekS22/os0xLgJ2BE4AzgXuBg5v3tI7FiwLW0W07laqWQptJ+nzwPfLY2fga1SzFuo2UL7dvoNqcOZngNXaEAfgCUkrU7ogJG1H9Q27ThMaWiH2oRo0e6Lt/wI2qDkWVM3zL7e9p+09qRJAA6+gmilRp3uArWxvU75VT6L6mXw91c9H3U6T9IY23Hco35X0eUnbS9p68FFzjE8ASwMfB7YB3sO87b1jFLIORUTr/gnMkHQODWs22P54G2LtRbUGwFW231/GH/yqDXHmlPUh3kvVrwztS5o+BZwMrC/pQqppse+sOcYESYuWJOl1wMEN59rxe3BN239vOL6vlD0oaU7NsTa0PWvwwPZ1kja2favUluUoLgFOKutRzKG9g0A3p/qAfy3zujxcjmth+3J4bn2Nj9t+rK5795skFBGtO7k8OuFJ2wOSnpW0POWDqg1x3g98GPiy7dvK4LjamumbzAJeA2xE9eE0m/pbT48F/izpfqoFki4AkLQB9beGAEyT9Cfgd+V4z1K2DPBwzbFmSfoxVTcOVC0w15WxNnUnLwDfourOmdnOaarFO4H1bD/TrgBl8ayfA8uV40eAA23XvhDZeJcxFBE1kLQU1SI4s9sc50dUa1/sC/wb8Dgww/b72xm3nYZaE6Id60SUrpTVgDNtP1HKNgSWbcO0UVElEa8sRRcCJ7bjA7j87H2EahzAYKwfUQ2cXNr24zXHOx/YqY4FrEYQ6w/AwXUPZG2KcQ3wL7YHk8wdgR/16hTpbkpCEdEiSW+lWtZ5cdvrlkV3vtiuWR4NcdehWh3zmjbcu+1z8yW9lGrp419RLYo02D6/PPAT2xsP99roHklHA+sBp/H8Lr52TBudRjXD6PKmWHVOG33B+i3tXvhsvEqXR0TrvgBMoVoLAtszJNW6KM78BqJJ2rrub9hUTcCDc/N3pszNrznGG4EDgDWomtEHPUbVCtOzymJTw+258m+2b60x1uBiZEPF+h/bD9QVq7itPBYvj3b6fJvvD1VX2BFU3WKm6jKaNvh/rg3/t8attFBEtEjSJba3a5r2WOvUuiEWZnref9y6F2bq5Nx8SXvaPrHu+3aTpC8BdwG/ofo325dqY7IrqaZy7lRjrK9RrX/ym1K0L9Wshb8BO9p+63CvHevK2J17B9daKd07q9q+vcYYQy39PshZ9GzkklBEtEjSz4BzgEOo+s0/Dixm+8M1xuj0wkwdnZsv6S3Apjy/e6XWrdI7SdLVtrdsKpthe9JQ51qMNewYlMaEsMZ4ZwHvHFzkTNJKwHG231hnnHLv6cAOg4MyVW3YdaGb9rOJsSFdHhGt+xhwKFUf77HAGcCXao7xE6pdHikLM32FeQszTaWaTlqnxrn5X6Lq9mjL3HxJPymxdgaOpHovl7UjVgf9U9LeVAkZVO9pcEXTur/FTZA0xfZlAJK2BSaUc7WvlAms0rhiqu2HVG0O1g6LNs7wsP1MSSpaJulT8zvfjjEh411aKCJ6QOO3Wkk/BP5h+wvluNZVJTttsHuo4c9lgdNsv6rbdVtYZQzNd6mmV5pq7YZ/pWrp2cb2X2qMtS1wFLAsVffKo8BBVNNx32L7+LpilXhXAG/3vA3q1gZOascgxtIa8n3bJ5fjPajWinhdDfceAGYwb3Dp8xbtcBuWEh/v0kIR0aIyj/0/eeEGRnVOO+v0wkydNPjN/Z+SXgY8QPtW5eyIMujyeWMXJG1r+2agtmSixLoc2FzSCuX4EUmrlqmxtSYTxaHAXyT9mepD+FU8/+exTh8Gfi3pB+X4LqqFruqwFdUOtG+hWhb9WOCcDqytMW6lhSKiRar28vgML9zAqLadOSUdCrwZuB9YC9jatsvCTMfYfuV8bzAGSfokcBHwBuAHVKsf/ojqG/2RZVnsnqZq59T9yuNh25PbGGtFqnE176Ja9vtlbYz1YmC7cniJ7fvbFavEWxag7jU1Gu6/A9W/0S7Afwy2iMToJKGIaJGkv9jeccFXthynkwszrdyG6YbNMb4B7AC8HLiGakGmi4GL2h27ncr6IINJxByqbe0n1zkzoSHWUsAeVEnEVlSrPb4NOL/uhackrTO/91AW81rd9l01xHo38Jvh3oOq3XBXq6PrSNIqwN5Uq3LOAf7L9iWt3rcfJaGIaJGk11F9eDTv5fH7rlWqRZJuoupf/jnVeIa2/aIog+wmUyUX25fHw7Y3aVfMdpF0MdXCXMdRzXy4SdJtttdtQ6zfUHU3nFninQvc3I5YJd7vqNYi+SNVF8E/qGblbEA1oPZ1wOdtn1VDrE8AB5Y4zbFeQ9VSd4jtm1qIcSBVIrEk1eDZ49u5Imc/6PW+14ix4P3AxlSbZzVuYNSzCQWwIVXz74HA9yQdDxxt+8Y2xFqK6kN4hfK4h3lbcfeav1Ot/rkq1SZnN1H/rI5BmwAPAdcD19ueK6ltiZ/td5YunP2pfi5Wo9oY73qqLe6/PLheRA2xvlvGTbyWavnyLaj2YLkeeM/ggNAWHUm1zfsdVIusvUENm6m1e6Xb8SgtFBEtkjS7XeszjAWSdqZaHnsZ4Gqqb4YX13DfqVRrTzwGXEo1E+IS2w+1eu9uKoMj30HVajURWBF44+C0zppjbVzi7EP1rX0jYDM/f6fTniNppXb/HEh6zfzO2/5zO+OPR0koIlok6efA121f1+261EXSysC7qUbU/x34GdWOqpOA39XRrC7pdODFVN8SL6IaP3HteBplX9Zn2JvqQ38t2+3YGXYw1jYlzt7AXbZ3qPn+E6n2rFmfqgXp07bvrjNGQ6z7qBKkC6l+Ni5sU+tY1CgJRUSLJF1P9Uv2NubNZ3fN00Y7StKNVNuV/7x5kJ2k/7D91ZriiKqVYofy2Ax4ELjYdif2cegYSWvXOfNnPnEEvMr2+TXf9wLgF8D5wO7A9rbfUWeMpngbMu/nYgeqLqRLqJKLr9Vw//+jWhTudNtzms6tR7XPzO22j2o1Vr9IQhHRorKwzwt04sOjXSSpky0Fktag6ivfAdgNWNn2ip2KHwvWvIDaUEt+tzH2+lTTpj9BNZNkqRru+VLgU1RTbR9k3sDPdYBbgB/Y/mOrcfpJEoqIGkjaEZho++dlGtqytm/rdr1Gq3xrG/aXQp0D1SR9nHnfPudQNW0PPmbWPe0xWiPpBqoulcGRi7+mYdv5Oqcul3UhBmf9rAncShljA1zZuBx3TfHWoRpk+iRwo+1/1nn/fpGEIqJFkj5PNe1xI9sbltUef9eji011bKCapG9R+shdNj2LsUvSNIZPNu0ad+Usy2JfCXybalnvtn/AS1oGeMr23HbHGq+SUES0SNIMqkWFrnSbti/vFElr1TQlr6+V/v8fU221vZmkLYDdbf9PG2KtCvwv8DLbbypTO7e3/bO6Y3VK6Y4YbKWYQrXEwZVUA3cvLkubtxpjEaqt3vcHtqUa/7QE1WDQU4AjylLpMUJJKCJaJOky21M0b8voZah+6fViQvFcv7ikE23v2e069aKyz8VnqD6UBpPMa21v1oZYp1EtQHao7S0lLQpc5fq3LZ/vAMx2LuQmaWmqtS8+Caxre8L8XzGie/4ZOJtqoa5rB7vYJL2IaqGud1G1jvyq1Vj9IgtbRbTueElHACtK+iDVL76fdrlOC6txx8X1ulaL3re07csaF0qiPVuJA7zY9vGSPgtg+1lJ7Wi2P4Fq9dQZ5bjxzdW6kFtZy2N75rVSbEW1SNj/UXWT1WGX5tkdALYfBE4ETpS0WE2x+kISioiFVDbmWtX2NyS9nmrb6I2otkM+tauVW3ge5nmMzv1lZoIBJO0FtGucyBNl3ZDBWNsBj7Qhzjuougi2oPpWf2wbuwRupnRvAF8ELrf9ZJ0BBpMJSd8EjrI9a7hrYmTS5RGxkCT9Cfis7ZlN5ZsD/2v7rUO/cuwq32yfoPr2uRTV0sowb22N5btVt15S1jGYSvXt+iGqNUre3aYNwrYGvk+1hse1VOs17GX7mrpjlXjLUG1Itg+wMlVXS9tWlezATqMHUS2fvyhV19GxttuRkI17SSgiFpKky21vO8y5mXX3YUfvKR++i9h+rM1xFqVqHRMwu53frCVNAHalaq3YnGq77zPaEOf/AZ+lWvJdVEu0f9X2j+qOVeJtRJVY7EfVrfJT2+e1I9Z4tUi3KxDRw1acz7mWF96J3iXpE5KWp2rh+bakKyW9oY0hpwBbAlsD+0l6b90BJL227L9yBdWgxe/antSmZOIw4K3ATrZXtj04UPJN5Vzd8SZQbfC3MdUsj6uBT0k6ru5Y41laKCIWkqRjgXNt/7Sp/CDg9bb36U7NotskXV1mXLwR+DBwGPDLdqwsKemXVEu/zwAGB2Pa9sdrjjMAXAP8hWq8xvM+POqMJ2k2sKWbdi+VtBRwte0Na4z1barVWc8Ffta4idt43/ivbhmUGbHwPgmcJGl/qm9tUC1wtTjw9m5VKsaEwRkQbwZ+YXuWmqZ81GgysEkHlko/kM4N1HVzMlEKnyyJTZ2uAQ6z/cQQ56bUHGtcS0IRsZBcbRG9Q9nee3B9gVNsn9vFasXYcIWkM4F1gc9KWg5o11Li1wIvpX2zSACwfXQ779/kbkmvs31OY6Gk11Lz+yzL5a8kaVOqvTwGy8/P4MzRSZdHRESNSkvEGlSzLW61/XCZ1rl6O2ZeSDqPalv5y6hWewTq3XelxGne58VU4w3Oq3vxp/Lh/keq7pXG1r9XAnsMNcWzhVgHUW06tgZVt9F2VAvT1baUeL9IQhERUbNOzvIZbv+VuqdyDhPnRcC7gZtsH1JzvCWpVqvctBRdR7Uh2Za2L60xzkyqpbcvsT1J0sZU077btjX7eJWEIiKiZpKOodr++vJu16XdygyJKxq3Nm9zvDttr1Xj/S63vW3Zk+cVtp+WNMv2pgt6bTxfxlBERNTvFcD+ku5g3kJhrnN/F0l/sb2jpMd4fldERxchsz23feNNh1R3sLskrQj8AThL0kPAHTXH6AtpoYiIqJmktYcqt13bB5Wkteu83wjivWiI4pWA9wIb2N6/Q/WotYWi6d6vAVYATrf9TDtijGdpoYiIqNngB72kl9Awc6BmJ1EtZNWpnWGvoGoJGWwhGByUOY1qrY3aDDEA9LlTVMt91xFjqARpcBn9ZYEH64jTT5JQRETUTNLuwDeBlwH3AWsD1zNvgGEtYRqet31nWNvrDlsR6bdUe3vU5RsLeW40GhOktaj2XBHVCrh3Uk35jVFIQhERUb8vUU0/PNv2VmWtknfXHGMs7Qy7fc33u832nTXf83kGEyRJPwVOsn1qOX4T8LZ2xh6vspdHRET95th+AFhE0iJlk6nJNcfYUtKjZVDmFuX5o5Iek/RozbE67Q+DTySd2OZY2w0mEwC2T6PaJTZGKS0UERH1e7hsu30B8GtJ91HN9qiN7Ql13m9ByjbpQ54CFqs7XMPzdnfn3FM2HBtcnGt/4J42xxyXMssjIqJmZdvyp6g+GPenmjnw69Jq0ZPKipzNgzKfO2175xpjXTm4kVrj83YogzM/D7y6FP0Z+KLtDMocpSQUERFtIGlVqhUYAS6zfV8369MqSVOAv9q+txy/D9gTuB34Qp0fwJLmMm/9jqWotoGHNq+xIWm1wfcXo5cxFBERNZO0N9XeGu8E9gYulbRXd2vVsp9Q9gqR9GrgK8AxwCPA1DoD2Z5ge3nby9letDwfPG7ngl2ntPHe417GUERE1O9QYNvBVglJqwBnAyd0tVatmdDQCrEPMNX2icCJZdnq8aCjS36ON2mhiIio3yJNXRwP0Pu/bydIGvwS+jrg3IZz4+XL6U+7XYFeNl5+CCIixpLTJZ0BHFuO9wFO62J96nAs8GdJ9wNPUs1gQdIGVN0ePUfSWsCjZYv5dYD7JG1m+9ouV60nZVBmREQbSHoHsGM5vMD2Sd2sTx0kbQesBpxp+4lStiGwrO0ru1q5UZJ0CPAhqnEh3wA+DVxItSDZz2x/q4vV60lJKCIialK+ra9q+8Km8h2Be23f0p2aRTNJs6gWG1uaaqbKerb/Uab8Xmp7s27Wrxf1ep9eRMRY8h1gqFUqHynnYuyYa/tJ4GGqLpwHAAZbXmL00kIREVETSZfb3naYczNtb97pOsXQJB0NLA4sQ7XOxbPA6cBrgeVs79292vWmDMqMiKjPivM5t1SnKhEjchDVOiGmms47BXgXMBv4YRfr1bPSQhERURNJxwLn2v5pU/lBwOtt17nFd8SYkoQiIqImZbntk4BngCtK8WSqpvW32/5bt+oWzydpeeCzwBrAabZ/03DuR7Y/0rXK9agkFBERNZO0MzA4S2CW7XPnd310XtkW/SbgEuBAYA7wLttPt3tDsvEqCUVERPQdSTNsT2o4PhR4M7A7cFYSitHLoMyIiOhHS0haxPYAgO0vS7obOB9YtrtV601ZhyIiIvrR/1FNEX2O7aOBf6MaAxOjlC6PiIiIaFlaKCIiou9IerekYT8DJa1flkyPEcoYioiI6EcrA1dJuoJqiu8/gCWBDYDXAPcDh3Sver0nXR4REdGXJE2gGkfxSqpdVJ8Erqdal+LObtatFyWhiIiIiJZlDEVERES0LAlFREREtCwJRURERLQsszwiIqJvSVoC2BNYh4bPRNtf7FadelUSioiI6Gd/BB6hmjr6dJfr0tMyyyMiIvqWpGttb7bgK2NBMoYiIiL62UWSNu92JcaDtFBERETfkTQTMFXX/0TgVqouDwG2vUUXq9eTklBERETfkbT2/M7bvqNTdRkvklBERETfkvRL2+9ZUFksWMZQREREP9u08aDs77FNl+rS05JQRERE35H0WUmPAVtIerQ8HgPuo5pKGqOULo+IiOhbkr5i+7Pdrsd4kIQiIiL6lqSthyh+BLjD9rOdrk8vS0IRERF9S9IlwNbANVRTRjcHrgVWAP6f7TO7WL2ekjEUERHRz+4BtrI92fY2wCSqNSleD3ytmxXrNUkoIiKin21oe9bgge3rgI1t39rFOvWkbA4WERH9bJakHwPHleN9gOvKLqRzulet3pMxFBER0bckLQV8BNixFF0I/Ah4Clja9uPdqluvSUIRERERLUuXR0RE9C1JrwS+AKxNw2ei7fW6VadelRaKiIjoW5JuAP4VuAKYO1hu+4GuVapHpYUiIiL62SO2T+t2JcaDtFBERETfknQ4MAH4PfD0YLntK7tWqR6VhCIiIvqWpPOGKLbt13a8Mj0uCUVERES0LCtlRkRE35K0qqSfSTqtHG8i6QPdrlcvSkIRERH97GjgDOBl5fhG4JPdqkwvS0IRERH97MW2jwcGAMqW5XPn/5IYShKKiIjoZ09IWhkwgKTtgEe6W6XelHUoIiKin30KOBlYX9KFwCrAXt2tUm9KQhEREX1J0gTgNeWxESBgtu3sMroQMm00IiL6lqTLbE/pdj3GgyQUERHRtyR9G1gM+C3wxGB5VsocvSQUERHRt7JSZn2SUERERN+StJ7tWxdUFguWaaMREdHPThii7Hcdr8U4kFkeERHRdyRtDGwKrCDpHQ2nlgeW7E6telsSioiI6EcbAbsBKwJvbSh/DPhgNyrU6zKGIiIi+pak7W1f3O16jAcZQxEREX1H0gclTbR9sSpHSXpE0jWStu52/XpREoqIiOhHnwBuL8/3A7YE1qNaivu7XapTT0tCERER/ejZhiW2dwN+YfsB22cDy3SxXj0rCUVERPSjAUmrSVoSeB1wdsO5pbpUp56WWR4REdGPPgdMByYAJ9ueBSDpNUAWtVoImeURERF9SdKiwHK2H2ooW4bqs/Hx7tWsNyWhiIiIiJZlDEVERES0LAlFREREtCwJRURE9K2yqNW7JX2uHK8laUq369WLMoYiIiL6lqQfAwPAa22/XNJKwJm2t+1y1XpOpo1GREQ/e4XtrSVdBWD7IUmLd7tSvShdHhER0c/mSJoAGEDSKlQtFjFKSSgiIqKffQ84CXiJpC8DfwH+t7tV6k0ZQxEREX1N0sZUy28LOMf29V2uUk9KQhEREX1L0veA42xf1O269Lp0eURERD+7AjhM0i2SviFpcrcr1KvSQhEREX1P0ouAPYF9gbVsT+xylXpOWigiIiJgA2BjYG3ghi7XpSelhSIiIvqWpK8BbwduAX4LnGT74a5WqkdlYauIiOhntwDb276/2xXpdWmhiIiIviNpY9s3SNp6qPO2r+x0nXpdEoqIiOg7kqbaPljSeUOctu3XdrxSPS4JRURERLQsszwiIqJvSXqnpOXK88Mk/V7SVt2uVy9KQhEREf3sv2w/JmlHYBfgZ8BPulynnpSEIiIi+tnc8udbgKm2TwGyfflCSEIRERH97G5JRwD7AKdKWoJ8Ni6UDMqMiIi+JWlpYFdgpu2bJK0GbG77zC5XreckoYiIiL4laX3gLttPS9oJ2AL4RVbLHL0060RERD87EZgraQNgKrAm8JvuVqk3JaGIiIh+NmD7WeAdwPdtfwZYrct16klJKCIiop/NkbQf8F7gT6VssS7Wp2cloYiIiH72fmB74Mu2b5O0LvDLLtepJ2VQZkRE9DVJSwFr2Z7d7br0srRQRERE35L0VmAGcHo5niTp5K5WqkcloYiIiH72BWAK8DCA7RnAet2rTu9KQhEREf1sju1HmsoGulKTHrdotysQERHRRbMkvQuYIGki8HHgoi7XqSelhSIiIvrZx4BNgaepFrR6BPhkNyvUqzLLIyIi+pKkCcDZtnfudl3Gg7RQREREX7I9FxiQtEK36zIeZAxFRET0s8eBmZLOAp4YLLT98e5VqTcloYiIiH72+/KIFmUMRURE9DVJiwMbAwZm236my1XqSUkoIiKib0l6M3AEcAsgYF3gQ7ZP62rFelASioiI6FuSbgB2s31zOV4fOMX2xt2tWe/JLI+IiOhnjw0mE8WtwGPdqkwvSwtFRET0LUk/BtYGjqcaQ/FO4E7gbADbGbA5QkkoIiKib0n6+XxO2/aBHatMj0tCERERES3LOhQREdG3JK1LtZ/HOjR8JtrevVt16lVJKCIiop/9AfgZ8H9k2/KWpMsjIiL6lqRLbb+i2/UYD5JQRERE35L0LmAicCbVFuYA2L6ya5XqUenyiIiIfrY58B7gtczr8nA5jlFIC0VERPQtSTcDm2T/jtZlpcyIiOhn1wIrdrsS40G6PCIiop+tCNwg6XKeP4Yi00ZHKQlFRET0s893uwLjRcZQREREX5O0KrBtObzM9n3drE+vyhiKiIjoW5L2Bi6j2hRsb+BSSXt1t1a9KS0UERHRtyRdDbx+sFVC0irA2ba37G7Nek9aKCIiop8t0tTF8QD5bFwoGZQZERH97HRJZwDHluN9gNO6WJ+elS6PiIjoa5LeAexYDi+wfVI369OrklBERETfkbQBsKrtC5vKdwTutX1Ld2rWu9JPFBER/eg7wKNDlD9SzsUoJaGIiIh+tKrtmc2FpWydzlen9yWhiIiIfrTifM4t1alKjCdJKCIioh9Nl/TB5kJJBwFXdKE+PS+DMiMiou+U5bZPAp5hXgIxGVgceLvtv3Wrbr0qCUVERPQtSTsDm5XDWbbP7WZ9elkSioiIiGhZxlBEREREy5JQRERERMuSUERERETLklBEREREy5JQRERERMv+P0Qv8FXp0fjKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(df.corr(),cmap='viridis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Cement', 'Slag', 'Fly ash', 'Water', 'SP', 'Coarse Aggr.',\n",
       "       'Fine Aggr.', 'SLUMP(cm)', 'FLOW(cm)',\n",
       "       'Compressive Strength (28-day)(Mpa)'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train | Test Split\n",
    "\n",
    "Alternatively you could also set this up as a pipline, something like:\n",
    "\n",
    "    >>> from sklearn.pipeline import make_pipeline\n",
    "    >>> from sklearn.preprocessing import StandardScaler\n",
    "    >>> from sklearn.svm import SVR\n",
    "\n",
    "    >>> clf = make_pipeline(StandardScaler(), SVR())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Cement', 'Slag', 'Fly ash', 'Water', 'SP', 'Coarse Aggr.',\n",
       "       'Fine Aggr.', 'SLUMP(cm)', 'FLOW(cm)',\n",
       "       'Compressive Strength (28-day)(Mpa)'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop('Compressive Strength (28-day)(Mpa)',axis=1)\n",
    "y = df['Compressive Strength (28-day)(Mpa)']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_X_train = scaler.fit_transform(X_train)\n",
    "scaled_X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support Vector Machines - Regression\n",
    "\n",
    "There are three different implementations of Support Vector Regression: SVR, NuSVR and LinearSVR. LinearSVR provides a faster implementation than SVR but only considers the linear kernel, while NuSVR implements a slightly different formulation than SVR and LinearSVR. See [Implementation details](https://scikit-learn.org/stable/modules/svm.html#svm-implementation-details) for further details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVR,LinearSVR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting C: C is 1 by default and it’s a reasonable default choice. If you have a lot of noisy observations you should decrease it: decreasing C corresponds to more regularization.\n",
    "\n",
    "LinearSVC and LinearSVR are less sensitive to C when it becomes large, and prediction results stop improving after a certain threshold. Meanwhile, larger C values will take more time to train, sometimes up to 10 times longer\n",
    "\n",
    "Epsilon: https://stats.stackexchange.com/questions/259018/meaning-of-epsilon-in-svm-regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model = SVR()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVR()"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "base_model.fit(scaled_X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_preds = base_model.predict(scaled_X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_absolute_error,mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.236902091259178"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_error(y_test,base_preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.695914838327133"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_test,base_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "36.26870967741935"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grid Search in Attempt for Better Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "param_grid = {'C':[0.001,0.01,0.1,0.5,1],\n",
    "             'kernel':['linear','rbf','poly'],\n",
    "              'gamma':['scale','auto'],\n",
    "              'degree':[2,3,4],\n",
    "              'epsilon':[0,0.01,0.1,0.5,1,2]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "svr = SVR()\n",
    "grid = GridSearchCV(svr,param_grid=param_grid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(estimator=SVR(),\n",
       "             param_grid={'C': [0.001, 0.01, 0.1, 0.5, 1], 'degree': [2, 3, 4],\n",
       "                         'epsilon': [0, 0.01, 0.1, 0.5, 1, 2],\n",
       "                         'gamma': ['scale', 'auto'],\n",
       "                         'kernel': ['linear', 'rbf', 'poly']})"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(scaled_X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1, 'degree': 2, 'epsilon': 2, 'gamma': 'scale', 'kernel': 'linear'}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid_preds = grid.predict(scaled_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5128012210762365"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_error(y_test,grid_preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.178210305119858"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_test,grid_preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great improvement!"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}