{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "___\n", "\n", "\n", "___\n", "
Copyright by Pierian Data Inc.
\n", "
For more information, visit us at www.pieriandata.com
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Grid Search \n", "\n", "We can search through a variety of combinations of hyperparameters with a grid search. While many linear models are quite simple and even come with their own specialized versions that do a search for you, this method of a grid search will can be applied to *any* model from sklearn, and we will need to use it later on for more complex models, such as Support Vector Machines." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\statsmodels\\tools\\_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n", " import pandas.util.testing as tm\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "df = pd.read_csv(\"../DATA/Advertising.csv\")" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TVradionewspapersales
0230.137.869.222.1
144.539.345.110.4
217.245.969.39.3
3151.541.358.518.5
4180.810.858.412.9
\n", "
" ], "text/plain": [ " TV radio newspaper sales\n", "0 230.1 37.8 69.2 22.1\n", "1 44.5 39.3 45.1 10.4\n", "2 17.2 45.9 69.3 9.3\n", "3 151.5 41.3 58.5 18.5\n", "4 180.8 10.8 58.4 12.9" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Formatting Data" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "## CREATE X and y\n", "X = df.drop('sales',axis=1)\n", "y = df['sales']\n", "\n", "# TRAIN TEST SPLIT\n", "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)\n", "\n", "# SCALE DATA\n", "from sklearn.preprocessing import StandardScaler\n", "scaler = StandardScaler()\n", "scaler.fit(X_train)\n", "X_train = scaler.transform(X_train)\n", "X_test = scaler.transform(X_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Model" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import ElasticNet" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on class ElasticNet in module sklearn.linear_model.coordinate_descent:\n", "\n", "class ElasticNet(sklearn.linear_model.base.LinearModel, sklearn.base.RegressorMixin, sklearn.base.MultiOutputMixin)\n", " | ElasticNet(alpha=1.0, l1_ratio=0.5, fit_intercept=True, normalize=False, precompute=False, max_iter=1000, copy_X=True, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic')\n", " | \n", " | Linear regression with combined L1 and L2 priors as regularizer.\n", " | \n", " | Minimizes the objective function::\n", " | \n", " | 1 / (2 * n_samples) * ||y - Xw||^2_2\n", " | + alpha * l1_ratio * ||w||_1\n", " | + 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2\n", " | \n", " | If you are interested in controlling the L1 and L2 penalty\n", " | separately, keep in mind that this is equivalent to::\n", " | \n", " | a * L1 + b * L2\n", " | \n", " | where::\n", " | \n", " | alpha = a + b and l1_ratio = a / (a + b)\n", " | \n", " | The parameter l1_ratio corresponds to alpha in the glmnet R package while\n", " | alpha corresponds to the lambda parameter in glmnet. Specifically, l1_ratio\n", " | = 1 is the lasso penalty. Currently, l1_ratio <= 0.01 is not reliable,\n", " | unless you supply your own sequence of alpha.\n", " | \n", " | Read more in the :ref:`User Guide `.\n", " | \n", " | Parameters\n", " | ----------\n", " | alpha : float, optional\n", " | Constant that multiplies the penalty terms. Defaults to 1.0.\n", " | See the notes for the exact mathematical meaning of this\n", " | parameter.``alpha = 0`` is equivalent to an ordinary least square,\n", " | solved by the :class:`LinearRegression` object. For numerical\n", " | reasons, using ``alpha = 0`` with the ``Lasso`` object is not advised.\n", " | Given this, you should use the :class:`LinearRegression` object.\n", " | \n", " | l1_ratio : float\n", " | The ElasticNet mixing parameter, with ``0 <= l1_ratio <= 1``. For\n", " | ``l1_ratio = 0`` the penalty is an L2 penalty. ``For l1_ratio = 1`` it\n", " | is an L1 penalty. For ``0 < l1_ratio < 1``, the penalty is a\n", " | combination of L1 and L2.\n", " | \n", " | fit_intercept : bool\n", " | Whether the intercept should be estimated or not. If ``False``, the\n", " | data is assumed to be already centered.\n", " | \n", " | normalize : boolean, optional, default False\n", " | This parameter is ignored when ``fit_intercept`` is set to False.\n", " | If True, the regressors X will be normalized before regression by\n", " | subtracting the mean and dividing by the l2-norm.\n", " | If you wish to standardize, please use\n", " | :class:`sklearn.preprocessing.StandardScaler` before calling ``fit``\n", " | on an estimator with ``normalize=False``.\n", " | \n", " | precompute : True | False | array-like\n", " | Whether to use a precomputed Gram matrix to speed up\n", " | calculations. The Gram matrix can also be passed as argument.\n", " | For sparse input this option is always ``True`` to preserve sparsity.\n", " | \n", " | max_iter : int, optional\n", " | The maximum number of iterations\n", " | \n", " | copy_X : boolean, optional, default True\n", " | If ``True``, X will be copied; else, it may be overwritten.\n", " | \n", " | tol : float, optional\n", " | The tolerance for the optimization: if the updates are\n", " | smaller than ``tol``, the optimization code checks the\n", " | dual gap for optimality and continues until it is smaller\n", " | than ``tol``.\n", " | \n", " | warm_start : bool, optional\n", " | When set to ``True``, reuse the solution of the previous call to fit as\n", " | initialization, otherwise, just erase the previous solution.\n", " | See :term:`the Glossary `.\n", " | \n", " | positive : bool, optional\n", " | When set to ``True``, forces the coefficients to be positive.\n", " | \n", " | random_state : int, RandomState instance or None, optional, default None\n", " | The seed of the pseudo random number generator that selects a random\n", " | feature to update. If int, random_state is the seed used by the random\n", " | number generator; If RandomState instance, random_state is the random\n", " | number generator; If None, the random number generator is the\n", " | RandomState instance used by `np.random`. Used when ``selection`` ==\n", " | 'random'.\n", " | \n", " | selection : str, default 'cyclic'\n", " | If set to 'random', a random coefficient is updated every iteration\n", " | rather than looping over features sequentially by default. This\n", " | (setting to 'random') often leads to significantly faster convergence\n", " | especially when tol is higher than 1e-4.\n", " | \n", " | Attributes\n", " | ----------\n", " | coef_ : array, shape (n_features,) | (n_targets, n_features)\n", " | parameter vector (w in the cost function formula)\n", " | \n", " | sparse_coef_ : scipy.sparse matrix, shape (n_features, 1) | (n_targets, n_features)\n", " | ``sparse_coef_`` is a readonly property derived from ``coef_``\n", " | \n", " | intercept_ : float | array, shape (n_targets,)\n", " | independent term in decision function.\n", " | \n", " | n_iter_ : array-like, shape (n_targets,)\n", " | number of iterations run by the coordinate descent solver to reach\n", " | the specified tolerance.\n", " | \n", " | Examples\n", " | --------\n", " | >>> from sklearn.linear_model import ElasticNet\n", " | >>> from sklearn.datasets import make_regression\n", " | \n", " | >>> X, y = make_regression(n_features=2, random_state=0)\n", " | >>> regr = ElasticNet(random_state=0)\n", " | >>> regr.fit(X, y) # doctest: +NORMALIZE_WHITESPACE\n", " | ElasticNet(alpha=1.0, copy_X=True, fit_intercept=True, l1_ratio=0.5,\n", " | max_iter=1000, normalize=False, positive=False, precompute=False,\n", " | random_state=0, selection='cyclic', tol=0.0001, warm_start=False)\n", " | >>> print(regr.coef_) # doctest: +ELLIPSIS\n", " | [18.83816048 64.55968825]\n", " | >>> print(regr.intercept_) # doctest: +ELLIPSIS\n", " | 1.451...\n", " | >>> print(regr.predict([[0, 0]])) # doctest: +ELLIPSIS\n", " | [1.451...]\n", " | \n", " | \n", " | Notes\n", " | -----\n", " | To avoid unnecessary memory duplication the X argument of the fit method\n", " | should be directly passed as a Fortran-contiguous numpy array.\n", " | \n", " | See also\n", " | --------\n", " | ElasticNetCV : Elastic net model with best model selection by\n", " | cross-validation.\n", " | SGDRegressor: implements elastic net regression with incremental training.\n", " | SGDClassifier: implements logistic regression with elastic net penalty\n", " | (``SGDClassifier(loss=\"log\", penalty=\"elasticnet\")``).\n", " | \n", " | Method resolution order:\n", " | ElasticNet\n", " | sklearn.linear_model.base.LinearModel\n", " | sklearn.base.BaseEstimator\n", " | sklearn.base.RegressorMixin\n", " | sklearn.base.MultiOutputMixin\n", " | builtins.object\n", " | \n", " | Methods defined here:\n", " | \n", " | __init__(self, alpha=1.0, l1_ratio=0.5, fit_intercept=True, normalize=False, precompute=False, max_iter=1000, copy_X=True, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic')\n", " | Initialize self. See help(type(self)) for accurate signature.\n", " | \n", " | fit(self, X, y, check_input=True)\n", " | Fit model with coordinate descent.\n", " | \n", " | Parameters\n", " | ----------\n", " | X : ndarray or scipy.sparse matrix, (n_samples, n_features)\n", " | Data\n", " | \n", " | y : ndarray, shape (n_samples,) or (n_samples, n_targets)\n", " | Target. Will be cast to X's dtype if necessary\n", " | \n", " | check_input : boolean, (default=True)\n", " | Allow to bypass several input checking.\n", " | Don't use this parameter unless you know what you do.\n", " | \n", " | Notes\n", " | -----\n", " | \n", " | Coordinate descent is an algorithm that considers each column of\n", " | data at a time hence it will automatically convert the X input\n", " | as a Fortran-contiguous numpy array if necessary.\n", " | \n", " | To avoid memory re-allocation it is advised to allocate the\n", " | initial data in memory directly using that format.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Static methods defined here:\n", " | \n", " | path = enet_path(X, y, l1_ratio=0.5, eps=0.001, n_alphas=100, alphas=None, precompute='auto', Xy=None, copy_X=True, coef_init=None, verbose=False, return_n_iter=False, positive=False, check_input=True, **params)\n", " | Compute elastic net path with coordinate descent\n", " | \n", " | The elastic net optimization function varies for mono and multi-outputs.\n", " | \n", " | For mono-output tasks it is::\n", " | \n", " | 1 / (2 * n_samples) * ||y - Xw||^2_2\n", " | + alpha * l1_ratio * ||w||_1\n", " | + 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2\n", " | \n", " | For multi-output tasks it is::\n", " | \n", " | (1 / (2 * n_samples)) * ||Y - XW||^Fro_2\n", " | + alpha * l1_ratio * ||W||_21\n", " | + 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2\n", " | \n", " | Where::\n", " | \n", " | ||W||_21 = \\sum_i \\sqrt{\\sum_j w_{ij}^2}\n", " | \n", " | i.e. the sum of norm of each row.\n", " | \n", " | Read more in the :ref:`User Guide `.\n", " | \n", " | Parameters\n", " | ----------\n", " | X : {array-like}, shape (n_samples, n_features)\n", " | Training data. Pass directly as Fortran-contiguous data to avoid\n", " | unnecessary memory duplication. If ``y`` is mono-output then ``X``\n", " | can be sparse.\n", " | \n", " | y : ndarray, shape (n_samples,) or (n_samples, n_outputs)\n", " | Target values\n", " | \n", " | l1_ratio : float, optional\n", " | float between 0 and 1 passed to elastic net (scaling between\n", " | l1 and l2 penalties). ``l1_ratio=1`` corresponds to the Lasso\n", " | \n", " | eps : float\n", " | Length of the path. ``eps=1e-3`` means that\n", " | ``alpha_min / alpha_max = 1e-3``\n", " | \n", " | n_alphas : int, optional\n", " | Number of alphas along the regularization path\n", " | \n", " | alphas : ndarray, optional\n", " | List of alphas where to compute the models.\n", " | If None alphas are set automatically\n", " | \n", " | precompute : True | False | 'auto' | array-like\n", " | Whether to use a precomputed Gram matrix to speed up\n", " | calculations. If set to ``'auto'`` let us decide. The Gram\n", " | matrix can also be passed as argument.\n", " | \n", " | Xy : array-like, optional\n", " | Xy = np.dot(X.T, y) that can be precomputed. It is useful\n", " | only when the Gram matrix is precomputed.\n", " | \n", " | copy_X : boolean, optional, default True\n", " | If ``True``, X will be copied; else, it may be overwritten.\n", " | \n", " | coef_init : array, shape (n_features, ) | None\n", " | The initial values of the coefficients.\n", " | \n", " | verbose : bool or integer\n", " | Amount of verbosity.\n", " | \n", " | return_n_iter : bool\n", " | whether to return the number of iterations or not.\n", " | \n", " | positive : bool, default False\n", " | If set to True, forces coefficients to be positive.\n", " | (Only allowed when ``y.ndim == 1``).\n", " | \n", " | check_input : bool, default True\n", " | Skip input validation checks, including the Gram matrix when provided\n", " | assuming there are handled by the caller when check_input=False.\n", " | \n", " | **params : kwargs\n", " | keyword arguments passed to the coordinate descent solver.\n", " | \n", " | Returns\n", " | -------\n", " | alphas : array, shape (n_alphas,)\n", " | The alphas along the path where models are computed.\n", " | \n", " | coefs : array, shape (n_features, n_alphas) or (n_outputs, n_features, n_alphas)\n", " | Coefficients along the path.\n", " | \n", " | dual_gaps : array, shape (n_alphas,)\n", " | The dual gaps at the end of the optimization for each alpha.\n", " | \n", " | n_iters : array-like, shape (n_alphas,)\n", " | The number of iterations taken by the coordinate descent optimizer to\n", " | reach the specified tolerance for each alpha.\n", " | (Is returned when ``return_n_iter`` is set to True).\n", " | \n", " | Notes\n", " | -----\n", " | For an example, see\n", " | :ref:`examples/linear_model/plot_lasso_coordinate_descent_path.py\n", " | `.\n", " | \n", " | See also\n", " | --------\n", " | MultiTaskElasticNet\n", " | MultiTaskElasticNetCV\n", " | ElasticNet\n", " | ElasticNetCV\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors defined here:\n", " | \n", " | sparse_coef_\n", " | sparse representation of the fitted ``coef_``\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data and other attributes defined here:\n", " | \n", " | __abstractmethods__ = frozenset()\n", " | \n", " | ----------------------------------------------------------------------\n", " | Methods inherited from sklearn.linear_model.base.LinearModel:\n", " | \n", " | predict(self, X)\n", " | Predict using the linear model\n", " | \n", " | Parameters\n", " | ----------\n", " | X : array_like or sparse matrix, shape (n_samples, n_features)\n", " | Samples.\n", " | \n", " | Returns\n", " | -------\n", " | C : array, shape (n_samples,)\n", " | Returns predicted values.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Methods inherited from sklearn.base.BaseEstimator:\n", " | \n", " | __getstate__(self)\n", " | \n", " | __repr__(self, N_CHAR_MAX=700)\n", " | Return repr(self).\n", " | \n", " | __setstate__(self, state)\n", " | \n", " | get_params(self, deep=True)\n", " | Get parameters for this estimator.\n", " | \n", " | Parameters\n", " | ----------\n", " | deep : boolean, optional\n", " | If True, will return the parameters for this estimator and\n", " | contained subobjects that are estimators.\n", " | \n", " | Returns\n", " | -------\n", " | params : mapping of string to any\n", " | Parameter names mapped to their values.\n", " | \n", " | set_params(self, **params)\n", " | Set the parameters of this estimator.\n", " | \n", " | The method works on simple estimators as well as on nested objects\n", " | (such as pipelines). The latter have parameters of the form\n", " | ``__`` so that it's possible to update each\n", " | component of a nested object.\n", " | \n", " | Returns\n", " | -------\n", " | self\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors inherited from sklearn.base.BaseEstimator:\n", " | \n", " | __dict__\n", " | dictionary for instance variables (if defined)\n", " | \n", " | __weakref__\n", " | list of weak references to the object (if defined)\n", " | \n", " | ----------------------------------------------------------------------\n", " | Methods inherited from sklearn.base.RegressorMixin:\n", " | \n", " | score(self, X, y, sample_weight=None)\n", " | Returns the coefficient of determination R^2 of the prediction.\n", " | \n", " | The coefficient R^2 is defined as (1 - u/v), where u is the residual\n", " | sum of squares ((y_true - y_pred) ** 2).sum() and v is the total\n", " | sum of squares ((y_true - y_true.mean()) ** 2).sum().\n", " | The best possible score is 1.0 and it can be negative (because the\n", " | model can be arbitrarily worse). A constant model that always\n", " | predicts the expected value of y, disregarding the input features,\n", " | would get a R^2 score of 0.0.\n", " | \n", " | Parameters\n", " | ----------\n", " | X : array-like, shape = (n_samples, n_features)\n", " | Test samples. For some estimators this may be a\n", " | precomputed kernel matrix instead, shape = (n_samples,\n", " | n_samples_fitted], where n_samples_fitted is the number of\n", " | samples used in the fitting for the estimator.\n", " | \n", " | y : array-like, shape = (n_samples) or (n_samples, n_outputs)\n", " | True values for X.\n", " | \n", " | sample_weight : array-like, shape = [n_samples], optional\n", " | Sample weights.\n", " | \n", " | Returns\n", " | -------\n", " | score : float\n", " | R^2 of self.predict(X) wrt. y.\n", " | \n", " | Notes\n", " | -----\n", " | The R2 score used when calling ``score`` on a regressor will use\n", " | ``multioutput='uniform_average'`` from version 0.23 to keep consistent\n", " | with `metrics.r2_score`. This will influence the ``score`` method of\n", " | all the multioutput regressors (except for\n", " | `multioutput.MultiOutputRegressor`). To specify the default value\n", " | manually and avoid the warning, please either call `metrics.r2_score`\n", " | directly or make a custom scorer with `metrics.make_scorer` (the\n", " | built-in scorer ``'r2'`` uses ``multioutput='uniform_average'``).\n", "\n" ] } ], "source": [ "help(ElasticNet)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "base_elastic_model = ElasticNet()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Grid Search\n", "\n", "A search consists of:\n", "\n", "* an estimator (regressor or classifier such as sklearn.svm.SVC());\n", "* a parameter space;\n", "* a method for searching or sampling candidates;\n", "* a cross-validation scheme \n", "* a score function." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "param_grid = {'alpha':[0.1,1,5,10,50,100],\n", " 'l1_ratio':[.1, .5, .7, .9, .95, .99, 1]}" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import GridSearchCV" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "# verbose number a personal preference\n", "grid_model = GridSearchCV(estimator=base_elastic_model,\n", " param_grid=param_grid,\n", " scoring='neg_mean_squared_error',\n", " cv=5,\n", " verbose=2)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fitting 5 folds for each of 42 candidates, totalling 210 fits\n", "[CV] alpha=0.1, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=0.1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=0.1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=0.1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=0.1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=0.1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=0.1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=0.1, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=0.1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=1, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=1, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=1, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=1, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=1, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=1, l1_ratio=1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.1 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.5 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.7 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.9 ...........................................\n", "[CV] ............................ alpha=5, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.95 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=5, l1_ratio=0.99 ..........................................\n", "[CV] ........................... alpha=5, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=5, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=5, l1_ratio=1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=5, l1_ratio=1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=5, l1_ratio=1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=5, l1_ratio=1, total= 0.0s\n", "[CV] alpha=5, l1_ratio=1 .............................................\n", "[CV] .............................. alpha=5, l1_ratio=1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=10, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=10, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=10, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=10, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=10, l1_ratio=1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=10, l1_ratio=1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=10, l1_ratio=1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=10, l1_ratio=1, total= 0.0s\n", "[CV] alpha=10, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=10, l1_ratio=1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.1 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.5 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.7 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.9 ..........................................\n", "[CV] ........................... alpha=50, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.95 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=50, l1_ratio=0.99 .........................................\n", "[CV] .......................... alpha=50, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=50, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=50, l1_ratio=1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=50, l1_ratio=1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=50, l1_ratio=1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=50, l1_ratio=1, total= 0.0s\n", "[CV] alpha=50, l1_ratio=1 ............................................\n", "[CV] ............................. alpha=50, l1_ratio=1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.1, total= 0.0s" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n", "[Parallel(n_jobs=1)]: Done 1 out of 1 | elapsed: 0.0s remaining: 0.0s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "[CV] alpha=100, l1_ratio=0.1 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.5 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.5, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.7 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.7, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.9 .........................................\n", "[CV] .......................... alpha=100, l1_ratio=0.9, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.95 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.95, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=100, l1_ratio=0.99 ........................................\n", "[CV] ......................... alpha=100, l1_ratio=0.99, total= 0.0s\n", "[CV] alpha=100, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=100, l1_ratio=1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=100, l1_ratio=1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=100, l1_ratio=1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=100, l1_ratio=1, total= 0.0s\n", "[CV] alpha=100, l1_ratio=1 ...........................................\n", "[CV] ............................ alpha=100, l1_ratio=1, total= 0.0s\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[Parallel(n_jobs=1)]: Done 210 out of 210 | elapsed: 0.1s finished\n" ] }, { "data": { "text/plain": [ "GridSearchCV(cv=5, error_score='raise-deprecating',\n", " estimator=ElasticNet(alpha=1.0, copy_X=True, fit_intercept=True,\n", " l1_ratio=0.5, max_iter=1000, normalize=False,\n", " positive=False, precompute=False,\n", " random_state=None, selection='cyclic',\n", " tol=0.0001, warm_start=False),\n", " iid='warn', n_jobs=None,\n", " param_grid={'alpha': [0.1, 1, 5, 10, 50, 100],\n", " 'l1_ratio': [0.1, 0.5, 0.7, 0.9, 0.95, 0.99, 1]},\n", " pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n", " scoring='neg_mean_squared_error', verbose=2)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "grid_model.fit(X_train,y_train)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "ElasticNet(alpha=0.1, copy_X=True, fit_intercept=True, l1_ratio=1,\n", " max_iter=1000, normalize=False, positive=False, precompute=False,\n", " random_state=None, selection='cyclic', tol=0.0001, warm_start=False)" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "grid_model.best_estimator_" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'alpha': 0.1, 'l1_ratio': 1}" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "grid_model.best_params_" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "# pd.DataFrame(grid_model.cv_results_)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using Best Model From Grid Search" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "y_pred = grid_model.predict(X_test)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import mean_squared_error" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2.3873426420874737" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_squared_error(y_test,y_pred)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }