{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Linear models\n",
    "sklearn.liner_model - [documentation on sklearn.linear_model](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model)\n",
    "\n",
    "**linear_model examples:**\n",
    "* RidgeClassifier\n",
    "* SGDClassifier\n",
    "* SGDRegressor\n",
    "* LinearRegression\n",
    "* LogisticRegression\n",
    "* Lasso\n",
    "\n",
    "[All the examples of sklearn.linear_model](http://scikit-learn.org/stable/modules/linear_model.html#linear-model)\n",
    "\n",
    "Module *metrics* is needed to evaluate the quality of aquired models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from matplotlib.colors import ListedColormap\n",
    "from sklearn import model_selection, datasets, linear_model, metrics\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Data generation\n",
    "We choose a dataset of 2 features, labels forming 2 clouds.\n",
    " - **centers** - number of classes\n",
    " - **cluster_std** - standart deviation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "blobs = datasets.make_blobs(centers = 2, cluster_std = 5.5, random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualize datasets in colors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1d01665d2b0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAI/CAYAAACrl6c+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABPZUlEQVR4nO3dd5iU1fnG8ftsL3REBFQQFBS7roi9oaBgwdi7iSKxJUaNNdFoNEZNIsZKAlFU7GIBu0ERg8KCgGAFRESQJlK2l/P742F/22ZhYd+Zd8r3c11z7e47y87DsOzec8pznPdeAAAACE5a2AUAAAAkGwIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAywi6grq222sr36NEj7DIAAAA2afr06Su9950i3RdXAatHjx4qLCwMuwwAAIBNcs5919R9TBECAAAEjIAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAIWAAAAAEjYAEAAASMgAUAABAwAhYAAEDACFgAAAABI2ABAAAEjIAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAywi4AABAC76WPPpKefVbKyJDOPlsqKAi7KiBpELAAIBVdeaX0n/9IxcWSc9LIkdLvfy/dckvYlQFJgSlCAEg1hYXS6NFSUZGNZFVXW9C66y5p/vywqwOSAgELAFLNK69IpaWR75swIba1AEmKgAUAqSYnR0pPb3w9Lc3uA9BiBCwASDVnnGEL2xvyXho6NPb1AEmIgAUAqaZXL+n++220qlUru+XmSk88IXXqFHZ1QFJgFyEApKKLLpJOPFF64w2bLhw8WGrXLuyqgKRBwAKAVNWpk3TeeWFXASQlpggBAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAhZIwHLOjXbOLXfOzalz7Vbn3A/OuZkbbscF8VgAAADxLqgRrMckDYpw/R/e+7023F4P6LEAAADiWiABy3s/SdJPQXwtAACARBftNViXO+dmb5hCbB/lxwIAAIgL0QxYD0vqJWkvSUsl/S3SJznnhjnnCp1zhStWrIhiOQAAALERtYDlvV/mva/y3ldL+pekfk183kjvfYH3vqATh4wCAIAkELWA5ZzrUufDoZLmNPW5AAAAySSQw56dc09LOlzSVs65xZJukXS4c24vSV7SQkmXBPFYAAAA8S6QgOW9PzPC5VFBfG0AAIBEQyd3AACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACkBg++0waP1764YewKwGATQrkLEIAiJpVq6TjjpPmzJEyM6WyMumCC6QHH5TSeI0IID7x0wlAfDvvPGnmTKm4WFqzRiotlcaMkR59NOzKAKBJBCwA8evnn6V335XKy+tfLy6W7r8/lJIAoDkIWADiV1FR09OAa9bEthYA2AwELADxq2tXqVOnxtczMqTBg2NfDwA0EwELQPxyTho9WsrLs1AlSTk5UocO0p/+FG5tALAR7CIEEN8GDJCmT7c1V998Ix1+uDR8uNSxY9iVAUCTCFgA4t/OO0sPPRR2FQDQbEwRAgAABIyABQAAEDACFgAAQMAIWAAAAAEjYAEAAASMgAUAQJDKy621yIIFYVeCEBGwAAAIylNP2ekDRxwh7babtN9+0pIlYVeFEBCwAAAIQmGhNGyYtHattG6dVFIiffqpNHCg5H3Y1SHGCFgAAATh/vul0tL616qqpG+/lWbNCqcmhIaABQBAEBYvlqqrG19PT5eWLYt9PQgVAQsAgCAce6yUm9v4enm5VFAQ+3oQKgIWAABBGDZM6txZys6uvZafL11/PYeTpyAOewYAIAht20ozZkj33SeNG2eh6qqrpBNOCLsyhMD5ONrZUFBQ4AsLC8MuAwAAYJOcc9O99xHnf5kiBAAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACFhG2AUAwCbNmiV99JHUubM0ZIiUnR12RQCwUQQsAPGrqko66yxp/HipulrKzJRycqT335f69g27OgBoElOEAOLXY49ZuCoulkpLpXXrpJUrpZNOkryP3uN6Ly1ZIq1fH73HAJDUCFgA4tfIkRau6vJe+uEH6ZtvovOY48dL224r9eolbbWVdMYZBC0Am40pQgDxq6ws8vW0NKm8PPjHmz5dOv30+qHu5ZeltWul118P/vEAJC1GsADEr3POkXJzG19v1So6a7DuvlsqKal/raxMmjhRWrQo+McDkLQIWADi12WXSbvtZoFKsgXu+fnSM8/YKFbQ5s2LvLYrO5uABWCzMEUIIH7l5kr/+5/02ms2itStm3TeeVKXLtF5vEMOkT77TKqoqH+9rEzaddfoPCaApETAAhDfMjKkoUPtFm3XXCM9/rituaqutmv5+TaS1r599B8fQNJgihDAps2aJV11lXTxxdKbb9aGj2Sz7ba1C9233lraZRdpxAjprrvCrgxAgmEEC8DGPfCAdN11Nk1WVWXrn445Rnr++eisgwpbz57S2LFhVxGs+fOladNsivXggyXnwq4ISHoELABNW77cps3qtktYv156+23pjTekwYPDqw2bVl0tXXih9Nxz1gXfe6lrV+m//7WwBSBqkvDlJ4DAvPuulJXV+Pr69dKLL8a+HmyekSOlF16o7YK/fr2NZp1+etiVAUmPgAWgaZF6UEk2NZifH9tasPkefLBxJ/yqKqmwUFq2LJyagBRBwALQtIEDI1/PyZEuuCCmpWALFBVFvp6e3jh4AQgUAQtA0/LypFdflVq3tlurVhaubrtN2nffsKvDppx8cuQp3g4dpB49Yl4OkEpY5A5g4w4/XPrxR1vUXlRkOwi32SbsqtAcN94ovfSSTQcWF1vYysyUxoxhJyEQZQQsAJuWlyf94hdhV4HN1aGDdaYfM8Y64e+wgzR8uL0FEFXORzp3KyQFBQW+sLAw7DIAAAA2yTk33XtfEOm+QNZgOedGO+eWO+fm1LnWwTn3jnPumw1vOWcCAACkhKAWuT8maVCDa9dLes97v5Ok9zZ8DAAAkPQCCVje+0mSfmpw+URJj294/3FJJwXxWAAAAPEumm0aOnvvl0rShrdbR/GxAAAA4kbofbCcc8Occ4XOucIVK1aEXQ4AAECLRTNgLXPOdZGkDW+XR/ok7/1I732B976gU6dOUSwHAAAgNqIZsF6VdP6G98+X9EoUHwsAACBuBNWm4WlJUyT1cc4tds79StJdko52zn0j6egNHwMAEI7Fi6X586U46v+I5BVIJ3fv/ZlN3HVUEF8fAIAt9u230imnSJ9/LqWlSR07SmPHSgcfHHZlSGIclQMASF6VldKhh0pLlkjV1XatuFgaNEj6+mupa9dw60PSCn0XIQAAUfPOO9KaNbXhqkZlpTR6dDg1ISUwggUAiMx7ado0acYMqUcP6eijpfT0sKvaPD/8IFVVNb5eViYtXBjzcpA6CFgAkEo++EB65BEb1TntNOmss6SsrMafV1oqDR4sffyxBa2MDGmrraTJkxNrWq1//8iL2lu1ko44Ivb1IGUwRQgAqeLOO6XjjpOeeUZ64w3p8sstZFRUNP7cv/xFmjLF1iuVlEjr1kmLFknnnRf7ultit92k44+X8vJqr+XkSNttZwvfgSghYAFAKli2TLr9dgtMNYqKpFmzpBdeaPz5o0ZZsKqrqkqaNMnCViIZO1a65x5p992lnXaSrrnGRuays8OuDEmMKUIASAWTJkmZmTb1V1dRkTRunHRmg247kUa1alRWBl9fNKWnS5deajcgRhjBAoCW+v576Te/kfbd14LKjBlhV9RY27aRr6enW1+ohk4+2QJZQ337Su3bB1sbkIScj6OOtgUFBb6wsDDsMgCg+ebNkwoKbOqtosIaWebk2LTbsceGXV2tykqpWzdpeYNjYfPybOH63nvXv75qlbTfftKKFdL69VJuri2GnzRJ2mOP2NUNxDHn3HTvfUGk+5giBICWuOEGW5NU02eputrC1iWXSN99JzkXbn01MjKsJ9SgQRaYnLNAOGJE43Al2ajW3LnSc8/ZeqXevW2Be6TRLgCNMIIFAC2x1VY22tNQdrZNHXbqFPuaNqaqSvrf/yxkHXyw1Lp12BUBCYsRLACIlg4dIgcsScrPj20tzZGeLh1ySNhVAEmPRe4A0BJXX12/x5Jko1enntr4erybNUt68klp6tTIzTkBNBsjWADQEsOG2UL3Bx6wYFVWJh11lHVLTxQlJdaMc8oUW6TvvbTrrtLbbze9+xDARjGCBQAt4Zw1sfzhB2nCBOnrr6Xx4+NzerApN98sffSRLc5fv956Y82caZ3ekVqqqqTHHpMOP9y6/D/xROSzHLFJLHIHgFTXvr3088+Nr2dl2ehWGq/FU4L30gknSBMnWsiW7IXCwIHWdiRedsTGkY0tcud/DQCkurKyyNcrK2vbTyD5TZ5cP1xJ9v5bb9m6PGwWAhaA5LR4sbR0adhVJIaBAxuPUjkn9e9v/bOQGiZOrH9WZY2yMrsPm4WABSC5zJplx7nstJO0ww7SXntJX30VdlXx7R//sHYTNbsec3KkNm2kkSPDrQux1bGj/ds3lJ1t/d6wWViDBSB5rFkjde9ub2s4Z78cFi2K/MsDZvVqafRomwraYw/p4oulrbcOuyrE0qpVUo8ettGhrtatrWkuO0obodEogNTwzDN2/Etd3kulpdLLL0tnnBHxj3kvffih9NRT9v4551gvzpRa09u+vfX0Qurq2NF2wJ5ySu26vNxc6aWXCFdbgIAFIHksWhR5DUlJib0Cb8Lvfif961+1f3TsWOmii6T77otOmUDcOuww6ccfpcJCe4Wx777W/R+bjTVYAJJH//5Sq1aNr+fkSPvvH/GPzJ4tPfqobZby3m5FRbb8aPbsKNcLxKP0dPv/0q8f4aoFCFgAksdxx0l9+tRfa5Wba6/Cmzh/7/XXG88qSnZtwoQo1Qkg6RGwACSP9HTpgw+k3/9e6tlT2nFH61L+1ltNLqjKy4vciSAjI/GOEoyouNg6c//ud7aIvW6PIwBRwy5CAClt6VKpVy9bplVXbq4dMdi1azh1BWLxYpvmWbfOdobl59uOsKlTpe22C7s6IOHRyR0AmtClizRmjI1WtW5tt9xc6fHHoxCuahZ4xeqF7ZVXSsuX1267LyqSVqyQLrssNo8PpDACFoCUd8opNpL173/bbelS6dRTA3wA76URI6wfV7t21l/qkUeC+/ozZ0pnny3tt5901VV28LRki8gaHtRbVSW98UbsQh6QomjTAACyxuWnnRalL/7ww9KNN9b2gVi50npO5eRIF1zQsq/95pvSL35hvb6qq62T/WOP2Tb7pnaApaenWJMvIPYYwQKAaLvttsb9uYqLpVtuadnX9V665BL7WjWHMldUSGvXWqA75RQpK6v+n8nMtEAGIKoYwQKAaKqulpYti3zfkiUt+9orVkT+2tXV0n//a2cwzpghffedBa+sLGnbbaX772/Z4wLYJAIWAARtxQpp3Djbmjh4sJ2P+N13jT9vxx1b9jiRmqrWaN/eDnCePVt6913piy+knXeWjj5aSmPyAog2AhYABGncOFtw7pyNJF1/vYWsFSvqTxPm5kr33NOyx8rLs+m+F1+sPTuu5vrvfmfvp6VJxxxjNwAxw8sYAAjKmjV2UnRJiYWp0lK7vfGGdOut0q67WvjZc08LRUOGtPwxH31UOuIIWzDftq29veQSu2HTKiuluXOtZxgQIEawACAor78eeedeaakdNj1nTvCP2aqVBbjvvrPDrvv2lTp2DP5xktFLL0kXXyyVl1vQ2mcfC77bbBN2ZUgCBCwACEpVVeT+Ut437kcVtO7d7YbmmT1bOvfc+tO2U6dKAwdaXzHaWKCFmCIEgKAMGmQjIQ3l5Umnnx77etC0+++vv25Nsn+7+fMtYAEtRMACgKBstZX00EO2gD0z0xaY5+VJ550nHXJI2NWhru++izyqmJ5urfyBFmKKEACCdOGF0uGHS888o0VLM/XAz+dozsJtdMCfpeHDpU6dwi4QkmxX5UcfNT7lu7xc2nffcGpCUnE+js6jKigo8IWFhWGXAQAtNm2adOSR9vu6vNw29+Xn2/Uddgi7OmjNGmn33a1Ra3m5XcvPly69VLr77nBrQ8Jwzk333hdEuo8pQgCIgosvltavr/3dXVoqrV4tXXttuHVhg7ZtpU8/lX77W6lPH6l/f2nUKOmvfw27MiQJRrAAIGDFxXZ4dKQlPq1b21GBofvpJwsUU6ZIu+1mfbO6dQu7KiChbGwEizVYABCwzExbKx0pYOXnx76eRhYtkgoKbIitpMT6d913n/T++9YLCkCLMUUIAAHLzJROPVXKzq5/PTfXFrqH7tprbQSrZoF3WZm0bp3NawIIBAELQOLyXrr3XqlzZykjw46gmTgx7KokWbeGfv2sS0ObNrbI/bjjpBtvDLsySW++GXl4bdas+o03AWwxpggBJK6bb7aprZpQMHu2Haw8caK0//6hltamjTRpkvTZZ9K8edIee0i9eoVaUq3c3MgLwdLTLagCaDFGsAAkppKS+uGq7vU//jGUkiLZfXdp6NA4CleSNGyYhay6srKkk06ytwBajIAFIDEtWdL0eXFz58a2lkRz883SgAEWslq3tpX3e+4pPfpo2JUBSYOxYACJqUsXqbo68n19+8a2lkSTlSW9+qr0xRc2h9mrl+0e5IBjIDCMYAFITHl50pVX2tuG1//0p3BqSjS77CKddpodDUO4AgJFwAKQuO6809ZbbbWVBYRdd5VeeUU64ICwKwOQ4ujkDiA5eM8oDICY4ixCAMmPcAUgjhCwAAAAAkbAAgAACBgBC8DGLVokjR9PbykA2Az0wQIQWVWV9KtfSc8+a32TKiulvfeWJkyQ2rYNuzoAiGuMYAGI7L77pOefl0pL7dy64mJp2jTpoovCrgwA4h4BC0BkDzzQ+Jy/8nLrAN7wOgCgHgIWgMjWrWv6vtLS2NUBJIvSUmniRGnyZJuCR1IjYAGIbOBAKT298fXu3aX27WNfD5DIXn1V2npr6aSTpOOOs7M0P/447KoQRQQsAJH95S8WpHJy7OOMDDvnb9QomnoCm2PRIunMM21UeO1ae7tihTRokFRUFHZ1iBICFoDItt9e+vxz6YYbpKOPloYPl2bNkg45JOzKgMQyZoztwm2outpGtpCUaNMAoGmdOtlhygC23KpVtkGkocpKafXq2NeDmGAECwCAaBo4UGrVKvJ9Rx0V21oQMwQsAEhVM2dKxx4rbbWVtMce0gsvhF1RcjrmGOmgg6T8/Npr+fnSBRdIffqEVhaiK+pThM65hZLWSaqSVOm9L4j2YwIANmH2bOngg2sXWa9aJZ1/vvTjj9Lll4dbW7JJS7Pjpp5+WnrySTsZ4eKLpeOPD7syRJHz3kf3ASxgFXjvV27qcwsKCnxhYWFU6wEASDrhBPul3/B3QNu2tsMtMzOcuoAE4pyb3tTAEYvcASAVTZvWOFxJUkWFtGSJ9TvbEitX2hFLa9bY1Ng++7SsTiBBxWINlpf0tnNuunNuWAweDwCwKT16RL7uva3J2hLvvGPB7OqrpT/8wVp6/PKXkYMckORiEbAO8t7vI+lYSZc55w6te6dzbphzrtA5V7hixYoYlAMA0B//aI1j68rLky68sP5i7OYqK5NOOcXOqSwpsRYExcXSc89Jr70WTM1AAol6wPLeL9nwdrmkcZL6Nbh/pPe+wHtf0KlTp2iXAwDBePNNacgQ6cADpb//PfE6ch97rPTwwzZalZMj5ebaaNN9923Z15s0KfL1oiLpsce2tEogYUV1DZZzLl9Smvd+3Yb3j5F0WzQfEwCi7k9/ku65pzZUzZwp/ec/0tSpFlQSxXnnSWefbeum2ratPRZpS2xsGpApQqSgaI9gdZY02Tk3S9JUSRO8929G+TEBIHqWL5fuuqv+iFVJibRggfTEE+HVtaXS06XOnVsWriTp0EMjX8/Pt/YPQIqJasDy3i/w3u+54bar9/6OaD4eAETdlCnWx6ih4uLUXmuUkyM9+6yt48rNtd5P+fnSSSdZSwggxdCmAUDiW7NG+uknabvtpIwo/1jr2DHylFd6utS1a3QfO94NGmQjec89V9umYb/9JOfCrgyIOQIWgMRVVGQLs195xQJObq4t0j7nnOg95oEHSh06SOvX1w9aWVnS8OHRe9xE0bmzdMUVYVcBhI6zCAEkrnPOsXBVVmZTdKtWSZdcIn3wQfQeMy1NevddaccdbQqsTRs7yPeRR6S9947e4wJIKIxgAUhMy5ZZq4SysvrXi4ttEfphh0XvsXfcUfrqKzvPb+1aqaAgsXYPAog6AhaAxLRkiU3LlZY2vm/hwug/vnPSnntG/3EAJCSmCAEkpj59rFt4QxkZ0R29wpbznp5YSBkELACJKS+v8XEvaWm2HuqGG8KrC42tXWubEfLypMxMacAA6Ztvwq4KiCoCFoDEdd110uOPS/vuK3XrJp15pjR9uh04jPjgvbVrGDvWpnOrqqT//lfq399aawBJijVYABLbKafYDfFp6lRpzpz6mxG8t+73o0ZJ114bXm1AFDGCBQCIni+/jHy9pMTOcASSFAELABA9fftGvp6XZ1O7QJIiYAEAoqegQNprLyk7u/ZaWpoFrAsvDK0sINoIWACA6HHOGsL+8pe2wzMz084s/OQTqX37sKsDosb5OOpJUlBQ4AsLC8MuAwAAYJOcc9O99wWR7mMECwAAIGAELAAAgIARsAAg2iZMkI44wnbUXX21HVQNIKnRaBQAounee6VbbpGKi+3j+fOlp56SPvtM6tQp3NoARA0jWABabNIk6ZxzpBNPtOxQURF2RXFi/Xo7L7EmXElSebn088/SffeFVRWAGCBgAWiRO+6Qjj3Wjpp79VXpkkukgQOlysqwK4sDs2dbW4KGysqsdQGApEXAArDFli6V/vxnG6Cp6fhSVGTHz73ySri1xYXOnZsezttuu9jW0lLz50vnnSf17Ckddpj01lthVwTENQIWgC32/vuRB2iKiqSXXop5OfGnVy9pn30aP0l5ebbYPVHMm2d/j6eekr791uaETz5Z+ve/w64MiFspF7Cqq8OuAEgebdpYo+6G0tOljh1jX09cevll6cADpZwcqXVru/3zn9Ihh4RdWfPdcoul5ro/QIuLpWuvZcEd0ISUCVgTJ0q77y5lZEjt2tnPi6qqsKsCEtvRR9v/qYaysqSLLop9PXFpq61sqO+bb2zkZ8UKOzYmkUyaFPkHZkWFtGhR7OsBEkBKBKzp06UhQ6Q5c2ydyJo1tnP6N78JuzIgsWVlSW+/bd0G2rSxW26udP/90h57hF1dnNl228aHHieKbt0iX6+stAC5OV57Terf39agnX669PXXLa8PiEMpcRbhSSfZ7qaGf9WcHOnHH6W2bQN/SCClVFZKH35os0iHHmpBC0nk1VelM8+s324iJ8fWYT31VPO/ziOP2Nqzmq+Tlibl50vTpkl9+gRbMxADKX8W4dy5jcOVZK++Gd0GWi4jwxqVDxlCuEpKJ5wg/fWvtn6sVSsbhTvxROlf/2r+1ygvl66/vn5Iq662VP6nPwVfMxCylAhYe+4ZeSFuebnUvXvs6wGAhHP55bZ+bNo0ackS6ZlnbDdkcy1aFLk5WnW1NHlycHUCcSIlAtYf/mDrQurKy5MuvZRX20BEn39u/0GGDJEeeMA6kiN1/fCDjVY984z19urQYfO/RqdOTXefTbSeYEAzpETA2nNP6Z13pP32s6mMrbe20yvuuSfsyoA49Mor9p9l5Eg7pPi662xx9s8/h10ZwvCPf0g77ij99rc2itWtmzRu3OZ/nbZtpdNOi/xq96abAikViCcpscgdQDNVVtorkNWr61/PzpZ+/3vpttvCqQvhmDNH6tdPKimpfz03V1q8ePNHskpLpWHDpOeft2ZpmZm2tmvYsOBqBmIo5Re5A2imzz+PPI1TVkZr9kiqq21n3O67W9f2665LrpG+sWNtsWpD6enWbmFz5eRIY8ZIy5bZOY3LlxOukLQIWABqtWnT9DqZeOtnMnWq7WTbeWfp3HOlr76KfQ3nn29tB+bMkRYskEaMsOnVujvlEll5eeTjL6qrIwev5mrTxs40jHTOEpAkCFgAavXoIfXtayMUdeXnS1deGUpJEb35pvWFeO01C1ZPPy0VFEizZsWuhm++kV58sX6YKiuzE7Cffjp2dUTTySdH3ilYXS0dd5wky1kjRtha1z32sCVbLcleQLIgYAGob9w4m+5q1cpGGrKzpUsusQXK8cB72+FYXFzb4K6qynY6XnNN7Or45JPGQVSyvk7vvhu7OqLpgAOk886zkOWc/X1zc6U775S6dZP30rHHSjfeaDN+n31m69UHDYrcexBIJRFOEQOQ0rbbTvrySwsQS5dK++8vde0adlW11q+Xvv8+8n0ffxy7Orp1i9xgLyvLpr+SgXPSQw/ZFOyLL1rYPussadddJdkRi1On1h/EKymxVlkTJ0pHHhlO2UA8IGABaMw5Oy8uHuXm2tqdSGvFNvdcvJY47DDr7VRcXP8g5MzM5Fu4fcABdmvgf/+LvNysqMjuI2AhlTFFCCCxZGRIF18cuZ/StdfGro60NBvCKSiw3XF5eXag8/jxKXNERJcujf8ZJHsqunSJfT1APKEPFoCEU1FUrvdOvF+rJ32mQ7M/UbeqRdJvfmNrgyJN20Xb0qU2lNOzZziPH5J162xGec2a+tfbtrWTcTgpA8luY32wmCIEkFA++0waMCBLpaXXyGdXq6JcuuY35br9LznhFZWiwzWtW9taq1NOkX780a517iy98ALhCiBgAYnE+5QaIWmoutp2rS1fXnPFVjn846EcHTJAOuaY0EpLWXvvLc2bZzfvpZ12SulvUeD/sQYLSAQzZtgi44wMGza45hrruZRipk6V1q5tfL2oyBqqIxzOWbDq3ZtwBdRgBAuIdwsW2I619evt4/Xrbev8woU2F5NCioqa/gUeKXgBQFgYwQLi3d//bofk1lVSIk2YYCuJU8gBB9TviFAjL08644zY1wMATSFgAfHu008j93zKzpa+/jr29YQoL8+mAnNza5uo5+fbMS3nnhtubQBQF1OEQLzbZx9rjV1RUf96WZnUp084NYXonHNsYfXIkdKKFdIJJ0i/+AXnBgOIL/TBAuLdt9/aEM26dbXXcnOl44+Xnn02vLoAIMVtrA8WU4RAvNthB2nSJOmQQ2xerF076corpSeeCLsyAEATmCIEEsFee1nIAgAkBEawAAAAAkbAQtz78EPbgj9ggLV/KikJuyIAADaOKULEtX/8Q7r5ZgtV3ktTptjusSlTbJ03EHPV1VJ5uZQT4tmHTfn+e+u42qePdf0HEBpGsBC3Vq+WbrxRKi62cCXZ+998I40ZE25tSEEVFdLVV9spxvn5di7MO++EXZVZssS6sPbuLfXvb4dPv/pq2FUBKY2AlaSqquzn6/DhNgI0f37YFW2+KVOkrKzG14uLpXHjYl8PUtzw4dLDD9t5PdXVlvRPOsl6lIXJe+noo62O0lI7SmnlSunMM6W5c8OtDUhhBKwkVFEhHXOMdPbZ0qOPSnffLe2+e+IdW9eunf0ea8g5qVOnmJeDVPbTT9LYsY0XAJaUSHfcEU5NNQoL7cikhmcIlZVJDzwQTk0ACFjJaOxY6ZNPas8Grqiw3wMXXphYC8T795c6dGh8uG9urnTppeHUhBS1aFHk4VTvpS++iH09dS1dKqVF+FFeVWUHgmPjFi6UnnxSeuutyEdSAVuIgJWEnnrKZjEaSkuzabdEkZZmS1y6d5datZLatrVwdffdttwkCF9+af06338/8mgZIEnq2dMWtjeUliYVRGziHDv77WejVQ3l5koDB8a+nkThvXTFFdIuu0i//rV06qnS9ttLX30VdmVIEmwzSUJ5eZGvex+fG582pndvacECW17y8882qtWmTcu/bmWlTaG+9lrtocFduljQ6tq15V8fSaZNG/tl/OCDtgiwRm6udNNN4dUl2TfupZfa9tqaV1bZ2VLnztKvfhVubfHsxRel//zH1q3VWL/ejqD66qvGQ+fAZmIEKwldfLFtcmooL0/af//Y19NSzkn9+tm6siDClST985/S+PE2Zbp+vd0WLLB1wUBEf/2rdOed0rbbWrA6/HDrrt+3b9iVSX/7m/Svf9l/8D59pKuukqZPl1q3Druy+PXQQ42H+r23HZmffx5OTUgqHPachLyXfvc76ZFHbAYjPd1a4rz9dvizGfGiTx/p668bX8/Kkn74Qdpqq9jXBCCG+vWLvAO0TRtbm9CvX+xrQsLZ2GHPTBEmIeesQecVV0j//a8tFD/uuMSbHoymphb7p6XVnzEAkKROP12aM6fxD4O0NGnvvcOpCUmFKcIk1rOndNFF0sknE64aGjo08qawrl2lbt1iXw+AGPv1r20ou2Y9RWamraN4/HF7HxtXUtK4NQjqIWAhJf3xjxakan62Zmfb+2PGsLY1UXlvy46efFKaMSPsahD38vKsn80jj0hnnSX99rfSzJnSCSeEXVl8++AD23nZurXdLruMYf8msAYLKauoyFpafPCB1KuXNGyYrV9G4lm/Xho0yH4/Omdha++9pTfesBYfAAIwZ45tpGi4k3bIEOm558KrK0QbW4NFwAKQ8IYNs9HHuu2gsrOlX/7SNosBCMC551on64ZNA3Ny7Dy2FOxxs7GAFfUpQufcIOfcV865ec6566P9eABSz5NPNu61WVbGoeBAoD7/PHJH5uxs6dtvY19PnItqwHLOpUt6UNKxkvpKOtM5FwdNYwAkC+8jN1mXIjc4B7CF+vWznj8NlZbahgHUE+0RrH6S5nnvF3jvyyU9I+nEKD8mgBTinHTEEY03J6SlSUcdFU5NQFK69lpbc1VXXp7NxdM8sJFoB6xukr6v8/HiDdcAxMikSdLRR0s9eki/+IX02WdhVxSgqippzhw9eOMPateuth1Jbq7Urp30wANhFgckmZ49pf/9z36g5OXZVuxbb+U/WhOi3Wg00ob3eqvqnXPDJA2TpO233z7K5QCp5eWX7czFmk0/ixZJb71lOyf33TfU0lpu/Hjpwgul0lL1rqrSN70P0KjjXtSMBe20zz52DF/HjmEXCSSZ3XazY0GwSVHdReicO0DSrd77gRs+vkGSvPd/ifT57CIEguO91L279P33je878kjpvfdiX1NgvvzSEmLd7eLp6fYKm4N6AcRImLsIp0nayTm3g3MuS9IZkl6N8mMCkLRunfTjj5Hvmzo1trUE7uGHG69sr6qSli61KYx4573N3d50kx3UvGRJ2BUBCFhUpwi995XOucslvSUpXdJo7/3caD4mAJOXZyd+VFQ0vq9z59jXE6jvvpMqKxtfT0uzkBWvqqulp5+WrrpKWrXKPs7OtqMFnn3WGjYCSApR74PlvX/de9/be9/Le39HtB8PSFaLF0u33WbnSz711KZbEGRkSMOHW9CqKy9Puj7RO9Idc0zjv5hkT0r//rGvp7nOP98Wh61YUdtPqKzMpjrPOosjR4AkwlmEQAL44ANp552lO++URo2SLrnEliCtW7fxP3fXXdZ8OSfHjozJz7dw9atfxabuqDn/fOsanZ1dey0/X7r44vg972jmTOmll5pOxs5JH34Y05IARA8BC4hz1dW2E7CoqPZ3c1GRnUzx979v/M9mZtpZtsuW2UHIK1ZIf/jD5q0BnzlTuu466Zpr7GzcuJCfL02bJv3+93bwbP/+0siR0v33h11Z0yZOjDytWVd6emxqARB1nEUIxLmvvrLRqqKixvf16WMb6qLljjvsVlZm67Jzc23a8W9/i95jJq0nnpAuvdROpo6kVStpv/2kH36QDj9cuvFG2wYKIG6FehYhgJbJyYl8/JfUuKlykBYskP78Z6mkxB7fe1sq9MgjNqqFzTR0qC3CjyQry3YjTJwoff21NHq0tNde0sKFsawQQIAIWEmorIwz2JJJ9+5S796Nfzfn5dloUrS89lrk66Wl1sAUm6lVK2vQuM02UuvW9g+Ylyf99reWouv+p62stAV2t90WWrkAWoaAlUQWLrQGkvn59rN84EDbeYbE9+KLUpcu9ns5P99Grk44wXYURktWVuS1Wunpdh+2wP7723/Kd9+125o1lpIjDVFWVdmIFoCExBqsJFFcbE2s6+7+Tk+3X8rz5/MLMRlUVkrvvGM9KQ84QOrbN7qPt2yZtMMONkVYV06ONGeO1KtXdB8/ZaxaZWe6RRp2PuggafLk2NcEoFlYg5UCXnjBFkHXfSFcVWUvkF+ld35SyMiQjj3WWixEO1xJ1ox09Oj6LR5ycqT77iNcBapjR+m44+q3nJBs+vC668KpCUCLRfuwZ8TIvHmRNyeVlNh9wJY44wzp6KNtPVZVlTR4sC0hQsDGjLFeHG+9ZcPN3tsOg+OPD7syAFuIgJUk9tzT1uc0bDyZkyPtsUc4NSE5dOwoXXBB2FUkuVatpFdekZYvt7nZnXay/7wAEhYBK0mccIKNLJSV1Z6Bm5Ul9ehhi90BJICtt7YbgITHGqwkkZkpffyxjTS0bSu1b29rdSZPpjk0AACxxi5CAACALbCxXYRMEQJIPT/8IP3739au/rDDpDPPjG5bfAAph4AFILV89JEtTKystEWLL74o/eUv0tSpNrcOAAFgDRaAmFi/Xnr6aenRR635bSi8l845x5rG1TT2LCqSFi2S7rwzpKKQdFautNYbTzxhjWSRkhjBAhB1kydbL03JBo68ly6/XLr77sjH8UTN999bG4SGysul55+X7rkn4h+rrJTGj5c++8zOhTzppMZ9QQFJFqwuucQ6A0v2zTNqlHTWWeHWhZhjkTuAqCovt67wP/9c/3p+vh0aPWBADItZvlzafvvIx9LsvLP0xReNLq9aJR14oLR0qY3C5edL7drZrt1u3aJfMhLI999LffpEPl9q/nypa9dw6kLUcFQOgNB88EHks4yLiuyFfUxtvbW0zz6Ne5fk5Um//nXEP3LNNdK331oTX+8tZC1dKg0bFoN6kViefz7yN7tka/2QUghYAKKqoqLp+yINJEXds8/aKFbr1jYclZtr85eXXhrx0198sfHfoapKevNN6b33pNLSGNSMxFBaalOCDVVV8Y2SgghYAKLq0EMj/87Jz7fuCDG33XZ2QOe4cdIDD0iFhTbykLF5S1Krq6WhQ21Q7KWXolQrEsvxx9sRGg1lZHCuZAoiYAGIqlatpNGjbaAoM9Ou5edLRx4pnXxySEWlpUlHHWVHH/Ttu9FP/cUvautuaN06u517rrXUQorbfXfpsstsyjktzXZw5OVJV11la/yQUljkDiAmFiywDVarV0tDhtji9pjuINxCq1ZJBxwg/fijrb+K9CMzM1O6/nrptttiX9/GrFljbTEmTJC23Va68kpp//3DrioFfPyx9Mwz9g1+5plSv35hV4Qo2dgidwIWAGxCTZuGxx6ztVeR1o5dcon0yCMxL61Jq1fbev5ly2xTm3M2ivjQQ9L554ddHZAc2EWYgNaulW64QerZ03b93ntv44W23tsP/eOPt9mOUaNsSzyAYGVkWO+rv/0t8qhbq1a1fb7ixX332W7Hmo4B3kvFxdIVV4S0uQBIMTQajUPl5TYlMX9+7Q/CP/5Rev99C1Q1rrnGhv+LiuzjTz6xV9gTJ272el0AzdCrl7VnGDWq9v9dfr60337S4MHh1tbQq69GDlLOSbNnW80AoocRrDj00kt2ckfdH44lJRacpk+3jxcutKH+mh/ykr0/c6b9YAUQHffdZ0f+DBliC/X/+U/prbcat9YKW8eOka9XVEgdOsS2FiAVEbDi0OTJtpi2oepqG6WSbDQr0ijV+vW2oBVAdDhn0/KvvWZ9sC68sOldhmG66irbwFZXerptmuzVK5yagFRCwIpDO+xgi1Ebysy0Fj6SvQKNtBYkM9P68gBIbYMHSzfeaKe0tG1rU5m77iq98krYlQGpgV2EcWjFCnuFuW5d7bW0NKlLF5sazMiw6cOuXaWffqr/Z3NzpVmzpJ12imnJAOLU6tXWS7VzZ2vTlAitMYBEwS7CBNOpk0097LijBabsbGnffaVJk2qnBbOz7XO6drUTP9q0sbePP064QnyYN0+66CJpjz2k006TPv007IpSU/v20tFH278D4QqIHUaw4pj30uLFNu23zTaRP6e6Wpo2zRbB9+9v0wFA2D77TDrwQPu+rKqyEdicHJueGjAg7OoAIBgbG8FiM38cc652zVVT0tLozIz4c/XV9TdqVFdbD6Zf/1r65pvw6qqxeLH05JM2HT9woIW+NMbzAQSIgAUgcFOmRL6+cKEFr1atYlpOPRMmSKeeaqGvrEwaOVI65BBrb0L/OABB4TUbgMC1bx/5emZmuNPYZWXSWWfZ1GVNn7n1621949NPh1cXgORDwAIQuGuuadyDKTfXekaFOUrU1MhaUZH0xBOxrQVAciNgpTDvpfvvt3VeeXnSoYfadm6gpS6/3A4/runBlJMjnXii9Pe/h1tXZqZ930eSlRXbWgAkN3YRprAbbrCAVVxcey0/X/r4Y2m33cKrC8lj9Wpb1N69u/VhCltlpfWTW7my/vX8fBvBGjo0nLoAJCb6YKGR9eulESPqhyvJ1qbcfns4NSE5zJoljR1ro6Ht2kn9+sVHuJJsevK116xvXKtWNrKWmyudfbZ00klhVwcgmbBnJkXVdIRvqLq69kBpYHMUF9sZfR9/bC0PvLfmlm++aYEmXvTvLy1ZIr38sp2EMGCAtMsuYVcFIDDV1dKXX9rhm717h9Zhl4CVorbdViovb3zdOX7ZYMvcdJP0v/9JpaW112bMkH77W2n06NDKiig/30atACSZKVPs6IjVq+1VXpcu0ksv2au9GGOKMEW1ayedf37knV433xxKSUhw//lP/XAlWSuEsWObXlgOAIFZuVI65hjrJFxUZMPq8+dLhx/eeD1MDBCwUtiDD9pur/x8m9LZcUcL+nSGx5ZoGK5qVFQQsADEwNixdjZXQxUVtiYgxghYKSwjQ/rrX6W1ay3sf/ONHRsCbImjj2583Ixz0sEHcwwNgBj44QfbqdVQebn0448xL4cfe/j/g3iBlhgxwjq45+baxzk5trj94YfDrQtAijjssMjncGVm2iu9GGORO4BA9Owpff21NGqUNG2atOee0sUXS1tvHXZlAFLCoEH2g2fGjNqRrLw86aijrF9MjBGwAASmQwfp2muj+xhPPmm92pYskfbaS7rnHmu9ACDFpaVJ774rPfSQNGaMtWm46CJ7pRcCOrkDzVRSIj3wgPTUU1J2th0Fc8EFrC+KpREjpBtvrL8hKC9Pev99ab/9QisLQIraWCd3RrCAZqistOn9OXNqR57nzrUXS2PHhltbqqiokG65pfFu6+JiC13vvBNOXQAQCa+9gWZ49VXpiy/qb1ApKrKdv3PmhFZWSlm+PHJzXEmaPTu2tQDAphCwgGZ47z07vzGSDz+MbS2pqmPHpk+82GGH2NYCAJtCwAKaYdttbd1VQxkZ0jbbxL6eVJSTI116aePTB/LypFtvDaUkAGgSAQtohvPPtw0pdTlnv/QHDw6nplR01112tmGrVtbapksX6d//tt3ZABBPCFhAM3TtKk2YIHXubL/c8/KknXaSPvhAysoKu7rUkZ4u3XGHneO6YoU1bj7zzLCrAoDG2EUINNPhh1vvpblzLVT17t30miBEV0aG1LZt2FUAQNMIWMBmSEuTdt897Cparrpa+vZbqXVrOq0DQDQwRQikmDfflLp1k/bYQ9p+exuZW7Ys7KoAILkQsIAU8uWX0i9+YQfLFxdLZWXSRx9JxxwjxdGhDgCQ8AhYQAC8tyN09ttP2nFH6eqrbRF2vPnnPxs366yslObPt/NRN+Xll6VddpFyc6XddpNeey0qZQJAwiNgAQG45ho7m7Cw0MLKAw9I++xjoeXf/5bGjbPRorAtWGCBqqH0dGnx4o3/2eeek84+20bBSkttsf/pp0uvvBKdWgEgkXHYM9BCP/5oncRLS+tfT0+3XYZZWfZ+ZqZ1hN9rr1DKlCTde6/0xz/WP/JHsn5e8+bZ2qym7LCDtHBh4+u9e0tffRVomQCQEDZ22DMjWEALzZgRuRdWVZWNFhUXS+vWST/9JB1/vO3gC8vFF9uRM3XrzcuTLrxw4+HK+8jhSrIROwBAfQQsoIW6drUw1Rw//9y8tU7R0ratPf6ll0o9ekh77mnTmQ8+uPE/51zTRwJtLJgBQKoiYAEttOeetrA9oxld5dLSGk8lxlqnTtI//mF9sGbOtNGr5jRMveWWyOcA/ulPUSkTABIaAQtoIeekt96SDjzQ1jLl59txOpEOh5akfv1iW19QLrlEuuceaautbE3Z1ltL991nxwf1728jXEOGWGgDgFQXtYDlnLvVOfeDc27mhttx0XosIGydO9u5hAsWSNOnS8uXW8uGVq3s/qwsa23wxBOJe3ahcza1uHy5rSn78UcLlKecIn3yiTUrff116aCDwp0GBYB4ELVdhM65WyWt997f29w/wy5CJJOqKusT9cYbNrpzwQW2Ey9ZVFfb3ytSv6+jj5befjv2NQFALG1sFyFnEQJRkp4unXSS3ZLRypXS2rWR75s+Pba1AEC8ifYarMudc7Odc6Odc+2j/FhA0pkxw9Y+DR0qjRnTuAt7mNq2tRAZSdeusa0FAOJNiwKWc+5d59ycCLcTJT0sqZekvSQtlfS3Jr7GMOdcoXOucEU8ni0ChORf/5IOOcQ6wb/8sq1/Ovjg+OgIL9ki/mHDIu8s/OMfw6kJAOJFTDq5O+d6SBrvvd9tY5/HGizArFtnC+cbdlzPy5NGjJAuuiicuhqqrJSuukoaNcoWwWdmSrffLl1xRdiVAcFZu1a6+27pmWfshcWvfy0NH9681ixIbqF0cnfOdanz4VBJc6L1WECymTLFwkpDxcV2JmC8yMiwA6RXrrQzClesIFwhuZSVWRuSe++1Uws+/1y67jrpjDPCrgzxLpr5+27n3F6SvKSFki6J4mMBSaV1azueJpJ27WJaSrPk5TWeKgSSwfPPS99/X39qvrjYWpJ89pm0++7h1Yb4FrWA5b0/N1pfG0h2++9vi8jXrat/PS/PpieQuNatk0aPlt58U9p+exvx222jiycQpg8+kNavb3zdOev/RsBCU+jkDsShtDT7Bdyli41mtWljTT1vukk64ojN/3orVti6rY4d7WveeGPj9V2IvtWr7WilG2+0f99RoyxMv/RS2JWhKT162P+9htLTOYcTGxeTRe7NxSJ3oL6qKmnSJPvFfMghdo7g5ioulvr2lZYskSoq7FpOjq0r+e9/m3cOYSx4b3/fZF44fPPNtpan4U7QDh2sE34y/90T1ZIlUu/eUlFR7bW0NGtF8u23/JululAWuQNoufR0G7E6+eQtC1eS9Oyztgi9JlxJduD0tGnS1KnB1NkSFRXStdfaSF1Wlo3wTJ4cdlXRMW5c5DYbFRW2eBrxp2tXO2u0e3ebos/JkfbZx174EK6wMXx7AEnu44/rv/quUV1tBzPvv3/MS6rnV7+SXnihdspy9mxp4EBb35Jsa5Oa2qBQUWFr7hCfDjrIRqu+/dbaNDA1iOZgBAtIcjvvbAdNN5SRIfXsGft66lq2zNpONFwPVloq/fWv4dQUTVdeKeXn17+Wnm4Lpbt3D6cmNI9z9v+FcIXmImABSe6882zqre5aq4wMO6j5qKPCq0uyEYFIC4irq20kK9mcdprtAs3JsY0LrVrZ+p5x48KuDEDQCFhAkuvY0dY07buvNS/NzJQGDLA1JGkh/wTYcUcbrWooPd3qTTbOSffcY8HyiSdsk8HcuYyKAMmIXYRAClm3zsJLPDUFHT7cwkZxce21Vq2k6dNtdAcA4hW7CAFIsp168RSuJOnBB60v1NZb21TmIYdYc0fCFYBExggWAADAFmAECwAAIIYIWAAAAAEjYAEAAASMgAUAABAwAhYAAEDACFgAAAABI2ABAJpn3Trpu++kysqwKwHiHgELALBxpaXShRdKnTpJfftaV9jRo8OuCohrBCwAwMZdcon07LNSWZmdabR6tXTFFdKbb4ZdGRC3CFgAgKatWWPhqqSk/vXiYumOO8KpCU2rrJSqqsKuAiJgAQA2ZvlyKTMz8n2LFsW2FjRtwQLpqKOknBy7nXSStGxZ2FWltIywCwAAxLHu3aW0CK/F09KkAw+MfT1obP166YADpJUrpepqu/b669LBB0tffimlp4dbX4piBAsA0LSsLOnOO6W8vNpraWlSfr50662hlYU6nnlGKiqqDVeSVFFhI1hvvRVeXSmOgAUA2LjLLpPGjpUKCqSuXaWTT5amTpX69Am7MkjSF19YwGqovFz6+uvY1wNJTBECAJrjxBPthviz115Sq1Y2VVhXZqa0++6hlARGsIDALFwo/fKXUo8e0v77S+PGhV0RgJRw6qlShw5SRp0xk6wsaccdpSOOCK+uFEfAAgLw/ffSPvtIY8ZYo+upU6VzzpHuvVfyXnrtNWnQIFsTfP/9jXe8o/lWrZL+9z9p6dKwKwHiRE6O/dA5/XQbyWrb1l7tffBB5A0KiAmmCIEA/OUvdopI3fYzxcXSLbdIS5ZII0fWLpGYNUt67DFpyhQpOzuUchNSdbV05ZXSqFH2vJWVSSecYKGW5xEpr3Nn6cknw64CdRBtgQB88EHk49nS0qQHH6y//rS42NadPvts7OpLBn//u/Sf/9ipLWvW2NvXXpOuuSbsygCgMQIWEIDtt498vawsco/GoiJpwoTo1pRs7rvPwmldJSU2okXjagDxhoAFBOC66+q3CZJs2mqffSIvgcjIkLbZJja1JYvVqyNfLy+3IAsA8YSABQTg8MOlhx+W2re3/ovZ2dLgwXYWbuvWknP1Pz8z087PRfM11TR8p50ah1sACBsBCwjIeefZsW2ffmoL2198UWrXTnrvPalnT9vc06aNBa7HHpP69g274sTy97/bc1izEz0tzYLVww+HWxcARMIuQiBAGRk2olLXzjtL33wjzZ5tfQALCtj1tiV2312aOVO6+27bkb7rrjY1Sx9FAPGIgAXEgHPSnnuGXUXi69VLevTRsKsAgE1jihAAACBgBCwAAICAEbAAAAACRsACAAAIGAELCNn69dJNN0k9elg7h9tvt2NgAETJ3LnSCy/YWyBK2EUIhKiyUjrkEOnLL2tD1V/+Ir3zjp1v2LBBKYAWKCmRTjxRmjzZuv1WVkoHHCC9+irdahE4RrCAEE2YIM2bV3/EqqREmjFDmjQpvLqApPT730sffmj/ydautcMtP/pIuvbasCtDEiJgASH65BObImyovFyaNi329QBJ7bHHGs+/l5badSBgBCwgRN27R56ZyMmRttsu9vUASa2pxY2lpZL3sa0FSY+ABYTojDOkrKz615yTcnNtqQiAAB16aOOFjc5Fvg60EAELCFHbtrbWarfd7HzC7Gxpn31sDW5OTtjVAUnmwQftxPWa/1zZ2fbxgw+GWxeSEgELCNnuu0uffSZ9+6303XdSYWHjA6MRmffSqFHS9tvbprDddpPeeCPsqhC3dt5Z+uorOyV8yBB7++WXUt++YVeGJOR8HM07FxQU+MLCwrDLAJAg/vlP6frrbTNYjdxc6ZVXpKOPDq8uAKnBOTfde18Q6T5GsAAkpOpq6dZb64cryXbg33hjKCUBwP8jYAFISGvWRG5xIdksEACEiYAFICG1aWPTgZHsuGNsawGAhghYABJSerp0881Sfn7967m50p//HE5NAFCDgAUgYV19tXTnnVKnTvZxjx7SE09Ixx0XalkAwC5CAPFrzRpp6lSpfXtp33033guyulpK4yUjgBhiFyGAhDNihNSli3TKKdIRR0i9e0sLFjT9+YQrAPGEH0kA4s6kSdZqoaREWrvWdgsuWCANGsSRcQASAwELQNx54AELV3VVV0tLl0ozZoRTEwBsDgIWkkpVlTRmjJ3detBB0qOPShUVYVeFzbViReSRqrQ0afXq2NcDAJsrI+wCgCCddZY0YYJUVGQfz5wpvfCC9PbbG18gjfhy0knSJ580HsWqqJD69QulJADYLIxgIWlMny6NH18briQ7RuXjj6X33guvLmy+iy6ylgt5efaxc/b+3Xdbg1EAiHeMYCFpvP++VFnZ+Pr69dJ//ysNGBDzkrCF8vOladOk0aOlceOkrbeWLr9cOvjgsCsDgOYhYCFpbL21lJUllZfXv56bK3XuHE5N2HL5+dIVV9gNABINU4RIGkOH2vEpDaWl2dosAABihYCFpNGqlfTuu1LXrvZ+69Z2hMr48bVHqQAAEAtMESKpFBRI338vzZplLRv23jvyqBYAANHUohEs59ypzrm5zrlq51xBg/tucM7Nc8595Zwb2LIygeZLS7NgVVBAuAIAhKOlI1hzJJ0s6dG6F51zfSWdIWlXSV0lveuc6+29r2rh4wEAAMS9FgUs7/0XkuQad3A8UdIz3vsySd865+ZJ6idpSkseD0gF3ltbiTfekDp0kM45R9p++7CrAgBsjmitweom6eM6Hy/ecA3ARlRV2W7IiROtf1dWlnTHHdLYsdKJJ4ZdHQCguTa5Bss5965zbk6E28Z+3Ec6lCTCyWKSc26Yc67QOVe4YsWK5tYNJKXnn7fRq/Xr7ePycutGf+65UmlpuLVtyk8/SbfdJh14oHTaadIUxqsBpLBNjmB577ek//ViSdvV+XhbSUua+PojJY2UpIKCgoghDEgVY8bUP+qnhnPS5Mnx241+1Sppr72klSstCDpnZ0I+9JB0/vlhVwcAsRetPlivSjrDOZftnNtB0k6SpkbpsYCkkZ3d9H0ZcdxU5d57pRUrakfZvLeRtyuukMrKwq0NAMLQ0jYNQ51ziyUdIGmCc+4tSfLez5X0nKTPJb0p6TJ2EAKb9qtf2RExDWVmxvc5fOPHNx2k5s6NbS0AEA9aFLC89+O899t677O995299wPr3HeH976X976P9/6NlpcKJL/Bg21KLTdXysmp7Uj/yivxPYLVVKf8ykqpY8fY1gIA8SCOf2QDqcc56cEHbWrt3Xeldu1s92Dr1mFXtnG/+530ySc2LVgjI8PWZXXvHlpZABAaAhYQh3be2W6JYsgQ6eabbRdhdrZUUSHtsos0blzYlQFAOAhYAAJxww3SpZdKn34qde5sAQsAUhUBC0Bg2raVDj887CoAIHzRatMAAACQsghYAAAAASNgAQAABIyABQAAEDACFoB6vJeqOHcBAFqEgAVAklRSYm0W8vPtaJ5+/aQZM8KuCgASEwELgCTplFOk//zHgpb30rRp0mGHSQsXhl0ZACQeAhYAzZsnTZwolZbWv15WJo0YEU5NAJDICFgA9M03UlZW4+sVFdKsWbGvBwASHQELgHbZxUarGsrKkgoKYl8PACQ6AhYA9ehhBzbn5tZec07KyZGuvDK0sgAgYRGwAEiSnnpKuuoqqUMHG7kaMECaMkXadtuwKwOAxOO892HX8P8KCgp8YWFh2GUAAABsknNuuvc+4kIKRrAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwgJCtWSN9+KE0b17YlQAAgkLAAkJ0++3SNttIxx8v7bGHdNBB0qpVYVcVW7NnS6+/Li1dGnYlABAcAhYQkpdeku66SyottVGskhJp2jTp9NPDriw2Vq6U9ttPOvBA6ayzpJ49pSuukLwPuzIAaDkCFhCSv/1NKi6uf62iQpo8OTVGc846S5o1SyoqsoBZWiqNHm03AEh0BCwgJCtWRL6emSmtXh3bWmJt5Upp0iQLlHUVF0sjRoRTEwAEiYAFhOTYYy1MNZSRIe20U+zriaV166T09Mj3/fxzTEsBgKggYAEhueEGqX17KTvbPnZOysuTHnggcvBKJt27S23bNr6emSmdcELs6wGAoBGwgJBss400Z4507bVSv37SKadI770nnX122JVFX1qaNGqUBcqakaycHKljR+nmm8OtDQCC4HwcbdkpKCjwhYWFYZcBIEY+/9zWXM2fLx15pDR8uNShQ9hVAUDzOOeme+8LIt2XEetiAKBG377So4+GXQUABI8pQgAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIBlhF0AAADNUVYmvfuuVFoqHXmk1L592BUBTSNgAQDi3uTJ0pAhkvd2q6iQRoyQhg0LuzIgMqYIAQBxrbhYGjxYWrNGWrtWWrfORrF++1tpzpywqwMiI2ABAOLaG2/YqFVD5eXS44/Hvh6gOQhYAIC4tn595IBVVWWjWkA8ImABAOLagAFSZWXj661aSUOHxr4eoDkIWACAuNatm/SHP0h5eZJzdi0/XzriCGngwHBrA5rCLkIAQNy78UYLVKNG2aL3006TTjhBSmOYAHGKgAUASAgHHGA3IBG0KPs75051zs11zlU75wrqXO/hnCtxzs3ccHuk5aUCAAAkhpaOYM2RdLKkRyPcN997v1cLvz4AAEDCaVHA8t5/IUmuZtUhAAAAorqLcAfn3KfOuQ+cc4dE8XEAAADiyiZHsJxz70raJsJdN3nvX2nijy2VtL33fpVzbl9JLzvndvXer43w9YdJGiZJ22+/ffMrBwAAiFObDFje+wGb+0W992WSyja8P905N19Sb0mFET53pKSRklRQUBChVy8AAEBiicoUoXOuk3MufcP7PSXtJGlBNB4LAAAg3rS0TcNQ59xiSQdImuCce2vDXYdKmu2cmyXpBUnDvfc/taxUAACAxNDSXYTjJI2LcP1FSS+25GsDAAAkKg4ZAAAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAZYRcAxLMff5TefFPKzpYGD5batAm7IgBAIiBgAU24/37puuuk9HQpLU2qqpKef1467riwKwMAxDsCFhDBnDnS9ddLpaX1r596qrRkidS2bTh1AQASA2uwgAieeEIqL298PS1NGj8+9vUAABILAQuIoLRUqq5ufN37xqNaAAA0RMACIjj5ZCkvr/H1qirp2GNjXw8AILEQsIAIDj3U1lvl50vO2UL33Fzpz3+WunYNuzoAQLxjkTsQgXPS6NHS+edLL7xg4eqcc6Q99wy7MgBAIiBgAU1wTjr8cLsBALA5mCIEAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIARsAAAAAJGwAIAAAgYAQsAACBgBCwAAICAEbAAAAACRsACAAAIGAELAAAgYAQsAACAgBGwAAAAAkbAAgAACBgBCwAAIGAELAAAgIARsAAAAALmvPdh1/D/nHMrJH0X44fdStLKGD9msuK5DA7PZXB4LoPDcxkcnsvghPlcdvfed4p0R1wFrDA45wq99wVh15EMeC6Dw3MZHJ7L4PBcBofnMjjx+lwyRQgAABAwAhYAAEDACFjSyLALSCI8l8HhuQwOz2VweC6Dw3MZnLh8LlN+DRYAAEDQGMECAAAIWMoGLOfcPc65L51zs51z45xz7ercd4Nzbp5z7ivn3MAQy0wIzrlTnXNznXPVzrmCOtd7OOdKnHMzN9weCbPORNDUc7nhPr4vt5Bz7lbn3A91vhePC7umROOcG7The2+ec+76sOtJZM65hc65zzZ8LxaGXU8icc6Nds4td87NqXOtg3PuHefcNxvetg+zxhopG7AkvSNpN+/9HpK+lnSDJDnn+ko6Q9KukgZJesg5lx5alYlhjqSTJU2KcN987/1eG27DY1xXIor4XPJ9GYh/1PlefD3sYhLJhu+1ByUdK6mvpDM3fE9iyx2x4Xsx7toLxLnHZD8D67pe0nve+50kvbfh49ClbMDy3r/tva/c8OHHkrbd8P6Jkp7x3pd577+VNE9SvzBqTBTe+y+891+FXUcy2MhzyfclwtRP0jzv/QLvfbmkZ2Tfk0BMee8nSfqpweUTJT2+4f3HJZ0Uy5qakrIBq4FfSnpjw/vdJH1f577FG65hy+zgnPvUOfeBc+6QsItJYHxfttzlG5YEjI6XKYQEwvdfsLykt51z051zw8IuJgl09t4vlaQNb7cOuR5JUkbYBUSTc+5dSdtEuOsm7/0rGz7nJkmVkp6q+WMRPj/lt1o257mMYKmk7b33q5xz+0p62Tm3q/d+bdQKTQBb+FzyfbkJG3teJT0s6XbZc3a7pL/JXlihefj+C9ZB3vslzrmtJb3jnPtyw8gMkkhSByzv/YCN3e+cO1/SEElH+dp+FYslbVfn07aVtCQ6FSaOTT2XTfyZMkllG96f7pybL6m3pJRe1Lklz6X4vtyk5j6vzrl/SRof5XKSDd9/AfLeL9nwdrlzbpxsCpaAteWWOee6eO+XOue6SFoedkFSCk8ROucGSbpO0gne++I6d70q6QznXLZzbgdJO0maGkaNic4516lmIbZzrqfsuVwQblUJi+/LFtjwQ7fGUNlmAjTfNEk7Oed2cM5lyTZcvBpyTQnJOZfvnGtd876kY8T3Y0u9Kun8De+fL6mpmYCYSuoRrE14QFK2bHhWkj723g/33s91zj0n6XPZ1OFl3vuqEOuMe865oZL+KamTpAnOuZne+4GSDpV0m3OuUlKVpOHe+4aLE1FHU88l35ctdrdzbi/ZtNZCSZeEWk2C8d5XOucul/SWpHRJo733c0MuK1F1ljRuw++dDEljvfdvhltS4nDOPS3pcElbOecWS7pF0l2SnnPO/UrSIkmnhldhLTq5AwAABCxlpwgBAACihYAFAAAQMAIWAABAwAhYAAAAASNgAQAABIyABQAAEDACFgAAQMAIWAAAAAH7PyQibJzAw6E+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "colors = ListedColormap(['red', 'blue'])\n",
    "\n",
    "pylab.figure(figsize(10, 10))\n",
    "pylab.scatter([x[0] for x in blobs[0]], [x[1] for x in blobs[0]], c=blobs[1], cmap=colors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we split the **blobs** dataset into the **train** and **test** sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data, test_data, train_labels, test_labels = model_selection.train_test_split(blobs[0], blobs[1], \n",
    "                                                                                    test_size = 0.3,\n",
    "                                                                                    random_state = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Linear classification models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 RidgeClassifier\n",
    "We'll create/build the linear classifier named **[RidgeClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.RidgeClassifier.html#sklearn.linear_model.RidgeClassifier)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "ridge_classifier = linear_model.RidgeClassifier(random_state = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train the classifier object: pass to it the train data and the train labels generated ealier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RidgeClassifier(random_state=1)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_classifier.fit(train_data, train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we check how well the classsifier has been trained. \n",
    "\n",
    "We apply the test data to get predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "ridge_predictions = ridge_classifier.predict(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we output the test labels (response values) and the predicted labels, ones generated by the classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Actual labels:\n",
      " [0 0 0 1 0 0 0 0 0 1 0 1 0 1 1 1 0 0 0 1 0 0 0 1 1 1 1 0 0 1]\n",
      "\n",
      "Predicted labels:\n",
      " [0 0 0 1 0 1 0 0 0 1 0 1 0 1 1 1 1 0 1 1 1 0 0 1 1 1 1 0 0 1]\n"
     ]
    }
   ],
   "source": [
    "print('Actual labels:\\n', test_labels)\n",
    "print('\\nPredicted labels:\\n', ridge_predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now **rate the classification model quality**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.87"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(metrics.accuracy_score(test_labels, ridge_predictions), 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We output the featurs weights and intercept. $y(x1, x2) = w0 + w1 * x1 + w2 * x2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[w1, w2]: [[-0.0854443  -0.07273219]]\n"
     ]
    }
   ],
   "source": [
    "print('[w1, w2]:', ridge_classifier.coef_ )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[w0]: [-0.31250723]\n"
     ]
    }
   ],
   "source": [
    "print('[w0]:',ridge_classifier.intercept_ )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 LogisticRegression\n",
    "The **[Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)** is another model that we'll train with the labeled data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a classifier with the default parameters. Some of them are shown below: \n",
    " - penalty = **L2**. Taken as a parameter of [regularization](/add-regularization-in-linear-regression-model/).  \n",
    " - tolerance = 0.0001 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "log_regressor = linear_model.LogisticRegression(random_state = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train the set with the method *fit()*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(random_state=1)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_regressor.fit(train_data, train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get predictions of the trained model. \n",
    "Besides prediction labels, the logistic regression build the **probability model**. \n",
    "That means that it may return a probability with which each object belongs to a certain class.\n",
    "Method *predict_proba()* returns the probability array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_predictions = log_regressor.predict(test_data)\n",
    "lr_proba_predictions = log_regressor.predict_proba(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Actual labels:\n",
      " [0 0 0 1 0 0 0 0 0 1 0 1 0 1 1 1 0 0 0 1 0 0 0 1 1 1 1 0 0 1]\n",
      "\n",
      "Predicted labels:\n",
      " [0 1 1 1 0 1 0 0 0 1 0 1 0 1 1 1 1 0 1 1 1 0 0 1 1 1 1 0 0 1]\n"
     ]
    }
   ],
   "source": [
    "print('Actual labels:\\n', test_labels)\n",
    "print('\\nPredicted labels:\\n', lr_predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We output the probability predictions.\n",
    "For each object we get 2 probability values. \n",
    "These define probability if it pertains to label \"0\" or \"1\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[9.99254667e-01 7.45333217e-04]\n",
      " [4.08279523e-01 5.91720477e-01]\n",
      " [4.90541791e-01 5.09458209e-01]\n",
      " [3.78296027e-03 9.96217040e-01]\n",
      " [7.32221196e-01 2.67778804e-01]\n",
      " [2.44262899e-01 7.55737101e-01]\n",
      " [9.93761113e-01 6.23888724e-03]\n",
      " [9.78405579e-01 2.15944205e-02]\n",
      " [9.55344987e-01 4.46550128e-02]\n",
      " [1.68318566e-01 8.31681434e-01]]\n"
     ]
    }
   ],
   "source": [
    "print(lr_proba_predictions[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now **rate the Logistic regression** and **RidgeClassifier models**; compare quality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic regression 0.8\n",
      "\n",
      "RidgeClassifier: 0.867\n"
     ]
    }
   ],
   "source": [
    "print('Logistic regression', round(metrics.accuracy_score(test_labels, lr_predictions), 3))\n",
    "print('\\nRidgeClassifier:', round(metrics.accuracy_score(test_labels, ridge_predictions), 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Quality assessment of cross-validation\n",
    "#### 3.1 Evaluate a score by cross-validation\n",
    "The library provides a useful function **[cross_val_score()](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html)** to evaluate a score of method by cross-validation.\n",
    " - [trained] method: **ridge_classifier**\n",
    " - data: blobs[0]\n",
    " - labels: blobs[1]\n",
    " - metric, **scoring** parameter. 'accuracy' - 1/N*&sum;[a(xi)==yi]  \n",
    " - cv -- a cross-validation splitting strategy.Cv = **K** parameter in the [KFold](/cross-validation-strategies-application/#1.-KFold) strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "ridge_scoring = model_selection.cross_val_score(ridge_classifier, blobs[0], blobs[1], scoring = 'accuracy', cv = 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_scoring = model_selection.cross_val_score(log_regressor, blobs[0], blobs[1], scoring = 'accuracy', cv = 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RidgeClassifier:\n",
      " [0.8 0.9 0.9 0.9 1.  1.  0.7 0.9 0.9 0.8]\n",
      "\n",
      "Logistic regression scoring:\n",
      " [0.8 0.9 0.9 0.9 1.  1.  0.7 0.9 0.9 0.8]\n"
     ]
    }
   ],
   "source": [
    "print('RidgeClassifier:\\n', ridge_scoring)\n",
    "print('\\nLogistic regression scoring:\\n', lr_scoring)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's view the scoring statistics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge mean:0.88, max:1.0, min:0.7, std:0.087\n"
     ]
    }
   ],
   "source": [
    "print('Ridge mean:{}, max:{}, min:{}, std:{}'.format(round(ridge_scoring.mean(), 3), ridge_scoring.max(), \n",
    "                                                     ridge_scoring.min(), round( ridge_scoring.std(), 3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log mean:0.88, max:1.0, min:0.7, std:0.087\n"
     ]
    }
   ],
   "source": [
    "print('Log mean:{}, max:{}, min:{}, std:{}'.format(round(lr_scoring.mean(), 3), lr_scoring.max(), \n",
    "                                                   lr_scoring.min(),  round( lr_scoring.std(), 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2 Evaluate a score by cross-validation with specified *scorer* и *cv_strategy*\n",
    "Suppose we want to **score a non-standart metric** and we want to **specify the cross-validation strategy**.\n",
    "\n",
    "We first define a *scorer* and specify cross-validation stratagy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scorer = metrics.make_scorer(metrics.accuracy_score)\n",
    "\n",
    "cv_strategy = model_selection.StratifiedShuffleSplit(n_splits=20, test_size = 0.3, random_state = 2)\n",
    "cv_strategy.get_n_splits(blobs[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "ridge_scoring = model_selection.cross_val_score(ridge_classifier, blobs[0], blobs[1], scoring = scorer, cv = cv_strategy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_scoring = model_selection.cross_val_score(log_regressor, blobs[0], blobs[1], scoring = scorer, cv = cv_strategy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge mean:0.87, max:1.0, min:0.7666666666666667, std:0.06\n"
     ]
    }
   ],
   "source": [
    "print('Ridge mean:{}, max:{}, min:{}, std:{}'.format(round(ridge_scoring.mean(), 3), ridge_scoring.max(), \n",
    "                                                     ridge_scoring.min(), round( ridge_scoring.std(), 3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log mean:0.87, max:1.0, min:0.7666666666666667, std:0.061\n"
     ]
    }
   ],
   "source": [
    "print('Log mean:{}, max:{}, min:{}, std:{}'.format(round(lr_scoring.mean(), 3), lr_scoring.max(), \n",
    "                                                   lr_scoring.min(),  round( lr_scoring.std(), 3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}