diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000..d6901fc Binary files /dev/null and b/.DS_Store differ diff --git a/.ipynb_checkpoints/BT2101_Default - EDIT-Copy1-checkpoint.ipynb b/.ipynb_checkpoints/BT2101_Default - EDIT-Copy1-checkpoint.ipynb new file mode 100644 index 0000000..fcf3597 --- /dev/null +++ b/.ipynb_checkpoints/BT2101_Default - EDIT-Copy1-checkpoint.ipynb @@ -0,0 +1,5439 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "-4Rm0wjQMUHi" + }, + "source": [ + "# BUILDING A DEFUALT DETECTION MODEL\n", + "\n", + "---\n", + "\n", + "\n", + "\n", + "## Table of Contents\n", + "1. Problem Description (Brief Write Up)\n", + "2. Exploratory Data Analysis (EDA)\n", + "3. Data Pre-processing\n", + "4. Model Selection\n", + "5. Evaluation\n", + "6. Discussion and Possible Improvements\n", + "\n", + "## 1. Problem Description\n", + "\n", + "The goal of this project is to predict a binary target feature (default or not) valued 0 (= not default) or 1 (= default). This project will cover the entire data science pipeline, from data analysis to model evaluation. We will be trying several models to predict default status, and choosing the most appropriate one at the end. \n", + "\n", + "The data set we will be working on contains payment information of 30,000 credit card holders obtained from a bank in Taiwan, and each data sample is described by 23 feature attributes and the binary target feature (default or not).\n", + "\n", + "The 23 explanatory attributes and their explanations (from the data provider) are as follows:\n", + "\n", + "### X1 - X5: Indivual attributes of customer\n", + "\n", + "X1: Amount of the given credit (NT dollar): it includes both the individual consumer credit and his/her family (supplementary) credit. \n", + "\n", + "X2: Gender (1 = male; 2 = female). \n", + "\n", + "X3: Education (1 = graduate school; 2 = university; 3 = high school; 4 = others). \n", + "\n", + "X4: Marital status (1 = married; 2 = single; 3 = others). \n", + "\n", + "X5: Age (year). \n", + "\n", + "### X6 - X11: Repayment history from April to Septemeber 2005\n", + "The measurement scale for the repayment status is: -1 = pay duly; 1 = payment delay for one month; 2 = payment delay for two months, . . . 8 = payment delay for eight months; 9 = payment delay for nine months and above.\n", + "\n", + "\n", + "X6 = the repayment status in September, 2005\n", + "\n", + "X7 = the repayment status in August, 2005\n", + "\n", + "X8 = the repayment status in July, 2005\n", + "\n", + "X9 = the repayment status in June, 2005\n", + "\n", + "X10 = the repayment status in May, 2005\n", + "\n", + "X11 = the repayment status in April, 2005. \n", + "\n", + "### X12 - X17: Amount of bill statement (NT dollar) from April to September 2005\n", + "\n", + "X12 = amount of bill statement in September, 2005; \n", + "\n", + "X13 = amount of bill statement in August, 2005\n", + "\n", + ". . .\n", + "\n", + "X17 = amount of bill statement in April, 2005. \n", + "\n", + "### X18 - X23: Amount of previous payment (NT dollar)\n", + "X18 = amount paid in September, 2005\n", + "\n", + "X19 = amount paid in August, 2005\n", + "\n", + ". . .\n", + "\n", + "X23 = amount paid in April, 2005. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "aM_aIU6UPHe4" + }, + "source": [ + "## EDA\n", + "\n", + "In this section we will explore the data set, its shape and its features to get an idea of the data.\n", + "\n", + "### Importing packages and the dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "Is0wEkk3LJCt" + }, + "outputs": [], + "source": [ + "import pandas as pd" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "x_Z7u_9vRC5m" + }, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import seaborn as sns" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "KhmX9KWWyrUW" + }, + "outputs": [], + "source": [ + "url = 'https://raw.githubusercontent.com/reonho/bt2101disrudy/master/card.csv'\n", + "df = pd.read_csv(url, header = 1, index_col = 0)\n", + "# Dataset is now stored in a Pandas Dataframe" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 255 + }, + "colab_type": "code", + "id": "FhJ2eAxVQhBm", + "outputId": "7f79bb40-f08f-4709-e7d4-1f747bb8af2f" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...BILL_AMT4BILL_AMT5BILL_AMT6PAY_AMT1PAY_AMT2PAY_AMT3PAY_AMT4PAY_AMT5PAY_AMT6Y
ID
1200002212422-1-1-2...000068900001
212000022226-12000...3272345532610100010001000020001
3900002223400000...1433114948155491518150010001000100050000
4500002213700000...2831428959295472000201912001100106910000
55000012157-10-100...2094019146191312000366811000090006896790
\n", + "

5 rows × 24 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 \\\n", + "ID \n", + "1 20000 2 2 1 24 2 2 -1 -1 \n", + "2 120000 2 2 2 26 -1 2 0 0 \n", + "3 90000 2 2 2 34 0 0 0 0 \n", + "4 50000 2 2 1 37 0 0 0 0 \n", + "5 50000 1 2 1 57 -1 0 -1 0 \n", + "\n", + " PAY_5 ... BILL_AMT4 BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 PAY_AMT3 \\\n", + "ID ... \n", + "1 -2 ... 0 0 0 0 689 0 \n", + "2 0 ... 3272 3455 3261 0 1000 1000 \n", + "3 0 ... 14331 14948 15549 1518 1500 1000 \n", + "4 0 ... 28314 28959 29547 2000 2019 1200 \n", + "5 0 ... 20940 19146 19131 2000 36681 10000 \n", + "\n", + " PAY_AMT4 PAY_AMT5 PAY_AMT6 Y \n", + "ID \n", + "1 0 0 0 1 \n", + "2 1000 0 2000 1 \n", + "3 1000 1000 5000 0 \n", + "4 1100 1069 1000 0 \n", + "5 9000 689 679 0 \n", + "\n", + "[5 rows x 24 columns]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#rename the target variable to \"Y\" for convenience\n", + "df[\"Y\"] = df[\"default payment next month\"] \n", + "df = df.drop(\"default payment next month\", axis = 1)\n", + "df0 = df #backup of df\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "zcuPyfM86AKj", + "outputId": "89bb2e37-a3ba-43e5-99a7-6917f24acc3f", + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data has 24 Columns and 30000 Rows\n" + ] + } + ], + "source": [ + "size = df.shape\n", + "print(\"Data has {} Columns and {} Rows\".format(size[1], size[0]))" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "QVaSnvJP3VbO", + "outputId": "4bf72e64-2d0c-41c3-85b5-3bd6e70920d3" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check for null values\n", + "df.isnull().any().sum() " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "eVYXnIGH9Zq6" + }, + "source": [ + "From the above analyses, we observe that:\n", + "1. The data indeed has 30000 rows and 24 columns\n", + "2. There are no null values\n", + "\n", + "We will now explore the features more in depth." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "W6hhPNl1Slau" + }, + "source": [ + "### Exploring the features" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "1Sp2F3gzXX2F" + }, + "source": [ + "**1) Exploring target attribute:**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "colab_type": "code", + "id": "DCSEICWwXWgX", + "outputId": "9545da56-f31b-48f2-a271-db0e18677beb" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "defaults : 22.12 %\n", + "non defaults : 77.88000000000001 %\n" + ] + }, + { + "data": { + "text/plain": [ + "Text(0, 0.5, 'Frequency')" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAapElEQVR4nO3deZxcZZ3v8c+XgALjAg6LCEhQo4iMsgTEcQNRBBwFRhz16oBeFHVwvDqLouOIG3fQGZfBBQcxF3BUFBFFQTGgiNcrSFBkEZWIKDEBoiyyKYu/+0c9LWWnOl056Uqn6c/79apXn/Oc55zzO51Kffs8p+pUqgpJkrpYZ7oLkCTNXIaIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEpNWU5NAkX27T6yepJFtNd10ASY5OcnybfnSSm6Zw2yckeWOb3ifJ4inc9jOT/HCqtqfRMUTUWZJb+x5/SHJH3/xL1nAtq/3ineTaJE/pm98uyd2TrVdVn6iq53bd77gazk/y0qnY1nhV9dOq2miIGl6d5Owhtveyqnrv6tY16N+uqs6uqies7rY1eutOdwGauarqAWPTSa4GXlFVk774DJJk3aqa9AV7bTNT615dSeZU1T3TXYemn2ciGpkkT05yQZKbkyxN8oEk67ZlY399vibJz4DLWvtzklyZ5KYkHxz/l3mSVyX5SZIbkpyRZMu26Lz28yftTOiAAfVsl+Tctu7yJCcmeWBbdgqwGfD1tv7r2jbn9J1d7dT+Sv9Gko8kuRE4YoK/3A9IcnXbz1FJ0vbzx+GlvprubtPvA3YFjm/7e19r36Ht88YkVww6tr7tPSrJd5LckuSrwMaD9tXmX9lqvCXJVUlekGQn4IPAHq2Ga1vfk5Mck+TrSW4DntTa3jpu/+9ov9+rkrygr338v2P/72yFf7vxw2NJ/iLJt9vz4pIk+/YtO7k9V85qx/KdJNtM9DvSFKsqHz5W+wFcDTxzXNtu9F4U5wCPBBYDr27L1gcKOAPYCNgAeChwK/BXwHrAG4G7gJe2dV4EXAE8ui1/N/DNcdvbaiU1bgc8A7hf29f5wNF9y68FnjKu/93jtvFq4G7gle24NmhtZ4+r46x2XNsCV/Udw9HA8RPto9X00r75BwHLgJe0/e0K3AA8aoJj/D7wb+0Y9wJuH9tf/77ohctNwCPb/JbAY/uO8exx2z257feJ9P74vH9re2tbvk/7vYzt+5lt39tOcFyDfmdb9S3fB1jct/wXwD+2f/dnt+fJtn21XQ/s3JZ/Hjhhuv9PzJaHZyIamar6XlVdWFX3VNXPgOOBp4/rdlRV3VRVdwDPAy6sqq9U1V3AfwA39vV9FfDu6o3t3wW8A3hKks2HrOfHVfWNqrqzqq6l9xf3+HqGcVVVfbwd1x0T9Pm3dlw/Bz4MvLjDfgAOBC6rqk+1/V0IfBl4/viOSR4NbA+8ox3jOcDXJtn+DknWr6pfVdUVk/T9fFVdUFV/qKrfD1h+d9++zwbOBg6a7ACH8NT28/1VdVdVnQUsBF7Y1+dzVfX99rz4NLDjFOxXQzBENDJJtk/y1STXJfkt8DZgk3Hdrumbflj/fFX9AfhV3/JtgI+1IY2bgOX0XriGupie5GFJTknyq1bP8QPqGcY1k3f5kz6/oHdsXWwDPG3smNtxPx/YYkDfhwHLq+p34/a9gqq6kd7ZzeuAa5OcnuRRk9Qy2XEP2nfX4+73MOCXVdV/t9hf0Dt7GnNt3/TtwAPQGmGIaJQ+Tm945ZFV9SDgnUDG9el/YVhGXyAkWYc/faG4BnhZVW3U99igqi4at52J/DtwG7BDq+cV4+oZv42JtjnMvrbum344sLRN3wZs2LfsoZNs+xrg6+OO+QFV9foB+1wGbJJk/XH7HqiqzqiqvWgv0sCxE9QwUW3jDdr3MMc92XaXsuJxPJw//QND08QQ0Sg9ELi5qm5N8jh61xFW5nTgiUn2axfg/4G+C8PAx4C3JnkMQJKNkzwfoA2v3Aw8YpJ6bgV+m+Thbfv9rhu3/vX0LqxP+EK8Em9K8uAkc4HXAp9t7RcDeybZMsnGwJsmqeGLwE5JXphkvST3S7J7G7oa76fAj4F/bf32pHdtYQVt/89JsiHwe3q/l7F3W10HbJ1kvVU85vX69v0M4FnAqX3HfVB6b6jYDnjZ2EpD/Nt9G1gnyeuTrJvkWcDewCmrWJ9GwBDRKL0BeEWSW4GPcO8L6UBVtYzetYNjgF/TOyu5lN6LHFX1GXrXF77QhqMupvdCNeZtwClt2Od5A3bxNuAp9F6wTuPeF7gxRwFHtfVf24Z83gtc1NpWZZz9DOCHwCJ6L3b/3df+FeBH9C42f3Hceh8ADm7vxHpvq+HZwMvpnWkspfeGghVe4NtwzwuBPeldBH9j337HmwO8md4w0G/oXbD/+7bsa/TeKHF9kiWrcMxX0xtevBZYALy8qq5qy95L7yMFy4HjBtQ14b9dGyL7K3rXV34DvB94YbvOpmmWPx1mlNYe7WzkWuC5VfXd6a5H0oo8E9FaJcm+bRhofeBIehdJL5rmsiRNwBDR2uZpwM/pXY/YCziwqu6c3pIkTcThLElSZ56JSJI6m3U3YNxkk01q7ty5012GJM0oF1100a+ratPx7bMuRObOncuiRYumuwxJmlGSDLz7gcNZkqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTOZt0n1meKuUecMd0l3GdcffRzprsE6T7LMxFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmcjC5EkWyf5ZpIrklye5H+19ockWZjkyvZz49aeJMckWZzkkiQ7923rkNb/yiSH9LXvkuTSts4xSTKq45EkrWiUZyJ3A/9YVY8FdgcOT7I9cARwTlXNA85p8wD7AvPa4zDgWOiFDnAk8ERgN+DIseBpfQ7rW2+fER6PJGmckYVIVS2rqu+36VuAK4Atgf2BE1u3E4ED2vT+wEnVcz6wUZItgGcDC6vqhqq6EVgI7NOWPaiqvltVBZzUty1J0hqwRq6JJJkL7ARcAGxeVcugFzTAZq3blsA1fastaW0ra18yoF2StIaMPESSPAA4FXh9Vf12ZV0HtFWH9kE1HJZkUZJFy5cvn6xkSdKQRhoiSdajFyCfqqovtObr2lAU7ef1rX0JsHXf6lsBSydp32pA+wqq6riqml9V8zfddNPVOyhJ0h+N8t1ZAT4BXFFV7+9bdDow9g6rQ4Av9bUf3N6ltTtwcxvuOgvYO8nG7YL63sBZbdktSXZv+zq4b1uSpDVg3RFu+8nA3wKXJrm4tb0FOBr4XJJDgV8CL2jLzgT2AxYDtwMvB6iqG5K8C7iw9XtnVd3Qpl8DnABsAHy1PSRJa8jIQqSq/i+Dr1sA7DWgfwGHT7CtBcCCAe2LgB1Wo0xJ0mrwE+uSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1NnIQiTJgiTXJ7msr+3tSX6V5OL22K9v2ZuTLE7ykyTP7mvfp7UtTnJEX/u2SS5IcmWSzya536iORZI02CjPRE4A9hnQ/oGq2rE9zgRIsj3wIuBxbZ2PJpmTZA7wEWBfYHvgxa0vwHvatuYBNwKHjvBYJEkDjCxEquo84IYhu+8PnFxVv6+qnwOLgd3aY3FVXVVVdwInA/snCfAM4PNt/ROBA6b0ACRJkxoqRJLsMIX7fG2SS9pw18atbUvgmr4+S1rbRO1/DtxUVXePax8oyWFJFiVZtHz58qk6Dkma9YY9E/lYku8l+bskG63G/o4FHgnsCCwD3tfaM6BvdWgfqKqOq6r5VTV/0003XbWKJUkTGipEquopwEuArYFFST6d5FmrurOquq6q7qmqPwAfpzdcBb0zia37um4FLF1J+6+BjZKsO65dkrQGDX1NpKquBN4KvAl4OnBMkh8n+etht5Fki77ZA4Gxd26dDrwoyf2TbAvMA74HXAjMa+/Euh+9i++nV1UB3wQOausfAnxp2DokSVNj3cm7QJLHAy8HngMsBJ5bVd9P8jDgu8AXBqzzGWAPYJMkS4AjgT2S7Ehv6Olq4FUAVXV5ks8BPwLuBg6vqnvadl4LnAXMARZU1eVtF28CTk7ybuAHwCdW+eglSatlqBABPkxv+OktVXXHWGNVLU3y1kErVNWLBzRP+EJfVUcBRw1oPxM4c0D7Vdw7HCZJmgbDhsh+wB19ZwfrAOtX1e1V9cmRVSdJWqsNe03kbGCDvvkNW5skaRYbNkTWr6pbx2ba9IajKUmSNFMMGyK3Jdl5bCbJLsAdK+kvSZoFhr0m8nrglCRjn8XYAnjhaEqSJM0UQ4VIVV2YZDvgMfQ+Lf7jqrprpJVJktZ6w56JAOwKzG3r7JSEqjppJFVJkmaEYT9s+El697y6GLinNRdgiEjSLDbsmch8YPt2uxFJkoDh3511GfDQURYiSZp5hj0T2QT4UZLvAb8fa6yq542kKknSjDBsiLx9lEVIkmamYd/i+60k2wDzqursJBvSu6uuJGkWG/brcV9J7/vM/6s1bQl8cVRFSZJmhmEvrB8OPBn4LfzxC6o2G1VRkqSZYdgQ+X1V3Tk2076W1rf7StIsN2yIfCvJW4AN2nernwJ8eXRlSZJmgmFD5AhgOXApva+0PZPe961LkmaxYd+d9Qd6X4/78dGWI0maSYa9d9bPGXANpKoeMeUVSZJmjFW5d9aY9YEXAA+Z+nIkSTPJUNdEquo3fY9fVdUHgWeMuDZJ0lpu2OGsnftm16F3ZvLAkVQkSZoxhh3Oel/f9N3A1cDfTHk1kqQZZdh3Z+056kIkSTPPsMNZ/7Cy5VX1/qkpR5I0k6zKu7N2BU5v888FzgOuGUVRkqSZYVW+lGrnqroFIMnbgVOq6hWjKkyStPYb9rYnDwfu7Ju/E5g75dVIkmaUYc9EPgl8L8lp9D65fiBw0siqkiTNCMO+O+uoJF8FntqaXl5VPxhdWZKkmWDY4SyADYHfVtV/AkuSbDuimiRJM8SwX497JPAm4M2taT3gv0dVlCRpZhj2TORA4HnAbQBVtRRveyJJs96wIXJnVRXtdvBJ/mx0JUmSZophQ+RzSf4L2CjJK4GzmeQLqpIsSHJ9ksv62h6SZGGSK9vPjVt7khyTZHGSS/pv+JjkkNb/yiSH9LXvkuTSts4xSbIqBy5JWn3D3gr+P4DPA6cCjwHeVlUfmmS1E4B9xrUdAZxTVfOAc9o8wL7AvPY4DDgWeqEDHAk8EdgNOHIseFqfw/rWG78vSdKITfoW3yRzgLOq6pnAwmE3XFXnJZk7rnl/YI82fSJwLr0L9vsDJ7Uhs/OTbJRki9Z3YVXd0GpZCOyT5FzgQVX13dZ+EnAA8NVh65Mkrb5Jz0Sq6h7g9iQPnoL9bV5Vy9p2lwGbtfYt+dP7cC1pbStrXzKgfaAkhyVZlGTR8uXLV/sgJEk9w35i/XfApe1M4Laxxqp63RTVMeh6RnVoH6iqjgOOA5g/f/6E/SRJq2bYEDmjPVbXdUm2qKplbbjq+ta+BNi6r99WwNLWvse49nNb+1YD+kuS1qCVhkiSh1fVL6vqxCna3+nAIcDR7eeX+tpfm+RkehfRb25Bcxbwv/supu8NvLmqbkhyS5LdgQuAg4HJLvRLkqbYZNdEvjg2keTUVdlwks8A3wUek2RJkkPphcezklwJPKvNA5wJXAUspvfW4b8DaBfU3wVc2B7vHLvIDrwGOL6t8zO8qC5Ja9xkw1n91x4esSobrqoXT7BorwF9Czh8gu0sABYMaF8E7LAqNUmSptZkZyI1wbQkSZOeiTwhyW/pnZFs0KZp81VVDxppdZKktdpKQ6Sq5qypQiRJM8+qfJ+IJEl/whCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM7Wne4CJM0sc484Y7pLuE+5+ujnTHcJq8UzEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKmzaQmRJFcnuTTJxUkWtbaHJFmY5Mr2c+PWniTHJFmc5JIkO/dt55DW/8okh0zHsUjSbDadZyJ7VtWOVTW/zR8BnFNV84Bz2jzAvsC89jgMOBZ6oQMcCTwR2A04cix4JElrxto0nLU/cGKbPhE4oK/9pOo5H9goyRbAs4GFVXVDVd0ILAT2WdNFS9JsNl0hUsDXk1yU5LDWtnlVLQNoPzdr7VsC1/Stu6S1TdS+giSHJVmUZNHy5cun8DAkaXabrlvBP7mqlibZDFiY5Mcr6ZsBbbWS9hUbq44DjgOYP3/+wD6SpFU3LWciVbW0/bweOI3eNY3r2jAV7ef1rfsSYOu+1bcClq6kXZK0hqzxEEnyZ0keODYN7A1cBpwOjL3D6hDgS236dODg9i6t3YGb23DXWcDeSTZuF9T3bm2SpDVkOoazNgdOSzK2/09X1deSXAh8LsmhwC+BF7T+ZwL7AYuB24GXA1TVDUneBVzY+r2zqm5Yc4chSVrjIVJVVwFPGND+G2CvAe0FHD7BthYAC6a6RknScNamt/hKkmYYQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHU240MkyT5JfpJkcZIjprseSZpNZnSIJJkDfATYF9geeHGS7ae3KkmaPWZ0iAC7AYur6qqquhM4Gdh/mmuSpFlj3ekuYDVtCVzTN78EeOL4TkkOAw5rs7cm+ckaqG022AT49XQXMZm8Z7or0DTx+Tm1thnUONNDJAPaaoWGquOA40ZfzuySZFFVzZ/uOqRBfH6uGTN9OGsJsHXf/FbA0mmqRZJmnZkeIhcC85Jsm+R+wIuA06e5JkmaNWb0cFZV3Z3ktcBZwBxgQVVdPs1lzSYOEWpt5vNzDUjVCpcQJEkaykwfzpIkTSNDRJLUmSGiVZbkhCQHTdJnuyQXJ/lBkkd22Mfbk/xTm35Zkod1rVczX//zYYLlmya5oD3fntph+y9L8uE2fYB3vhieIaJROQD4UlXtVFU/W81tvQwwRLQyewE/bs+3b6/mtg6gdxslDcEQuQ9JMjfJFUk+nuTyJF9PskFbtmOS85NckuS0JBu39nOTvCfJ95L8dNBfcen5cJIfJTkD2Kxv2S5JvpXkoiRnJdkiyX7A64FXJPlm6/fF1ufydgeBsfVv7Zs+KMkJ4/Z9EDAf+FQ7s9lgKn9nWnsl+Zd2c9Wzgce0tkcm+Vp7Ln27nfHuCLwX2G/sOZLk2CSL2vPtHX3bvDrJJm16fpJzx+3zL4HnAf/etrXKZ9GzjSFy3zMP+EhVPQ64CXh+az8JeFNVPR64FDiyb511q2o3ei/8/e1jDqT3n/gvgFcCfwmQZD3gQ8BBVbULsAA4qqrOBD4GfKCq9mzb+J+tz3zgdUn+fJiDqarPA4uAl1TVjlV1xzDraWZLsgu9z33tBPw1sGtbdBzw9+259E/AR6vqYuBtwGf7niP/0j6t/njg6UkeP8x+q+r/0fus2T+3ba3uWfR93oz+nIgG+nn7TwVwETA3yYOBjarqW639ROCUvnW+0N9/wDafBnymqu4Blib5Rmt/DLADsDAJ9D6rs2yCul6X5MA2vTW9sPvNqhyYZpWnAqdV1e0ASU4H1qf3B8wp7fkGcP8J1v+bdsa7LrAFveGpS0Za8SxliNz3/L5v+h5gmOGfsXXuYeLnxKAPFAW4vKqetLKNJ9kDeCbwpKq6vQ0hrD9gu+sj3Wv8c24d4Kaq2nFlKyXZlt5Zyq5VdWMbIh17bt3NvSMwPt+mgMNZs0BV3Qzc2He942+Bb61klfHOA16UZE6SLYCxIaqfAJsmeRL0hreSPG7A+g8GbmwBsh2we9+y65I8Nsk69IbNBrkFeOAq1KuZ7zzgwHZ944HAc4HbgZ8neQH88VrdEwas+yDgNuDmJJvT+76hMVcDu7Tp5zOYz7dVYIjMHofQu1h4CbAj8M5VWPc04Ep611KOpQVQ+w6Xg4D3JPkhcDHtesk4XwPWbft+F3B+37IjgK8A32DiobATgI95YX32qKrvA5+l95w6FRh7x9VLgEPb8+1yBnx/UFX9EPhBW74A+E7f4ncA/5nk2/TOvAc5Gfjnrm9Pn2287YkkqTPPRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISKNSJKHJjk5yc/afcfOTPLoJJdNd23SVPET69IIpHdfjtOAE6vqRa1tR2DzaS1MmmKeiUijsSdwV1V9bKyh3dPsmrH5dtflbyf5fnuM3dhyiyTntQ9XXpbkqe1uASe0+UuTvGHNH5K0Is9EpNHYgd4NLVfmeuBZVfW7JPOAz9C7y/H/AM6qqqOSzAE2pHeXgS2rageAJBuNrnRpeIaINH3WAz7chrnuAR7d2i8EFrRb7X+xqi5OchXwiCQfAs4Avj4tFUvjOJwljcbl3Hujv4m8AbgOeAK9M5D7AVTVefRuv/8r4JNJDq6qG1u/c4HDgeNHU7a0agwRaTS+Adw/ySvHGpLsCmzT1+fBwLKq+gO9OyvPaf22Aa6vqo8DnwB2bt/Gt05VnQr8K7DzmjkMaeUczpJGoKqqfQnXB5McAfyO3m3IX9/X7aPAqe3W5t+kd/tygD3o3UX2LuBW4GBgS+D/tFvmA7x55AchDcG7+EqSOnM4S5LUmSEiSerMEJEkdWaISJI6M0QkSZ0ZIpKkzgwRSVJn/x+T6PNo2DnX9gAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "All = df.shape[0]\n", + "default = df[df['Y'] == 1]\n", + "nondefault = df[df['Y'] == 0]\n", + "\n", + "x = len(default)/All\n", + "y = len(nondefault)/All\n", + "\n", + "print('defaults :',x*100,'%')\n", + "print('non defaults :',y*100,'%')\n", + "\n", + "# plotting target attribute against frequency\n", + "labels = ['non default','default']\n", + "classes = pd.value_counts(df['Y'], sort = True)\n", + "classes.plot(kind = 'bar', rot=0)\n", + "plt.title(\"Target attribute distribution\")\n", + "plt.xticks(range(2), labels)\n", + "plt.xlabel(\"Class\")\n", + "plt.ylabel(\"Frequency\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "tysR0WHw4SGU" + }, + "source": [ + "**2) Exploring categorical attributes**\n", + "\n", + "Categorical attributes are:\n", + "- Sex\n", + "- Education\n", + "- Marriage" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 323 + }, + "colab_type": "code", + "id": "s61SSRII00UB", + "outputId": "69df981f-8c36-43a9-d155-a6553adbba0b", + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2 60.373333\n", + "1 39.626667\n", + "Name: SEX, dtype: float64\n", + "--------------------------------------------------------\n", + "2 46.766667\n", + "1 35.283333\n", + "3 16.390000\n", + "5 0.933333\n", + "4 0.410000\n", + "6 0.170000\n", + "0 0.046667\n", + "Name: EDUCATION, dtype: float64\n", + "--------------------------------------------------------\n", + "2 53.213333\n", + "1 45.530000\n", + "3 1.076667\n", + "0 0.180000\n", + "Name: MARRIAGE, dtype: float64\n" + ] + } + ], + "source": [ + "print(df[\"SEX\"].value_counts().apply(lambda r: r/All*100))\n", + "print(\"--------------------------------------------------------\")\n", + "print(df[\"EDUCATION\"].value_counts().apply(lambda r: r/All*100))\n", + "print(\"--------------------------------------------------------\")\n", + "print(df[\"MARRIAGE\"].value_counts().apply(lambda r: r/All*100))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Uudv5XE828nb" + }, + "source": [ + "**Findings**\n", + "\n", + "- Categorical variable SEX does not seem to have any missing/extra groups, and it is separated into Male = 1 and Female = 2\n", + "- Categorical variable MARRIAGE seems to have unknown group = 0, which could be assumed to be missing data, with other groups being Married = 1, Single = 2, Others = 3\n", + "- Categorical variable EDUCATION seems to have unknown group = 0,5,6, with other groups being graduate school = 1, university = 2, high school = 3, others = 4 " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 357 + }, + "colab_type": "code", + "id": "U3IJzhwwe5KK", + "outputId": "cb61e112-a3ec-4a37-c1a0-0ffc9ebcbf89", + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Total target attributes:\n", + "non defaults : 77.88000000000001 %\n", + "defaults : 22.12 %\n", + "--------------------------------------------------------\n", + "SEX Male Female\n", + "Y \n", + "non defaults 75.832773 79.223719\n", + "defaults 24.167227 20.776281\n", + "--------------------------------------------------------\n", + "EDUCATION 0 1 2 3 4 5 \\\n", + "Y \n", + "non defaults 100.0 80.765234 76.265146 74.842384 94.308943 93.571429 \n", + "defaults 0.0 19.234766 23.734854 25.157616 5.691057 6.428571 \n", + "\n", + "EDUCATION 6 \n", + "Y \n", + "non defaults 84.313725 \n", + "defaults 15.686275 \n", + "--------------------------------------------------------\n", + "MARRIAGE unknown married single others\n", + "Y \n", + "non defaults 90.740741 76.528296 79.071661 73.993808\n", + "defaults 9.259259 23.471704 20.928339 26.006192\n" + ] + } + ], + "source": [ + "#proportion of target attribute (for reference)\n", + "print('Total target attributes:')\n", + "print('non defaults :',y*100,'%')\n", + "print('defaults :',x*100,'%')\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with Sex\n", + "sex_target = pd.crosstab(df[\"Y\"], df[\"SEX\"]).apply(lambda r: r/r.sum()*100).rename(columns = {1: \"Male\", 2: \"Female\"}, index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(sex_target)\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with education\n", + "education_target = pd.crosstab(df[\"Y\"], df[\"EDUCATION\"]).apply(lambda r: r/r.sum()*100).rename(index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(education_target)\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with marriage\n", + "marriage_target = pd.crosstab(df[\"Y\"], df[\"MARRIAGE\"]).apply(lambda r: r/r.sum()*100).rename(columns = {0: \"unknown\",1: \"married\", 2: \"single\", 3: \"others\"},index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(marriage_target)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "kOriUQ0wxbhD" + }, + "source": [ + "**Conclusion**\n", + "\n", + "From the analyses above we conclude that\n", + "\n", + "1. The categorical data is noisy - EDUCATION and MARRIAGE contains unexplained/anomalous data.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "77GAylGWnPJO" + }, + "source": [ + "**3) Analysis of Numerical Attributes**\n", + "\n", + "The numerical attributes are:\n", + " \n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 669 + }, + "colab_type": "code", + "id": "HEcCl5Rj-N0T", + "outputId": "a59f7092-366e-47ec-c67b-e18f02d84ac4", + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
012345678910111213141516171819
0LIMIT_BALAGEPAY_0PAY_2PAY_3PAY_4PAY_5PAY_6BILL_AMT1BILL_AMT2BILL_AMT3BILL_AMT4BILL_AMT5BILL_AMT6PAY_AMT1PAY_AMT2PAY_AMT3PAY_AMT4PAY_AMT5PAY_AMT6
\n", + "
" + ], + "text/plain": [ + " 0 1 2 3 4 5 6 7 8 \\\n", + "0 LIMIT_BAL AGE PAY_0 PAY_2 PAY_3 PAY_4 PAY_5 PAY_6 BILL_AMT1 \n", + "\n", + " 9 10 11 12 13 14 15 \\\n", + "0 BILL_AMT2 BILL_AMT3 BILL_AMT4 BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 \n", + "\n", + " 16 17 18 19 \n", + "0 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6 " + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#printing numerical attributes\n", + "pd.DataFrame(df.drop(['SEX', 'EDUCATION', 'MARRIAGE','Y'], axis = 1).columns).transpose()" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
LIMIT_BAL26245.00.2843370.2378750.00.0816330.2244900.4081631.0
AGE26245.00.3685950.2324220.00.1842110.3421050.5263161.0
PAY_026245.00.3721090.7657300.00.0000000.0000000.0000008.0
PAY_226245.00.3373210.8148780.00.0000000.0000000.0000008.0
PAY_326245.00.3246330.8114910.00.0000000.0000000.0000008.0
PAY_426245.00.2782240.7863140.00.0000000.0000000.0000008.0
PAY_526245.00.2387500.7439230.00.0000000.0000000.0000008.0
PAY_626245.00.2432080.7403450.00.0000000.0000000.0000008.0
BILL_AMT126245.00.2001680.1893800.00.0575900.1229230.2652971.0
BILL_AMT226245.00.2549170.1745860.00.1236710.1857490.3089291.0
BILL_AMT326245.00.2981590.1609770.00.1777390.2370200.3517731.0
BILL_AMT426245.00.3176480.1568050.00.2018360.2611940.3706151.0
BILL_AMT526245.00.3300720.1554220.00.2174530.2741640.3743861.0
BILL_AMT626245.00.4279510.1332430.00.3326990.3777600.4635631.0
PAY_AMT126245.00.0727790.1059660.00.0152700.0413830.0880841.0
PAY_AMT226245.00.0551330.0923160.00.0091910.0306370.0615961.0
PAY_AMT326245.00.0618930.1056440.00.0062510.0302270.0694351.0
PAY_AMT426245.00.0617080.1070400.00.0033210.0265660.0690271.0
PAY_AMT526245.00.0628740.1075020.00.0018550.0275580.0710461.0
PAY_AMT626245.00.0553390.1033570.00.0000000.0224120.0601761.0
S_026245.0-0.1335870.879876-2.0-1.0000000.0000000.0000001.0
S_226245.0-0.3004380.883472-2.0-1.0000000.0000000.0000001.0
S_326245.0-0.3273000.895264-2.0-1.0000000.0000000.0000001.0
S_426245.0-0.3644120.886115-2.0-1.0000000.0000000.0000001.0
S_526245.0-0.3959990.877789-2.0-1.0000000.0000000.0000001.0
S_626245.0-0.4281580.900723-2.0-1.0000000.0000000.0000001.0
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "LIMIT_BAL 26245.0 0.284337 0.237875 0.0 0.081633 0.224490 0.408163 1.0\n", + "AGE 26245.0 0.368595 0.232422 0.0 0.184211 0.342105 0.526316 1.0\n", + "PAY_0 26245.0 0.372109 0.765730 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_2 26245.0 0.337321 0.814878 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_3 26245.0 0.324633 0.811491 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_4 26245.0 0.278224 0.786314 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_5 26245.0 0.238750 0.743923 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_6 26245.0 0.243208 0.740345 0.0 0.000000 0.000000 0.000000 8.0\n", + "BILL_AMT1 26245.0 0.200168 0.189380 0.0 0.057590 0.122923 0.265297 1.0\n", + "BILL_AMT2 26245.0 0.254917 0.174586 0.0 0.123671 0.185749 0.308929 1.0\n", + "BILL_AMT3 26245.0 0.298159 0.160977 0.0 0.177739 0.237020 0.351773 1.0\n", + "BILL_AMT4 26245.0 0.317648 0.156805 0.0 0.201836 0.261194 0.370615 1.0\n", + "BILL_AMT5 26245.0 0.330072 0.155422 0.0 0.217453 0.274164 0.374386 1.0\n", + "BILL_AMT6 26245.0 0.427951 0.133243 0.0 0.332699 0.377760 0.463563 1.0\n", + "PAY_AMT1 26245.0 0.072779 0.105966 0.0 0.015270 0.041383 0.088084 1.0\n", + "PAY_AMT2 26245.0 0.055133 0.092316 0.0 0.009191 0.030637 0.061596 1.0\n", + "PAY_AMT3 26245.0 0.061893 0.105644 0.0 0.006251 0.030227 0.069435 1.0\n", + "PAY_AMT4 26245.0 0.061708 0.107040 0.0 0.003321 0.026566 0.069027 1.0\n", + "PAY_AMT5 26245.0 0.062874 0.107502 0.0 0.001855 0.027558 0.071046 1.0\n", + "PAY_AMT6 26245.0 0.055339 0.103357 0.0 0.000000 0.022412 0.060176 1.0\n", + "S_0 26245.0 -0.133587 0.879876 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_2 26245.0 -0.300438 0.883472 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_3 26245.0 -0.327300 0.895264 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_4 26245.0 -0.364412 0.886115 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_5 26245.0 -0.395999 0.877789 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_6 26245.0 -0.428158 0.900723 -2.0 -1.000000 0.000000 0.000000 1.0" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.drop(['SEX', 'EDUCATION', 'MARRIAGE','Y'], axis=1).describe().transpose()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Analysis of PAY_0 to PAY_6**\n", + "\n", + "We observe that the minimum value of PAY_0 to PAY_6 is -2. The dataset's author has explained these factors (PAY_0 to PAY_6) as the number of months of payment delay, that is, 1= payment delay of one month; 2= payment delay of two months and so on. \n", + "\n", + "However, the presence of -2, -1 in these columns indicates that\n", + "1. There is anomalous data, OR \n", + "2. The numbers do not strictly correspond to the number of months of payment delay. \n", + "\n", + "This means we must conduct some data transformation.\n", + "\n", + "According to **(link)**, the numeric value in these attributes shows the past history of a credit card holder, where -2 means: No consumption of credit card, -1 means that holder paid the full balance, and 0 means the use of revolving credit.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 669 + }, + "colab_type": "code", + "id": "awXnqvLOS-wB", + "outputId": "a77b53b8-011e-4f53-b7b7-20d80bbc1777", + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df7BcZZ3n8ffHIC4VpEiM3IoEDboZZ/mhCHcIrlUz17EIAZwKlj+WHyXBQWMpKWdXtoowyxYK6Ga2lnXBHzhRY0ItENmpYZNRNGZZe3FnRZMoS4KARLzKJUiEQJaLOBj87h/naWgufW/f7j7d/Zx7P6+qru5++ulznpP+5vme55znnqOIwMzMLDevGHQDzMzMmnGCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzMyy5AQ1TZJGJT0raVzSY5K+Lunwhs83SDoo6XUNZaenugsayl4l6T5JH22xvpMk7ZT02/R8Um+2zHqlnzEj6Y8kbZb0G0n7JW2V9ObebZ2Vrc/xskDSP0p6QtJTkn4g6R2927rOOEG15y8i4nDgZOBPgCsAJM0F3gscAC6oV46IbcA3gesalnEF8CiwbrKVSDoU2Az8V2AesBHYnMqtWvoSM8CRwBbgzcAQ8COKGLJq6Ve8jAN/CbyWoo/5G+AfJB1S2paUwAmqAxHxCPBt4IRU9F7gKeAqYOWE6p8E/kzS2ZJOAFYDH4mprzE1AhwC/JeI+KeIuB4Q8OflbYX1U69jJiJ+FBFfi4j9EfF74HPAmyW9puxtsd7rQ7z8LiIeiIg/UPQtz1Mkqvnlbkl3nKA6IOkY4CzgJ6loJXALsAn4Y0kn1+tGxAHgY8CXgfXApyPi5y1WcTxwz4QAuyeVWwX1IWYm+lPg1xHxRLdtt/7rV7xIugf4HcXo+6sRsa+0jSiBE1R7/rukp4D/Dfwv4LOSXg+8E7g5Ih4D7mDCHk5E/ANwF8W/9/XTWM/hFEP5RgeAV3fXfBuAfsXMCyQtAr5IsWdt1dLXeImItwBHAOendWbFCao950TEkRHxhoj4eEQ8C3wQuC8i7k51bgLOl/TKCd+9F7g/DalbGacImkZHAE9303gbiH7FDACSXgt8F/hSRNxSxgZYX/U1XuCFw323AGskvbXrLShRVifEKupC4PWSfp3eHwK8BjiTYtjciXuBSyWp4TDfWyj2iq36ehEzSJpHkZy2RMRnum6l5aIn8dLEK4E3Av+3xGV2xQmqC5LeDrwJeBvwm4aPrqUYgncaPDWKk5afkPRl4COp/H92uDzLRK9iRtIRwFbgHyNiTbfttDz0MF5Oo+j/fwTMAT5BMfvzh920t2xOUN1ZCWyOiF2NhZKuA74vaX5E7G93oRHxnKRzgK8Ca4H7KIb+z5XRaBuonsQM8B6KacnHS7qoofy4iPhVx621QetVvLyK4lzVG4HfA7uAsyNib7cNLpN8R10zM8uRJ0mYmVmWnKAGRNIF6ZImEx/3DrptlifHjLVjJsSLD/GZmVmWWk6SkLQeeDewLyJOSGWfophZVp9V8tcRcXv67HLgYtIstIjYmsqXU1wvag7FXyyvTeXHUvx19Hzgx8AHpzMZYMGCBbF48eJpb2grzzzzDHPnzi1teb0yiHbu3Lnz8Yh4bV9XWrKy4wWqETOOl865j+mfSWMmIqZ8UFwy5WRgd0PZp4B/26TucRRz6F8FHAv8nCIhzUmv3wgcmuocl75zK3Buev1l4GOt2hQRnHLKKVGm733ve6Uur1cG0U5gR0zjN8n5UXa8RFQjZhwv+cRMFeIlIq+YaXkOKiLuBKY7jXEFsCmKC5z+AtgDnJoeeyLioShGR5uAFZLqF0D9u/T9jcA501yXmZnNYN38HdRqSRcCO4BLI+JJ4GiK60HVjaUygIcnlC+l+GvopyLiYJP6LyNpFbAKYGhoiFqt1kXzX2p8fLzU5fVKVdppZtatThPUDcDVQKTnaynuLaImdYPmswVjivpNRcQ60j1OhoeHY2RkpK1GT6VWq1Hm8nqlKu00M+tWRwkqiivqAiDpKxQ3zIJiBHRMQ9VFQP0vk5uVPw4cKemQNIpqrN9Xux45wEVrvjVlndG1Z/epNVYFrWLG8WKNHC/t6+jvoCQtbHj7HmB3er0FODfdcvhYYAnFtZ62A0skHZvuCnsuxQUtA/ge8L70/ZX4LqCVJ2m9pH2SdjeUfUrSI5LuTo+zGj67XNIeSQ9IOqOhfHkq2yNpTUP5sZJ+KOlBSd/wnYarzfFik2mZoCTdAvyA4u6cY5IuBv6jpF3pZlfvBP4NQETcSzEr76fAd4BLIuL5NDpaTXExy/uAW1NdgMuAT0raQ3FO6mulbqENwgZgeZPyz0XESelR/7OE4yh2WI5P3/mSpDmS5lBcvf1Mitmh56W6UNye+nMRsQR4kuLPGqy6NuB4sSZaHuKLiPOaFE+aRKK4zP/LLvWfAuz2JuUPUczysxkiIu6UtHia1V+Y+Qn8Iu2o1ONhT4oPJNVnft5HMfPz/FRnI8WfPdxQTuut3xwvNhlfzdz6qa8zP3s56xNg6DC49MSDk36ew2zLis/6nFEzhasQL5BXzDhBWb/0feZnL2d9Anz+ps1cu2vy/0KjF5S7vk5UeNbnjJspXIV4gbxixgnK+mImzvy03nG8GPhq5tYnnvlp7XC8GHgEZT2QZn6OAAskjQFXAiOSTqI4vDIKfBSKmZ+S6jM/D5Jmfqbl1Gd+zgHWT5j5uUnSNcBP8MzPSnO82GScoKx0nvlp7XC82GR8iM/MzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllqWWCkrRe0j5JuxvK5kvaJunB9DwvlUvS9ZL2SLpH0skN31mZ6j8oaWVD+SmSdqXvXC9JZW+kmZlVz3RGUBuA5RPK1gB3RMQS4I70HuBMYEl6rAJugCKhUdzGeSnFnS2vrCe1VGdVw/cmrsvMzGahlgkqIu4E9k8oXgFsTK83Auc0lN8YhbuAIyUtBM4AtkXE/oh4EtgGLE+fHRERP4iIAG5sWJaZmc1ih3T4vaGIeBQgIh6VdFQqPxp4uKHeWCqbqnysSXlTklZRjLYYGhqiVqt12PyXGzoMLj3x4JR1ylxfp8bHx7Noh5lZr3WaoCbT7PxRdFDeVESsA9YBDA8Px8jISAdNbO7zN23m2l1T/3OMXlDe+jpVq9Uoc7t7QdJ64N3Avog4IZXNB74BLAZGgQ9ExJPpnON1wFnAb4GLIuLH6TsrgSvSYq+JiI2p/BSKQ8+HAbcDf5VG4FZBjhebTKez+B5Lh+dIz/tS+RhwTEO9RcDeFuWLmpRbtW3A5y1t+jbgeLEmOk1QW4D6TLyVwOaG8gvTbL7TgAPpUOBWYJmkeSlolgFb02dPSzot7Rld2LAsqyift7R2OF5sMi0P8Um6BRgBFkgao9hLWQvcKuli4FfA+1P12ymG3nsoht8fAoiI/ZKuBraneldFRD0gP8aLw+9vp4fNPH0/b9nLc5bQ+rxlDucKK3zOctad587ld8opZlomqIg4b5KP3tWkbgCXTLKc9cD6JuU7gBNatcNmrJ6dt+zlOUtofd7S5yx7Ysae584hXiCvmPGVJKxffN7S2uF4MSco6xuft7R2OF6s9GnmZj5vaW1xvNhknKCsdD5vae1wvNhkfIjPzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZWnW3LBw8ZpvTfn5pSf2qSFWGY4Zs8HqKkFJGgWeBp4HDkbEsKT5wDeAxcAo8IGIeFKSgOsobtf8W+CiiPhxWs5K4Iq02GsiYmM37TIz6zfv0JSvjEN874yIkyJiOL1fA9wREUuAO9J7gDOBJemxCrgBICW0K4GlwKnAlZLmldAuMzOrsF6cg1oB1EdAG4FzGspvjMJdwJGSFgJnANsiYn9EPAlsA5b3oF2WAUmjknZJulvSjlQ2X9I2SQ+m53mpXJKul7RH0j2STm5YzspU/8E0ArcZyPEyu3WboAL4rqSdklalsqGIeBQgPR+Vyo8GHm747lgqm6zcZi6Puq0djpdZqttJEu+IiL2SjgK2Sbp/irpqUhZTlL98AUUSXAUwNDRErVabdkMvPfHglJ8PHda6Tjvr65Xx8fEs2lGyFcBIer0RqAGX0TDqBu6SVB91j5BG3QCS6qPuW/rbbBsQx8ss0VWCioi96XmfpNso9k4ek7QwIh5NwbEvVR8Djmn4+iJgbyofmVBem2R964B1AMPDwzEyMtKsWlMXtTyBeZBrd039zzF6wfTX1yu1Wo12tjtD9VF3AH+bftOXjLrTDg941G2Ol1mt4wQlaS7wioh4Or1eBlwFbAFWAmvT8+b0lS3AakmbKIbaB1JwbQU+2zDkXgZc3mm7LHt9G3V3M+KG7kfdOYx0Z8CIe9Ycpcnld8opZroZQQ0BtxWzxzkEuDkiviNpO3CrpIuBXwHvT/Vvp5hivodimvmHACJiv6Srge2p3lX1objNPP0cdXcz4obuR90ecXdvNh2lySFeIK+Y6XiSREQ8FBFvTY/jI+IzqfyJiHhXRCxJz/tTeUTEJRHxpog4MSJ2NCxrfUT88/T4evebZTmSNFfSq+uvKUbLu3lx1A0vH3VfmGZnnUYadQNbgWWS5qWR97JUZjOI48VmzZUkLAsedVs7HC+znBOU9U1EPAS8tUn5E8C7mpQHcMkky1oPrC+7jZYPx4s5QZmZTUOrSxlZ+Zyg2tAqQEfXnt2nlli3cuhsptMGx5Q1mm19kG+3YWZmWXKCMjOzLDlBmZlZlpygzMwsSzNmkkQOJ73NzKw8MyZB5WC2zbAxM+slH+IzM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmlqVsEpSk5ZIekLRH0ppBt8fy5nixdjlmqieLq5lLmgN8ETgdGAO2S9oSET8dbMvKNZ1bgviK563NlngBXyG/LLMpZmaSLBIUcCqwJyIeApC0CVgBOHisGceLtWtWxMxM26HJJUEdDTzc8H4MWDqxkqRVwKr0dlzSA2U14BOwAHi8rOV1Sn/Tssog2vmGPq+vlYHHC7SOmWn8ll1zvEzbwGOmIvECGcVMLglKTcriZQUR64B1PWmAtCMihnux7DJVpZ09NvB4gWr8FlVoY58MPGaq8lvk1M5cJkmMAcc0vF8E7B1QWyx/jhdrl2OmgnJJUNuBJZKOlXQocC6wZcBtsnw5XqxdjpkKyuIQX0QclLQa2ArMAdZHxL19bkbPDgWVrCrt7JlM4gWq8VtUoY09l0nMVOW3yKadinjZYVgzM7OBy+UQn5mZ2Us4QZmZWZacoKjGJVAkjUraJeluSTsG3Z7ZrArxAo6ZnFQhZnKMl1l/DipdAuVnNFwCBTgvt0ugSBoFhiNi4H9MPJtVJV7AMZOLqsRMjvHiEVTDJVAi4jmgfgkUs2YcL9Yux0yHnKCaXwLl6AG1ZSoBfFfSznQ5FhuMqsQLOGZyUZWYyS5esvg7qAGb1iVQMvCOiNgr6Shgm6T7I+LOQTdqFqpKvIBjJhdViZns4sUjqIpcAiUi9qbnfcBtFIcNrP8qES/gmMlIJWImx3hxgqrAJVAkzZX06vprYBmwe7CtmrWyjxdwzGQm+5jJNV5m/SG+TC6B0soQcJskKH6zmyPiO4Nt0uxUkXgBx0w2KhIzWcbLrJ9mbmZmefIhPjMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKCmSdKopGcljUt6TNLXJR3e8PkGSQclva6h7PRUd0FD2ask3Sfpo9Nc70pJIenD5W6R9Vq/YybFyTNpfeOSvtqbLbNeGEC8zJF0jaS9kp6W9BNJR/Zm6zrjBNWev4iIw4GTgT8BroAX7kD5XuAAcEG9ckRsA74JXNewjCuAR4F1rVYmaR5wOZDbzc1s+voaM8BbI+Lw9PBOTfX0M14+DfxL4O3AEcAHgd+VshUlcYLqQEQ8AnwbOCEVvRd4CrgKWDmh+ieBP5N0tqQTgNXAR2J6d4r8D8D1wOOlNNwGpo8xYzNAr+Ml7fz+61Tvl1HYHRFOUFUn6RjgLOAnqWglcAuwCfhjSSfX60bEAeBjwJeB9cCnI+Ln01jHqcBw+p5VXD9iJrlT0q8l/b2kxSU13/qsD/FyInAQeF+Kl59JuqTkzeheRPgxjQcwCoxT7MX8EvgScBjweuAPwEmp3lbguibf/2/ADuAV01jXnFT37el9DfjwoP8N/Mg3ZlL9PwUOBY4EvgDsBg4Z9L+DH/nFC3A+EMDX0jreAvwGOH3Q/w6ND4+g2nNORBwZEW+IiI9HxLMUx23vi4i7U52bgPMlvXLCd+8F7o+IP0xjPR8H7omIH5TXdBuQfsUMEXFnRDwXEU8BfwUcC/yLkrbD+qNf8fJser4qIp6NiHsoRmdnlbERZTlk0A2YAS4EXi/p1+n9IcBrgDOBLR0u810Ux5TrwTIfeJukkyJidVettRz0ImaaCUAlLs8Goxfxck96zvq8phNUFyS9HXgT8DaK4XHdtRTHjDsNnouAf9bw/u+Bv6MYjluF9SpmJB0PvBLYRXHI5hrgEeC+btprg9WreImIn0v6PvDvJH0CeCPwr4DzumtxuZygurMS2BwRuxoLJV0HfF/S/IjY3+5C0yGaxuU9B/y/KE6GWrX1JGaAIeAGYBHwDPB/gHdHxO+7bbANVK/iBYpk9DXgCWAf8O8j4o6uWlsypRNmZmZmWfEkCTMzy5IT1IBIuqDhkjSND181wppyzFg7ZkK8+BCfmZllqbKTJBYsWBCLFy8ubXnPPPMMc+fOLW15vTKIdu7cufPxiHhtX1dasrLjBaoRM46XzrmP6Z/JYqayCWrx4sXs2LGjtOXVajVGRkZKW16vDKKdkn7Z1xX2QNnxAtWIGcdL59zH9M9kMeNzUGZmliUnKDMzy5ITlJmZZamy56DKtuuRA1y05ltT1hlde3afWmNV0CpmHC/WyPHSPo+gzMwsS05QZmaWJScoMxsoSesl7ZO0u6HsU5IekXR3epzV8NnlkvZIekDSGQ3ly1PZHklrGsqPlfRDSQ9K+oakQ/u3ddaNlgnKwWNmPbYBWN6k/HMRcVJ63A4g6TjgXOD49J0vSZojaQ7wRYp7JB0HnJfqAvxNWtYS4Eng4p5ujZVmOiOoDTh4zKxHIuJOYLq3jFgBbIqIf4qIXwB7gFPTY09EPBQRz1HcHXaFJAF/TnE/NYCNwDmlboD1TMsE5eAxswFZLemedBRnXio7Gni4oc5YKpus/DXAUxFxcEK5VUA308xXS7oQ2AFcGhFPUvzwdzXUaQyGicGzlDaDR9IqYBXA0NAQtVqti+a/1NBhcOmJB6esU+b6OjU+Pp5FO6YiaT3wbmBfRJyQyj4FfIQX7wr61w0j78spRs7PA5+IiK2pfDlwHTAH+GpErE3lx1Ls5MwHfgx8MO342MxxA3A1xS3Jr6a4g+xf0vwW9kHzne3Jbnk/6RWyB9nH5PL/Oqc+ptMENZDgiYh1wDqA4eHhKPN6UZ+/aTPX7pr6n2P0gvLW16mKXM9rA/AF4MYJ5Z+LiP/UWDDhsPDrgP8h6Y/Sx18ETqfYcdkuaUtE/JQXDwtvkvRliuR2Q682xvovIh6rv5b0FeCb6e0YcExD1UXA3vS6WfnjwJGSDkk7wo31m613YH1MDv0L5NXHdDSLLyIei4jnI+IPwFcoDuHB5MEzWfkLwTOh3CrMh4WtW5IWNrx9D1CfpLUFOFfSq9JIegnwI2A7sCRNujqUYqdnSxT3E/oe8L70/ZXA5n5sg3WvoxGUpIUR8Wh6OzF4bpb0nyn2huvBI1LwAI9QBM/5ERGS6sGzCQfPTNfXw8K9PFwD1Thkk9PhmslIugUYARZIGgOuBEYknURxRGUU+ChARNwr6Vbgp8BB4JKIeD4tZzWwleKQ8PqIqN+Y7zJgk6RrgJ8AX+vTplmXWiYoB4+VpO+HhXt5uAaqccgmp8M1k4mI85oUT9oPRMRngM80Kb8duL1J+UO8eJTHKqRlgnLwWBkGdU7BzKrLV5KwvvA5BTNrl69mbqXzYWEzK4MTlJXOh4XNrAw+xGdmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzAZK0npJ+yTtbiibL2mbpAfT87xULknXS9oj6R5JJzd8Z2Wq/6CklQ3lp0jalb5zvST1dwutUy0TlIPHzHpsA7B8Qtka4I6IWALckd4DnAksSY9VwA1Q9EkUd25eSnEzyyvr/VKqs6rhexPXZZmazghqAw4ea4N3aqwdEXEnsH9C8QpgY3q9ETinofzGKNwFHClpIXAGsC0i9kfEk8A2YHn67IiI+EFEBHBjw7Iscy1v+R4Rd0paPKF4BTCSXm8EasBlNAQPcJekevCMkIIHQFI9eGqk4Enl9eD5djcbZQO3AfgCRWdQV9+pWStpTXp/GS/dqVlKscOytGGnZhgIYKekLanzqe/U3EVxS/jlOGZmmqGIeBQgIh6VdFQqPxp4uKHeWCqbqnysSXlTklZRxBZDQ0PUarXutqLB0GFw6YkHJ/28zHV1Y3x8PJu2tExQk5h1wQN5BFBOwTMZ79RYDzUbLUcH5U1FxDpgHcDw8HCMjIx00MTmPn/TZq7dNXmXO3pBeevqRq1Wo8zt7kanCWoyMzZ4II8Ayil42jSQnRqrrMckLUyxshDYl8rHgGMa6i0C9qbykQnltVS+qEl9q4BOE5SDx8rSs52aXo64oRqHbKow4p7EFmAlsDY9b24oXy1pE8Uh4QOpH9oKfLbh3PYy4PKI2C/paUmnAT8ELgQ+388Nsc51mqAcPNauvu/U9HLEDdU4ZFOFEbekWyh+6wWSxijOPa4FbpV0MfAr4P2p+u3AWcAe4LfAhwBSX3I1sD3Vu6p+eBj4GMV50cMoDgX7cHBFtExQDh4riXdqrKmIOG+Sj97VpG4Al0yynPXA+iblO4ATummjDcZ0ZvE5eKwt3qkxszKUPUnCzDs1ZlYKX+rIzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZck3LDQzK8HiNd+a8vNLT+xTQ2aQrhKUpFHgaeB54GBEDEuaD3wDWAyMAh+IiCclCbiO4vbevwUuiogfp+WsBK5Ii70mIjZ2065mHDxm1VOlPsbKV8YI6p0R8XjD+zXAHRGxVtKa9P4y4ExgSXosBW4AlqZguxIYBgLYKWlLRDxZQtssM1XqcLxTkw33MbNUL85BrQDqncVG4JyG8hujcBdwpKSFwBnAtojYnwJmG7C8B+2yfLwzIk6KiOH0vt7hLAHuSO/hpR3OKooOh4YOZylwKnClpHl9bL8NlvuYWaLbEVQA35UUwN9GxDpgKCIeBYiIRyUdleoeDTzc8N2xVDZZ+ctIWkXRUTE0NEStVpt2Qy898eCUnw8d1rpOO+vrlfHx8SzaUbIVwEh6vRGoUewRv9DhAHdJqnc4I6QOB0BSvcO5pb/Ntj6YNX1MLv+vc+pjuk1Q74iIvSlAtkm6f4q6alIWU5S/vLAIznUAw8PDMTIyMu2GXtTycM1Brt019T/H6AXTX1+v1Go12tnuDPWtw+mms4GZ0eHk1Nl0KJs+ptUh31bdaas+Jof+BfLqY7pKUBGxNz3vk3QbxeGWxyQtTB3NQmBfqj4GHNPw9UXA3lQ+MqG81k27LGt963C62aGB7ndqcuhwcupsOuE+Znbr+ByUpLmSXl1/DSwDdgNbgJWp2kpgc3q9BbhQhdOAA2mveSuwTNK8dB5hWSqzGaixwwFe0uEAtNHhNCu3GcR9jHUzghoCbismWnEIcHNEfEfSduBWSRcDvwLen+rfTjEbaw/FjKwPAUTEfklXA9tTvavq5xba0Xr4bYOWOplXRMTTDR3OVbzY4azl5R3OakmbKCZEHEh7zVuBzzZMjFgGXN5OWxwvlZBVH2P913GCioiHgLc2KX8CeFeT8gAumWRZ64H1nbbFKsMdjk2b+xjzlSSsb9zhmFk7fC0+MzPLkhOUmZllyQnKzMyy5HNQJWo1M2x07dl9aomZWfV5BGVmZlnyCMrMrCJm21Eaj6DMzCxLHkGZDch0rmYx0/aIzdrhEZSZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMspRNgpK0XNIDkvZIWjPo9ljeHC/WLsdM9WSRoCTNAb4InAkcB5wn6bjBtspy5XixdjlmqimXi8WeCuyJiIcAJG0CVgA/HWirSjadi4O2smH53J63oQIXKJ0V8QLdx0y38TKdNlQgXmCWxMwM+a1eoIgYdBuQ9D5geUR8OL3/ILA0IlZPqLcKWJXevhl4oMRmLAAeL3F5vTKIdr4hIl7b53VOKpN4gWrEzKyPF8gmZqoQL5BRzOQyglKTspdlzohYB6zrSQOkHREx3Itll6kq7eyxgccLVOO3qEIb+2TgMVOV3yKndmZxDgoYA45peL8I2Dugtlj+HC/WLsdMBeWSoLYDSyQdK+lQ4Fxgy4DbZPlyvFi7HDMVlMUhvog4KGk1sBWYA6yPiHv73IyeHQoqWVXa2TOZxAtU47eoQht7LpOYqcpvkU07s5gkYWZmNlEuh/jMzMxewgnKzMyy5ARFNS6BImlU0i5Jd0vaMej2zGZViBdwzOSkCjGTY7zM+sWNACwAAAEnSURBVHNQ6RIoPwNOp5iKuh04LyKy+gtzSaPAcERU4Q/9ZqyqxAs4ZnJRlZjJMV48gmq4BEpEPAfUL4Fi1ozjxdrlmOmQExQcDTzc8H4sleUmgO9K2pkux2KDUZV4AcdMLqoSM9nFSxZ/BzVg07oESgbeERF7JR0FbJN0f0TcOehGzUJViRdwzOSiKjGTXbx4BFWRS6BExN70vA+4jeKwgfVfJeIFHDMZqUTM5BgvTlAVuASKpLmSXl1/DSwDdg+2VbNW9vECjpnMZB8zucbLrD/El8klUFoZAm6TBMVvdnNEfGewTZqdKhIv4JjJRkViJst4mfXTzM3MLE8+xGdmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZen/A6K07c+5e7/dAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEYCAYAAADmugmLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfbRdVXnv8e9PEKS8lMRImhLqoR3RK8KQQoZJL7Y91EEIYA0doyroNSfIbVovjNqWcWuwtlBBG22xJdXLJbZpkqvyMlpfooIhl+EptQMwwYsEREygUY5JSWkAE17aJn3uH2tusrKz93nZZ7/MvfbvM8Yee++518tc2U/Os9dcc86liMDMzCxHr+h1BczMzJpxkjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTMzy5aTlJmZZctJyszMslWZJCVph6QXJe2T9Iykr0k6JX22VtL16fWQpJB0ZINtXCvpsy3ufzTt9+i68rVpf2+vK/+LVL5M0odSvfdJeknSgdL7R9Ly10naKmm/pGtbqaMdVOV4kXSSpFsk7ZT0nKR/lLSglXraQVWOmbT8NyT9i6QfS/qOpCWt1LPdKpOkkl+NiOOAOcBTwF92Y6eShoBfBAJ4e4NFvg+MlJY/EngH8DhARHwsIo5Ldf8t4N7a+4h4Y1ptO/D7wNc6dRwDqKrxchywGTgbmAmsA74m6biOHdTgqGrMAHwAmBMRJwDLgc9KmtOhQ5q0qiUpACLiJeBvgdO6tMulwH3AWkqBUvIV4BxJM9L7xcBDwD9PdgcRsS4i7gT2Tq+qVq9q8RIRT0TEJyNiV0QciIjVwFHA66ddcwOqFzMAEfFQROyvvQVeCZzSaoXbpZJJStJPAO+i+FK7YSnwufQ4X9Lsus9fAjYAl5SWX9+lutkEqh4vks6kSFLbW92GHaqqMSPpq5JeAu4HRoEtrVa4XaqWpL4k6Vngx8B5wJ92eoeS3gK8Frg9Ih6gOL1+d4NF1wNLJf0k8MvAlzpdN5tQ5eNF0gnA/wH+OCKea63WVlLpmImItwHHAxcCGyPiP1uueJtULUldHBEnAkcDVwJ/L+mnOrzPEeCuiHg6vf88DU7HI+KbwGuADwNfjYgXO1wvm1il40XSMRTNQPdFxJ+0XmUrqXTMpO38R7q0cH59Z4xeOKz3SRVExAHgC5JuBt7Sqf2kPwLvBI6QVGv7PRo4UdKbIuI7dat8Fvgj4NxO1cmmrorxknqAfQn4EfCbrdfaGqlizDRwJPBzbdjOtFTtTAoAFZYAM4BHmyx2tKRXlR61f4tX1JUf3WR9gIuBAxQXT89MjzcA/0DRJlxvFUUTwT0tHNMrJb2K4js7MtXtiKluxw5XtXiR9EqKi/ovAktzaLKpmgrGzH+RdIGkY9Lfmv8G/BLw91PZTidU7UzqK5IOUPRM+QEwEhGPSGq07L669+el50vTo+ZHwNwm+xsB/iYiflgulPQpYJWkD5bLI2IPcPdkDqSBz3DoKf4fAJdR9Pax1lQ1Xv4r8DaKJPVs6XguiIh/aGF7dlBVY0bAtRTJ8ACwDXhXRHy7hW21lXxnXjMzy1Ulm/vMzKwaqtbc13aSfgb4bpOPT6s/DbfB5nixqXLMjM/NfWZmlq2+PZOaNWtWDA0NTXr5559/nmOPPbZzFcpA7RgfeOCBpyPiNb2uT04mipdBiI96jpfx9VvMdLM+3YyZvk1SQ0NDbNky+Rk7RkdHGR4e7lyFMlA7Rkk/6HVdcjNRvAxCfNRzvIyv32Kmm/XpZsy444SZmWXLScrMzLLlJGVmZtnq22tSkzG04uD9Aa86Yz/LSu93rLyoF1WyDJTjoqYWH44La6RZzAx3vyoDx2dSZmaWLScpMzPLlpOUmfWUpDWSdkt6uFR2raQfSXowPS4sfXa1pO2SHpN0fql8cSrbLmlFqfxUSfdL2ibpNklHde/obLomTFKSTpH0DUmPSnpE0gdS+UxJm9IXv0nSjFQuSatSoDwk6azStkbS8tskjZTKz5a0Na2zSk2mFDazSloLLG5Q/ucRcWZ63AEg6TSKW6S/Ma3zvyQdkW5b82ngAoqZvC9NywJ8PG1rHvAMcHlHj8baajJnUvuBqyLiDcBC4Ir05a8A7k5f/N3pPRRBMi89lgM3QZHUgGuABcCbgWtqiS0ts7y0XqOANbMKioh7gD2TXHwJcGtE/FtE/BOwneLvyZuB7RHxRET8O3ArsCT94P0VivtrAayjuEeT9YkJk1RE7KrdUyQi9lLc4OtkimBZlxYrf/FLgPVRuI/iDpJzgPOBTRGxJyKeATYBi9NnJ0TEvVFMJLgeB5GZwZWpNWZN6QftycCTpWXGUlmz8lcDz0bE/rpy6xNT6oIuaQj4eeB+YHZE7IIikUk6KS021SA6Ob2uLzezwXUTcB3FzQWvA24A3kdxc756QeMf3DHO8g1JWk7RqsPs2bMZHR0Fiu7m9WYfw8uf52Dfvn1Z1addJp2kJB0H/B3wOxHx43EuGzULiqmWN6pDwwBqphxYs4859H0Vv8yqBqkNnoh4qvZa0meAr6a3Y8AppUXnAjvT60blT1O05hyZzqbKyzfa72pgNcD8+fOjNhfesibjpN45oHP3ddOkkpSkV1IkqM9FxBdS8VOS5qSzqDnA7lTeLIjG4JCxb3OB0VQ+t8Hyh2kWQM0sqxvMe8PWg4e74z3jr9uPqhqkNnhqf1vS218Daj3/NgCfl/RJ4KcprmF/i+LH7jxJp1Lcjv0S4N0REZK+Afw6xXWqEeDL3TsSm67J9O4T8NfAoxHxydJHGyi+cDj0i98ALE29/BYCz6Vg2wgskjQjtS8vAjamz/ZKWpj2tRQHkdnAkHQLcC/wekljki4HPpF6/D4EnAv8LkBEPALcTnGTwK8DV0TEgXSWdCXF35lHgdvTsgAfBH5P0naKa1R/3cXDs2mazJnUOcB7ga2SHkxlHwJWArengPoh8I702R3AhRS9bl4ALgOIiD2SrgM2p+U+EhG1Hj3vp+iGegxwZ3qY2QCIiEsbFDdNJBHxUeCjDcrvoPj7U1/+BEXvP+tDEyapiPgmja8bAby1wfIBXNFkW2uANQ3KtwCnT1QX6w+S1gBvA3ZHxOmp7FrgN4B/SYt9qDT25WqKsSsHgN+OiI2pfDFwI3AE8FcRsTKVn0rRdDMT+Dbw3tTt2MwqxjNOWCesxYMzzawNnKSs7Tw408zapdK36rDsXClpKbCFYhaTZyjGxN1XWqY8Tq5+XN0CpjA4c6pjXq46Y/9AdeH3kAXrB05S1i1dH5w51TEvN2w9spJDE5rxkAXrB05S1hW9GpxpZv3N16SsK9KA75r6wZmXSDo69dqrDc7cTBqcmW6tcAmwIfUerQ3OBA/ONKs0n0lZ26XBmcPALEljFLPfD0s6k6Jpbgfwm1AMzpRUG5y5nzQ4M22nNjjzCGBN3eDMWyVdD/w/PDjTrLKcpKztPDjTBsVQg+ubNTtWXtTFmlSXm/vMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLLlJGVmZtlykjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTPrKUlrJO2W9HCpbKakTZK2pecZqVySVknaLukhSWeV1hlJy2+TNFIqP1vS1rTOqnTjTOsTTlJm1mtrgcV1ZSuAuyNiHnB3eg9wAcVM+fMobmh5ExRJjWIi4wUU8zpeU0tsaZnlpfXq92UZc5Iys56KiHuAPXXFS4B16fU64OJS+foo3Edxb7E5wPnApojYk+74vAlYnD47ISLuTbd5WV/alvUBz4JuZjmaHRG7ACJil6STUvnJwJOl5cZS2XjlYw3KG5K0nOKsi9mzZzM6OgoUd24+rILHNC6vqa3bLfv27ev6PrvBScrM+kmj60nRQnlDEbEaWA0wf/78GB4eBmBZg1tyXHXGfm7Y2vxP6I73DDf9rBNGR0ep1bdK3NxnZjl6qnY35/S8O5WPAaeUlpsL7JygfG6DcusTTlLWdu6tZW2wAah95yPAl0vlS1PcLASeS82CG4FFkmak2FoEbEyf7ZW0MMXJ0tK2rA84SVknrMW9tWySJN0C3Au8XtKYpMuBlcB5krYB56X3UNyp+QlgO/AZ4H8ARMQe4Dpgc3p8JJUBvB/4q7TO48Cd3Tguaw9fk7K2i4h7JA3VFS8BhtPrdcAo8EFKvbWA+yTVemsNk3prAUiq9dYaJfXWSuW13lr+w9OnIuLSJh+9tcGyAVzRZDtrgDUNyrcAp0+njtY7TlLWLV3vrdVKT60q9o5qpqq9waxanKSs1zrWW6uVnlrd7pHVS1XtDWbV4mtS1i3urWVmU+YkZd3i3lpmNmVu7rO2S721hoFZksYoeumtBG5PPbd+CLwjLX4HcCFFz6sXgMug6K0lqdZbCw7vrbUWOIaiw4Q7TZhVlJOUtV0/99YaanC9CmDHyos6sTszm8CEzX0emGlmZr0ymWtSa/HATDMz64EJk5Sn0Tczs15p9ZpUVtPoN1MetFk/rX4VBzF6cKaZVU27O070ZBr9ZsqDNuun1a/ioE0PzjSzqml1nJQHZpqZWce1mqQ8MNPMzDpuwuY+D8w0M7NemTBJ9fPATDMz62+eu8/MzLLlJGVmZtlykjIzs2w5SZmZWbacpMwsW5J2pAmoH5S0JZW1bYJry5+TlJnl7tyIODMi5qf37Zzg2jLnJGVm/aYtE1x3u9LWGt/00LpK0g5gL3AA2B8R89Mv3duAIWAH8M6IeCbNQnIjxQDxF4BlEfHttJ0R4MNps9dHxDqsigK4S1IAN6f5O9s1wfVhmk1iXZ6cuqZ+0up63Z7suaoTTDtJWS+cGxFPl97Xmm9WSlqR3n+QQ5tvFlA03ywoNd/Mp/gj9oCkDelXslXLORGxMyWiTZK+N86y057Iutkk1ssa3LG5ftLqet2exLqqE0y7uc9y4OYbaygidqbn3cAXKa4ptWuCa+sDA3smNdTglxHAjpUXdbkmA6drzTf93HTTDbk3D0k6FnhFROxNrxcBH+HgBNcrOXyC6ysl3Upx5v1ciqeNwMdKnSUWAVd38VBsGgY2SVnPdK35pp+bbrqhD5qHZgNfLC5NciTw+Yj4uqTNtG+Ca8uck5R1Vbn5RtIhzTfpV+9km2+G68pHO1x167KIeAJ4U4Pyf6VNE1xb/nxNyrpG0rGSjq+9pmh2eZg23Z+si4diZl3iMynrJjffmNmUOElZ17j5xsymys19ZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtzzhhNgnNbu0Cvr2LWSf5TMrMzLLlJGVmZtlykjIzs2z5mlQdX3swM8uHk5SZWQc0+8HrH7tT4+Y+MzPLVjZJStJiSY9J2i5pRa/rY3lzvNhUOWb6UxbNfZKOAD4NnAeMAZslbYiI7/a2Zpaj3OLFzTr5yy1mbPKySFLAm4Ht6fbiSLoVWAJkFUD+Y5SNvo4XcMz0QDYx47iYmlyS1MnAk6X3Y8CC+oUkLQeWp7f7JD022R38NswCnp5OJZvRxzux1ZbUjvG1va5Ih7U9XjoZH41kEjODEi/QJzEzzbjoZgx3LWZySVJqUBaHFUSsBla3tANpS0TMb2XdfjEIx5i0PV4G6N/uZQN2zJWPmdzq0y65dJwYA04pvZ8L7OxRXSx/jhebKsdMn8olSW0G5kk6VdJRwCXAhh7XyfLleLGpcsz0qSya+yJiv6QrgY3AEcCaiHikzbtpqZmwzwzCMXYqXgbi367OwBzzgMRMbvVpC0Uc1ixrZmaWhVya+8zMzA7jJGVmZtkaiCSV43QoktZI2i3p4VLZTEmbJG1LzzNSuSStSvV/SNJZpXVG0vLbJI2Uys+WtDWts0qSWt1HleUYG404XvIy3bjx9zkFEVHpB8VF0seBnwWOAr4DnJZBvX4JOAt4uFT2CWBFer0C+Hh6fSFwJ8VYj4XA/al8JvBEep6RXs9In30L+IW0zp3ABa3so8qPXGPD8ZL3ox1x4+9zCv9Wva5AFwLqF4CNpfdXA1f3ul6pLkN1QfoYMCe9ngM8ll7fDFxavxxwKXBzqfzmVDYH+F6p/OXlprqPXv8bDWpsOF7yfbQrbvx9Tu4xCM19jaZDOblHdZnI7IjYBZCeT0rlzY5hvPKxBuWt7KPK+v2YHS+90alj9/fZwCAkqUlNh5K5Zscw1fJW9lFlVT1mx0tndfvYB/r7HIQk1U/ToTwlaQ5Aet6dypsdw3jlcxuUt7KPKuv3Y3a89Eanjt3fZwODkKT6aTqUDUCth84I8OVS+dLUA2ch8Fw6Vd8ILJI0I/XSWUTRVr4L2CtpYerVs7RuW1PZR5X1U2w04njpjU7Fjb/PRnp9UawbD4qeK9+n6JHzB72uT6rTLcAu4D8ofsVcDrwauBvYlp5npmVFccO2x4GtwPzSdt4HbE+Py0rl84GH0zqf4uDsIlPeR5UfOcaG4yX/x3Tjxt/n5B+eFsnMzLI1CM19ZmbWp5ykzMwsW05SZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLJVmSQlaYekFyXtk/SMpK9JOiV9tlbS9en1kKSQdGSDbVwr6bMt7n807ffouvK1aX9vryv/i1S+TNKHUr33SXpJ0oHS+0fq1vvltN71rdTTClWPl7rj2yfprlbqaQdVPWbSOh+Q9E+Snpf0qKTXtVLXdqpMkkp+NSKOA+YATwF/2Y2dShoCfhEI4O0NFvk+B++GSQred1DcYIyI+FhEHJfq/lvAvbX3EfHG0nqvBG4E7u/QoQyaSscL6fjSY1FnjmbgVDZmJP13ipsvXgQcB7wNeLpTxzRZVUtSAETES8DfAqd1aZdLgfuAtZQCpeQrwDnpFs8Ai4GHgH+e4n6uAu4CvtdaNa2RCseLdUjVYkbSK4BrgN+NiO9G4fGI2DPtmk9TJZOUpJ8A3kXxpXbDUuBz6XG+pNl1n78EbAAuKS2/fio7kPRailtFf2R6VbV6VYyX5HOS/kXSXZLe1HJt7TAVjJm56XG6pCdTk98fp+TVUz2vQJt9SdKzwI+B84A/7fQOJb0FeC1we0Q8QHF6/e4Gi64Hlkr6SeCXgS9NcVergD+MiH3Tqa8dosrx8h5gKO3rG8BGSSe2Wm97WVVjZm56XgScAZwLXErR/NdTVUtSF0fEicDRwJXA30v6qQ7vcwS4KyJqbbefp8HpeER8E3gN8GHgqxHx4mR3IOlXgeMj4rY21NcOqmS8pPX/MSJejIgXIuJPgGcprmnY9FQ1ZmrLfiIino2IHcDNwIUt17pNDut9UgURcQD4gqSbgbd0aj+SjgHeCRwhqdb2ezRwoqQ3RcR36lb5LPBHFL9SpuKtwPzSPn4SOCDpjIhY0mL1LalgvDQSgNqwHaOSMfMY8O8UcZKVqp1JAaDCEmAG8GiTxY6W9KrSo/Zv8Yq68qObrA9wMXCA4uLpmenxBuAfKNqE662iaCK4Z4qH9IfA60r72AB8BrhsituxBqoWL5J+RtI5ko5KdfqfwCzgH6eyHWuuajETES8AtwG/L+l4SXOB3wC+OpXtdELVzqS+IukAxa+BHwAjEfGI1PAHZP21nfPS86XpUfMjDrbX1hsB/iYiflgulPQpYJWkD5bLU0+ZuydzIHXr7QX2lrb/IvB8Dj1v+lwl4wU4HrgJ+DmKC+oPAhdExL+2sC07VFVjBormy9XATorm4c8Aa1rcVtsoIruzOzMzM6CizX1mZlYNVWvuaztJPwN8t8nHp9Wfhttgc7zYVDlmxufmPjMzy1bfnknNmjUrhoaGprTO888/z7HHHtuZCmW03wceeODpiHhN13bYByaKl17FRg77d7w01srfmPH0OsbaoXYM3YyZvk1SQ0NDbNmyZUrrjI6OMjw83JkKZbRfST/o2s76xETx0qvYyGH/jpfGWvkbM55ex1g71I6hmzHjjhNmZpYtJykzM8vWhElK0imSvpFugPWIpA+k8pmSNknalp5npHJJWiVpu6SHJJ1V2tZIWn6bpPK9T86WtDWts0pNRsaZmdlgmcw1qf3AVRHxbUnHAw9I2gQsA+6OiJWSVgArgA8CFwDz0mMBxcj3BZJmUtyvZD7FaO0HJG2IiGfSMssppr2/g+JeKHdO9+CGVnztkPdXnbGfZalsx8qLprt561P1cQEHY8NxYe3S7O+PY2xqJjyTiohdEfHt9HovxTxVJwNLgHVpsXUUc0yRytenm2bdRzER4hzgfGBTROxJiWkTsDh9dkJE3BtFf/j1pW1ZH5K0RtJuSQ+Xyq6V9CNJD6bHhaXPrk5n0Y9JOr9UvjiVbU8/hGrlp0q6P52R3ybpqO4dnZl105R696m4hfHPU9y+fHZE7IIikUk6KS12MvBkabWxVDZe+ViD8kb7X05xxsXs2bMZHR0dt75XnbH/kPezjzlYNtG67bRv376u7i8Da4FPcfhN1/48Iv6sXCDpNIobtb0R+Gng/0p6Xfr40xTznY0Bm9OZ93eBj6dt3Srpf1Pc8+amTh2MmfXOpJOUpOOAvwN+JyJ+PM5lo0YfNLtNwHjlhxdGrKaYAJH58+fHRN05lzU43b5ha3HIO94z/rrtVIWup1MREfekHzSTsQS4NSL+DfgnSduBN6fPtkfEEwCSbgWWSHoU+BUO3vRtHXAtTlJmlTSp3n2SXkmRoD4XEV9IxU+lpjrS8+5UPgacUlp9LsWsuuOVz21QbtVzZepMs6bW0Yapn3m/Gng2IvbXlZtZBU14JpV62v018GhEfLL00QaKaeRXpucvl8qvTL98FwDPpebAjcDHSn+cFgFXR8QeSXslLaRoRlwK/GUbjs3ychNwHcVZ8nXADcD7aH4m3egH1JTOvJs1D9c3A8PBpuBeNcsOYJOw2aRMprnvHOC9wFZJD6ayD1Ekp9slXQ78EHhH+uwOilsObwdeIN2YLyWj64DNabmPlO6H9H6K6xjHUPTqm3bPPstLRDxVey3pMxy8mVqzM2yalD9N0RnnyHQ21fTMu1nzcH0zMBxsCu5mM3DZoDUJm03WhEkqIr5J89tOv7XB8gFc0WRba2hwE62I2AKcPlFdrH9JmlPraAP8GlDr+bcB+LykT1J0nJgHfIsi5uZJOpXipnCXAO+OiJD0DeDXgVs59Cze+pCkUyg62fwU8J/A6oi4MQ1buQ0YAnYA74yIZ1Lrzo0UP4ZfAJbVeiCn8ZcfTpu+PiLWpfKzOfhD+A7gA+HZtfuCZ5ywtpN0C3Av8HpJY+ls+xNpwPZDwLnA7wJExCPA7RS3Kvg6cEVEHEhnSVcCGymGPdyeloViPN7vpU4Wr6Zojrb+VRuL+QZgIXBF6vW5gmIs5jyKu83WhiGUx2IuJ3WaKY3FXEDR+eaa0uWF2ljM2nqLu3Bc1gZ9O8Gs5SsiLm1Q3DSRRMRHgY82KL+D4ldvffkTHOwBaH0unWHXhrPsTT04a2Mxh9Ni64BRih8oL4/FBO6TVBuLOUwaiwmQJh1YLGmUNBYzldfGYvqyQh/wmZSZZWO8sZhAx8diWn58JmVmWchhLOZUJwwYT7PJBPq5F2cveqE6SZlZz403FjMNYZnsWMzhuvJRpjAWc6oTBoyn2WQCvepB2g696IXq5j4z66lJjMWEw8diLk13XFhIGotJ0clmkaQZqcPEImBj+myvpIVpX0txj9C+4TMpM+s1j8W0ppykzKynPBbTxuPmPjMzy5aTlJmZZctJyszMsuUkZWZm2XKSMjOzbDlJmZlZtpykzMwsW05SZmaWLScpMzPLlpOUmZlly9MimZUM1c1cXbNj5UVdromZgc+kzMwsY05SZmaWLScpMzPLlpOUtZ2kNZJ2S3q4VDZT0iZJ29LzjFQuSaskbZf0kKSzSuuMpOW3SRoplZ8taWtaZ5XGuc+4mfU3JynrhLXA4rqyFcDdETEPuDu9B7gAmJcey4GboEhqwDXAAuDNwDW1xJaWWV5ar35fZlYRTlLWdhFxD7CnrngJsC69XgdcXCpfH4X7gBMlzQHOBzZFxJ6IeAbYBCxOn50QEfemm9+tL23LzCrGXdCtW2ZHxC6AiNgl6aRUfjLwZGm5sVQ2XvlYg/LDSFpOccbF7NmzGR0dBeCqM/YfXrljGpfX1NbtlH379nV8H2b9yEnKeq3R9aRoofzwwojVwGqA+fPnx/DwMADLGoyFuuqM/dywtfl/hx3vGW76WTuMjo5Sq5+ZHeTmPuuWp1JTHel5dyofA04pLTcX2DlB+dwG5WZWQU5S1i0bgFoPvRHgy6XypamX30LgudQsuBFYJGlG6jCxCNiYPtsraWHq1be0tC0zqxg391nbSboFGAZmSRqj6KW3Erhd0uXAD4F3pMXvAC4EtgMvAJcBRMQeSdcBm9NyH4mIWmeM91P0IDwGuDM9zKyCnKSs7SLi0iYfvbXBsgFc0WQ7a4A1Dcq3AKdPp45m1h/c3GdmZtlykjIzs2w5SZmZWbacpMzMLFsTJilPFmpmZr0ymTOptXiyUDMz64EJk5QnCzUzs15pdZxU1ycLheYThjZTP2FoeRLRbk7m6clDzcxa0+7BvB2bLBSaTxjaTP1EouVJRDs9YWiZJw81a07SGuBtwO6IOD2VzQRuA4aAHcA7I+KZdM36RopZSl4AlkXEt9M6I8CH02avj4h1qfxsDs5QcgfwgdRyY32g1d59nizUzNplLb7ubU20mqQ8WaiZtYWve9t4Jmzu82ShZtYDfXHdezzNron38/XpXlxfnzBJebJQM8tIVte9x9Psmng3r4e3Wy+ur3vGCTPLka97G+AkZWZ58nVvA3w/KTPrMV/3tvE4SZlZT/m6t43HzX1mZpatgT2TGqrreVOzY+VFXa6JmZk14zMp6ypJO9KtWR6UtCWVte3WL2ZWLU5S1gvnRsSZETE/vW/nFDhmViFOUpaDtkyB0+1Km1nnDew1KeuZAO6SFMDNaYR/u6bAOUSzKW7qp6uBQ2/j0kinp4Lx7VzMGnOSsm47JyJ2pkS0SdL3xll2WlPdNJvipn66Gjj0Ni6NdHoqG9/OxawxN/dZV0XEzvS8G/gixTWldk2BY2YV467mbUoAAATPSURBVCRlXSPpWEnH115TTF3zMG2aAqeLh2JmXeLmPuum2cAXiynUOBL4fER8XdJm2jcFjplViJOUdU1EPAG8qUH5v9KmKXDMctdsIgHwZAKNuLnPzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLLlJGVmZtnytEhmk+CpbMx6w2dSZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcu++Ou7FZWaWD59JmZlZtpykzMwsW27uMzPLRLPLDYN8qSGbJCVpMXAjcATwVxGxssdVsozlFC/+w9IfcooZm7wskpSkI4BPA+cBY8BmSRsi4ru9rdmh/McoD/0SL5YPx0z/yiJJAW8GtkfEEwCSbgWWAH0RQE5eXdcX8eKeolnpi5hpZpD/xuSSpE4Gniy9HwMW1C8kaTmwPL3dJ+mxqezkt2EW8HSrlZwqffzll13dL/DaLu6rF9oeLz2MjZpux0hZ1eMFuvQ3ZjydiLEGcdRptWPoWszkkqTUoCwOK4hYDaxueSfSloiY3+r6/bbfCmt7vPT6O+r1/gdAV/7GjFuBCnzHvTiGXLqgjwGnlN7PBXb2qC6WP8eLTZVjpk/lkqQ2A/MknSrpKOASYEOP62T5crzYVDlm+lQWzX0RsV/SlcBGiu6hayLikQ7sqiOn8Rnvt5I6FC+9/o56vf9K6+LfmPFU4Tvu+jEo4rBmWTMzsyzk0txnZmZ2GCcpMzPL1sAkKUmLJT0mabukFVNYb4ekrZIelLQllc2UtEnStvQ8I5VL0qq0j4cknVXazkhafpukkVL52Wn729O6Gm8f1n6txkZpfceINTXd+JrGftdI2i3p4VJZz+JyvH2MKyIq/6C4UPo48LPAUcB3gNMmue4OYFZd2SeAFen1CuDj6fWFwJ0UYzIWAven8pnAE+l5Rno9I332LeAX0jp3AheMtw8/8okNx4gf3Yivaez7l4CzgIdLZT2Ly2b7mOgxKGdSL0+JEhH/DtSmRGnVEmBder0OuLhUvj4K9wEnSpoDnA9siog9EfEMsAlYnD47ISLujeJbXF+3rUb7sPZqd2zUOEYMOhdfE4qIe4A9dcW9jMtm+xjXoCSpRlOinDzJdQO4S9IDKqZMAZgdEbsA0vNJE+xnvPKxJvVqtg9rr+nERo1jxJppR3y1Uy/jsqV/iyzGSXXBpKZEaeKciNgp6SRgk6TvtbCfqZZb97TjO3CMWDP98v11Iy5b+rcYlDOplqdEiYid6Xk38EWK0/enaqep6Xn3BPsZr3xuk3o124e117Sny3GM2Dhym46pl3HZ0r/FoCSplqZEkXSspONrr4FFwMNp3VovlxHgy+n1BmBp6sWyEHgune5uBBZJmpF6uiwCNqbP9kpamHrGLK3bVqN9WHtNa7ocx4hNILfpmHoZl832Mb5u9DLJ4UHRs+T7FD1t/mCS6/wsRW+c7wCP1NYDXg3cDWxLzzNTuShurPY4sBWYX9rW+4Dt6XFZqXw+xR+1x4FPcXAWkIb78COP2HCM+NGN+Jrmfm8BdgH/QXEWc3kv43K8fYz38LRIZmaWrUFp7jMzsz7kJGVmZtlykjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTMzy9b/Bx/Qh33ndxUNAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAEYCAYAAADWNhiqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfbAcdZ3v8ffH8CAbxCRGTsUka8KaukuEe4XkQly3ds+KhIBVN9xSVijKnKvczRbCqrVqbVzrXlZRL24V7hV0paJkSdisiE+buEtuNmKOSJU8KiSEyOYEoxwTiBgIHHyAsN/7R/9GO8PMmYfzMPOb83lVdU3Pt3/d/euZ7+nvTHefHkUEZmZmOXtZpztgZmY2Vi5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2pkwxk7RP0i8ljUh6QtI/SDqxNP0mSUckvaYUOze1nV2KHS9pt6Q/b2Kd09P6bqvTn+fLy07xBySFpAWStqT5RyS9kNpXnt8g6ThJX03LCkn9bb9ANqoezZ9lkrZJOiTpZ5K+ImlO+6+S1dKjubNY0n2SnkrDtyQtbv9VGgcRMSUGYB/wljQ+F3gIuCY9nw48C/wc+FDVfDcCG0vPrwa+DaiJdQ6kZR4B5tTozyPAX5Rip6dYAAuq2t8EfLwqdhzwfuAPgQNAf6df514dejR/zgcuAk4CfgdYB/y/Tr/WvTb0aO7MABYAAqYB7wV2dPJ1njLfzMoi4qfAFuC0FHob8DTwMYokKPtL4I8lvVXSacCVwJ9FekcbGABuAHYAl9aYfjOwqqr9hha24/mI+L8RcSfwYrPz2dj0UP5siYivRMQzEfEL4LPAm5qd31rXQ7nzdETsS30Rxf7ndc3OPxGmZDGTNB+4APhBCg0AXwJuAX5f0pmVthFxGLicIjHWAR+NiL1NrON3gX5gYxpW1Wh2F3CSpFMlTQPeAfxjm5tlk6SH8+ePgF1jmN8a6LXckfQ08CvgeuCTrc4/nqZaMfvn9OLfCXwH+GR64/8E+KeIeAK4napPSBHxTYo3/2XAdU2uaxXF1+6HKZL19ZLOqNGu8gnpXOCHwE9b3iqbLD2bP5L+M/C/gQ+1M7811JO5ExEzgFdSfGv8QYPmE+qYTq68Ay6MiG+VA5LeCeyOiAdSaCNwraQPRsQLpaa7gF9HxH80ua5VwBcAImK/pO9QJGr1G34zcAewkBa+5ltH9GT+SHodxaGv90XEd9tZhjXUk7mT1vGcpBuAn0k6NSIOtrussZhq38xqWQWcIulxSY8DnwZmU5wcb4ukPwAWAR8uLfds4BJJR32AiIgfAz+iOPTw9XbXaR2Tdf5Iei3wLeDqiLi53T5bW7LOnSovo7iIaO4Yl9O2qfbN7CiS3gj8HnAG8LPSpGspPslsbnPRA8A2jj5WfQLFydjzgW9Wtb8MmJk+4bT0nkg6nuIELMBxkl5O8SnOP1Q3wXLPH0lzKa6O+1xE3NBmX60NPZA75wJPpuVOBz4OPAXsbrPfYzalixnFG78pInaWg5I+A3xX0qyIONTKAlMx+VNgVUQ8XjXt5rTOoxKqmZO6o3gEeG0a35oeF1JcfmsTK/f8+Z/AKcBVkq4qLe/E+rPYOMk9d2ZQXPQxD/glcC+wIiJ+1ebyxkz+AG9mZrnzOTMzM8uei1mbJF1aur1LefD/6VhDzh9rl3OnNh9mNDOz7GV7Acjs2bNjwYIFADz33HNMnz69sx1qUTf2+f77738yIl7d6X5MtHLuQHe+F83opn47d7pfN/d1XPKn0c0bgZcD9wAPUvzz3kdTfCFwN7AH+DJwXIofn54PpekLSsv6cIo/ApxXiq9IsSFgTTM3lVyyZElUbN++PXLTjX0G7ovxvcFq1+dORHe+F83opn6Pd+5El+ZPzrnTzX0dj/xp5pzZr4E3R8R/Ad4ArJC0DPgU8HcRsYji/wsuS+0vA56KiNcBf5fakX4e4GLg9SmB/l7StHRfsM9R/A/EYop/7uvsTwnYeHHu2Fg4f6xpDYtZKpwj6emxaQjgzcBXU3w9cGEaX5mek6afI0kpfktE/DoifkTxSeisNAxFxKMR8TzFDTdXjnnLrOOcOzYWzh9rRVPnzNInmPspbvH/OWAv8HREHElNhvntbUzmAo8BRMQRSYeBV6X4XaXFlud5rCp+dp1+rAZWA/T19TE4OAjAyMjIb8ZzkWOf29EtuWN56ob8qbffgbz+jnPqazuaKmYR8SLwBkkzgG8Ap9Zqlh5VZ1q9eK1vhzUvsYyItcBagKVLl0Z/fz8A12/cxLV3Ple3//uueWvdaZ0yODhIpf+9rFtyZ7Qd0sFDh7l+46Y6WwCnz31l3Wmd1Os7J+iO/Km334G89j29vs9p6WrGiHha0iCwDJgh6Zj0CWkesD81GwbmA8PpXl+vBA6V4hXleerFrUd0Onca7pB21v9T2Hdpf91pndTrO6eyTuePdb+G58wkvTp9KkLSCcBbKG4muR14e2o2AFQ+2m7mt7/J83bg2+lqlc3AxZKOl7SQ4s7O91Dc02uRpIWSjqM4UdvuTTatizh3bCycP9aKZr6ZzQHWp2PXLwNujYh/kfQwcIukj1P8Ts6Nqf2NwM2Shig+FV0MEBG7JN0KPAwcAa5IhxCQdCXFTXKnAesiYkr/J3sPce7YWDh/rGkNi1lE7KD4mYLq+KMUVwNVx38FXFRnWZ8APlEjfhtwWxP9tYw4d2wsnD/WCt+b0czMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsNi5mk+ZK2S9otaZek96X4LEnbJO1JjzNTXJKukzQkaYekM0vLGkjt90gaKMWXSNqZ5rlOkiZiY80sH973WCua+WZ2BPhARJwKLAOukLQYWAPcHhGLgNvTc4DzgUVpWA18HooEBK4CzgbOAq6qJGFqs7o034qxb5p1mndGNkbe91jTGhaziDgQEd9P488Cu4G5wEpgfWq2Hrgwja8ENkThLmCGpDnAecC2iDgUEU8B24AVadpJEfG9iAhgQ2lZljfvjKxt3vdYK1o6ZyZpAXAGcDfQFxEHoEg64OTUbC7wWGm24RQbLT5cI26Z887Ixov3PdbIMc02lHQi8DXg/RHxzChHc2pNiDbitfqwmuJTOH19fQwODgLQdwJ84PQjdfteaddNRkZGurJfE2W0nZGkCd8Z1csdyDN/YOrkUKf3Pb2SO72eL00VM0nHUiTTxoj4ego/IWlO2hnNAQ6m+DAwvzT7PGB/ivdXxQdTfF6N9i8REWuBtQBLly6N/v5icddv3MS1O+tvyr5L++tO65TBwUEq/e91nd4ZQf3cgTzzB6ZGDnXDvqdXcqfX86WZqxkF3AjsjohPlyZtBion4geATaX4qnQyfxlwOH0K3woslzQzne9YDmxN056VtCyta1VpWZa50XZGaXqzO6N68aY+CFl+vO+xVjRzzuxNwDuBN0t6IA0XANcA50raA5ybngPcBjwKDAFfAN4DEBGHgKuBe9PwsRQDuBz4YppnL7BlHLbNOsw7Ixsj73usaQ0PM0bEndQ+nANwTo32AVxRZ1nrgHU14vcBpzXqi2WnsjPaKemBFPtrip3PrZIuA34CXJSm3QZcQLFj+QXwLih2RpIqOyN46c7oJuAEih2Rd0Y9wvsea0XTF4CYtco7IzObLL6dlZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfYaFjNJ6yQdlPRQKTZL0jZJe9LjzBSXpOskDUnaIenM0jwDqf0eSQOl+BJJO9M810nSeG+kdY7zx9rl3LFWNPPN7CZgRVVsDXB7RCwCbk/PAc4HFqVhNfB5KBIQuAo4GzgLuKqShKnN6tJ81euyvN2E88facxPOHWtSw2IWEXcAh6rCK4H1aXw9cGEpviEKdwEzJM0BzgO2RcShiHgK2AasSNNOiojvRUQAG0rLsh7g/LF2OXesFce0OV9fRBwAiIgDkk5O8bnAY6V2wyk2Wny4RrwmSaspPknR19fH4OBg0ZkT4AOnH6nb2Uq7bjIyMtKV/Zokk54/9XIH8swfmLI55NxpU6/nS7vFrJ5ax5yjjXhNEbEWWAuwdOnS6O/vB+D6jZu4dmf9Tdl3aX/daZ0yODhIpf/2GxOWP/VyB/LMH3AOVXHuNNDr+dLu1YxPpK/ppMeDKT4MzC+1mwfsbxCfVyNuvc35Y+1y7lhN7RazzUDlqqABYFMpvipdWbQMOJwOCWwFlkuamU6+Lge2pmnPSlqWriRaVVqW9S7nj7XLuWM1NTzMKOlLQD8wW9IwxZVB1wC3SroM+AlwUWp+G3ABMAT8AngXQEQcknQ1cG9q97GIqJzYvZziqqUTgC1psB7h/LF2OXesFQ2LWURcUmfSOTXaBnBFneWsA9bViN8HnNaoH5Yn54+1y7ljrfAdQMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLXNcVM0gpJj0gakrSm0/2xfDh3bCycP72hK4qZpGnA54DzgcXAJZIWd7ZXlgPnjo2F86d3dEUxA84ChiLi0Yh4HrgFWNnhPlkenDs2Fs6fHnFMpzuQzAUeKz0fBs6ubiRpNbA6PR2R9Eganw08WW/h+tQ49XJ8jdrnDnltpzvQhrHmDuSZP9BdOZRj7kAT+dNDudNN+VJtzPnTLcVMNWLxkkDEWmDtS2aW7ouIpRPRsYmSY5+71JhyB/J9L3Ltd5dpmD+9kjs59bUd3XKYcRiYX3o+D9jfob5YXpw7NhbOnx7RLcXsXmCRpIWSjgMuBjZ3uE+WB+eOjYXzp0d0xWHGiDgi6UpgKzANWBcRu1pYRM1DAF0uxz53nXHIHcj3vci1311jiu17cupryxTxktMLZmZmWemWw4xmZmZtczEzM7PsZV/MOnErGkn7JO2U9ICk+1JslqRtkvakx5kpLknXpf7tkHRmaTkDqf0eSQOl+JK0/KE0r0Zbh7VnMnNH0jpJByU9VIp1LGdGW4c1Z5Lzx/ucRiIi24HihO1e4BTgOOBBYPEkrHcfMLsq9rfAmjS+BvhUGr8A2ELx/yzLgLtTfBbwaHqcmcZnpmn3AG9M82wBzh9tHR66P3eAPwLOBB7qhpyptw4PXZs/3uc0GHL/ZtZNt6JZCaxP4+uBC0vxDVG4C5ghaQ5wHrAtIg5FxFPANmBFmnZSRHwvigzaULWsWuuw1k1q7kTEHcChqnAnc6beOqw53bDv8T6nJPdiVutWNHMnYb0B/Juk+1Xc6gagLyIOAKTHkxv0cbT4cI34aOuw1nUqd8o6mTPdsP05m+zXz/ucBrri/8zGoKlbGU2AN0XEfkknA9sk/XCUtvX62Grcxlc3v86TkTPdvP05mOzXz/ucBnL/ZtaRW9FExP70eBD4BsUhhycqh2nS48EGfRwtPq9GnFHWYa3rhtsYdTJnumH7czapr5/3OY3lXswm/VY0kqZLekVlHFgOPJTWW7k6aADYlMY3A6vSFUbLgMPp6/pWYLmkmekKoeXA1jTtWUnL0hVFq6qWVWsd1rpuuI1RJ3Om3jqsOZOWP97nNKnTV6CMdaC4cuffKa4s+sgkrO8UiiuXHgR2VdYJvAq4HdiTHmeluCh+/G8vsBNYWlrWu4GhNLyrFF9Kkax7gc/y2zu11FyHh+7PHeBLwAHgBYpPwpd1MmdGW4eH7sof73OaG3w7KzMzy17uhxnNzMxczMzMLH8uZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyyN2WKmaR9kn4paUTSE5L+QdKJpek3SToi6TWl2Lmp7exS7HhJuyX9eRPrnJ7Wd1ud/jxfXnaKPyApJC2QtCXNPyLphdS+8vyGqvmuSvO9pdXXxhrrxfxJbaIUG5H0v9p/layWXsyd1P53JP29pCclHZZ0R7uv0bjo9A+qTeIP6e0D3pLG51L8EN016fl04Fng58CHqua7EdhYen418G3Sj9c1WOdAWuYRYE6N/jwC/EUpdnqKBbCgqv1NwMfrrOf3KH6Eb39lGz04fxrlD7AgtT2m069vLw+9mDsp/o/ALcCrgWnAkk6+zlPmm1lZRPwU2AKclkJvA54GPsZvfyK84i+BP5b0VkmnAVcCfxbp3WxgALgB2AFcWmP6zRQ/UV5uv6HZ7Sj5LPBXwPNtzGst6sH8sUnSK7kj6T8B/w1YHRE/i4gXI+L+ZuefCFOymEmaT/GT5z9IoQGKn7W/Bfh9SWdW2kbEYeByisRYB3w0IvY2sY7fBfqBjWlYVaPZXcBJkk6VNA14B8WnnVa25SLg+Yh4yeEEmxi9lD/JjyUNp8Nfsxs3t3b1UO6cDfwY+Gg6zLhT0ttamH/cTbVi9s+SngbuBL4DfDK98X8C/FNEPAHcTtUnpIj4JsWb/zLguibXtQrYEREPUyTr6yWdUaNd5RPSucAPgZ82uzHpuPsngfc3O4+NSU/lD/Ak8F+B1wJLgFdQ7Pxs/PVa7syj+HZ5GHgNxbfG9ZJObWEZ4+qYTq24Qy6MiG+VA5LeCeyOiAdSaCNwraQPRsQLpaa7gF9HxH80ua5VwBcAImK/pO9QJOoPqtrdDNwBLKT1Q0QfBW6OiB+1OJ+1p6fyJyJGgPvS0yckXQkckHRSRDzTyrKsoZ7KHeCXwAsU59KOAN+RtB1YDuxucVnjYqp9M6tlFXCKpMclPQ58GpgNnN/uAiX9AbAI+HBpuWcDl0g66gNERPwY+BHFoYevt7iqc4D3ltYxH7hV0l+123drWc75U61yLkZjXI41J+fc2dFuHyfKVPtmdhRJb6S4EvAM4GelSddSfJLZ3OaiB4BtHH2s+gSKBDgf+GZV+8uAmRHxXHXCNXAOcGzp+b0UJ423tNxja1nu+SPpbIqLD/YAMykOYw2mczU2gXLPHYpvdD+hKJr/h6Jg9gMfarPfYzalixnFG78pInaWg5I+A3xX0qyIONTKAiW9HPhTYFVEPF417ea0zqMSqpmTurVExM+rlv8i8FQ6fGQTL+v8AU6hOOd6MvAMxU7wkjaXZa3JOnci4gVJK4EvAmsoLgZZFRE/bGd540HNXeVpZmbWvXzOzMzMsudi1iZJl+ro2wBVhl2d7pt1P+ePtcu5U5sPM5qZWfYaXgCSTireARyf2n81Iq6StJDiv9ZnAd8H3hkRz0s6nuJ/FpZQ3BvsHRGxLy3rwxRXz7wIvDcitqb4CuAzFPf3+mJEXNOoX7Nnz44FCxYA8NxzzzF9+vQWNrt3jOe233///U9GxKvHZWHkkTuQZ/50W5/HO3egO/OnF3KnExq9TuOSP41u3kjxPycnpvFjgbuBZcCtwMUpfgNweRp/D3BDGr8Y+HIaXww8SJGYC4G9FAk0LY2fAhyX2ixu1K8lS5ZExfbt22OqGs9tB+6L8b3Batfnzni/hpOl2/o83rkTXZo/vZA7ndDodRqP/Gl4ziytq3Kp97FpCODNwFdTfD1wYRpfmZ6Tpp8jSSl+S0T8Ooo7VgwBZ6VhKCIejYjnKT5xrWzUL+t+zh0bC+ePtaKp/zNLN6K8H3gd8DmKTzNPR3EbE4Bhip82ID0+BhARRyQdBl6V4neVFlue57Gq+Nktb4l1pW7JHUmrgdUAfX19DA4O/mbayMjIUc9zkGOf29EN+dNrudMJk/E6NVXMIuJF4A2SZgDfAGrdTHK0W+HEKPFa3w5rXpVSL6mmckJ1+7Z3S+5ExFpgLcDSpUujv7//N9MGBwcpP89Bjn1uRzfkT6/lTidMxuvU0h1AIuJpSYMUx61nSDomfUKaR/HDkFB8upkPDKfbo7wSOFSKV5TnqRevXn/NpLp+4yauvfO5uv3ed81bm97G3OTyx9Tp3BnNzp8e5n+s+de603s5f3LRrfnj3OkeDc+ZSXp1+lSEpBOAt1DcFXk78PbUbADYlMY389ufMXg78O10gm8zcLGKn/5eSHEzzHso7ie4SNJCScdRnLht975k1kWcOzYWzh9rRTPfzOZQ/E7NNIrid2tE/Iukh4FbJH2c4qcFbkztbwRuljRE8anoYoCI2CXpVuBhip/yviIdQkDFT09spbi6aF1ETOl//ushzh0bC+ePNa1hMYuIHRR3dq6OP0pxNVB1/FfARXWW9QngEzXitwH+peQe49yxsXD+WCt8OyszM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZjZhJE0X9J2Sbsl7ZL0vhSfJWmbpD3pcWaKS9J1koYk7ZB0ZmlZA6n9HkkDpfgSSTvTPNdJ0uRvqU0E54+1omExc0LZGBwBPhARpwLLgCskLQbWALdHxCLg9vQc4HxgURpWA5+HIteAq4CzgbOAqyr5ltqsLs23YhK2yyaH88ea1sw3MyeUtSUiDkTE99P4s8BuYC6wElifmq0HLkzjK4ENUbgLmCFpDnAesC0iDkXEU8A2YEWadlJEfC8iAthQWpZlzvljrWhYzJxQNh4kLQDOAO4G+iLiABT5BZycms0FHivNNpxio8WHa8Stxzh/rJFjWmk8WkJJmvCEkrSa4hscfX19DA4OAtB3Anzg9CN1+11p14tGRka6fvsknQh8DXh/RDwzylHkWhOijXitPtTMHcgzf3J438dLp/On13KnEyYjX5suZp1OKICIWAusBVi6dGn09/cDcP3GTVy7s/6m7Lu0v+603A0ODlJ5HbqRpGMp8mZjRHw9hZ+QNCd9CJoDHEzxYWB+afZ5wP4U76+KD6b4vBrtX6Je7kCe+dPt7/t46Yb86bXc6YTJyNemrmYcLaHS9GYTql68qR2S5SVdyHMjsDsiPl2atBmoXAA0AGwqxVeli4iWAYfTt/+twHJJM9N51uXA1jTtWUnL0rpWlZZlmXP+WCuauZrRCWXtehPwTuDNkh5IwwXANcC5kvYA56bnALcBjwJDwBeA9wBExCHgauDeNHwsxQAuB76Y5tkLbJmMDbNJ4fyxpjVzmLGSUDslPZBif02RQLdKugz4CXBRmnYbcAFFcvwCeBcUCSWpklDw0oS6CTiBIpmcUD0gIu6k9mFkgHNqtA/gijrLWgesqxG/DzhtDN20LuX8sVY0LGZOKDMz63a+A4iZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2GhYzSeskHZT0UCk2S9I2SXvS48wUl6TrJA1J2iHpzNI8A6n9HkkDpfgSSTvTPNdJ0nhvpHWO88fa5dyxVjTzzewmYEVVbA1we0QsAm5PzwHOBxalYTXweSgSELgKOBs4C7iqkoSpzerSfNXrsrzdhPPH2nMTzh1rUsNiFhF3AIeqwiuB9Wl8PXBhKb4hCncBMyTNAc4DtkXEoYh4CtgGrEjTToqI70VEABtKy7Ie4Pyxdjl3rBXHtDlfX0QcAIiIA5JOTvG5wGOldsMpNlp8uEa8JkmrKT5J0dfXx+DgYNGZE+ADpx+p29lKu140MjKS4/ZNev7Uyx3IM38yfd/Hg3MnQ5ORr+0Ws3pqHXOONuI1RcRaYC3A0qVLo7+/H4DrN27i2p31N2Xfpf11p+VucHCQyuvQAyYsf+rlDuSZPz32vo8H504Xm4x8bfdqxifS13TS48EUHwbml9rNA/Y3iM+rEbfe5vyxdjl3rKZ2i9lmoHJV0ACwqRRfla4sWgYcTocEtgLLJc1MJ1+XA1vTtGclLUtXEq0qLct6l/PH2uXcsZoaHmaU9CWgH5gtaZjiyqBrgFslXQb8BLgoNb8NuAAYAn4BvAsgIg5Juhq4N7X7WERUTuxeTnHV0gnAljRYj3D+WLucO9aKhsUsIi6pM+mcGm0DuKLOctYB62rE7wNOa9QPy5Pzx9rl3LFW+A4gZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7HVNMZO0QtIjkoYkrel0fywfzh0bC+dPb+iKYiZpGvA54HxgMXCJpMWd7ZXlwLljY+H86R3HdLoDyVnAUEQ8CiDpFmAl8PB4LHzBmn8ddfq+a946HquxzpjQ3IHG+TMa51bX876nR3RLMZsLPFZ6PgycXd1I0mpgdXo6IumRND4beLLdletT7c7ZFca07VVeO07LmUxjzR0Y39fw6PVOXG5NWJ/blGPuQBP5M5G5k/m+pxWNXqcx50+3FDPViMVLAhFrgbUvmVm6LyKWTkTHut1U3vZkTLkDeb6GOfa5SzXMn17LnU6YjNepK86ZUXwaml96Pg/Y36G+WF6cOzYWzp8e0S3F7F5gkaSFko4DLgY2d7hPlgfnjo2F86dHdMVhxog4IulKYCswDVgXEbtaWETNQwBTxFTe9vHIHcjzNcyxz13H+55JM+GvkyJecnrBzMwsK91ymNHMzIT8XAIAAAPVSURBVKxtLmZmZpa97ItZzreikbRP0k5JD0i6L8VmSdomaU96nJniknRd2s4dks4sLWcgtd8jaaAUX5KWP5Tm1WjrmGo6kTuS5kvaLmm3pF2S3pfifyPppykXHpB0QWmeD6c+PiLpvEb9Txcz3J3e3y+nCxuQdHx6PpSmL5iMbe5VOe97WpHNfioish0oTtjuBU4BjgMeBBZ3ul8t9H8fMLsq9rfAmjS+BvhUGr8A2ELxfzHLgLtTfBbwaHqcmcZnpmn3AG9M82wBzh9tHVNp6FTuAHOAM9P4K4B/p7iN0t8AH6zRfnHq2/HAwtTnaaP1H7gVuDiN3wBcnsbfA9yQxi8Gvtzp9yHXIfd9T4vbmsV+KvdvZr+5FU1EPA9UbkWTs5XA+jS+HriwFN8QhbuAGZLmAOcB2yLiUEQ8BWwDVqRpJ0XE96LIhg1Vy6q1jqmkI7kTEQci4vtp/FlgN8VdKOpZCdwSEb+OiB8BQxR9r9n/9Kn2zcBX0/zVOVR5378KnFP5FGwt68V9Tyu6bj+VezGrdSua0XYM3SaAf5N0v4pb5gD0RcQBKHZ8wMkpXm9bR4sP14iPto6ppOO5kw7znQHcnUJXpkMz60qHVFp9318FPB0RR6riRy0rTT+c2lvrOp4/kyiL/VRX/J/ZGDR1K6Mu9qaI2C/pZGCbpB+O0rbetrYat0JHXx9JJwJfA94fEc9I+jxwderD1cC1wLtH6WetD6KN3nfnxPiZSq9lFvup3L+ZZX0rmojYnx4PAt+gOHTxRPrqTXo8mJrX29bR4vNqxBllHVNJx3JH0rEUhWxjRHwdICKeiIgXI+I/gC9Q5MJo/awXf5Li0M4xVfGjlpWmvxI4NL5bN2Vkve9pRS77qdyLWba3opE0XdIrKuPAcuAhiv5XrvQZADal8c3AqnS10DLgcPrqvRVYLmlmOjS1HNiapj0raVk6L7Kqalm11jGVdCR30ntxI7A7Ij5dis8pNfvvFLlA6tPF6UrEhcAiihPmNfufzjtsB96e5q/Oocr7/nbg26m9tS7bfU8rstpPdfpKmXG40uYCiivC9gIf6XR/Wuj3KRRXQD0I7Kr0neIcxu3AnvQ4K8VF8SOCe4GdwNLSst5NcWHAEPCuUnxpSry9wGf57R1faq5jqg2dyB3gDykOo+wAHkjDBcDN6X3dkf6I55Tm+Ujq4yOkK71G63/KrXtSPnwFOD7FX56eD6Xpp3T6Pch5yHXf0+I2ZrOf8u2szMwse7kfZjQzM3MxMzOz/LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZll7/8DgrC436upn5EAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "def draw_histograms(df, variables, n_rows, n_cols, n_bins):\n", + " fig=plt.figure()\n", + " for i, var_name in enumerate(variables):\n", + " ax=fig.add_subplot(n_rows,n_cols,i+1)\n", + " df[var_name].hist(bins=n_bins,ax=ax)\n", + " ax.set_title(var_name)\n", + " fig.tight_layout() # Improves appearance a bit.\n", + " plt.show()\n", + "\n", + "PAY = df[['PAY_0','PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6']]\n", + "BILLAMT = df[['BILL_AMT1','BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6']]\n", + "PAYAMT = df[['PAY_AMT1','PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6']]\n", + "\n", + "draw_histograms(PAY, PAY.columns, 2, 3, 10)\n", + "draw_histograms(BILLAMT, BILLAMT.columns, 2, 3, 10)\n", + "draw_histograms(PAYAMT, PAYAMT.columns, 2, 3, 10)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "C6c_Gz6wUrJ8" + }, + "source": [ + "We observe that the \"repayment status\" attributes are the most highly correlated with the target variable and we would expect them to be more significant in predicting credit default. In fact the later the status (pay_0 is later than pay_6), the more correlated it is.\n", + "\n", + "Now that we have an idea of the features, we will move on to feature selection and data preparation." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "AQBksEyEf4Sf" + }, + "source": [ + "## Data Preprocessing\n", + "\n", + "It was previously mentioned that our data had a bit of noise, so we will clean up the data in this part. Additionally, we will conduct some feature selection.\n", + "1. Removing Noise - Inconsistencies\n", + "2. Dealing with negative values of PAY_0 to PAY_6\n", + "3. Outliers\n", + "4. One Hot Encoding\n", + "5. Train Test Split\n", + "6. Feature selection\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Removing Noise\n", + "First, we found in our data exploration that education has unknown groups 0, 5 and 6. These will be replaced with Education = Others, which has value 4" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([2, 1, 3, 4], dtype=int64)" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['EDUCATION'].replace([0,5,6], 4, regex=True, inplace=True)\n", + "df[\"EDUCATION\"].unique()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly, for Marriage" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([1, 2, 3], dtype=int64)" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['MARRIAGE'].replace([0], 3, regex=True, inplace=True)\n", + "df[\"MARRIAGE\"].unique()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Separating negative and positive values for PAY_0 to PAY_6" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Second, we are going to extract the negative values of PAY_0 to PAY_6 as another categorical feature. This way, PAY_0 to PAY_6 can be thought of purely as the months of delay of payments.\n", + "\n", + "The negative values will form a categorical variable. e.g. negative values of PAY_0 will form the categorical variable S_0." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(0,7):\n", + " try:\n", + " df[\"S_\" + str(i)] = [x if x < 1 else 1 for x in df[\"PAY_\" + str(i)]]\n", + " except:\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dummy variables for negative values\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
S_0S_2S_3S_4S_5S_6
ID
111-1-1-2-2
2-110001
3000000
4000000
5-10-1000
.....................
29995111111
29996000000
29997-1-1-1-100
29998111-100
30000000000
\n", + "

26245 rows × 6 columns

\n", + "
" + ], + "text/plain": [ + " S_0 S_2 S_3 S_4 S_5 S_6\n", + "ID \n", + "1 1 1 -1 -1 -2 -2\n", + "2 -1 1 0 0 0 1\n", + "3 0 0 0 0 0 0\n", + "4 0 0 0 0 0 0\n", + "5 -1 0 -1 0 0 0\n", + "... ... ... ... ... ... ...\n", + "29995 1 1 1 1 1 1\n", + "29996 0 0 0 0 0 0\n", + "29997 -1 -1 -1 -1 0 0\n", + "29998 1 1 1 -1 0 0\n", + "30000 0 0 0 0 0 0\n", + "\n", + "[26245 rows x 6 columns]" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "print('Dummy variables for negative values')\n", + "df[[\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\"]]" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "#attributes representing positive values\n", + "for col in [\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"]:\n", + " df[col].replace([0,-1,-2], 0, regex=True, inplace=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Outliers\n", + "Next, we would like to remove outliers from the continuous variables. Assuming that all the data points are normally distributed, we will consider a point an outlier if it falls outside the 99% interval of a distribution. (Critical value = 2.58) " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...PAY_AMT4PAY_AMT5PAY_AMT6YS_0S_2S_3S_4S_5S_6
count26245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.000000...26245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.000000
mean149324.8999811.6089541.8527341.56471735.0065920.3721090.3373210.3246330.2782240.238750...2787.4250712778.8306732822.2850070.230177-0.133587-0.300438-0.327300-0.364412-0.395999-0.428158
std116558.6165300.4879940.7385720.5219368.8320280.7657300.8148780.8114910.7863140.743923...4835.0819064751.2632875271.1981000.4209540.8798760.8834720.8952640.8861150.8777890.900723
min10000.0000001.0000001.0000001.00000021.0000000.0000000.0000000.0000000.0000000.000000...0.0000000.0000000.0000000.000000-2.000000-2.000000-2.000000-2.000000-2.000000-2.000000
25%50000.0000001.0000001.0000001.00000028.0000000.0000000.0000000.0000000.0000000.000000...150.00000082.0000000.0000000.000000-1.000000-1.000000-1.000000-1.000000-1.000000-1.000000
50%120000.0000002.0000002.0000002.00000034.0000000.0000000.0000000.0000000.0000000.000000...1200.0000001218.0000001143.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000
75%210000.0000002.0000002.0000002.00000041.0000000.0000000.0000000.0000000.0000000.000000...3118.0000003140.0000003069.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000
max500000.0000002.0000004.0000003.00000059.0000008.0000008.0000008.0000008.0000008.000000...45171.00000044197.00000051000.0000001.0000001.0000001.0000001.0000001.0000001.0000001.000000
\n", + "

8 rows × 30 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean 149324.899981 1.608954 1.852734 1.564717 35.006592 \n", + "std 116558.616530 0.487994 0.738572 0.521936 8.832028 \n", + "min 10000.000000 1.000000 1.000000 1.000000 21.000000 \n", + "25% 50000.000000 1.000000 1.000000 1.000000 28.000000 \n", + "50% 120000.000000 2.000000 2.000000 2.000000 34.000000 \n", + "75% 210000.000000 2.000000 2.000000 2.000000 41.000000 \n", + "max 500000.000000 2.000000 4.000000 3.000000 59.000000 \n", + "\n", + " PAY_0 PAY_2 PAY_3 PAY_4 PAY_5 \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean 0.372109 0.337321 0.324633 0.278224 0.238750 \n", + "std 0.765730 0.814878 0.811491 0.786314 0.743923 \n", + "min 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "25% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "50% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "75% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "max 8.000000 8.000000 8.000000 8.000000 8.000000 \n", + "\n", + " ... PAY_AMT4 PAY_AMT5 PAY_AMT6 Y \\\n", + "count ... 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean ... 2787.425071 2778.830673 2822.285007 0.230177 \n", + "std ... 4835.081906 4751.263287 5271.198100 0.420954 \n", + "min ... 0.000000 0.000000 0.000000 0.000000 \n", + "25% ... 150.000000 82.000000 0.000000 0.000000 \n", + "50% ... 1200.000000 1218.000000 1143.000000 0.000000 \n", + "75% ... 3118.000000 3140.000000 3069.000000 0.000000 \n", + "max ... 45171.000000 44197.000000 51000.000000 1.000000 \n", + "\n", + " S_0 S_2 S_3 S_4 S_5 \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean -0.133587 -0.300438 -0.327300 -0.364412 -0.395999 \n", + "std 0.879876 0.883472 0.895264 0.886115 0.877789 \n", + "min -2.000000 -2.000000 -2.000000 -2.000000 -2.000000 \n", + "25% -1.000000 -1.000000 -1.000000 -1.000000 -1.000000 \n", + "50% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "75% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "max 1.000000 1.000000 1.000000 1.000000 1.000000 \n", + "\n", + " S_6 \n", + "count 26245.000000 \n", + "mean -0.428158 \n", + "std 0.900723 \n", + "min -2.000000 \n", + "25% -1.000000 \n", + "50% 0.000000 \n", + "75% 0.000000 \n", + "max 1.000000 \n", + "\n", + "[8 rows x 30 columns]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from scipy import stats\n", + "#we are only concerned with the ordinal data\n", + "o = pd.DataFrame(df.drop(['Y','EDUCATION', 'MARRIAGE', \"SEX\",\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\",\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"], axis=1))\n", + "#rows where the absolute z score of all columns are less than 2.58 (critical value)\n", + "rows = (np.abs(stats.zscore(o)) < 2.58).all(axis=1)\n", + "df = df[rows]\n", + "df.describe()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Feature Scaling\n", + "The models used subsequently may have difficulty converging before the maximum number of iterations allowed\n", + "is reached if the data is not normalized. Additionaly, Multi-layer Perceptron is sensitive to feature scaling, so we will use StandardScaler for standardization. We only want to scale the numerical factors." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.preprocessing import MinMaxScaler\n", + "scaler = MinMaxScaler()\n", + "cols = df.drop(['Y','EDUCATION', 'MARRIAGE', \"SEX\",\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\",\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"], axis =1)\n", + "df1 = df.copy()\n", + "df1[cols.columns] = scaler.fit_transform(cols)\n", + "df = df1" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...PAY_AMT4PAY_AMT5PAY_AMT6YS_0S_2S_3S_4S_5S_6
ID
10.0204082210.07894722000...0.0000000.0000000.000000111-1-1-2-2
20.2244902220.13157902000...0.0221380.0000000.0392161-110001
30.1632652220.34210500000...0.0221380.0226260.0980390000000
40.0816332210.42105300000...0.0243520.0241870.0196080000000
50.0816331210.94736800000...0.1992430.0155890.0133140-10-1000
\n", + "

5 rows × 30 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 \\\n", + "ID \n", + "1 0.020408 2 2 1 0.078947 2 2 0 0 \n", + "2 0.224490 2 2 2 0.131579 0 2 0 0 \n", + "3 0.163265 2 2 2 0.342105 0 0 0 0 \n", + "4 0.081633 2 2 1 0.421053 0 0 0 0 \n", + "5 0.081633 1 2 1 0.947368 0 0 0 0 \n", + "\n", + " PAY_5 ... PAY_AMT4 PAY_AMT5 PAY_AMT6 Y S_0 S_2 S_3 S_4 S_5 S_6 \n", + "ID ... \n", + "1 0 ... 0.000000 0.000000 0.000000 1 1 1 -1 -1 -2 -2 \n", + "2 0 ... 0.022138 0.000000 0.039216 1 -1 1 0 0 0 1 \n", + "3 0 ... 0.022138 0.022626 0.098039 0 0 0 0 0 0 0 \n", + "4 0 ... 0.024352 0.024187 0.019608 0 0 0 0 0 0 0 \n", + "5 0 ... 0.199243 0.015589 0.013314 0 -1 0 -1 0 0 0 \n", + "\n", + "[5 rows x 30 columns]" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### One-Hot Encoding for Categorical attributes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In some models, categorical variables which are encoded numerically will be erroneously treated as ordinal data. To understand why this is a problem, consider the \"Education\" column for our dataset.\n", + "\n", + "A logistic regression model, for example, will assume that the difference in odds of default between education = 1 and education = 2 is the same as the difference between education = 2 and 3. This is wrong because the difference in odds between a graduate degree and university (1 and 2) is likely to be different from that between univeristy education and high school education (2 and 3).\n", + "\n", + "One hot encoding will allow our models to treat these columns explicitly as categorical features.\n", + "\n", + "The following categorical columns will be one-hot encoded\n", + "\n", + "1. EDUCATION\n", + "2. MARRIAGE\n", + "3. S0 - S6\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.preprocessing import OneHotEncoder" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "onenc = OneHotEncoder(categories='auto')" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
GRADUNIHSMARRIEDSINGLE
00.01.00.01.00.0
10.01.00.00.01.0
20.01.00.00.01.0
30.01.00.01.00.0
40.01.00.01.00.0
\n", + "
" + ], + "text/plain": [ + " GRAD UNI HS MARRIED SINGLE\n", + "0 0.0 1.0 0.0 1.0 0.0\n", + "1 0.0 1.0 0.0 0.0 1.0\n", + "2 0.0 1.0 0.0 0.0 1.0\n", + "3 0.0 1.0 0.0 1.0 0.0\n", + "4 0.0 1.0 0.0 1.0 0.0" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#one hot encoding for EDUCATION and MARRIAGE\n", + "onehot = pd.DataFrame(onenc.fit_transform(df[['EDUCATION', 'MARRIAGE']]).toarray())\n", + "onehot.columns= names = [\"GRAD\",\"UNI\",\"HS\",\"OTHER-EDU\",\"MARRIED\",\"SINGLE\",\"OTHER_MS\"]\n", + "#drop one of each category to prevent dummy variable trap\n", + "onehot = onehot.drop([\"OTHER-EDU\", \"OTHER_MS\"], axis = 1)\n", + "onehot.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PAY_0_No_TransactionsPAY_0_Pay_DulyPAY_0_Revolving_CreditPAY_2_No_TransactionsPAY_2_Pay_DulyPAY_2_Revolving_CreditPAY_3_No_TransactionsPAY_3_Pay_DulyPAY_3_Revolving_CreditPAY_4_No_TransactionsPAY_4_Pay_DulyPAY_4_Revolving_CreditPAY_5_No_TransactionsPAY_5_Pay_DulyPAY_5_Revolving_CreditPAY_6_No_TransactionsPAY_6_Pay_DulyPAY_6_Revolving_Credit
00.00.00.00.00.00.00.01.00.00.01.00.01.00.00.01.00.00.0
10.01.00.00.00.00.00.00.01.00.00.01.00.00.01.00.00.00.0
20.00.01.00.00.01.00.00.01.00.00.01.00.00.01.00.00.01.0
30.00.01.00.00.01.00.00.01.00.00.01.00.00.01.00.00.01.0
40.01.00.00.00.01.00.01.00.00.00.01.00.00.01.00.00.01.0
\n", + "
" + ], + "text/plain": [ + " PAY_0_No_Transactions PAY_0_Pay_Duly PAY_0_Revolving_Credit \\\n", + "0 0.0 0.0 0.0 \n", + "1 0.0 1.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 1.0 0.0 \n", + "\n", + " PAY_2_No_Transactions PAY_2_Pay_Duly PAY_2_Revolving_Credit \\\n", + "0 0.0 0.0 0.0 \n", + "1 0.0 0.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_3_No_Transactions PAY_3_Pay_Duly PAY_3_Revolving_Credit \\\n", + "0 0.0 1.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 1.0 0.0 \n", + "\n", + " PAY_4_No_Transactions PAY_4_Pay_Duly PAY_4_Revolving_Credit \\\n", + "0 0.0 1.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_5_No_Transactions PAY_5_Pay_Duly PAY_5_Revolving_Credit \\\n", + "0 1.0 0.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_6_No_Transactions PAY_6_Pay_Duly PAY_6_Revolving_Credit \n", + "0 1.0 0.0 0.0 \n", + "1 0.0 0.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 " + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#one hot encoding for S_0 to S_6\n", + "onehot_PAY = pd.DataFrame(onenc.fit_transform(df[['S_0', 'S_2', 'S_3', 'S_4', 'S_5', 'S_6']]).toarray())\n", + "onehot_PAY.columns= onenc.fit(df[[\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\"]]).get_feature_names()\n", + "#drop one of each category to prevent dummy variable trap\n", + "#onehot = onehot.drop([\"OTHER-EDU\", \"OTHER_MS\"], axis = 1)\n", + "names = []\n", + "for X in range(0,7):\n", + " if X == 1:\n", + " continue\n", + " names.append(\"PAY_\"+str(X)+\"_No_Transactions\")\n", + " names.append(\"PAY_\"+str(X)+\"_Pay_Duly\")\n", + " names.append(\"PAY_\"+str(X)+\"_Revolving_Credit\")\n", + " try:\n", + " onehot_PAY = onehot_PAY.drop(\"x\" + str(X) +\"_1\", axis =1)\n", + " except:\n", + " onehot_PAY = onehot_PAY.drop(\"x1_1\", axis =1)\n", + "onehot_PAY.columns = names\n", + "onehot_PAY.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Index(['LIMIT_BAL', 'SEX', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5',\n", + " 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4',\n", + " 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3',\n", + " 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'Y', 'GRAD', 'UNI', 'HS', 'MARRIED',\n", + " 'SINGLE', 'PAY_0_No_Transactions', 'PAY_0_Pay_Duly',\n", + " 'PAY_0_Revolving_Credit', 'PAY_2_No_Transactions', 'PAY_2_Pay_Duly',\n", + " 'PAY_2_Revolving_Credit', 'PAY_3_No_Transactions', 'PAY_3_Pay_Duly',\n", + " 'PAY_3_Revolving_Credit', 'PAY_4_No_Transactions', 'PAY_4_Pay_Duly',\n", + " 'PAY_4_Revolving_Credit', 'PAY_5_No_Transactions', 'PAY_5_Pay_Duly',\n", + " 'PAY_5_Revolving_Credit', 'PAY_6_No_Transactions', 'PAY_6_Pay_Duly',\n", + " 'PAY_6_Revolving_Credit'],\n", + " dtype='object')" + ] + }, + "execution_count": 97, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1 = df.drop(['EDUCATION', 'MARRIAGE','S_0', 'S_2', 'S_3', 'S_4', 'S_5', 'S_6'], axis = 1)\n", + "df1 = pd.concat([df1.reset_index(drop=True), onehot], axis=1)\n", + "df1 = pd.concat([df1.reset_index(drop=True), onehot_PAY], axis=1)\n", + "df1.columns" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXAGEPAY_0PAY_2PAY_3PAY_4PAY_5PAY_6BILL_AMT1...PAY_3_Revolving_CreditPAY_4_No_TransactionsPAY_4_Pay_DulyPAY_4_Revolving_CreditPAY_5_No_TransactionsPAY_5_Pay_DulyPAY_5_Revolving_CreditPAY_6_No_TransactionsPAY_6_Pay_DulyPAY_6_Revolving_Credit
\n", + "

0 rows × 45 columns

\n", + "
" + ], + "text/plain": [ + "Empty DataFrame\n", + "Columns: [LIMIT_BAL, SEX, AGE, PAY_0, PAY_2, PAY_3, PAY_4, PAY_5, PAY_6, BILL_AMT1, BILL_AMT2, BILL_AMT3, BILL_AMT4, BILL_AMT5, BILL_AMT6, PAY_AMT1, PAY_AMT2, PAY_AMT3, PAY_AMT4, PAY_AMT5, PAY_AMT6, Y, GRAD, UNI, HS, MARRIED, SINGLE, PAY_0_No_Transactions, PAY_0_Pay_Duly, PAY_0_Revolving_Credit, PAY_2_No_Transactions, PAY_2_Pay_Duly, PAY_2_Revolving_Credit, PAY_3_No_Transactions, PAY_3_Pay_Duly, PAY_3_Revolving_Credit, PAY_4_No_Transactions, PAY_4_Pay_Duly, PAY_4_Revolving_Credit, PAY_5_No_Transactions, PAY_5_Pay_Duly, PAY_5_Revolving_Credit, PAY_6_No_Transactions, PAY_6_Pay_Duly, PAY_6_Revolving_Credit]\n", + "Index: []\n", + "\n", + "[0 rows x 45 columns]" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check for perfect collinearity\n", + "corr = df1.corr()\n", + "for i in range(len(corr)):\n", + " corr.iloc[i,i] = 0\n", + "#corr[corr == 1] = 0\n", + "corr[corr.eq(1).any(1)]" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data has 45 Columns and 26245 Rows\n" + ] + } + ], + "source": [ + "size = df1.shape\n", + "print(\"Data has {} Columns and {} Rows\".format(size[1], size[0]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Train Test Split\n", + "\n", + "Before we conduct feature selection and model selection, we split the data using a train test split according to the project description." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.metrics import *\n", + "from sklearn.model_selection import *" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "VOB68z_hM1jW" + }, + "outputs": [], + "source": [ + "#using holdout sampling for train test split\n", + "ft = df1.drop(\"Y\", axis = 1)\n", + "target = df1[\"Y\"]\n", + "X_train,X_test,y_train,y_test = train_test_split(ft,target,test_size=0.33333)\n", + "#make the results reproducible (according to instructions)\n", + "np.random.seed(123) " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Filter method for feature selection\n", + "The filter method for feature selection entails selecting relevant attributes before moving on to learning phase.\n", + "We will utitlise univariate feature selection to reduce the features to the fewer more significant attributes. " + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Significant values are:\n", + " 0 pval\n", + "LIMIT_BAL 81.747784 3.335170e-04\n", + "PAY_0 4215.816943 0.000000e+00\n", + "PAY_2 3730.186349 0.000000e+00\n", + "PAY_3 3017.725262 0.000000e+00\n", + "PAY_4 3020.555618 0.000000e+00\n", + "PAY_5 2812.887854 0.000000e+00\n", + "PAY_6 2575.321192 0.000000e+00\n", + "PAY_0_No_Transactions 63.305610 2.349730e-02\n", + "PAY_0_Revolving_Credit 454.630595 0.000000e+00\n", + "PAY_2_Pay_Duly 76.591060 1.225866e-03\n", + "PAY_2_Revolving_Credit 241.568619 0.000000e+00\n", + "PAY_3_Pay_Duly 88.958886 4.790331e-05\n", + "PAY_3_Revolving_Credit 140.257620 3.064993e-12\n", + "PAY_4_Pay_Duly 82.933933 2.446182e-04\n", + "PAY_4_Revolving_Credit 98.729838 2.851222e-06\n", + "PAY_5_Pay_Duly 67.720209 9.456393e-03\n", + "PAY_5_Revolving_Credit 70.202609 5.486203e-03\n", + "PAY_6_Revolving_Credit 73.167377 2.782902e-03\n" + ] + } + ], + "source": [ + "from sklearn.feature_selection import SelectKBest\n", + "from sklearn.feature_selection import chi2\n", + "\n", + "selector = SelectKBest( score_func = chi2, k=10)\n", + "selector.fit(X_train, y_train)\n", + "np.set_printoptions(precision=10)\n", + "chi2data = pd.DataFrame(selector.scores_)\n", + "chi2data[\"pval\"] = 1 - stats.chi2.cdf(chi2data, 43)\n", + "chi2data.index = X_train.columns\n", + "\n", + "print(\"Significant values are:\")\n", + "print(chi2data[chi2data[\"pval\"] < 0.05])\n", + "\n", + "cols = chi2data[chi2data[\"pval\"] < 0.05].index\n", + "X_train_filter = X_train[cols]\n", + "X_test_filter = X_test[cols]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "mbhlIlQzZz7c" + }, + "source": [ + "## Model Selection\n", + "\n", + "In this part, we will fit machine learning models learnt in BT2101 to this classification problem, and pick the model that can produce the best results.\n", + "\n", + "We will be attempting to fit the following models:\n", + "\n", + "\n", + "- Decision Tree \n", + "- Logistic Regression\n", + "- Support Vector Machine\n", + "- Neural Network\n", + "\n", + "To make things easier, we define a get_roc function that will plot an ROC curve for all the models we evaluate, and a confusion matrix function.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "def get_roc(model, y_test, X_test, name):\n", + " try:\n", + " fpr = roc_curve(y_test,model.predict_proba(X_test)[:,1])[0]\n", + " tpr = roc_curve(y_test,model.predict_proba(X_test)[:,1])[1]\n", + " thresholds = roc_curve(y_test,model.predict_proba(X_test)[:,1])[2]\n", + " except:\n", + " fpr = roc_curve(y_test,model.predict(X_test))[0]\n", + " tpr = roc_curve(y_test,model.predict(X_test))[1]\n", + " thresholds = roc_curve(y_test,model.predict(X_test))[2]\n", + " plt.plot([0, 1], [0, 1], color='navy', linestyle='--')\n", + " plt.xlim([0.0, 1.0])\n", + " plt.ylim([0.0, 1.05])\n", + " plt.xlabel('False Positive Rate')\n", + " plt.ylabel('True Positive Rate')\n", + " plt.title('Receiver operating characteristic for ' + name)\n", + " plt.plot(fpr,tpr,label='ROC curve (AUC = %0.2f)' % (auc(fpr, tpr)))\n", + " plt.legend(loc=\"lower right\")\n", + " \n", + " #find- best threshold\n", + " optimal_idx = np.argmax(tpr - fpr)\n", + " optimal_threshold = thresholds[optimal_idx]\n", + " print(\"Optimal Threshold: \" + str(optimal_threshold))\n", + " \n", + " plt.show()\n", + " \n", + " return auc(fpr, tpr)" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "def confusion(y_test, predictions, name):\n", + " conf = pd.crosstab(y_test,predictions, rownames=['Actual'], colnames=['Predicted'])\n", + " print(\"Of \" + str(conf[0][1] + conf[1][1]) + \" Defaulters, the \" + name + \" identified \" + str(conf[1][1])) \n", + " return conf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Evaluation \n", + "We will select the model based on the model evaluation. The key metrics we will compute are:\n", + "\n", + "1. Accuracy\n", + "2. Recall\n", + "3. AUROC\n", + "\n", + "Because of the nature of a default detection problem, we would like to prioritise **recall** for defaults. \n", + "This means we will place more importance in correctly identifying a defaulter than avoiding misclassifying a non-defaulter. (Assumming that the bank loses more money when lending to a defaulter than not lending to a non-defaulter)\n", + "\n", + "However, simply predicting every data point as a defaulter will give us 100% recall. We have to also consider accuracy and AUROC to get a better idea of how our model performs.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation = pd.DataFrame(columns=['Model', 'Recall-1', 'AUROC'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "H89tM6NvaN17" + }, + "source": [ + "### Decision Trees\n", + "\n", + "#### Theory:\n", + "The decision tree algorithm aims to recursively split the data points in the training set until the data points are completely separated or well separated. At each iteration, the tree splits the datasets by the feature(s) that give the maximum reduction in heterogeneity, which is calculated by a heterogeneity index.\n", + "\n", + "Below is a binary decision tree that has been split for a few iterations.\n", + "\n", + "![image.png](https://elf11.github.io/images/decisionTree.png)\n", + "\n", + "Since the target for this project is binary (fraud = yes or no) we will be building a binary decision tree, using the the GINI Index as the Heterogeneity index. The GINI is given by:\n", + "\n", + "![image.png](https://miro.medium.com/max/664/1*otdoiyIwxJI-UV0ukkyutw.png)\n", + "\n", + "The GINI index measures how heterogenous a single node is (0 being completely homogenous and 1 being heterogenous). For each possible split, we will calculate the *weighted sum* of the GINI indices of the child nodes, and choose the split that results in the maximum information gain. i.e. reduction in the weighted sum of the GINI Index.\n", + "\n", + "#### Training\n", + "We will now construct a simple decision tree using the GINI index." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.tree import DecisionTreeClassifier" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n", + " max_features=None, max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, presort=False,\n", + " random_state=None, splitter='best')" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tree = DecisionTreeClassifier()\n", + "tree.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 1.00 1.00 1.00 13475\n", + " 1 1.00 1.00 1.00 4021\n", + "\n", + " accuracy 1.00 17496\n", + " macro avg 1.00 1.00 1.00 17496\n", + "weighted avg 1.00 1.00 1.00 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, tree.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training set accuracy is 1, which means the datapoints are completely separated by the decision tree. We evaluate on the test set below." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (GINI) identified 828\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
054151314
11192828
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 5415 1314\n", + "1 1192 828" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, tree.predict(X_test), \"Decision Tree (GINI)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.6666666666666666\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e9NAoEQekINIQkt9A4iVQVFFEFBFBFUUAQUO2L72VCxtxcsgIoiCgIqVQRUTGjSeyehhN5CGgkp5/1jJrjEkCwhm005n+fhITs7O3NmdnbOzD0zd4yIoJRSSl2Oh7sDUEoplb9polBKKZUlTRRKKaWypIlCKaVUljRRKKWUypImCqWUUlnSRJEJY8wAY8wid8fhbsaYQGNMnDHGMw/nGWSMEWOMV17N05WMMduMMV1y8LkcbYPGmJLGmLnGmHPGmBlX+vm85Oz2VVh/j8aYEsaYncaYSnk4z6eMMW9c8QdFJF//A/YD54E44BgwGfB1d1yF8Z+9rru6OYYgQACvfLA+BKhdkJYZGAiszq31B3QB0uzfXxwQBfwEtHb393OVyzXAYZnOZ1jGuDyK4UlgXIZhbYAFQLT9bxswBihnv/8gsNRh/CjgKODjMGwYsMT+28vetoLs1z7AYaDilcRaUM4oeoqIL9AMaA487+Z4csSdR8mF5Qj9ShTR9V0T2C0iKVf6wSxiPmL//koD1wA7gXBjzA05D9O9RGSqiPjay3Uz9jI6DLuEi77Ph4EpDvPoCPwJLAXqikg5oIf9dqMsplMceNSZGYpIArAI64DCee7O7E5k3f04HOUC7wLzHV57A+8DB4HjwBdASYf3ewEbgRhgH9DdHl4W+AorGx8G3gA87ffuB5bZf38BvJ8hptnAU/bf1YBZwEkgEnjMYbxXgZnA9/b8H8xk+coC39mfPwC8BHg4xLEc+D/gHNYP9IYMn81qGZYDHwFn7PdqYW2Ip4FTwFT+PVKZgnVUlX729iwZjnSxNuAx9nRjsTY4P4d4BtnLcBr4X8bvLsNylwQ+sMc/Byyzh6XP8z77Oz0FvJjhiGsl1tHWUWAcUNzhfQEeAfYAkfawT4BD9newDujoML4n8IK9bcTa79cAwuxpxdvr4y57/FuxtqdoYAXQJMO2OhrYDCRhHc1dXAd27GvtOI4DH9rDD9rzSj+ibYfDNmiP0xBYbH+Xx4EXMlmnrwEXgGR7OkOwmpdfstfzCaxtraw9fvq6HmLHEJbJNLsAUZkMHwesdXgd6hDfLqDfFXzX6dvX/UCE/T1EAgMy/h7t19cCa+xprQGudXhvKVlso5fZFi+3jFHAKGALcMEeFgD8wr+/90ccxvfg323pFDANKH+ZeYZgbVueDsNWAR9lE2tmZxSj7fmVsYdd9ozCHnYfsPiK9sM52Xnn5T8u/aEF2F/aJw7vfwzMASpgHfHMBcY6/DDPAd3sL7E6EGq/9yvwJVAKqIR1uv5wxg0T6IS1kzH26/JYO9Nq9jTXAS9jZfUQe0O/yR73VawfbW973JKZLN93WImnNNYPZzcwxCGOFKxT1GLAXfbyVHByGVKAkfbGUhKoba8Lb8Afa2f4cWbrOsOOxDFR7APq2tNbCrxtv9cAa+fUwV4X79vLfrlEMd7+fHWsnfW1dlzp85xoz6Mp1k63vv25llhHtV72uDuAJxymK1g7rArp6xu4F6hof+ZprCbMEvZ76TuCeoCx51fRYVq1HabdAmtn29aO+T57nXk7rL+NWImmZMZ1ipXgBtp/+wLXZLaeM9kGS2MlxaeBEvbrtpdZr68C3zu8Hgzsxdo2fYGfgSkZ5vsd1jaU2fbZhcx3otdjHViUsv8dAh6w13ELrB1XQye/ay97GjFAPfszVR0+77guKgBnsY6IvYD+9uv072wpl9lGs9jHXG4Zo7B+3wH2tDzt7/cFrG28tv393mCP/wxWgqpuf09fpa/rTKbdC9jk8LqMvT47ZBNrZomiC9Y+8FV7WHaJog1w4or2w67YuefmP/uLiMM6OhDgD/49CjZYWbmWw/jt+PdI8ksyydBAZaydj+OZR3/gr0w2TIN1tNXJfv0Q8Kf9d1vgYIZpPw984/Cj/c9RmsO4nnYcDRyGPZy+IdhxHMFOUvaw1faPxJllOHi5edvj9AY2ZFjX2SWKlxzeHwEstP9+GfjR4T0frKPb/yQKrKR5HmiayXvp8wzIsMx3X2YZngB+cXgtwPXZLPfZ9HljHf32usx4GRPF58CYDOPsAjo7rL/BmWy/6YkiDOuo3y/DOJes50y2wf6O31M2y/YqlyaKP4ARDq/rYSVwL4f5hmQxvS5kvhMNtT9bHesAJjzD+18Crzj5XacnimigDxkSVoZ1MRBYneH9lcD92W2jOVjGKGCQw+v2QESGcf4HTLT/3pO+Ldiva2D9Rj0ymfZ9XHqWlL4uHLe3D+11Eg88Zw+7XKJoao9bkewTRX3sMyRn/xWUGkVvESmNtUJCAT97uD/WDmmdMSbaGBMNLLSHg/VF7ctkejWxjtCPOnzuS6yj8kuItWanYf1YAe7BarJJn0619GnY03kBayee7lAWy+WHdWRywGHYAawfX7rDdgyO71dzchkumbcxppIxZpox5rAxJgarScyPK3PM4e8ErKNU7Jguzk+sttDTl5mGH9YRV2bfTZbzMcbUNcbMM8Ycs5fhLf67DBmX+2ljzA77SqBorCa79M9cbhvJTE3g6Qzfdw2sZc903hkMwTrS3WmMWWOMudXJ+V5JjBlV47/blxfOb6OXUx1rBxSNtV7aZlgvA4AqOPddIyLxWAlnGNY2Pd8YE+rE8sB/fzOX20ZzwnHd1AQCMyzns1jLCRAIzHV4bwvWOsrsqqazWGeG6c7Y41ZNHyAiT4lVp5iL9Z1dlohsAn6348lOaazvzWkFJVEAICJ/Y1319L496BTW0UpDESln/ysr/xajDmG1y2d0CCvT+zl8royINLzMrH8E+hpjamKdRcxymE6kwzTKiUhpEenh8FnJODEHp7CO7mo6DAvEqjekq26MMRneP+LkMmSc91h7WBMRKYPVJGOyGP9KHMU6RQesyzSxjm4ycwpIJPPvJjufY9Vq6tjL8AKXLgM4LIddIBwN9MNqLy6H1XyX/pnLbSOZOQS8meH79hGRHzObd0YiskdE+mPtON4BZhpjSmX1mRzEmNER/rt9pWDVOS6GloPp3g6st3fwh4C/M6wXXxEZzhV81yLyu4h0w9pZ7sRqfsxueeC/v5nc5LhuDgF7Mvm997TfjwK6ZXi/hIgc++9k2QzUSr80WERisOpXd1xFrC8Dw/k3cV1OfWDTlUy4QCUK28dAN2NMMxFJw9qYPkq/FtkYU90Yc5M97lfAA8aYG4wxHvZ7oSJyFKvI9YExpoz9Xi1jTOfMZigiG7CKV5OA30UkPRuvBmKMMaPt69c9jTGNjDGtnVkQEUnFutTwTWNMaTsRPYV1pJ+uEvCYMaaYMeZOrC95wZUug600VjNetDGmOlb7vKPjWG3ZOTET6GmMudYYUxyriSXjDhwA+3v7GvjQGFPNXm/tjDHeTsynNFZbdpx9xDncifFTsL4/L2PMy1jtwekmAWOMMXWMpYkxJj3BZVwfE4Fhxpi29riljDG3GGMcjwwvyxhzrzHG317+9G0o1Y4tjcuv+3lAFWPME8YYb3tbaevMPLEOcp40xgQbY3yxzsCmS86uijL2b+gVrCaQFxziq2uMGWhvp8WMMa2NMfWd/a6NMZWNMbfZiTMJaztNzSSMBfa87jHGeBlj7sKqj8270uXJgZXABfsMtYS9LI2NMS3t978A3jLGBMLFM/jbMpuQiOzHatJu6TB4FDDUGPOsMcbfnkYN/psYMyUiu7AOYkdmM2pn4DdnppmuwCUKETmJVXz7nz1oNFaxbpXdFLEEqx0WEVmNVWD7COso8m/+XemDsJp9tmOdBs7E4bQvEz8CXYEfHGJJBXpiXbYbiXX0NAmracNZI7HaICOwrgb5AeuHle4foI497TeBviKS3qRzpcvwGlah8RwwH6uw6Wgs8JJ96vzMFSwDIrLNXpZpWGcXsViF36TLfOQZrFPzNVin3e/g3Pb4DFbzXyzWjnt6NuP/jvWj2I3VRJHIpc0JH2Il60VYCegrrMIlWO3939rro5+IrMWqUY3DWt97sdrPndUd2GaMicO6EutuEUm0m+neBJbb87rG8UMiEot1EUJPrGaVPcB1Ts7za6wr2sKwttFEst+RZFTNjjkO6/tqDHQRkUUO8d0I3I11xH8M6/tMTwbOfNceWMX6I/Y4nbHqC5ewt/1b7XFPYzW13Coip65wma6YnVx7YBWD92P9Jr/k3wOPD7Gavv8wxsRiXRWX1UHjlzhcpmq3mHTF+m732s1Xv2Ht0z5zMszXyKKpzT7T7461D3Va+pU8Kh8yxtyPdUltB3fHcqXso9dorCaiSHfHo1R+Y4wpAWzAKoCfyKN5Pgn4i8gL2Y7soMjdhKVcxxjTE+sqG4NVR9qCdeSllMpARBKxmpLzcp4f5eRzBa7pSeVrvbCaDo5gNZfdLXrKqlSBp01PSimlsqRnFEoppbJU4GoUfn5+EhQU5O4wlFKqQFm3bt0pEfHPfsz/KnCJIigoiLVr17o7DKWUKlCMMRnvaHeaNj0ppZTKkiYKpZRSWdJEoZRSKkuaKJRSSmVJE4VSSqksaaJQSimVJZclCmPM18aYE8aYrZd53xhjPjXG7DXGbDbGtHBVLEoppXLOlWcUk7G6s72cm7H6A6oDDMV6II1SSqlclpic2aM9nOeyG+5EJMwYE5TFKL2A7+xO41YZY8oZY6raD+RRSil1lU7HJfHiN+v44+CZq5qOO+/Mrs6lD5CJsof9J1EYY4ZinXUQGBiYJ8EppVRBtf9UPB//tpPZW44iHgav04lXNT13FrMze0xmpl3ZisgEEWklIq38/XPUVYlSShV66w+eZdiUdVz3wVJ+3XKU8zvPMKBCGXZOuJpHcbv3jCIKqOHwOgDrOQZKKaWclJomLNlxnIlhEaw9cJbSJbwY3rkWLcv4ULdGWWrUuJInM2fOnYliDvCoMWYa0BY4p/UJpZRyTmJyKjPXRfHVskgiT8VTIlU489chBtwSyrPdQ3N1Xi5LFMaYH4EugJ8xJgp4BSgGICJfAAuwHlS+F0gAHnBVLEopVVicjktiyqoDfLfyAGfiL1CtRDES/4wiasNxnnqyHS882z7X5+nKq576Z/O+AI+4av5KKVWYRJ6K56tlEcxYG0VSSho3hFYidfsZvn1tJddeW4PF6x+mcePKLpl3gXsehVJKFSXrDpxhQlgEi7Yfp5iHB7c1qcpdzarTup4/u3adon0dP4YMaYGHR2bXB+UOTRRKKZXPpKYJi7cfZ0LYPtYfjKZsyWI80qU2NRKFF55aRGSzKsya1Y969fyoV8/P5fFoolBKqXzi/IVUZq6P4qvwCPafTqBGhZK8dltDOlQvx/OjFjNqxnbq1avIo4+2ztO4NFEopZSbnYpL4ruVB5iycj9nE5JpGlCW8fe0oHujKiz9K5LmjT/nwoVUxoy5jlGjrsXbO2933ZoolFLKTSJOxjFpWSSz1lkF6q71KzG0Uy1aB5UnJSUNTw9D06ZV6NGjDm+8cT21a1dwS5yaKJRSKg+JCGsPnGVCWARLdhynmKcHfVpUZ0iHEGpX8iUmJoknnljIP/8cZvnywfj5+TBtWl+3xqyJQiml8kBqmrBo2zEmhEew4WA05XyKMfK62gxsF4R/aW9EhBkztvH44ws5diyOESNak5SUio+P+x8bpIlCKaVc6PyFVGasO8Sk8EgOnkkgsIIPr/dqSN+WAfgUt3bBJ0/Gc999v/Lbb3tp3rwKs2ffTevW1d0c+b80USillAucjE1iysr9TFl1gLMJyTSrUY7nbw7lxoZV8Mxwz0OZMt6cOpXAxx/fxCOPtMHLy/1nEY40USilVC7adzKOSeERzFp/mOTUNLrWr8zQTiG0qlkeY/5NEGFhB3jzzXBmzeqHr29xVq160KU3zV0NTRRKKXWVRIQ1+/8tUBf38qBvywCGdAimlr/vJeOeOpXAqFGLmTx5I0FB5di/P5pGjSrl2yQBmiiUUirHUtOEhVutAvWmQ9GU9ynGYzfUYVC7mvj5el8yrojwzTcbGTVqMTExSTz/fAdeeqkTPj7F3BS98zRRKKXUFUq4kMKMtVFMWhbBoTPnCarow5jejejbIoCSxT0v+7nvv99Mgwb+fPHFLTRsWCkPI746miiUUspJJ2IT+W7FAaasOsC588k0DyzHiz3q063BfwvUAAkJybz1VjjDhrUiIKAMs2b1o2zZEvm6mSkzmiiUUiobe0/EMik8kp/XHyY5LY0bG1gF6pY1L3+n9IIFe3jkkQXs3x9N9eqlGT68NeXLl8zDqHOPJgqllMqEiPBP5BkmhkXwx84TeHt5cGcrq0AdkqFA7SgqKoYnnljIrFk7qF/fj7//vp9OnWrmYeS5TxOFUko5SElNY+G2Y0wMi2BT1DkqlCrOE13rMPCamlTMUKDOzJtvhjF//h7eeut6nn76WopnUbMoKIz1oLmCo1WrVrJ27Vp3h6GUKmTik1L4ae0hvloWSdTZ8wT7lWJIh2D6tgygRLGsd/arVx+mZEkvGjeuzOnTCZw7l0RISPk8itw5xph1ItIqJ5/VMwqlVJF2IiaRb1fu5/tVBzl3PplWNcvzv1sb0LV+5UwL1I7OnUvkhRf+4PPP13LrrXWZM6c/FSv6ULGiT94En0c0USiliqQ9x2OZGB7BrxuOkJyWxk0NqvBQpxBa1sz+TEBEmD59G08++TsnTsQzcmQbxoy5Pg+idg9NFEqpIkNEWBVxhonhEfy58wQlinlwV+saDOkQTJBfKaen8/33mxk06FdatarGvHn9admymgujdj9NFEqpQi8lNY0FW60C9ZbD56hYqjhPdavLvdfUpEKp4k5NIykphYiIs9Sv70+/fg1JSUlj0KCmeHrmrw78XEEThVKq0IpLSuGnNVaB+nD0eUL8SvHW7Y25o0X1bAvUjv76K5Lhw+eTkJDMnj0j8fb24oEHmrsw8vxFE4VSqtA5EZPINyv2M3XVAWISU2gdVJ5XeloF6iu5K/rEiXieeWYRU6ZsJiSkPBMm9Mzz51XnB0VviZVShdbu47FMDIvg142HSU0TujeqwoMdQ2gReOWXqu7de4Y2bSYSF3eBF1/syIsvdqRkyfzfgZ8raKJQShVoIsLKfaeZEB7B0l0nKVHMg/5tAhnSIZiaFZ0vUKeLiUmiTBlvatUqz5AhzRk8uDn16/u7IPKCQxOFUqpASk5NY8GWo0wMj2Dr4Rj8fIvztF2gLu9kgdpRfPwFXn/9byZOXM/mzcMJCCjDe+/d6ILICx5NFEqpAiUuKYVpqw/yzfL9VoHavxRv39GY3s2vrEDtaO7cXTz66G8cPHiOIUOaF4hnROQlTRRKqQLh2LlEJq/Yz9R/DhCbmEKb4Aq8dltDrg/N+dPhUlLS6NdvBr/8spOGDf0JD3+ADh0Ccznygk8ThVIqX9t5LIaJYZHM2WQVqG9uXJWHOobQrEa5HE9TRDDG4OXlQdWqvrz99g08+WS7QtGBnytoolBK5Tsiwop9p/kyLIKw3ScpWcyTAW1rMrh9MIFX2Y/SqlVRPPLIAiZO7EmLFlUZP/6WXIq68NJEoZTKN5JT05i/+SgTwiLYfjQGP19vRt1UjwFtAynnc+UFakdnz57nhRf+4Msv11GtWmnOnj2fS1EXfi5NFMaY7sAngCcwSUTezvB+IPAtUM4e5zkRWeDKmJRS+U9sYjLT1xzi62WRHDmXSC3/UrzTpzG9muW8QO1o+vStPPbYQk6dSuCJJ67htde6ULp09s+WUBaXJQpjjCcwHugGRAFrjDFzRGS7w2gvAT+JyOfGmAbAAiDIVTEppfKXY+cS+WZ5JD/8c5DYpBTaBlfgjdsb0aVuzgvUmdm58xRBQeVYuHAAzZtXzbXpFhWuPKNoA+wVkQgAY8w0oBfgmCgEKGP/XRY44sJ4lFL5xI6jMUwMj2DOxiOkidDDLlA3vYoCtaPExBTeeWcZLVpUpWfPerzwQkdeeqlTkejAzxVcmSiqA4ccXkcBbTOM8yqwyBgzEigFdM1sQsaYocBQgMBAvXRNqYJIRFi29xQTwiII33MKn+KeDGxnFahrVMi9B/0sWRLBiBHz2bPnDE8/3Y6ePetRLBear4oyVyaKzM4bMz53tT8wWUQ+MMa0A6YYYxqJSNolHxKZAEwA61GoLolWKeUSyalpzNt8hAlhkew4GoN/6dwrUDs6fjyOp55axA8/bKF27QosWnQv3brVyrXpF2WuTBRRQA2H1wH8t2lpCNAdQERWGmNKAH7ACRfGpZTKAzGJyRfvoD56LpE6lXx5t28TejWrhrdX7h/hL14cwcyZ23n55U48/3xHSpTQizpziyvX5BqgjjEmGDgM3A3ck2Gcg8ANwGRjTH2gBHDShTEppVzsSPR5Jq/Yzw//HCQuKYV2IRV56/bGdK7rn6sFaoBNm46xZ88Z+vZtwIABjWnfvgbBwVfeU6zKmssShYikGGMeBX7HuvT1axHZZox5HVgrInOAp4GJxpgnsZql7hcRbVpSqgDafsQqUM/ddAQBbrEL1I0Dyub6vOLiLvDKK3/xySf/EBRUjt69Q/Hy8tAk4SIuPTez74lYkGHYyw5/bwfauzIGpZTriAjhe04xMfzfAvWgdkEM7hBEQPncK1A7+vXXnYwc+RtRUTEMHdqCsWO74uWlVzO5kjbiKaWu2IWUNOZuOsLE8Ah2HoulUmlvRncP5Z42gZR1Yc+rW7Yc5/bbp9O4cSWmT+/LtdfWyP5D6qppolBKOS0mMZkf/7EK1MdiEqlb2Zf3+jbhNhcVqAGSk1MJDz/I9dcH07hxZebPv4du3UL0ktc8pIlCKZWtw9Hn+WZZJNPWHCIuKYX2tSvydh+rQG1M7haoHa1YcYhhw+axbdtJdu16lNq1K9CjRx2XzU9lThOFUuqyth4+x6TwCOZuPgrArU2sAnWj6rlfoHZ05sx5nntuCRMnrqdGjTL8/HM/ateu4NJ5qsvTRKGUuoSI8Pfuk0wMj2D53tOUKu7JA9cG8UCHYKqXK+ny+ScmptCs2RccORLL00+349VXu+Drm3s35qkrp4lCKQVYBeo5m44wMSyCXcdjqVzGm+duDqV/m0DKlnT9o0GjomIICChDiRJejBlzHc2aVaFp0youn6/KniYKpYq4c+eT+eGfg0xeEcnxmCRCq5Tmgzub0rNpNYrnwWWn588nM3bsMt55ZzkzZ95Jz571uO++Zi6fr3KeU4nCGFMcCBSRvS6ORymVR6LOJvD1sv1MX3OQ+AupdKjtx3t9m9Kxjp9LC9SOFi3ax4gR89m37yz33tuENm2q58l81ZXJNlEYY24BPgSKA8HGmGbAKyJyu6uDU0rlvq2HzzEhLIL5W45igJ5Nq/Fgx2AaVnNtgTqjkSMXMG7cGurUqcCSJQO54YaQPJ2/cp4zZxSvY3UP/heAiGw0xtR2aVRKqVwlIizdfZKJYRGs2HcaX28vBrcP4oH2wVTLgwJ1utRUq2NoT08PrrkmAD8/H0aP7qAd+OVzznw7ySISneFUVPtjUqoASEpJZfbGI0wKj2D38TiqlCnBCz1CubtNIGVKuL5A7Wj9+qMMGzaPgQObMHJkWwYMaJKn81c550yi2GGM6Qd42D3BPg6scm1YSqmrcS4hmamrDzB5+X5OxFoF6g/7NeXWJnlToHYUG5vEyy//xaefrsbf34eqVUvn6fzV1XMmUTwKvAykAT9j9Qb7vCuDUkrlzKEzCXy9PJLpaw6RcCGVjnX8+KBfUzrUzrsCtaNFi/YxePBsjhyJZdiwVrz11g2UK1ciz+NQV8eZRHGTiIwGRqcPMMbcgZU0lFL5wOaoaCaERbBgy1E8jOG2ptV4sGMIDaqVyf7DLlS8uCeVKpVi1qx+tG0b4NZYVM6Z7B7/YIxZLyItMgxbJyItXRrZZbRq1UrWrl3rjlkrla+kpQlLd59gQlgEqyLOUNrbi3vaBnJ/+yCqls27ArWj5ORUPvxwJTExSbz55g0X48ztBxapK2fvt1vl5LOXPaMwxtyE9ZjS6saYDx3eKoPVDKWUcoOklFRmbzjChPAI9p6Io1rZErx0S33ual2D0nlcoHa0bNnBix343Xlng4sJQpNEwZdV09MJYCuQCGxzGB4LPOfKoJRS/xWdcIGp/xxk8or9nIxNokHVMnx8VzNuaVKVYp7ue3DP6dMJjB69hK++2kBgYFnmzu3PrbfWdVs8KvddNlGIyAZggzFmqogk5mFMSikHh84k8NWySH5aaxWoO9f1Z+hdIVxbq6JbCtQZnT59nmnTtvLss9fy8sudKVVKO/ArbJwpZlc3xrwJNAAuXq4gInrIoJQLbToUzYTwCH7bchRPD8NtTavzUKdgQqu4t0ANsGPHSX76aRuvvNKFunUrcvDgk1So4J66iHI9ZxLFZOAN4H3gZuABtEahlEukpQl/7TrBl2ERrI60CtQPdQrhgWuDqVLW/ZeVJiQk8+abYbz33gp8fYszZEgLAgLKaJIo5JxJFD4i8rsx5n0R2Qe8ZIwJd3VgShUlicmp/LrhMBPDI9h3Mj7fFKgdLVy4lxEj5hMZGc199zXlvfe64e9fyt1hqTzgTKJIMlZD6D5jzDDgMFDJtWEpVTScjb/A96sO8O3K/ZyKu0DDamX45O5m9Gjs3gJ1RnFxFxg48BcqVizJX3/dR5cuQe4OSeUhZxLFk4Av8BjwJlAWGOzKoJQq7A6eTuCrZRH8tDaK88mpdKnnz9COIbTLJwVqsDrw+/HHrfTv3whf3+IsWTKQ0FA/vL21A7+iJttvXET+sf+MBQYCGGP0FkulcmDjoWgmhO1j4dZjeHoYejerzoMdQ6hXJX/1f7Ru3REefnge69YdpWRJL/r0aaBPmyvCskwUxpjWQHVgmYicMsY0xOrK43pAk4VSTkhLE/7YeYKJYRGs3n+G0iW8eLhzLe6/NojKZdxfoHZ07lwi//vfX4wfv4ZKlUoxbVof7rijvrvDUm6W1Z3ZY4E+wHLVKeQAACAASURBVCasAvYvWD3HvgMMy5vwlCq4EpNT+cUuUEecjKd6uZK8fGsD+rWugW8+bb7p0+cn/vwzkkceac0bb1xP2XxwpZVyv6y21l5AUxE5b4ypAByxX+/Km9CUKpjO2AXq71ZaBepG1cvwaf/m9GhUBa98VKBOFxFxFn9/H0qX9ubNN6/Hw8PQurU+klT9K6tEkSgi5wFE5IwxZqcmCaUu78DpeCaFRzJj3SESk9O4PrQSD3UM4ZqQCvmmQO3owoVU3n9/BWPGhPHYY214551u2sOrylRWiSLEGJPelbgBghxeIyJ3uDQypQqI9QfPMjEsgoXbjlHMw4Peza0uvutWzl8FakdhYQcYNmweO3acom/fBjz2WFt3h6TysawSRZ8Mr8e5MhClCpK0NGHxjuNMDItg7YGzlCnhxXC7QF0pnxWoM/roo5U89dQigoLKMX/+PfToUcfdIal8LqtOAf/Iy0CUKggSk1OZtT6KSeGRRJ6KJ6B8SV7p2YB+rWpQKp8WqMFKbPHxFyhd2ptbbqnLyZMJvPRSJ3x88sdd3yp/y/bBRfmNPrhIucOZ+At8t3I/3608wJn4CzQJKMvQTiF0b5g/C9SOtm07wbBh8y8+aU4VTS55cFFuMMZ0Bz4BPIFJIvJ2JuP0A14FBNgkIve4MialrkTkqXi+WhbBzHVRJCancUNoJR7qFELb4PxZoHaUkJDMmDF/8/77Kylb1pvBg5shIvk+bpX/OJ0ojDHeIpJ0BeN7AuOBbkAUsMYYM0dEtjuMUwd4HmgvImeNMdqHlMoX1h04y4SwfSzafpxiHh7c0aI6D3YMpnal/FugdrRhw1HuuOMn9u+P5oEHmvHuu93w8/Nxd1iqgMo2URhj2gBfYfXxFGiMaQo8KCIjs/loG2CviETY05mGdW/GdodxHgLGi8hZABE5ceWLoFTuSE0TFm8/zsTwCNYdOEvZksV4pEttBl1bk0ql83eBOl36GUNgYFkCA8vy7be96dSpprvDUgWcM2cUnwK3Ar8CiMgmY8x1TnyuOnDI4XUUkPEavLoAxpjlWM1Tr4rIQiemrVSuOX8hlZnro/gqPIL9pxOoUaEkr93WkDtbBeBTPP8WqB2lpKQxbtxq5szZxeLFA6lY0Ye//77f3WGpQsKZX4GHiBzI0K6Z6sTnMmsIzVg59wLqAF2w+o4KN8Y0EpHoSyZkzFBgKEBgYKATs1Yqe6fjkvhu5QGmrLIK1E1rlOOz7qHc1LAKnh4Fpx1/9erDDBs2jw0bjnHzzbWJiUmifHl9kJDKPc4kikN285PYdYeRwG4nPhcF1HB4HYDVDUjGcVaJSDIQaYzZhZU41jiOJCITgAlgXfXkxLyVuqyIk3FMWhbJrHVRJKWk0bV+ZYZ2CqF1UPkCVeiNi7vA6NGL+fzztVStWpoZM+6kT5/6BWoZVMHgTKIYjtX8FAgcB5bYw7KzBqhjjAnGetjR3UDGK5p+BfoDk40xflhNURHOha6U80TELlBHsHjHcYp5etCnRXWGdAihdiVfd4eXI8WKebB06QFGjmzDmDHXU6aMt7tDUoWUM4kiRUTuvtIJi0iKMeZR4Hes+sPXIrLNGPM6sFZE5tjv3WiM2Y7VnDVKRE5f6byUupzUNGHRtmNMCI9gw8FoyvkUY+R1tRnYLgj/0gVvx7p37xlef/1vxo/vQenS3qxbN5QSJQpGHUUVXNnecGeM2QfsAqYDP4tIbF4Edjl6w51yxvkLqcxcd4hJyyI5cDqBwAo+PNgxmL4tC06B2lFSUgrvvrucN98Mp3hxT+bPv4eOHfVqJuU8l95wJyK1jDHXYjUdvWaM2QhME5FpOZmhUq50Ki6J71bsZ8qqA5xNSKZZjXI81z2UGwtYgdrRX39FMnz4fHbtOs1ddzXkww9volq1gnE/hyocnDq0EpEVwApjzKvAx8BUQBOFyjf2nYxjUngks9ZHkZz6b4G6Vc2CVaDOSER4881wkpPTWLhwADfdVNvdIakiyJkb7nyxbpS7G6gPzAaudXFcSmVLRFiz3ypQL9lxnOJeHvRtGcCQDsHU8i+YBWqwOvD76qv1dO9emxo1yjJlyu2UK1eCkiW1Az/lHs6cUWwF5gLviki4i+NRKlupacLv244xISyCjYeiKe9TjMduqMOgdjXx8y14BWpHmzcfZ9iweaxcGcXLL3fitdeuo2pVbWZS7uVMoggRkTSXR6JUNhIupDBjbRRfLYvk4JkEgir6MKZ3I/q2CKBkcU93h3dV4uIu8NprS/noo1WUL1+SyZN7MWhQU3eHpRSQRaIwxnwgIk8Ds4wx/7k0Sp9wp/LKydgkvltpFaijE5JpHliOF3qE0q1BwS1QZ/Tqq0v54IOVPPhgc95+uysVK2oHfir/yOqMYrr9vz7ZTrnF3hOxTAqP5OcNh0lOTePGBlaBumXNCu4OLVccOnSO+PhkQkP9eO65DvTuHUqHDtpFjcp/snrC3Wr7z/oickmysG+k0yfgqVwnIqyOPMPE8AiW7DiBt5cHd9oF6pACXKB2lJKSxqef/sPLL/9Fy5bV+Pvv+/Hz89EkofItZ2oUg/nvWcWQTIYplWMpqWn8vu04E8L2sSnqHBVKFeeJrnUYeE1NKhbwArWjVauiGDZsHps2HeeWW+owblwPd4ekVLayqlHchXVJbLAx5meHt0oD0Zl/SqkrE5+Uwoy1h/hqeSSHzpwn2K8Ub/RuRN+WAZQoVrAL1BnNn7+bnj1/pFq10vz8cz969w4t0Pd4qKIjqzOK1cBprF5fxzsMjwU2uDIoVfidiE3k2xX7+X7VQc6dT6ZVzfK8dEsDutavXGgK1GA1pR05Ekv16mXo2jWE11+/jscfb0vpAtjPlCq6su3rKb/Rvp4Ktj3HrQL1LxsOk5yWxk0NqvBQpxBa1izv7tBy3e7dpxkxYj67d59m+/ZH8PUt7u6QVBHmkr6ejDF/i0hnY8xZLn3gkAFERArHpSfK5USEVRFWgfrPnScoUcyDu1rXYEiHYIL8Srk7vFyXmJjC228vY+zYZZQs6cXYsTdQsmTB64hQqXRZbb3pjzv1y4tAVOGTkprGb1uPMTE8gs1R56hYqjhPdavLvdfUpEKpwnl0fexYHJ06fcOePWfo378RH354E1WqFI6rtVTRldXlsel3Y9cAjojIBWNMB6AJ8D0QkwfxqQIoPimF6WsO8dWySA5HnyfErxRv3d6YO1pUL3QF6nTJyakUK+ZJ5cql6NSpJuPH96Bbt1ruDkupXOHM8yg2Aq2xnnC3GJgPBIvIra4P77+0RpF/nYhJZPKK/Xy/6gAxiSm0DirP0E61uCG0Eh6FqEDtKC1NmDBhHW+9Fc6KFUMICCjj7pCUypRLn0cBpIlIsjHmDuBjEfnUGKNXPamLdh+PZWJYBLM3HiElLY3ujarwYMcQWgQWvgK1o02bjvHww/P455/DXH99MMnJqe4OSSmXcOpRqMaYO4GBQG97mPZ3XMSJCCsjTjMxLIK/dp2kRDEP7m5jFahrVix8BWpHIsKoUYv5+ONVVKhQkilTbmfAgMZ6T4QqtJy9M3sEVjfjEcaYYOBH14al8quU1DTmbznKxPAIth6Owc+3OE/bBeryhbRAnZExhrNnzzNkiNWBX/nyJd0dklIu5dR9FMYYLyD90Vp7RSTFpVFlQWsU7hFnF6i/Ti9Q+5diaMcQejcvvAVqRwcORPP44wt5+eXOtGhRlbQ0KbR1F1U4ubRGYYzpCEwBDmPdQ1HFGDNQRJbnZIaqYDkek8g3y/cz9Z8DxCam0Ca4Aq/d1pDrC3GB2lFycioffbSK1177G4C77mpIixZVi8SyK5XOmaanj4AeIrIdwBhTHytx5CgzqYJh17FYJoZHMHvjYVLThJsbV+WhjiE0q1HO3aHlmRUrDvHww/PYuvUEvXrV49NPbyYwsKy7w1IqzzmTKIqnJwkAEdlhjCkajdFFjIiwYt9pJoRF8Pfuk5Qs5smAtjUZ3D6YwCL4IJ0lSyI4dy6RX3+9i169Qt0djlJu48x9FJOBJKyzCIABgI+I3Ofa0DKnNYrcl5yaxoItR5kQFsG2IzH4+XrzQPsgBrQNpJxP0TkmEBGmTNmMv78PN99ch6SkFJKT07SPJlUouPo+imHAY8CzWDWKMOD/cjIzlb/EJaUwbfVBvl4WyZFzidTyL8U7fRrTq1nRKFA72rnzFMOHz2fp0v3ceWcDbr65Dt7eXnhrJ69KZZ0ojDGNgVrALyLybt6EpFzt2LlEvlkRyQ//HCQ2MYW2wRV44/ZGdKlbNArUjs6fT+att8J5553llCpVnC+/vJUHH2zh7rCUyley6j32Bawn2a0HWhtjXheRr/MsMpXrdhyNYWJ4BHM2HiFNhB52gbppESpQZzR37m7eeCOce+9twvvvd6NyZe3AT6mMsjqjGAA0EZF4Y4w/sADQRFHAiAjL955mQngEYbtP4lPck4HtrAJ1jQpFr0ANVg+vGzceo3v32tx5ZwOCgh6kTZvq7g5LqXwrq0SRJCLxACJy0hjjkUcxqVyQnJrGvM1HmBAWyY6jMfiX9mbUTfWKXIHaUWpqGl9+uY7nn/+D4sU9OXjwCUqWLKZJQqlsZJUoQhyelW2AWo7PzhaRO1wamcqR2MRkpq0+xNfLIzl6LpE6lXx5t28TejWrhrdX0SpQO1q//ijDhs1jzZojdO0awmef9aBkSe2yTClnZJUo+mR4Pc6VgaircyT6PJNX7OfHfw4Sm5RCu5CKvHV7YzrX9S9yBeqMIiPP0qbNRPz8fPjhhzu4++5G2oGfUlcgqwcX/ZGXgaic2X7EKlDP3XQEAW6xC9SNA4r2HcQiwpYtJ2jSpDLBweX55pte9OxZj3LlSrg7NKUKHH2QbwEkIoTvOcXE8AjC95zCp7gng9oFMbhDEAHli2aB2lFk5FkeffQ3Fi7cy4YND9OkSWUGDmzq7rCUKrBcmiiMMd2BTwBPYJKIvH2Z8foCM4DWIqK3XWdh38k4Hvp2LRGn4qlU2pvR3UO5p00gZX20vf3ChVQ+/HAlr7/+Nx4ehvff70aDBv7uDkupAs/pRGGM8RaRpCsY3xMYD3QDooA1xpg5jv1G2eOVxrrz+x9np11UXUhJ45kZm4g4Fc8LPUK579qgIl2gdpSamsa1137FunVHueOO+nz88U3UqFG0m9+Uyi3ZXvJqjGljjNkC7LFfNzXGONOFRxusZ1dEiMgFYBrQK5PxxgDvAonOh100jf1tBxsORjPunuYM7VRLkwQQE2Mdu3h6ejB4cHPmzu3PrFn9NEkolYucuTfiU+BW4DSAiGwCrnPic9WBQw6vo+xhFxljmgM1RGReVhMyxgw1xqw1xqw9efKkE7MufOZuOsI3y/fzQPsgbm1Szd3huJ2IMHnyRkJCPmH27J0AjBjRmltvrevmyJQqfJxJFB4iciDDMGeeIp/Z9YcXu6q1b+D7CHg6uwmJyAQRaSUirfz9i16b894TsYyetZmWNcvz/M313R2O223ffpIuXb7lgQdmExrqR61aFdwdklKFmjM1ikPGmDaA2HWHkcBuJz4XBdRweB0AHHF4XRpoBCy1r2mvAswxxtymBe1/xSelMOz79ZQs5sn4e1pQ3Kto3yD/7rvLefHFPylTxptJk3rywAPNi/x9Ikq5mjOJYjhW81MgcBxYYg/LzhqgjjEmGOsxqncD96S/KSLnAL/018aYpcAzmiT+JSI89/MWIk7GMWVIW6qULbr3AIgIxhiqVPFlwIDGvPdeN/z9S7k7LKWKhGwThYicwNrJXxERSTHGPAr8jnV57Nciss0Y8zqwVkTmXHG0Rcy3K/Yzd9MRRt1Uj/a1/bL/QCF05Egsjz++kI4dA3nssbYMGtSUQYP0ngil8lK2icIYMxGH2kI6ERma3WdFZAFWr7OOw16+zLhdspteUbLuwFneXLCDG0IrMbxzLXeHk+dSU9P47LM1vPjinyQnp3HttQHuDkmpIsuZpqclDn+XAG7n0quZVC47HZfEI1PXU6VsCT7s16zItcFv3HiMBx+cw7p1R7nxxlp89lkPLVgr5UbOND1Nd3xtjJkCLHZZREVcaprw2LQNnEm4wM/Dry2Sd1yfO5fIkSOxTJ/elzvvbKAd+CnlZjnpwiMYqJnbgSjLx0t2s3zvad7p05hG1YvGTWMiwowZ29mz5zQvvtiJzp2DiIh4nBIltCsypfIDZ+7MPmuMOWP/i8Y6m3jB9aEVPX/uPM7//bmXfq0CuKt1oLvDyRP79p2hR48fuOuumcyevYvkZOsWHU0SSuUfWf4ajXXO3xTr8laANBH5T2FbXb1DZxJ4cvomGlQtw+u9Grk7HJdLSkrh/fdX8MYb4RQr5sEnn3RnxIjWeBXx+0SUyo+yTBQiIsaYX0SkZV4FVBQlJqcyfOo60kT44t6WlChW+PtwOnQohjFjwujZsx4ff3wT1auXcXdISqnLcObwbbUxpoXLIynCXpu7ja2HY/iwXzMCKxbe50mcPBnPuHGrAahduwLbtz/CjBl3apJQKp+77BmFMcZLRFKADsBDxph9QDxWH04iIpo8csGMtYf4cfUhRnSpRbcGld0djkukpQnffLOBZ59dQmxsEt26hVCvnh8hIeXdHZpSyglZNT2tBloAvfMoliJn+5EYXvp1K+1CKvJUt8LZ6+nWrScYPnw+y5YdpGPHQL744lbq1Suad5krVVBllSgMgIjsy6NYipRz55MZPnUd5XyK8Wn/5nh5Fr4i7oULqdx44xQuXEjl669v4/77m+k9EUoVQFklCn9jzFOXe1NEPnRBPEWCiPDMjE0cPnueaUOvwb+0t7tDylV//hlJ5841KV7ck59+upPQUD/8/Apv7UWpwi6rw1hPwBerO/DM/qkc+jIsgsXbj/N8j/q0Cio8XVNERcXQp89P3HDDd3z33SYAOnQI1CShVAGX1RnFURF5Pc8iKSJW7jvNuwt3ckvjqgxuH+TucHJFSkoa48at5n//+4vU1DTGjr2BAQOauDsspVQuybZGoXLP8ZhERv64gSC/UrzTt0mhaa8fOPAXpk3bys0312b8+B4EB+vVTEoVJlklihvyLIoiIDk1jUd/WE98Ugo/PNQWX++C3UVFdHQiXl4e+PoW55FHWtOnT3369KlfaJKfUupfl61RiMiZvAyksHvnt52s2X+Wt/s0pm7lglviERGmTdtK/frj+d///gSsOkTfvtrLq1KFVeG7JjMfWrDlKJOWRTKoXU16Navu7nBybO/eM9x00/f07z+LgIAy3Huv1iGUKgoKdvtHAbDvZBzPztxMsxrlePGW+u4OJ8d++GELgwfPxtvbi3HjbmbYsFZ4FsJ7P5RS/6WJwoUSLqQw/Pt1FPM0fDagBd5eBa+zv+TkVIoV86RVq2r07duAd9/tRrVqBbfpTCl15TRRuIiI8OIvW9lzIo7vBrehWrmS7g7pipw4Ec/TTy8iPv4CP/98F3XrVuT77+9wd1hKKTfQtgMX+f6fg/yy4TBPdq1Lxzr+7g7HaWlpwoQJ66hXbxzTp2+lYUN/UlPT3B2WUsqN9IzCBTYeimbM3O10qefPo9fVdnc4TouIOMu99/7MypVRdOkSxOef30JoqHbgp1RRp4kil52Jv8AjU9fjX9qbj+9qhodHwblktGxZb6KjE/n2294MHFh4bghUSl0dbXrKRalpwhPTN3IyNonP721BOZ/i7g4pW3Pm7OKOO6aTmppGxYo+bN06gkGDmmqSUEpdpIkiF/3fn3sI232SV29rSJOAcu4OJ0sHD56jd+9p9Oo1jd27T3P0aBxAgToDUkrlDW16yiVLd53gkz/2cEeL6vRvU8Pd4VxWSkoaH3+8ildeWYqI8M47XXnyyWsoVgSe062UyhlNFLkg6mwCT0zfSL3KpXmzd+N83WyTmprGpEnruf76YP7v/24mKCh/n/kopdxPm56uUlJKKo9MXU9qqvD5vS0pWTz/HZmfPXue0aMXExubhLe3F8uXD2bOnLs1SSilnKKJ4iqNmbedTVHneO/OpgT7lXJ3OJcQEaZO3Uxo6Hg++GAlf/21H4CKFX3y9VmPUip/0aanq/DLhii+X3WQhzuF0L1RFXeHc4ndu08zYsR8/vgjkjZtqvP77/fSrFn+ilEpVTBoosihncdieP7nLbQJrsCom+q5O5z/eOKJhaxde4TPPuvB0KEttQM/pVSOaaLIgdjEZIZ/v57SJYox7p7meOWTnfDixfsIDfWjRo2yfP75LXh7e1Gliq+7w1JKFXAu3cMZY7obY3YZY/YaY57L5P2njDHbjTGbjTF/GGNqujKe3CAijJqxmYNnEhjXvzmVSpdwd0gcOxbHPffM4sYbv+edd5YDULNmOU0SSqlc4bJEYYzxBMYDNwMNgP7GmAYZRtsAtBKRJsBM4F1XxZNbvloWycJtx3iueyhtQyq6NZa0NOGLL9YSGjqOWbN28MornXn//RvdGpNSqvBx5RlFG2CviESIyAVgGtDLcQQR+UtEEuyXq4AAF8Zz1VZHnmHsbzvp3rAKD3YMdnc4jB0bzvDh82nZshqbNw/j1Ve7UKKEtiYqpXKXK/cq1YFDDq+jgLZZjD8E+C2zN4wxQ4GhAIGBgbkV3xU5EZvIIz+sJ7CCD+/e6b4O82Jjkzh1KoHg4PIMG9aK4ODy9O/fSC93VUq5jCvPKDLbc0mmIxpzL9AKeC+z90Vkgoi0EpFW/v55/2yHlNQ0Rv6wgdjEZD6/twVlShTL8xhEhF9+2UGDBp9x110zEREqVvThnnvy953gSqmCz5WJIgpw7PQoADiScSRjTFfgReA2EUlyYTw59t6iXfwTeYa3bm9MaJUyeT7/Aweiue22adxxx09UqFCSTz+9WZODUirPuLLpaQ1QxxgTDBwG7gbucRzBGNMc+BLoLiInXBhLjv2+7Rhf/h3BgLaB3NEi70soK1ceomvXKQC8/343Hn/8Gry88sfluEqposFliUJEUowxjwK/A57A1yKyzRjzOrBWROZgNTX5AjPsI+SDInKbq2K6UvtPxfPMT5toElCWl3tmvGDLtWJikihTxpsWLaoyeHAzRo1qT2Bg2TyNQSmlAIxIpmWDfKtVq1aydu1al8/n/IVUbv9sOcdiEpn7aAdqVPBx+TwBTp9O4LnnlrBoUQTbto3A1zf/P/xIKZX/GWPWiUirnHxWr6XMhIjw0q9b2XU8lm/ub50nSUJEmDJlM08/vYizZ8/z1FPt0DKEUio/0ESRiWlrDjFrfRSP3VCHLvUquXx+584l0rv3dJYu3U+7dgF88cWtNGlS2eXzVUopZ2iiyGBL1Dlemb2NjnX8ePyGOi6dl4hgjKFMGW/8/HyYMOFWhgxpoY8jVUrlK3r5jIPohAsMn7oOP9/ifHJ3czxduMP+/fe9tGgxgaioGIwxzJhxJw891FKThFIq39FEYUtLE56cvpHjMYl8dm9LKpRyTRH56NFY7r57Jt27TyUhIZkTJ+JdMh+llMot2vRkG//XXv7adZIxvRrSrIZrHhE6fvxqXnjhT5KSUnjttS6MHt0eb2/9CpRS+ZvupYDwPSf5cMluejerxr3XuK6n83XrjtK2bXXGj+9BnTru7XlWKaWcVeQTxZHo8zw+bSN1Kvny1h25229STEwSL7/8FwMHNqFly2p89tkteHt7avcbSqkCpUgnigspaYyYup6k5FQ+v7clPsVzZ3WICLNm7eDxxxdy9GgsgYFladmymnYBrpQqkIr0nuutBTvYeCiazwa0oJZ/7jwNLjLyLI8++hsLFuyhWbMq/PxzP9q2zdeP2VBKqSwV2UQxe+NhJq/Yz5AOwfRoXDXXpjt16hbCwg7w0Uc38eijbbQDP6VUgVck+3raczyW28Ytp2G1Mvw49BqKeV7dzjw8/ABJSal07RpCUlIKJ08mEBCQ992RK6XU5VxNX09F7nA3LimFYd+vo5S3J+MHtLiqJHHqVAKDB8+mU6fJvP763wB4e3tpklBKFSpFqulJRBg9azORp+KZ+uA1VC5TIsfTmTx5I6NGLebcuSRGj27P//7XKZejVYVBcnIyUVFRJCYmujsUVUSUKFGCgIAAihXLvSdxFqlE8c3y/czffJTR3UNpVyvn9zEsWLCHwYPn0L59Db744lYaNXJ9x4GqYIqKiqJ06dIEBQXpZdHK5USE06dPExUVRXBwcK5Nt8g0Pa07cIa3Fuyga/3KDOsccsWfT0hIZvnygwD06FGH2bPvJizsAU0SKkuJiYlUrFhRk4TKE8YYKlasmOtnsEUiUZyKS2LE1PVUK1eSD/o1veIf7W+/7aFRo8+4+eapREcnYozhttvqaQd+yimaJFRecsX2VugTRWqa8NiPG4hOSObze1tQtqTz7XaHD8dw550z6NHjB7y9vZg7tz/lyuWsrqGUUgVVoU8UHy7exYp9pxnTuxENqzn/zOkTJ+Jp0OAz5s3bzRtvXMemTcPo3DnIdYEq5SKenp40a9aMRo0a0bNnT6Kjoy++t23bNq6//nrq1q1LnTp1GDNmDI6XzP/222+0atWK+vXrExoayjPPPOOORcjShg0bePDBBy8Z1qtXL9q1a3fJsPvvv5+ZM2deMszX998bbXfv3k2PHj2oXbs29evXp1+/fhw/fvyqYjtz5gzdunWjTp06dOvWjbNnz2Y63sGDB7nxxhupX78+DRo0YP/+/QCMGzeO2rVrY4zh1KlTF8efN28er7zyylXFdkVEpED9a9mypThr8bZjUnP0PBk9c5PTn4mKOnfx708+WSV79552+rNKZbR9+3Z3hyClSpW6+PegQYPkjTfeEBGRhIQECQkJkd9//11EROLj46V79+4ybtw4ERHZsmWLhISEyI4dO0REJDk5WcaPH5+rsSUnJ1/1NPr27SsbN268+Prs2bMSEBAgoaGhEhERcXH4fffdJzNmzLjks+nr5vz581K7dm2ZM2fOxff+Q002TQAAEWVJREFU/PNP2bJly1XFNmrUKBk7dqyIiIwdO1aeffbZTMfr3LmzLFq0SEREYmNjJT4+XkRE1q9fL5GRkVKzZk05efLkxfHT0tKkWbNmF8fLKLPtDlgrOdzvFtqrng6eTuCpnzbSqHoZXr2tYbbjnzuXyEsv/cmXX65j1aoHadGiKo891jYPIlVFxWtzt7H9SEyuTrNBtTK80jP77Ttdu3bt2Lx5MwA//PAD7du358YbbwTAx8eHcePG0aVLFx555BHeffddXnzxRUJDQwHw8vJixIgR/5lmXFwcI0eOZO3atRhjeOWVV+jTpw++vr7ExcUBMHPmTObNm8fkyZO5//77qVChAhs2bKBZs2b88ssvbNy4kXLlrO79a9euzfLly/Hw8GDYsGEcPGhdRPLxxx/Tvn37S+YdGxvL5s2badq06cVhs2bNomfPnlSuXJlp06bx/PPPZ7tefvjhB9q1a0fPnj0vDrvuuuucXq+XM3v2bJYuXQrAfffdR5cuXXjnnXcuGWf79u2kpKTQrVs34NKznObNm2c6XWMMXbp0Yd68efTr1++q48xOoUwUicmpDJ+6DoDPB7SkRDHPy44rIsyYsZ0nnljIsWNxPPpoG2rVKp9XoSqVZ1JTU/njjz8YMmQIYDU7tWzZ8pJxatWqRVxcHDExMWzdupWnn3462+mOGTOGsmXLsmXLFoDLNq842r17N0uWLMHT05O0tP9v796Do6qzBI5/z4K8M6AgqSBiwIg2CQECOKIgCoz4WtAMEuVN4aiIUoJg7Ra7Jaur4oyPlQUXnRkrODWEiA6IL1AQBlSCgMQQCWYixIgiiRghIhAgZ/+4N50m6aQ7mXQn6ZxPVVf1ff/6VPc9fX/33nPLWL16NdOnT2f79u3ExsYSHR3NhAkTmDNnDkOHDqWgoIDRo0eTk5Nzznp27txJQkLCOePS0tJ49NFHiY6OZty4cUEliuzs7Cqx8KekpIRhw4b5nbZixQr69OlzzrjDhw8TE+OUCIqJiaGwsLDKcrm5uXTq1Ink5GQOHDjAqFGjWLRoES1aVL/fAhg0aBBbt261RFFXj775BV98d4w/Tx3ExRe0q3Y+VSU5+TXWrNlHUlIMa9fexaBB3cLYUtOc1Oaff306ceIE/fv3Jz8/n4EDB3r/uar7zHZ/anPlzIYNG1i5cqV3+PzzA//RuuOOO7w7wpSUFB577DGmT5/OypUrSUlJ8a5379693mWOHTtGSUkJUVFR3nGHDh3iwgsv9A4fPnyYvLw8hg4diojQsmVLsrOzSUhI8PuZanuFUFRUFJmZmbVaJpAzZ86wdetWdu/eTY8ePUhJSSE1NdWb0KvTtWtXvvvuu3ptS3Ui7mT2azu+IX3nNzxwfRwjPdF+5zl9+izgfEmGDr2YxYtv5NNP77YkYSJS27ZtyczM5Ouvv6a0tJSlS5cCEB8fT+W6afv376dDhw5ERUURHx/Prl27Aq6/uoTjO67ydf3t27f3vh8yZAh5eXkUFRWxZs0akpOTASgrK2Pbtm1kZmaSmZnJt99+e06SKP9svutOT0+nuLiYnj17EhsbS35+vjeJde7c+ZyjnR9//JEuXbp4YxHMZy0pKaF///5+X75JrVx0dDSHDh0CnKTWtWvV+666d+/OgAED6NWrFy1btuS2227js88+C9iWkydP0rZt24Dz1YeIShTZ3x7lP9/M5pq4zsz5TW+/82zenE9i4jLefHMfAA8/fDUPPvhrWvyThQGNaew6duzI4sWLeeaZZzh9+jQTJ07ko48+YsOGDYBz5DF79mweeeQRAObPn8+TTz5Jbm4u4Oy4n3vuuSrrveGGG1iyZIl3uHxnHB0dTU5OjrdrqToiwu23387cuXPxeDx07tzZ73r9/ZP3eDzk5eV5h9PS0li3bh35+fnk5+eza9cub6K47rrrSE9Pp7S0FIDU1FTveYgJEybwySef8M4773jXtW7dOm93WrnyIwp/r8rdTgBjxoxh+fLlACxfvpyxY8dWmWfw4MEUFxdTVFQEwIcffuh3XZXl5uZW6XYLmbqeBW+oV3VXPf30S6kOe/pD/fUTG7So5GSV6YWFP+uUKasVFmrPnv+jGzfu97MWY+pXY7vqSVX11ltv1VdffVVVVbOysnT48OHau3dvvfTSS3XhwoVaVlbmnfett97SpKQkveKKK9Tj8ei8efOqrL+kpESnTJmi8fHxmpiYqG+88Yaqqq5atUp79eqlw4cP11mzZunUqVNV1f/VRzt27FBAU1NTveOKiop0/Pjx2rdvX/V4PHrvvff6/XwJCQl67NgxPXDggHbr1u2c9quqDhgwQDMyMlRVdeHChZqQkKD9+vXT5ORkLSws9M6Xk5Ojo0eP1ri4OPV4PJqSkqLff/99jbEN5IcfftARI0ZoXFycjhgxQo8cOeL9vDNmzPDO9/7772vfvn01ISFBp06dqqdOnVJV1RdeeEEvuugibdGihcbExJyzzC233KJZWVl+t1vfVz1FRJnxsjLlnr/sZPOXRaTfO4SBl5zbR5qWtodZs97l559LmT//ahYsuJZ27eqvYJYx1cnJycHj8TR0MyLa888/T1RUVJV7KSLZ4cOHmTBhAhs3bvQ73d/3rtmXGV+25Ss25BSy4BZPlSQBcOZMGQkJXcnMvI8nnhhpScKYCDJz5kxat27d0M0Iq4KCAp599tmwba/JX/X0yVc/8Mz6L7k1MYZpV8cCcPx4KY8/voUePTpy//2DmTQpkUmTEq3mjjERqE2bNkyePLmhmxFWgwcPDuv2mvQRxfdHTzI7bTe9LuzA0791EsHbb+cSH/8iTz/9Mbm5RwDnZJklCdNQmlr3rmnaQvF9a7JHFKfPljFrxWf8UnqWlfckUVz0C5Nnv8fq1fvo0+dCtmyZxrBhlzR0M00z16ZNG44cOWKlxk1YqPs8ijZt6rd4aZNNFE+9u49dXxez+K4BxHWNYsuWr1m//iueemokc+cOoVWrmu9qNCYcunfvzsGDB72XPhoTauVPuKtPTTJRvJN1iFc+PsBNl3bhwOYC6NeNa6+9hIKCh+jcufo7sY0Jt/POO69enzRmTEMI6TkKEblRRL4UkTwR+Tc/01uLSLo7fbuIxAZa56kzZcx//XM6nYGXZq7nuecyOH7cuYHGkoQxxtS/kCUKEWkBLAVuAvoAd4lI5dsNZwDFqhoHPA88TQD7C3/m+LFT7P1jFrMfuJI9e2bSvn2r+m6+McYYVyi7nq4E8lR1P4CIrATGAr4FUcYCC933rwNLRES0htP2Z8qUTnt/YvWmaSQlxYSm5cYYY7xCmSguAr7xGT4IVH7Ag3ceVT0jIkeBzsAPvjOJyD3APe7gqawPpmUP/CAkbW5qulApVs2YxaKCxaKCxaLC5XVdMJSJwt+1gJWPFIKZB1V9GXgZQER21vU29EhjsahgsahgsahgsaggIjsDz+VfKE9mHwQu9hnuDlQunu6dR0RaAh2BH0PYJmOMMbUUykSxA7hMRHqKSCvgTmBtpXnWAlPd9+OAD2s6P2GMMSb8Qtb15J5zeABYD7QAXlHVL0TkMZxyt2uBPwN/EZE8nCOJO4NY9cuhanMTZLGoYLGoYLGoYLGoUOdYNLky48YYY8KrSRcFNMYYE3qWKIwxxtSo0SaKUJT/aKqCiMVcEdkrIlkislFEIrZsbqBY+Mw3TkRURCL20shgYiEi493vxhcisiLcbQyXIH4jPURkk4jsdn8nNzdEO0NNRF4RkUIRya5muojIYjdOWSKSFNSK6/oM1VC+cE5+fwX0AloBnwN9Ks1zP7DMfX8nkN7Q7W7AWFwPtHPfz2zOsXDniwK2ABnAoIZudwN+Ly4DdgPnu8NdG7rdDRiLl4GZ7vs+QH5DtztEsbgWSAKyq5l+M/Aezj1sVwHbg1lvYz2i8Jb/UNVSoLz8h6+xwHL3/evASInMgv8BY6Gqm1T1F3cwA+eelUgUzPcC4HHg98DJcDYuzIKJxe+ApapaDKCqhWFuY7gEEwsFfuW+70jVe7oigqpuoeZ70cYCr6ojA+gkIgFrITXWROGv/MdF1c2jqmeA8vIfkSaYWPiagfOPIRIFjIWIDAAuVtW3w9mwBhDM96I30FtEPhaRDBG5MWytC69gYrEQmCQiB4F3gQfD07RGp7b7E6DxPo+i3sp/RICgP6eITAIGAcND2qKGU2MsRORfcKoQTwtXgxpQMN+LljjdT9fhHGVuFZEEVf0pxG0Lt2BicReQqqrPisgQnPu3ElS1LPTNa1TqtN9srEcUVv6jQjCxQERGAQuAMap6KkxtC7dAsYgCEoDNIpKP0we7NkJPaAf7G3lTVU+r6gHgS5zEEWmCicUM4DUAVd0GtMEpGNjcBLU/qayxJgor/1EhYCzc7paXcJJEpPZDQ4BYqOpRVe2iqrGqGotzvmaMqta5GFojFsxvZA3OhQ6ISBecrqj9YW1leAQTiwJgJICIeHASRXN8Pu1aYIp79dNVwFFVPRRooUbZ9aShK//R5AQZiz8AHYBV7vn8AlUd02CNDpEgY9EsBBmL9cANIrIXOAvMV9UjDdfq0AgyFg8DfxSROThdLdMi8Y+liKThdDV2cc/HPAqcB6Cqy3DOz9wM5AG/ANODWm8ExsoYY0w9aqxdT8YYYxoJSxTGGGNqZInCGGNMjSxRGGOMqZElCmOMMTWyRGEaHRE5KyKZPq/YGuaNra5SZi23udmtPvq5W/Li8jqs4z4RmeK+nyYi3Xym/UlE+tRzO3eISP8glnlIRNr9s9s2zZclCtMYnVDV/j6v/DBtd6Kq9sMpNvmH2i6sqstU9VV3cBrQzWfa3aq6t15aWdHOFwmunQ8BlihMnVmiME2Ce+SwVUQ+c19X+5knXkQ+dY9CskTkMnf8JJ/xL4lIiwCb2wLEucuOdJ9hsMet9d/aHb9IKp4B8ow7bqGIzBORcTg1t/7qbrOteyQwSERmisjvfdo8TUT+t47t3IZPQTcR+T8R2SnOsyf+yx03GydhbRKRTe64G0RkmxvHVSLSIcB2TDNnicI0Rm19up1Wu+MKgd+oahKQAiz2s9x9wAuq2h9nR33QLdeQAlzjjj8LTAyw/X8F9ohIGyAVSFHVvjiVDGaKyAXA7UC8qiYC/+27sKq+DuzE+effX1VP+Ex+HUj2GU4B0uvYzhtxynSUW6Cqg4BEYLiIJKrqYpxaPter6vVuKY//AEa5sdwJzA2wHdPMNcoSHqbZO+HuLH2dByxx++TP4tQtqmwbsEBEugN/U9V/iMhIYCCwwy1v0hYn6fjzVxE5AeTjlKG+HDigqrnu9OXALGAJzrMu/iQi7wBBlzRX1SIR2e/W2fmHu42P3fXWpp3tccpV+D6hbLyI3IPzu47BeUBPVqVlr3LHf+xupxVO3IypliUK01TMAQ4D/XCOhKs8lEhVV4jIduAWYL2I3I1TVnm5qv57ENuY6FtAUET8Pt/ErS10JU6RuTuBB4ARtfgs6cB4YB+wWlVVnL120O3EeYrbImApkCwiPYF5wGBVLRaRVJzCd5UJ8IGq3lWL9ppmzrqeTFPRETjkPj9gMs6/6XOISC9gv9vdshanC2YjME5EurrzXCDBP1N8HxArInHu8GTg726ffkdVfRfnRLG/K49KcMqe+/M34DacZySku+Nq1U5VPY3ThXSV2231K+A4cFREooGbqmlLBnBN+WcSkXYi4u/ozBgvSxSmqXgRmCoiGTjdTsf9zJMCZItIJnAFziMf9+LsUN8XkSzgA5xumYBU9SROdc1VIrIHKAOW4ex033bX93eco53KUoFl5SezK623GNgLXKKqn7rjat1O99zHs8A8Vf0c5/nYXwCv4HRnlXsZeE9ENqlqEc4VWWnudjJwYmVMtax6rDHGmBrZEYUxxpgaWaIwxhhTI0sUxhhjamSJwhhjTI0sURhjjKmRJQpjjDE1skRhjDGmRv8P5tHaIz14QuoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(tree, y_test, X_test, \"Decision Tree (GINI)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.82 0.80 0.81 6729\n", + " 1 0.39 0.41 0.40 2020\n", + "\n", + " accuracy 0.71 8749\n", + " macro avg 0.60 0.61 0.60 8749\n", + "weighted avg 0.72 0.71 0.72 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, tree.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[0] = ([\"Decision Tree\" , \n", + " classification_report(y_test, tree.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Random Forest Classifier\n", + "\n", + "#### Theory\n", + "Random Forest is an ensemble method for the decision tree algorithm. It works by randomly choosing different features and data points to train multiple trees (that is, to form a forest) - and the resulting prediction is decided by the votes from all the trees. \n", + "\n", + "Decision Trees are prone to overfitting on the training data, which reduces the performance on the test set. Random Forest mitigates this by training multiple trees. Random Forest is a form of bagging ensemble where the trees are trained concurrently. \n", + "\n", + "#### Training\n", + "To keep things consistent, our Random Forest classifier will also use the GINI Coefficient.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.ensemble import RandomForestClassifier\n", + "randf = RandomForestClassifier(n_estimators=300)" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n", + " max_depth=None, max_features='auto', max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, n_estimators=300,\n", + " n_jobs=None, oob_score=False, random_state=None,\n", + " verbose=0, warm_start=False)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "randf.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 1.00 1.00 1.00 13475\n", + " 1 1.00 1.00 1.00 4021\n", + "\n", + " accuracy 1.00 17496\n", + " macro avg 1.00 1.00 1.00 17496\n", + "weighted avg 1.00 1.00 1.00 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, randf.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training set has also been 100% correctly classified by the random forest model. Evaluating with the test set:" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (Random Forest) identified 749\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06310419
11271749
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6310 419\n", + "1 1271 749" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, randf.predict(X_test), \"Decision Tree (Random Forest)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.26\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEWCAYAAADcsGj7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KCCQhjRBCIBASIPROpAsoVUSxYG/YEBv27rWiV71c27WLfnZRsSOIIAI2pErvNaGkkt6T/f0xEzyEQA7l5JycrPd5eMiZumZmz6yZPTN7xBiDUkop5Yl83B2AUkopdSSapJRSSnksTVJKKaU8liYppZRSHkuTlFJKKY+lSUoppZTH8sgkJSKXichP7o7D3UQkRkRyRcS3BucZKyJGROrV1DxdSUTWicjQ4xjvuMqgiASIyPcikiUiXxzr+DXJ2fLlrfujiPiLyEYRiXTDvOvZ+1lsTc/bE4hILxH51Zlhq01SIrJTRArswrxfRN4TkaATD/PIjDEfG2NGunIenshe18MrfhtjdhtjgowxZe6My13snbjtiUzDGNPZGLOgmvkclphPoAyOB5oCjY0xFxzH+JVjGyoi5fb+lysiSSLyuYiccqLTdrZ8uWJ/tBNfxTIVVFrG3JM5r6O4EZhnjEmxY/pIRIrtGDJE5CcRaVdDsbiMiPwmIoWO6/dklJ9jmP9hCdkYswIoEJEzqhvf2Sups4wxQUAPoCfwwHHE6nbuvDrwliuTY1FH13crYLMxpvRYRzxKzHvt/S8Y6AdsBH4VkWHHH6Z72YkvyF6uM7CX0aHbIVy0PW8APqzU7Wl7/tFACvC2C+brDpMc168xZumxjCwiPiJysmvePsbaBkdnjDnqP2AnMNzh93PADw6/GwBTgd1AMvAGEODQfxzwN5ANbANG291DgXeAfcAeYArga/ebAPxm//0GMLVSTN8Cd9p/Nwe+BFKBHcBkh+EeA2YAH9nzv66K5QsFPrDH3wU8DPg4xPE78D8gC+vgMKzSuEdbht+BF4AMu18bYD6QDqTZGynMHv5DoBwoAHKBe4FYwAD17GEWAE/a080BfgIiHOK50l6GdOBflbddpeUOAP5rD58F/GZ3q5jnVfY2TQMechivD/AnkGkv9ytAfYf+BrgZ2ALssLu9BCTa22A5cKrD8L7Ag3bZyLH7twQW2dPKs9fHRfbwY7HKUybwB9CtUlm9D1gNFAH1HNeBHfsyO45k4Hm7+257Xrn2v/44lEF7mM7AXHtbJgMPVrFOHweKgRJ7OtdinQg+bK/nFKyyFmoPX7Gur7VjWFTFNIcCSVV0fwVY5vC7g0N8m4ALj2FbV5SvCcB2ezvsAC6rvD/avwcAS+1pLQUGOPRbwFHK6BHK4pGWMQm4B1gDFNvdWgBf88/+frPD8D78U5bSgOlAoyPMszVW2fJ16PYR8JjD77OBLIff8cAv/LP/flixLR3ivdOONwv4FGjg0P9+YD/WseJae93H2v3C7PmnYpXZBwCx+10HLARexir3W4G+9jQSscrj5UdZv78BE47QbxDWPpEFLAH6VhrvSaz9vcAuL2HA/2Ht+0nAE/xzvGyHtd9m2evnE7v7Hxy6L59vd29ld/M7avk4Wk+HHb9iJ29hb4CXHPq/CHwHhGOd6X0P/NvhoJAFjLALUDTQwe73DfAm0BCItFfQDZV3CmCwvSEqNlgje4U1t6e5HHgEqI9V8LYDoxySVAlwjj1sQBXL9wFW0gu2N8Jm4FqHOEqBOwA/4CJ7ecKdXIZS4Fasg2UA0NZeFw2AJvYGfbGqdV3pIOaYpLbZhSHA/v2M3a+TXQAG2etiqr3sR0pSr9rjR2MligF2XBXzfNueR3esA35He7zeWGfz9exhNwC3V0pSc7HKQ4Dd7XKgsT3OXVg7qr/dr+Ig1B4Qe36NHabV1mHavbAO9H3tmK+y11kDh/X3N1aSC6i8TrF2tivsv4OAflWt5yrKYDDWTnkX4G//7nuE9foY8JHD72uwDiqt7Xl+BXxYab4fYJWhqsrnUKo+gJ+OdVLT0P6XCFxtr+NeWAeJzk5u63r2NLKB9vY4zRzGd1wX4cAB4Ap7vEvs3xXbbAFHKKPHmaSWYx13AuzY/8ZKRPWx9qed2CeOwN1YyTHa3k7vVKzrKqY9DlhVqdvBJGVvq0+B5Q792wHD7HlH2vOaWinexUAUVnnfjH1ijHVytQ9rP20IfM6hSeoTu2wE22VlK3CVQ5Iqtde5L/AM1gnHy/Z2HIN1XAo8liQFRNjjXWJvy8uxEnAjh/F2Ah2xjn/1gJnAa0CgvZzL+ed4+QXWSaKPvf4H2t3rOS5rpRjygU4nI0nlYp0VGeBn/jn7F6xM2MZh+P78cwb9JvBCFdNsinXgc7ziugT4pYqdQrDOMgfbv68H5tt/9wV2V5r2A8D/ORwwDjs7dRjW146jk0O3G4AFDnHsxU6QdrcldmFxZhl2H2ne9jDnACsrrevqktTDDv1vAn60/34E+NShXyDWWf1hScouRAVA9yr6VcyzRaVlvvgIy3A78LXDbwOcXs1yH6iYN9ZZ/7gjDFc5Sb0OPFlpmE3AEIf1d00V5bciSS3CutqJqDTMIeu5ijJ4ieN2qmbZHuPQJPUzcJPD7/ZYJw/1HObb+ijTG0rVB/AO9rjRWCdPv1bq/ybwqJPbuiJJZQLnUylZVloXVwBLKvX/E/sgeLQyehzLmARc6fB7ILC90jD/At62/95SURbs3y2x9lGfKqZ9FQ5Xh3a3j4BCez0YrBPeLkeJezywtFK8Fzv8fh54xf77A2CKQ79O9jxisRJAKdDOof/NWPfLwEpSGxz69bTHbezQLetIsWIlm3x7uTIrth/WSc0flYZdin1VZo/3iEO/aLssOV4dXgHMtf/+BGsfja40zaMlqWQcrsSr+udsHeM5xphgrMLUASsDg3U1EAgsF5FMEckEfrS7g1VItlUxvVb2htnnMN6bWGcnhzDWkkzHOlAAXIpVTVYxneYV07Cn8yBWAqmQeJTlisA6K9rl0G0X1saosMeOwbF/cyeX4ZB5i0ikiEwXkT0iko21U0RwbPY7/J2PdcaHHdPB+Rlj8rHOiqoSgXWmU9W2Oep8RKSdiMy0H6LJBp7m8GWovNx3icgG+4m3TKxq0opxjlRGqtIKuKvS9m6JtexVzruSa7HOhjeKyFIRGevkfI8lxsqac3j5qofzZfRIorF2/Eys9dK30nq5DOtM15ltjTEmDyvZTcIq0z+ISAcnlgcO32eOVEaPh+O6aQXEVFrOe7GWEyAG+N6h3xqsdVTV03sHsK5aKnvGGBMGxGGd5MVX9BCRKPuhlYr99z0OL/tO7Z8cug4jsU6Yj3YcSnb4uwAoM8akV+p2tPV8kzEmzP7XxyGm6rZl5fXfAEh2WMev8k9ZvgvrmLhMRNaIyFVHiadCMFYZPqJjuhFmjFmItWGm2p3SsFZOZ4cVEGr+ufGZiHUfprJErDOcCIfxQowxnY8w60+B8SLSCuvq6UuH6exwmEaYMSbYGDPGMeyjLFIa1lltK4duMVh1xhWiRUQq9d/r5DJUnve/7W7djDEhWJfXcpThj8U+rGoRwHoUGqvKoSppWGeMVW2b6ryOdW8u3l6GBzl0GcBhOUTkVKwqgAuxqhHCsM76KsY5UhmpSiLwVKXtHWiM+bSqeVdmjNlijLkE66DwLDBDRBoebZzjiLGyvRxevko59KBzPNv9XGCFnVwSgYWV1kuQMeZGjmFbG2PmGGNGYFX1baTqhwYqLw8cvs+cTI7rJhHYUsX+fpbdPwkYUam/vzFm/+GTZTXQ5kiP3xtjdmJV8/9PRBrYnZ/F2ue72mV/AoeX/SPZh3WyUyHG4e8UoIyjH4dcwZltWXn952Pd7nA85nUDMMbsM8ZcZ4xphnUl+JaIxHGE8m0fz8G6Aj6i43la40VghIj0MMaUYxXkFyreNRCRaBEZZQ/7DnC1iAyznw6JFpEOxph9WDdU/ysiIXa/NiIypKoZGmNWYt1QnAbMMcZUZN4lQLaI3CfW+ym+ItLF2ccrjfXo7efAUyISbK+0O7GucCpEApNFxE9ELsCqn511rMtgC8aqOs0UkWis+zGOkrHqo4/HDOAsERkgIvWxqrWq3IHs7fYu8LyINLfXW3+HnfFogrHuXeTaZ9o3OjF8Kdb2qycijwAhDv2nAU+KSLxYuolIRXKtvD7eBiaJSF972IYicqaIVHVGfBgRuVxEmtjLX1GGyuzYyjnyup8JRInI7SLSwC4rfZ2ZJ9YJ1h0iEifWqxtPA5+Z43v6T+x96FGsKqAHHeJrJyJX2OXUT0ROEZGOzm5rEWkqImfbSbsIq5xW9Wj6LHtel4r1aPFFWFVXM491eY7Dn0CxfWXuby9LVxHpbfd/A3haRGLgYM3F2VVNyE5Cu7HusVbJGDMbq2xcZ3cKxrq9kSUiLbHugTnrc+AaEelgr+NHHeZTgrX/Pi0iQfaB/Q4OPQ65wkygs4hcZG/LS7Hu882qamBjTCLWAxxTHY55bUVkMICIXGgf1+CfKtMy+zibzuH71xCsKs2SowV5zEnKGJOKVb/6L7vTfVg3+Rbbl8DzsOrdMcYswar3fAHr7Hkh/2TuK7Gq2tZjXXrPwDqDO5JPgeFY9Z4VsZQBZ2E9Gr8D66xxGlZ1krNuxSp427HqYD/B2qkr/IV1yZ8GPAWMd7jMPtZleBzrpnYW8APWjVJH/wYeti+lj2UHwBizzl6W6VhnbTlYZ2hFRxjlbqzqkKVYT4Q9i3Pl4W6sKtccrKTxWTXDzwFmY91E3oV1Vu9YhfA81g78E1byewfrJjlY93fet9fHhcaYZVj3JF/BWt9bsc5mnTUaWCfWezgvYd0/KLSrRp8Cfrfn1c9xJGNMDtYDL2dhVedsAU5zcp7vYj0FtgirjBZibadj0dyOORdre3UFhhpjfnKIbyRwMdbZ8X6s7VmRiJzZ1j5Y1TV77WGGYN1POoRd9sfaw6ZjVbeNNcakHeMyHTM7sY/BeiBrJ9Y++Sb/nPQ8j3W74WcRycF6quxoJ6xvYt1TOZqpwH32id+j/PMw2Hf8U6PjTOzfY1WNLcTaF+ZWGuQmrOrFHfYw72MdZ13GPpafjXUMT8dKjGONMRlHGe1yrPuXFce8L/inurUvsFRE8rCObTcbY3bb/R4FPrH3r/PsbpdhnVgcVcUTc6oKIjIB6+mcQe6O5VjZZ+2ZWNVyO9wdj1KeRkT8gZVYD1ukuDueukREegL/c+bY6pHNIqnjIyJniUigXZ0wFevsead7o1LKM9lX0R01QdU8Y8xKZ0/+NUl5l3FY1TV7saooLzZ6qayUqsW0uk8ppZTH0isppZRSHqtWNnoaERFhYmNj3R2GUkrVKsuXL08zxjSpfkjPUSuTVGxsLMuWLXN3GEopVauISOUWJjyeVvcppZTyWJqklFJKeSxNUkoppTyWJimllFIeS5OUUkopj6VJSimllMdyeZISkXdFJEVE1h6hv4jIyyKyVURWi0gvV8eklFKqdqiJK6n3sD6RcCRnYLUzFw9MxPqonlJKqZOotjaB5/KXeY0xi0Qk9iiDjAM+sBtCXSwiYSLSzP6ooFJKqaMwxpBbVEpKThEp2UWk5BSSmlNk/y5kf3YhiWn57M0scHeox8UTWpyI5tCP4CXZ3Q5JUiIyEetKi5gYxy8vK6WUdyoqLSMlu4hd6fls2JfNhv3ZpOYUUVRSTmFpGVkFJaRkF1FQcvhHlOvX86FJUANKs4vYuSEdKTjmj0F7BE9IUlV94vyw61JjzFvAWwAJCQm187pVKaUqySooYeXuA6zdk8XerEKSswrZl1VIcnYh6XnFhwzbNKQBzcMC8K/nS3jD+rRq3JDI4AbWv5AGRAb727/9CQmox9Spf3Dvv3/loos68/yro4iOvtpNS3n8PCFJJQEtHX63wPoeklJKeYXSsnJyCks5kF/MjrQ8tqbksiUll1WJmWxJyT04XHjD+jQN8adZqD/dW4bRLNSfqBB/WoQH0CEqhPCG9audV0pKHts3pdGzZzNuvrkPPXs2Y/jw1q5cPJfyhCT1HXCLiEwH+gJZej9KKVVbbUnO4Yc1+5i/MYWU7CKyC0vILz68Oi4iqAFdokM4u3tzerdqRLeWYQQ1OP5Dcnm54Z13VnDfffOIigpi7dqbCAz0q9UJCmogSYnIp8BQIEJEkoBHAT8AY8wbwCxgDLAVyAdq3/WoUqpOKis37EjLZd3ebNbtzWb+xhS2puQiAgmtGjG4XQTB/n6E+PsRElCPEH8/WjUOpG1kEGGB1V8VOWv16mQmTZrJn38mMXRoLK+/fiY+PlXdSal9auLpvkuq6W+Am10dh1JKnYiC4jI27reS0fp92azfm83G/dkUlpQDUN/Xh16twriqf2dGdYkiMti/RuL6668kBg58l0aNAnj//XO44opuiHhHggLPqO5TSimPkp5bxPp9dkKyk9L21FzK7Ue2Qvzr0bl5KJf1bUXn5iF0ah5CmyZB+PnWXCM+SUnZtGgRwimnRPPkk6dxww0JhIcH1Nj8a4omKaVUnVVebkg8kM/6vYdeIe3PLjw4THRYAJ2ah3Bm12YHE1J0WIDbrlZ2785i8uTZLFy4i02bbiEysiEPPHCqW2KpCZqklFJeKyu/hF0ZeWQVlJBdUEp2YQnZBSXsyypk/d5sNuzLJqfIen/I10do2ySIAW0a08lORp2ahZzUe0cnoqSkjJde+otHH12AMYbHHhtKo0Y1U6XoTpqklFJeITEjn792ZLBpfzabknPZvD/nkCsiR4H1fenYLIRze0XTqVkInZuHEt80CH8/3xqO2jm5ucUMHPguq1cnM3ZsO/73vzOIjQ1zd1g1QpOUUqrWKC0rt5r7ySkiObuQlJwi1iZl8cf2NBIzrGZ/GtTzIb5pEAPaNqZd02BaRzQkvGF9QgL+ecouwM+3VjxcUFJShp+fL0FB9Rk2LI7HHhvCOed0qBWxnyyapJRSHq2wpIxft6Qxa80+5q1PPlg9VyHEvx792zTmukGt6de6MW0jg/Ct5Y9fG2P45JM13H//z8yefRldukTy/POj3B2WW2iSUkp5lILiMvvJuiyW7TzA/I0p5BaVEhrgx+guUfSMaUTTiiaAQhoQEdSg1iclR5s2pXHTTbOYP38HffpEU4cumqqkSUop5TZWQspiTVIWa/Zks3ZPFltTcymzn/WOCKrPmK5RjOnajIFtI2r0EW93eOqpRTzxxCICAurx2mtjmDixN75evszV0SSllKoReUWlrN+XzZqkLNbuyWLt3iy2pvzz7lFEUAO6RocwqnNTukSH0iU6lGah/nXq/ktubjHjx3fiv/8dSVRUkLvD8QiapJRSJ1VZuWF3Rj6b9uewOTmHTck5bNqfw7bUXCq+uxcZ3ICu0aGM7tKMrtGhdI0OpWlIgzqVkAD278/lrrt+4soruzFqVFueemqY1zRndLJoklJKHRdjDPuyCtmcbCej/blsTs5hS0rOwaaCRCAmPJB2TYMZ2+2fhBQZ4v3v9xxNebnhzTeX8cADP1NQUMrAgdaHIDRBHU6TlFLqqIwxJGcXHUxGW5Jz2ZySw9bk3EOetIsMbkD7qGAu79uKdlHBtG8aTHzTIALr62HG0d9/72fSpJn89dceTj89jtdeG0P79hHuDstjaelRSh1kjGFXej5LdmawcncmW+zElF34TzIKb1if+MggzukZTbumQcQ3tRJSIye+daSsBmF37Mjko4/O5dJLu9a5Ks5jJcbUvo/cJiQkmGXLlrk7DKVqvfJyw6bkHJbsyGDJzgyW7MggNacIsN4/ah8VTLum1r/4pkG0axpMRFADN0dduxhj+OabjRQWlnLJJV0pLzdkZxcRFlbzVZ4istwYk1DjMz4BeiWlVB1SVm5YsyeLxdvTWbojg6U7Mw5eJTUL9WdAm8acEhtO37hw2jQJ0nskJ2jXrkxuuWU2M2duZujQWC6+uAs+PuKWBFVbaZJSysslZxeycHMqizan8tvWNDLzSwBo3aQhY7o245TYcPrEhdOikfta9vY2JSVlvPDCYh5/fCEiMHXqCG67rZ+u3+OgSUopL5SWW8SsNfv47u+9LNt1AIAmwQ04vUMkQ9o1YUCbCJoEa7Wdq/z+eyL33TePc87pwEsvjSYmJtTdIdVamqSU8gIVDzz8tSOdWWv289vWNMrKDe2aBnH3yHac3qEpHZsF65m8C6Wn57No0S7OPbcjQ4fGsmTJdZxySrS7w6r1NEkpVUttT83l1y1pLLHvLaXYDzxEhwUwcXBrxvVoToeoEDdH6f2MMXz44Wruuusn8vKK2b37DiIiAjVBnSSapJSqRbam5DBrzX5mrdnHxv05ADQP9ae//cBDn7hw2uoDDzVm48Y0brzxBxYs2En//i14442xREQEujssr6JJSqla4I9taTz+3Xo2JecgAgmtGvHI2E6M6NSUluF6UHSHjIwCEhLews/PlzffHMt11/XSkwMX0CSllAdLyy3ihbmb+fiv3dT39eH+Mzpwbs9omtbxZoXcadWq/XTvHkV4eAD/93/jGDIklsjIhu4Oy2tpklLKA6XlFvHZ0kReX7CNgpIyJgyIZfKweMK1VQe32bcvhzvumMNnn61j/vwrOe20OC64oLO7w/J6mqSU8hC5RaX8tG4/3/699+DTecM7RnL/GR1pG6mfbXCXsrJyXn99GQ89NJ+iolIef3woAwa0dHdYdYYmKaXcbHVSJm8t2s68DckUlpQTHRbADYNbM65HNO2jgt0dXp1mjGHMmE/46adtDB/emtdeG0N8fGN3h1WnaJJSyk12pOUxdc4mflizj9AAP8b3bsE5PaLpFdNIb8C7WU5OEQ0b1sfHR7jyym5MmNCdiy/uou+ZuYEmKaVq0J7MApbtzGDR5jS++XsPDer5MPn0tlw/uDXB/n7uDq/OM8bw5ZcbuO22H3nssSFcf31vLrusm7vDqtM0SSnlQoUlZSzcnMqPa/ezeHs6+7IKAWhY35dL+8QweVi8Nk/kIXbsOMAtt8xm1qwt9OgRRY8eUe4OSaFJSqmTLr+4lF+3pDFrzT7mrU8mr7iMRoF+DGwbQUKrRiTEhtMhKph6vj7uDlXZpk1bweTJs/H19eGFF0Zxyy19qFdPt48n0CSl1EmQklPIzxtSmLc+md+2plFUWk5YoB9ndW/Omd2a0a91Y/w0KXkcYwwiQsuWIYwe3ZaXXz6DFi20KSlPoklKqeNUXm74eWMKby3axtKdVkvj0WEBXNInhhGdmtInLlwTk4dKS8vn3nvn0rx5MFOmnM6oUW0ZNaqtu8NSVdAkpdQxKi0r54c1+3jtl21sSs6hZXgAd45ox4hOTekQpS2NezJjDO+99zf33DOXrKwi7r9/oLtDUtVweZISkdHAS4AvMM0Y80yl/jHA+0CYPcz9xphZro5LqWNVVFrG1yv28PrCbexKzyc+MogXLurOWd2a6/2lWmDz5nSuu+47fv11NwMHtuSNN8bSpUuku8NS1XBpkhIRX+BVYASQBCwVke+MMesdBnsY+NwY87qIdAJmAbGujEupY1FQXManS3bz1qLt7M8upGt0KG9c3puRnZrq+0y1SHFxGZs3pzNt2llcfXVP3Xa1hKuvpPoAW40x2wFEZDowDnBMUgaouFMZCux1cUxKOSW7sIQP/9zFO7/tICOvmD5x4Tw3vhunxkdolV4tMXv2Fn75ZSfPPTeCLl0i2bXrdho00LsctYmrt1Y0kOjwOwnoW2mYx4CfRORWoCEwvKoJichEYCJATEzMSQ9UqQrpuUW8+/sOPvhjFzlFpQxt34SbT2vLKbHh7g5NOWnPnmxuv30OM2asp0OHCB5+eDAhIQ00QdVCrt5iVZ1umkq/LwHeM8b8V0T6Ax+KSBdjTPkhIxnzFvAWQEJCQuVpKHVCEjPy+XlDMvM2pLB4ezplxnBGlyhuGtqWLtGh7g5POam0tJzXXlvKww/Pp6SknClTTuOeewZSv76vu0NTx8nVSSoJcGwuuAWHV+ddC4wGMMb8KSL+QASQ4uLYVB23Kz2Pmav3MXP1PjbsywagbWQQ154axwW9W2rL47VQRkYBjzzyCwMGtOTVV8fQpo1e/dZ2rk5SS4F4EYkD9gAXA5dWGmY3MAx4T0Q6Av5AqovjUnVUak4RX61IYubqfazZkwVA71aNeGhMR4Z3akpchH68rrbJyirk7bdXcOed/YmMbMjKlTcQGxum9w29hEuTlDGmVERuAeZgPV7+rjFmnYg8ASwzxnwH3AW8LSJ3YFUFTjDGaHWeOqn2ZxXy5qJtfPLXbopKy+neIpSHxnRkTLdmRIcFuDs8dRyMMXz++Tpuv30Oycm5DBjQkgEDWhIX18jdoamTyOV3Ee13nmZV6vaIw9/rAX2jTp1Uxhi2pOTy1/Z0Fm/PYO76ZMqM4dye0dw4tA1tmmhVXm22bVsGN988izlzttGrVzO+//4SEhKauzss5QL6qIvyCuXlhk3JOfy1PZ2/dmSwZEcG6XnFAESF+HNBQgtuGNyGmMaBbo5UnShjDOPGTWf37ixefnk0N910Cr76MrXX0iSlai1jDEt3HuCDP3fy29Y0MvNLAKv9vCHtm9AvrjF9W4cTEx6o9ye8wK+/7qJ37+YEBvrx3nvn0Lx5MM2b65eLvZ0mKVXrpOUWMX9DCh8s3snaPdmEBfoxomNT+rZuTN+4cFqG69WSN0lNzeOee+by/vur+Pe/h3H//YO0aq8O0SSlPF5pWTl/J2ayYFMqCzenHnwqr21kEE+f25Vze0YToO/BeJ3ycsO7767k3nvnkptbzIMPDmLy5MptAShvp0lKeays/BI+WbKb9//Yyf7sQnwEesU04u6R7Rjcrgldo0O1Gs+L3X77j/zvf0s49dQY3nhjLJ06NXF3SMoNNEkpj1JWblidlMk3K/fw+bIkCkrKGNi2MQ+P7cipbZsQGujn7hCVC+XlFVNUVEZ4eADXX9+LXr2acdVV3fVkpA7TJKXczhjD3PXJ/Lh2Pws2p5KRV4yfr3B292iuHRRHp+b6pdS64IcfNnPzzbMYODCGjz8+j65dm9K1a1N3h6XczOkkJSL1gRhjzFYXxqPqmLV7shMjzc4AACAASURBVHj8+3Us3XmARoF+DG0fydD2TRgc34RGDeu7OzxVA5KSsrntth/56qsNdOrUhEmTers7JOVBnEpSInIm8DxQH4gTkR7Ao8aYc10ZnPJe+7IKePnnrUxfuptGgfV55ryuXJDQEl/9xk+d8uOPW7nggi8oKyvn3/8exp139tfGYNUhnL2SegLrExu/ABhj/haRti6LSnmtzck5vLlwO9/+vcdqA2tALLcPb0dogN5rqktKSsrw8/OlR48oxoyJ55lnhmlzRqpKziapEmNMZqWbl9q+nnJKaVk58zak8MmS3SzanEqAny+X92vFtYPi9J2mOiYzs5AHH/yZtWtTWLBgAlFRQXz22Xh3h6U8mLNJaoOIXAj42C2a3wYsdl1YyhsYY/hhzT6e+mED+7IKiQrx584R7biiXyu931THGGOYPn0td9wxh9TUfG69tQ/FxWX4++uzW+ronC0htwCPAOXAV1itmj/gqqBU7bcvq4B/fbOWeRtS6BodyhPjunBa+ybU0zbW6pz9+3O58sqvmTt3OwkJzZk16zJ69Wrm7rBULeFskhpljLkPuK+ig4ich5WwlAKsJ/Xmrk/mz23prEw8gK+P8PCZHZkwIFaTUx0WEtKAtLR8XnnlDCZNStDGYNUxEWc+3SQiK4wxvSp1W26MccuzogkJCWbZsmXumLWqwpqkLF6ct5mfN6YgAl2jQ+nfpjGX9WmlrY7XUfPn72Dq1D/48ssLCQjwo7zc4KNPbrqdfdxOcHccx+KoV1IiMgrr0+7RIvK8Q68QrKo/VYet3ZPFi/O2MG9DMqEBftwzqj2X9Y0hLFDvN9VVycm53H33XD76aDVt2jRi9+4s2reP0ASljlt11X0pwFqgEFjn0D0HuN9VQSnPtn5vNi/O28xP65MJ8a/HXSPaMWFgLMH++hh5XVVebnj77eXcf//P5OUV869/DeaBBwYRoK8WqBN01CRljFkJrBSRj40xhTUUk/JQm/bn8OK8zcxeu59g/3rcPjyeawbFEaLJSQEffLCaHj2ieP31M+nQIcLd4Sgv4eyDE9Ei8hTQCfCv6GiMaeeSqJRH2Z6ay0s/b+G7VXtpWL8ek4fFc+2gOH0Bt47LzS3m6ad/5bbb+tK0aRDff38JjRr5a2Ow6qRyNkm9B0wBpgJnAFej96S8XmJGPi//vIWvVu6hvq8Pk4a0YeKprfUdJ8W3327k1ltnk5iYTdu24VxzTU/CwwPcHZbyQs4mqUBjzBwRmWqM2QY8LCK/ujIw5T77swp55ZctfLY0ERHhqv6x3Di0DU2CG7g7NOVmu3dnMXnybL79dhNdu0Yyffp4Bgxo6e6wlBdzNkkViXUNv01EJgF7gEjXhaXcIS23iNcXbOPDxbsoLzdc3KclN5/WlmaheoasLI8+uoC5c7fz3HPDuf32fvj5aWOwyrWcfU+qL7AeaAQ8BYQCzxpjfndteFXT96ROnryiUr5btZdfNqbw65Y0ikrLOL9XCyYPi9d29RQAf/6ZSGioP506NSE5OZfCwlJatQpzd1jqOHjde1IVjDF/2X/mAFcAiEgLVwWlXC8lu5D3/9zJR4t3k1VQQnRYAOf1iuaaQXG0aRLk7vCUBzhwoID775/HW2+tYPz4TnzxxQU0baplQ9WsapOUiJwCRAO/GWPSRKQzVvNIpwOaqGqR0rJyFm/P4OuVe/h+1V5KyssZ3TmK605tTa+YMH0qSwFWY7Aff7yGO++cQ0ZGAXfe2Y/HHhvq7rBUHVVdixP/Bs4HVmE9LPE1VgvozwKTXB+eOlmW7zrAPTNWsT01j6AG9bjwlBZcN6g1sREN3R2a8jDTpq1g4sSZ9OkTzU8/XUGPHlHuDknVYdVdSY0DuhtjCkQkHNhr/97k+tDUiUrPLeKzZYks33mAXzal0Cw0gP9d0pMRnZrirze8lYPCwlJ27cqkffsILrusG/Xq+XDlld21MVjldtUlqUJjTAGAMSZDRDZqgqod1u7JYuIHy9ibVUjriIZcMzCO24bHa9NF6jBz527jpptmUV5u2LjxZgID/bj66p7uDkspoPok1VpEKj7HIUCsw2+MMee5LDJ1XFJyCnntl218/NcuIoIaMPPWQXSJDnV3WMoD7d+fy513zuHTT9cSHx/Om2+O1UfKlcepLkmdX+n3K64KRJ2YlJzCgx8ZBLjolJbcOaIdEUH6Aq463IYNqfTv/w4FBaU8+ugQ7r9/kH4lV3mk6hqY/bmmAlHHxxjDH9vSufPzv8kqKGHi4NZcmNCSOH0gQlUhO7uIkJAGtG8fwTXX9GTSpATatWvs7rCUOiI9darFdqblcdcXq1i+6wAx4YF8fVMfOjYLcXdYygPl5BTx6KML+PDD1axdeyNNmwbx/POj3B2WUtVyeZISkdHAS4AvMM0Y80wVw1wIPAYYYJUx5lJXx1WblZaV88mS3fxnziZ8fYQnx3VmfO+WBNTX+wnqUMYYvv56I5Mnz2bv3hxuuKE3DRrouamqPY6ptIpIA2NM0TEM7wu8CowAkoClIvKdMWa9wzDxwAPAQGPMARHRNgGPYkdaHjd+tJyN+3Po37oxz43vps0XqSoVF5dx/vmfM3PmZrp3b8qMGRfSr5++f69qF6eSlIj0Ad7BarMvRkS6A9cZY26tZtQ+wFZjzHZ7OtOx3r1a7zDM9cCrxpgDAMaYlGNbhLrjp3X7uffL1QjwxuW9GNU5SluJUIcxxiAi1K/vS1RUQ/7735FMntyXevX0nSdV+zhbal8GxgLpAMaYVcBpTowXDSQ6/E6yuzlqB7QTkd9FZLFdPagcJGcXMunD5Uz8cDnNQgP45uaBjO7STBOUOszvv++md++3WLvWOtd7++2zufPO/pqgVK3lbHWfjzFmV6WDYpkT41V1FK3c7Ho9IB4YitUW4K8i0sUYk3nIhEQmAhMBYmJinAy79puxPInHv1tHcVk5945uz/WntsZPWwFQlaSn53P//fOYNm0lMTGhZGQUuDskpU4KZ5NUol3lZ+z7TLcCm50YLwlw/CJaC6ymlSoPs9gYUwLsEJFNWElrqeNAxpi3gLfA+lSHk3HXWttSc/nPj5v4cd1++saF8+z53bSdPVWljz9eze23z+HAgQLuuWcAjzwyhKAg/Xqy8g7OJqkbsar8YoBkYJ7drTpLgXgRicP6UOLFQOUn974BLgHeE5EIrOq/7U7G5ZVWJWZy5btLyCooYfKweCaf3pZ6evWkjmDdulTi48N5442xdOvW1N3hKHVSOZukSo0xFx/rxI0xpSJyCzAH6xH0d40x60TkCWCZMeY7u99IEVmPVYV4jzEm/Vjn5S1WJ2Vy+bS/CAnw48Nr+9CthX5cTh2qoKCEp576lcGDWzFyZBsee2wo9er54OOj9yiV93E2SS21q+E+A74yxuQ4OwNjzCxgVqVujzj8bYA77X912lcrkrjvy9VEBvvzxaT+NA/Tz7arQ82Zs5WbbprF9u0HKC83jBzZhvr6fpzyYk7VIRlj2gBTgN7AGhH5RkSO+cpKHVlWfgmPf7+ebi3C+O6WgZqg1CH27cvhootmMHr0x/j5+TB//pU8/fQwd4ellMs5faPDGPOHMWYy0AvIBj52WVR1TNKBfK55fynZhSU8Oa4LjbVRWFXJzJmb+fbbjTzxxFBWrZrEaafFuTskpWqEsy/zBmG9hHsx0BH4FhjgwrjqjK0puVz05p8Ul5bz4kU96NRc295TluXL95KYmM0553Tg2mt7MWJEG2Jj9R6lqlucvSe1FvgeeM4Y86sL46lz3vltB3nFpcy89VTaRga5OxzlAbKzi/jXv+bzyitLad++MWef3R4fH9EEpeokZ5NUa2NMuUsjqWP+2JbG07M2sHZPNuf2jNYEpTDGMGPGem677Uf278/lxhsTeOqpYfrUnqrTjpqkROS/xpi7gC9F5LAXaPXLvMcuv7iUe2esZubqfbRoFMDjZ3dmfG9t9FPB8uX7uPDCGfToEcU331xMnz6VWxBTqu6p7krqM/t//SLvSZCVX8Itn67g961p3D48nklD2uCvn+uu04qLy/jjj0SGDo0lIaE5M2dewqhRbbWtPaVs1X2Zd4n9Z0djzCGJyn5JV7/c66SU7EIufmsxuzPyeea8blx4SsvqR1JebdGiXUyaNJMtWzLYtm0yMTGhnHlmO3eHpZRHcfZ07Zoqul17MgPxZttSczn/jT/Yn13IJ9f30wRVx6Wl5XPNNd8yZMh7FBSU8s03FxETE+rusJTySNXdk7oI67HzOBH5yqFXMJBZ9VjK0ZIdGVz/wTL8fIVPru9Hj5b6hFZdlpdXTNeur5OWls999w3kkUeGEBjo5+6wlPJY1d2TWoL1DakWWF/YrZADrHRVUN7AGMNXK/bwwFdraBEewPtX99Ev6NZhSUnZtGgRQsOG9Zky5TT69m1Bly76EWqlqlPdPakdwA6sVs+Vk4pKy7jni9V8t2ovfePCefOK3oQF6qcT6qL8/BKmTFnE1Kl/MHPmpYwc2YZrr+3l7rCUqjWqq+5baIwZIiIHOPRjhYLVNmy4S6OrhfKKSpn00XJ+3ZLGXSPacePQNvqZjTpq1qwt3HzzLHbuzGTChB707Bnl7pCUqnWqq+6r+ER8hKsD8QYpOYVc895S1u/N5rnz9Qm+umzixO95++0VdOwYwYIFVzFkSKy7Q1KqVqquuq+ilYmWwF5jTLGIDAK6AR9hNTSrbA98uYZtKXlMuyqB0zvox+fqmrKyckQEHx+hf/8WxMaGcffdA/RTGkqdAGfrob7B+nR8G+ADrEZmP3FZVLXQ34mZ/LwxhVtOb6sJqg5aunQPffpMY9q0FQBcfXVPHnzwVE1QSp0gZ5NUuTGmBDgPeNEYcyugbbbYvlu1l8mfrqRRoB9XDYh1dziqBmVlFXLLLbPo23ca+/blEBnZ0N0hKeVVnP58vIhcAFwBnGN305c7gEWbU5n86UriI4OYcnFPgho4u0pVbffDD5u57rrvSUnJ45Zb+jBlyumEhOi3wJQ6mZw9ol4D3IT1qY7tIhIHfOq6sGoHYwzPzN5ITHgg3986SNvhq2N8fX2Ijg7m++8vISGhubvDUcorOZWkjDFrRWQy0FZEOgBbjTFPuTY0z7dwcyrr92Xz9LldNUHVAUVFpfznP39QVlbOo48OZfTotowc2UY/paGUCzn7Zd5TgQ+BPVjvSEWJyBXGmN9dGZwny8wv5tZPVtK6SUPO6aln0d5uwYKdTJo0k02b0rn00q4YYw4+yaeUch1nq/teAMYYY9YDiEhHrKSV4KrAPFlRaRlPztxATlEp0y/pR2B9vQ/lrVJT87j77rl88MEq4uLCmDXrUs44I97dYSlVZzh7dK1fkaAAjDEbRKROtvOzN7OAGz5czpo9WUwYEEvn5tp6tTdLTs5jxoz1PPTQqTz00KkEBOjzQkrVJGeT1AoReRPr6gngMupgA7Nr92Qx8YNl5BSW8tYVvRnZWZu58UZr1iTz3XebeOihwXTpEkli4h2Ehwe4Oyyl6iRn35OaBGwD7gXuA7YDN7gqKE+UU1jCVe8uoaTcMP2GfpqgvFBeXjH33TeXXr3e4oUXFpOSkgegCUopN6r2SkpEugJtgK+NMc+5PiTPU1ZuuPmTlWQWlPD5Df21is8Lff/9Jm65ZTa7d2dxzTU9eO65ETRurJ9WUcrdqmsF/UGsL/CuAE4RkSeMMe/WSGQeZObqvSzanMqUc7rQu1Ujd4ejTrLMzEKuvPIbmjcPZtGiCZx6ait3h6SUslV3JXUZ0M0YkyciTYBZQJ1KUsYYXvtlG/GRQVzaJ8bd4aiTpLS0nOnT13LppV0JC/Nn/vwr6dw5UtvaU8rDVHdPqsgYkwdgjEl1Yniv88e2dDYl5zBxcGt9J8ZL/PVXEgkJb3HFFV8ze/YWAHr2bKYJSikPVN2VVGsR+cr+W4A2Dr8xxpznssg8xJuLttO4YX3O6q4v7NZ2mZmFPPjgz7zxxjKaNQtmxowLGDNG33lSypNVl6TOr/T7FVcF4ok+WryLRZtTeWhMR232yAuMHfsJf/6ZxOTJfXnyydMIDtbGYJXydNV99PDnmgrE02QVlPDcjxsZ1DaCawbFuTscdZy2bEknOjqEwEA/nn12OAEBfvTq1czdYSmlnFTn7jE564M/dpJdWMoDYzrgq/eiap2iolIef3wBXbu+zrPP/gbAwIExmqCUqmVcnqREZLSIbBKRrSJy/1GGGy8iRkQ8oj3AhZtT6dEyTN+JqoXmz99Bt25v8NhjCzn33I5MmuQRRUopdRyOKUmJyDFV4ouIL/AqcAbQCbhERDpVMVwwMBn461im7yofLt7Fsl0H6BMX7u5Q1DF65pnfGDbsA8rKypkz53I+/fR8mjULdndYSqnj5OynOvoA7wChQIyIdAeusz8jfzR9sL49td2eznRgHLC+0nBPAs8Bdx9D7C5hjOGpH9bTJy6cW09v6+5wlBPKyw35+SUEBdVn7Nh25OeX8MADg7QxWKW8gLNXUi8DY4F0AGPMKuA0J8aLBhIdfifZ3Q4SkZ5AS2PMzKNNSEQmisgyEVmWmprqZNjHbvrSRApLyrmsbwzB/nqQ83SrVu1n4MB3ueEGq/h06RLJE0+cpglKKS/hbJLyMcbsqtStzInxqnriwBzsKeKD9a2qu6qbkDHmLWNMgjEmoUmTJk7M+tgZY/j4r110ahbC2fpelEfLzS3m7rt/onfvt9i2LYMzztCrXqW8kbOf6ki0q/yMfZ/pVmCzE+MlAS0dfrcA9jr8Dga6AAtEBCAK+E5EzjbGLHMytpPCGMMj365j7Z5sHj6zI3Y8ygMtWbKH8eM/JzExm+uv78UzzwzXlsqV8lLOJqkbsar8YoBkYJ7drTpLgXgRicP69PzFwKUVPY0xWUBExW8RWQDcXdMJCuD3rel8uHgXV/ZvxTUD9b0oT1TxyfaYmFBiY8OYPn08Awa0rH5EpVSt5VSSMsakYCWYY2KMKRWRW4A5gC/wrjFmnYg8ASwzxnx3rNN0lY8W7yIs0I8Hx3TUNvo8TElJGS+99Bc//bSNH3+8nKioIBYtutrdYSmlaoCzT/e9jcO9pArGmInVjWuMmYXVerpjt0eOMOxQZ+I52X7ZlMKP6/YzeVi8Nn/kYf74I5FJk2ayZk0KY8e2IyeniNBQf3eHpZSqIc5W981z+NsfOJdDn9qr1V6cu5nYxoHcfFobd4eibNnZRdxzz0+89dYKWrQI4euvL2LcuPZ6r1CpOsbZ6r7PHH+LyIfAXJdEVMN2peexKimLh8Z0pEE9vYryFPXq+bBgwS7uuqs/jz02lKCg+u4OSSnlBs5eSVUWB3jF50sf/349fr7C2O7appu7bdqUxtNP/8brr59JYKAfq1ZNwt//eIuoUsobOPWelIgcEJEM+18m1lXUg64NzfXScotYuDmVK/rF0ixUH2F2l8LCUh599Be6dXuDb7/dyJo1yQCaoJRS1V9JiXUToDvWI+QA5caYwx6iqI2+XrGHsnLDJX30MWZ3+emnbdx00w9s23aAyy7rytSpI4mKCnJ3WEopD1FtkjLGGBH52hjTuyYCqil/bkvn37M30L1lGPFNtQFSdzDG8MQTC/HxEebNu4Jhw1q7OySllIdxtj5liYj0MsascGk0NeiTJbvx9/Pltct6uTuUOqWsrJxp01YwblwHoqKC+Oyz8TRuHKhVe0qpKh31yCAi9YwxpcAg4HoR2QbkYbXJZ4wxtfIIX1pWzsJNKZzZtRnRYXovqqasXLmPSZN+YMmSPWRkFPDAA6cSHR3i7rCUUh6sutPXJUAv4JwaiKXGrN2bTXZhKUPau6ahWnWonJwiHnnkF15+eQkREYF8/PF5XHJJF3eHpZSqBapLUgJgjNlWA7HUmK9WJAHoRw1ryAMP/Mxrry3lhht68/TTw2jUSK9elVLOqS5JNRGRO4/U0xjz/EmOx+W2pebywZ+7OK9nNJHB2ryOq+zcmUlJSRnx8Y15+OHBXH55N/r1a+HusJRStUx170n5AkFYn9So6l+tUlZueHLmeurX8+GBMR3dHY5XKikp49lnf6NTp1e55ZbZAERFBWmCUkodl+qupPYZY56okUhqwId/7mTBplSeHNeZJsEN3B2O1/n9993ccMNM1q1L5ZxzOvDyy6PdHZJSqpZz6p6UN8gpLOH5uZsZ1DaCy/t5RYtOHuXrrzdw3nmfExMTyrffXszZZ7d3d0hKKS9QXZIaViNR1IB19hN91w6K05a0TxJjDHv35hAdHcKoUW2ZMuU0br+9Hw0bamOwSqmT46j3pIwxGTUViKv9uiUVH4FuLULdHYpX2LAhldNOe5/Bg9+joKCEwEA/HnposCYopdRJ5VQDs7WdMYZZa/YzoE0EjYP0XtSJKCgo4eGH59O9+xusXp3MAw8MokEDbS1CKeUadeLosn5fNjvS8pg4WNuGOxGJiVkMHfo+27cf4IorujF16kgiIxu6OyyllBerE0nqh9X78PURRnWOcncotVJJSRl+fr5ER4dw6qkxTJt2FqedFufusJRSdUCdqO6buz6Z/q0bE673S45JWVk5//vfX7Rt+z+Sk3Px8RHee+8cTVBKqRrj9Ukqq6CELSm59GutTSAdi+XL99K37zQmT/6R9u0bU1xc5u6QlFJ1kNdX963dkwVA95Zhbo6kdigvN9xxx4+88spSIiMbMn36+Vx4YWd9bF8p5RZen6RWJ1lJqlu0Jiln+PgI6ekF3HhjAlOmnE5YmLZvqJRyH6+v7lu8PZ2mIQ0IDfRzdygea/v2A5x99qesW5cCwAcfnMsrr4zRBKWUcjuvTlIp2YUs3JzKJX1i3B2KRyouLuPpp3+lc+fX+OWXnWzcmAZYV1NKKeUJvLq67+/ETAAGtY1wcySeZ9GiXUyaNJMNG9I4//yOvPjiaFq00K/kKqU8i1cnqV82peLv50OXaG0KqbIff9xKQUEpM2dewplntnN3OEopVSWvre7LKyrl0yW7GdOlGf5+vu4Ox+2MMfzf/61k3rztAPzrX4NZt+4mTVBKKY/mtUlqxe4DAAxp38TNkbjfunUpDBnyHtdc8x3vv78KgIAAPwL1YRKllIfz2uq+ZTsP4CNweodId4fiNvn5JTz55EKmTv2TkJAGvPPO2UyY0MPdYSmllNO8N0ntyqBDVAjB/nX3auGLL9bxzDO/M2FCD/7znxFERAS6OySllDomXpmkSsvKWbk7kwt6t3B3KDVuz55s1q9PZcSINlxxRXc6dmxCnz7R7g5LKaWOi8vvSYnIaBHZJCJbReT+KvrfKSLrRWS1iPwsIif8bfdft6aRX1zGKXF1p72+0tJyXnppMR06vMpVV31DcXEZPj6iCUopVau5NEmJiC/wKnAG0Am4REQ6VRpsJZBgjOkGzACeO9H5vvvbDqLDAhjRqemJTqpWWLp0D337TuP22+cwaFAMv/12DfXr6xONSqnaz9VXUn2ArcaY7caYYmA6MM5xAGPML8aYfPvnYuCE6uiy8ktYsiODEZ2a0qCe9x+oN21Ko1+/d9i3L4fPPx/PrFmX0rp1I3eHpZRSJ4Wr70lFA4kOv5OAvkcZ/lpgdlU9RGQiMBEgJubIzRx9tTKJotJyxnvx/ShjDGvXptC1a1Pat4/g3XfP5txzOxIS0sDdoSml1Enl6iupqhqBM1UOKHI5kAD8p6r+xpi3jDEJxpiEJk2qfvfJGMMnf+2me4tQr21lYuvWDEaP/phevd462NbeVVf10ASllPJKrk5SSUBLh98tgL2VBxKR4cBDwNnGmKLjndnc9clsScn1ygZli4pKmTJlEV26vMaffybywgujiI+vOw+GKKXqJldX9y0F4kUkDtgDXAxc6jiAiPQE3gRGG2NSTmRm8zYkA3Bmt2YnMhmPU1JSximnvM2aNSlccEEnXnxxNM2bB7s7LKWUcjmXJiljTKmI3ALMAXyBd40x60TkCWCZMeY7rOq9IOAL++uvu40xZx/P/P5OzGR4x6Ze8wJvdnYRISEN8PPz5dpre9KuXWPOOCPe3WEppVSNcfnLvMaYWcCsSt0ecfh7+MmYT3FpOfsyCzkltvZXgZWXG955ZwX33TeP6dPHM3JkG267rZ+7w1JKqRrnNS1O/LhuPzlFpbX+3ag1a5KZNOkH/vgjkcGDW9GypX7jSSlVd3lNkvp7dyb+fj4Mjq+9rZ4/9dQiHntsIWFh/rz33jiuvLI7dhWoUkrVSV6RpHIKS/hqZRKdm4fWyk+fG2MQESIjG3LVVd159tnhNG6sjcEqpZRXfE9qw74cMvNLmDAg1t2hHJPExCzOO+8z3n57BQDXX9+badPO1gSllFI2r0hSFR847Ne6sZsjcU5paTnPP/8nHTu+yo8/bqW4uMzdISmllEfyiuq+zck5RIX40yTY81tdWLZsL9dd9x2rViVz5pnxvPLKGGJjw9wdllJKeSSvSFI70vKIi2jo7jCckpFRQFpaPl9+eSHnnttBH4xQSqmj8IoktTMtjzO6emYrE8YYPv10LXv2ZHPPPQMZObINW7dOxt/fK1a9Ukq5VK2/J7UrPY8D+SV0iPK8ZoK2bEln5MiPuOyyr/j2202UlZUDaIJSSikn1fok9dtWqyXwgW0j3BzJP4qKSnn88QV07fo6S5bs4dVXx7Bw4QR8fWv96lZKqRpV60/ptyTnUt/Xh9YedE9q27YDTJnyK+PHd+L550fSrJnnXeUppVRtUKuTlDGGBZtS6NemsdsfQEhOzuXLLzdw002n0KlTEzZuvJk2bWp/O4JKKeVOtbr+aW9WITvT8zm9vfuaQiovN7zxxjLat3+FO+6Yw44d1jtbmqCUUurE1eorqZ1peQC0j3JPI6yrVu1n0qQfWLw4idNOi+W1184kLq6RW2JRnq2kpISkpCQKCwvdHYqqA/z9/WnRogV+frX/s0W1Okll5BUDEBFUv8bnXVBQwvDhHyIC4MnDzwAAEzVJREFUH354Lpdd1tXtVY7KcyUlJREcHExsbKyWE+VSxhjS09NJSkoiLi7O3eGcsFqdpHZn5AMQFlhzSWr+/B0MHRpLQIAfM2ZcQNeuTQkPD6ix+avaqbCwUBOUqhEiQuPGjUlNTXV3KCdFrb4n9f2qvbRp0rBGrqR27cpk3LjpDBv2AdOnrwVgyJBYTVDKaZqgVE3xprJWa5NUUWkZG/fnMLxTU5dukJKSMv7zn9/p1Ok15s3bzn/+M4ILLujksvkppZT6R61NUjmFpQBEh7n2SubCC2dw773zGD68NRs23Mzddw/Az8/XpfNUyhV8fX3p0aMHXbp04ayzziIzM/Ngv3Xr1nH66afTrl074uPjefLJJzHGHOw/e/ZsEhIS6NixIx06dODuu+92xyIc1cqVK7nuuusO6TZu3Dj69+9/SLcJEyYwY8aMQ7oFBQUd/Hvz5s2MGTOGtm3b0rFjRy688EKSk5NPKLaMjAxGjBhBfHw8I0aM4MCBA4cN88svv9CjR4+D//z9/fnmm28AOPXUUw92b968Oeeccw4AM2fO5NFHHz2h2DyeMabW/evdu7fZvD/btLpvpvlmZZI52dLT801eXrExxpj587ebr7/ecNLnoeqW9evXuzsE07Bhw4N/X3nllWbKlCnGGGPy8/NN69atzZw5c4wxxuTl5ZnRo0ebV155xRhjzJo1a8z/t3fvwVFVeQLHvz8eQiIBESSFg2NAkCQEiAIuLLAGGRHl4YBIeOgAi6j4qhGEwsIqZtQVZcQHgjI4q8mskiBohMUZHB5hROQVhggRFLPYRjSFGDMQEIaY/PaPe0k6L9JR+snvU9VVfc+9fe8vp7r7l3Pu6XM6deqkBw44n4PS0lJdsmTJeY2ttLT0Z59jzJgxmpubW7FdXFysHTp00Pj4eD106FBF+aRJk3TlypVVXnu2bk6dOqWdO3fWNWvWVOzbtGmT7tu372fFNmvWLJ0/f76qqs6fP19nz559zuOLioq0devWevLkyRr7Ro8erenp6aqqWl5ersnJybUeV9t7DsjREPgOb8gjbAdOfOEOP7+yzfmbaUJVeeONvcyc+TfuuutannpqMIMGhf/oGBNafv+/n7D/m+Pn9ZyJl7dk3ohuPh/fr18/9u7dC8Dy5cvp378/Q4YMASA6OprFixeTkpLC/fffz4IFC5g7dy7x8fEANGnShPvuu6/GOU+cOMGDDz5ITk4OIsK8efO47bbbaNGiBSdOnABg1apVrF27lrS0NCZPnsyll17Knj17SE5OJisri9zcXC65xFm6pnPnzmzdupVGjRpx7733UlBQAMALL7xA//79q1y7pKSEvXv30rNnz4qyt99+mxEjRhAbG0tmZiaPPvpovfWyfPly+vXrx4gRIyrKBg0a5HO91mX16tVs3rwZgEmTJpGSksIzzzxT5/GrVq3i5ptvJjq66gKoJSUlbNq0iddffx1w7j2lpKSwdu1axo4d+7PjDEVhm6QOuUnqfC3R8dln3zF9+ntkZ3vo27cDqam+f+CNCSdlZWVs3LiRqVOnAk5XX69evaocc9VVV3HixAmOHz9OXl4eM2fOrPe8TzzxBK1atWLfvn0AtXZpVXfw4EE2bNhA48aNKS8vJysriylTprBjxw7i4uKIjY1lwoQJPPzwwwwYMICCggJuuukmDhw4UOU8OTk5JCUlVSnLyMhg3rx5xMbGMmbMGJ+SVF5eXo26qE1JSQkDBw6sdd/y5ctJTKx63/rIkSO0b++s1NC+fXu+/fbbc54/MzOTGTNm1CjPyspi8ODBtGxZ+dvQ3r17s2XLFktSoeaLoydp2+IiWkX9/B+rpaXlcs89a4mObsrSpcOYNq0XjRpFzugYE1oa0uI5n06dOkVycjIej4devXpx4403Ak4PQl2DjxoyKGnDhg1kZmZWbLduXf8P22+//XYaN3bu8aampvL4448zZcoUMjMzSU1NrTjv/v37K15z/PhxSkpKiImpnBOzsLCQyy6rnHnmyJEj5OfnM2DAAESEJk2akJeXR1JSUq1/U0MHX8XExJCbm9ug1/iqsLCQffv2cdNNN9XYl5GRUeO+W7t27fjmm2/8EksoCNuBE4e+O0Gnti3qP/AcSkudZdt7976cceOS+PTT+7nnnt6WoExEioqKIjc3ly+//JIzZ86wZMkSALp160ZOTk6VYw8dOkSLFi2IiYmhW7du7N69u97z15XsvMuqz7hx8cWVPSH9+vUjPz+fo0eP8u677zJ69GgAysvL2bZtG7m5ueTm5vL1119XSVBn/zbvc69YsYLi4mI6duxIXFwcHo+nIoG2adOmSivv+++/p23bthV14cvfWlJSUmWQg/fDO6GeFRsbS2FhIeAkoXbt2tV57rfeeotRo0bVmC2iqKiInTt3MmzYsCrlp0+fJioqcn8KE75J6uhJOl3207r6CgtLGD/+bSZNckbOJCW1Iz3918TG/rykZ0w4aNWqFYsWLeLZZ5+ltLSUiRMn8uGHH7JhwwbAaXE99NBDzJ49G4BZs2bx1FNPcfDgQcBJGs8991yN8w4ZMoTFixdXbJ9NBLGxsRw4cKCiO68uIsKoUaOYMWMGCQkJtGnTptbz1taCSUhIID8/v2I7IyODdevW4fF48Hg87N69uyJJpaSksGLFCs6ccWasSUtLq7jvNGHCBD766CPee++9inOtW7euogvzrLMtqdoe1bv6AEaOHEl6ejoA6enp3HrrrXXWQ0ZGBuPHj69RvnLlSoYPH07z5s2rlB88eLBGV2ckCcskVVauFJ080+AkVVZWzssv7yI+fglZWQeIj29bZZitMReKa665hp49e5KZmUlUVBSrV6/mySefpGvXrnTv3p0+ffrwwAMPANCjRw9eeOEFxo8fT0JCAklJSRWtAm+PPfYYxcXFJCUl0bNnT7KzswF4+umnGT58ODfccEPFfZm6pKam8sYbb1R09QEsWrSInJwcevToQWJiIkuXLq3xuvj4eI4dO0ZJSQkej4eCggL69u1bsb9jx460bNmSHTt2MHz4cAYOHEivXr1ITk5m69atFYMYoqKiWLt2LS+99BJdunQhMTGRtLS0c7Z8fDFnzhzWr19Ply5dWL9+PXPmzAGce2ne3Xcej4evvvqK66+/vsY5MjMza01e2dnZNVpXkUTC8Us6vnuynh72X6T/53Vcf7VvM6B//nkREye+w65d3zB4cEdeeWUYXbq08XOkxjgOHDhAQkJCsMOIaM8//zwxMTE17tlEsiNHjjBhwgQ2btxYY19t7zkR2a2qvQMV3/kQli2p0+69pIYsGd+yZTNKSs7w5pujWb/+TktQxkSY6dOn06xZs2CHEVAFBQUsXLgw2GH4VViO7jtdWs4VMc2Ibdm8zmNUlaysT8nMzCMzcwyxsS345JP7bFCEMRGqefPm3HnnncEOI6D69OkT7BD8LmxbUl1j625FeTz/ZMSIDG677S0OHizi6FHnN1WWoEwwhWPXuglPkfReC8skdaasnF+2ia5RXlpaxjPPfEhi4hI2b/bw3HNDyMm520btmaBr3rw5RUVFEfXlYUKTuutJVR8FGK7CsruvrFxrnVj2xx/LWbbsHwwd2pkXXxzKFVe0CkJ0xtTUoUMHDh8+HDFr/JjQdnZl3kgQlkkK4Eq3JVVU9AMLFmxl3rwUoqObsnPnXbSppZVlTDA1bdo0IlZJNSbQ/N7dJyJDReQzEckXkTm17G8mIivc/TtEJM6X87a5+CLS0nLp2nUxCxdu44MPvnTKLUEZY0zE8GuSEpHGwBLgZiARGC8i1X+OPRUoVtXOwPNA3VMDe5nx0DqmTFlN165t2bPnHoYO7Xw+QzfGGBMC/N2Sug7IV9VDqnoGyASqzwdyK5DuPl8FDBYfZnv8/NPvePXVEWzZMoXu3WPPa9DGGGNCg7/vSf0C+Mpr+zDwb3Udo6o/isgxoA3wnfdBInI3cLe7+a8iHs6bNg2mTfNL3OGkLdXq6gJmdVHJ6qKS1UWlrsEOoKH8naRqaxFVH4PryzGo6jJgGYCI5ITb1B7+YnVRyeqiktVFJauLSiKSU/9RocXf3X2HgSu8tjsA1Rc+qThGRJoArYDv/RyXMcaYMODvJLUL6CIiHUXkImAcsKbaMWuASe7zMcAmtV88GmOMwc/dfe49pgeA94HGwGuq+omIPA7kqOoa4L+B/xGRfJwW1DgfTr3Mb0GHH6uLSlYXlawuKlldVAq7ugjLpTqMMcZcGMJy7j5jjDEXBktSxhhjQlZIJyl/TakUjnyoixkisl9E9orIRhG5MhhxBkJ9deF13BgRURGJ2OHHvtSFiIx13xufiMjyQMcYKD58Rn4pItkissf9nNwSjDj9TUReE5FvRSSvjv0iIovcetorItcGOsYGUdWQfOAMtPg/oBNwEfAxkFjtmPuApe7zccCKYMcdxLoYBES7z6dfyHXhHhcDfABsB3oHO+4gvi+6AHuA1u52u2DHHcS6WAZMd58nAp5gx+2nuvgP4Fogr479twB/xfmNal9gR7BjPtcjlFtSfptSKQzVWxeqmq2qP7ib23F+kxaJfHlfADwBLABOBzK4APOlLqYBS1S1GEBVvw1wjIHiS10o0NJ93oqav9mMCKr6Aef+remtwJ/VsR24RETaBya6hgvlJFXblEq/qOsYVf0RODulUqTxpS68TcX5TykS1VsXInINcIWqrg1kYEHgy/viauBqEdkqIttFZGjAogssX+rid8AdInIY+AvwYGBCCzkN/T4JqlBeT+q8TakUAXz+O0XkDqA3cL1fIwqec9aFiDTCmU1/cqACCiJf3hdNcLr8UnBa11tEJElV/+nn2ALNl7oYD6Sp6kIR6Yfz+8wkVS33f3ghJay+N0O5JWVTKlXypS4QkV8Bc4GRqvqvAMUWaPXVRQyQBGwWEQ9On/uaCB084etnZLWqlqrqF8BnOEkr0vhSF1OBtwBUdRvQHGfy2QuNT98noSKUk5RNqVSp3rpwu7j+iJOgIvW+A9RTF6p6TFXbqmqcqsbh3J8bqaphN7GmD3z5jLyLM6gGEWmL0/13KKBRBoYvdVEADAYQkQScJHU0oFGGhjXAb9xRfn2BY6paGOyg6hKy3X3qvymVwo6PdfEHoAWw0h07UqCqI4MWtJ/4WBcXBB/r4n1giIjsB8qAWapaFLyo/cPHupgJvCoiD+N0b02OxH9qRSQDp3u3rXv/bR7QFEBVl+Lcj7sFyAd+AKYEJ1Lf2LRIxhhjQlYod/cZY4y5wFmSMsYYE7IsSRljjAlZlqSMMcaELEtSxhhjQpYlKROyRKRMRHK9HnHnODaurlmfG3jNze5M2h+7Uwl1/QnnuFdEfuM+nywil3vt+5OIJJ7nOHeJSLIPr/mtiET/3GsbE0iWpEwoO6WqyV4PT4CuO1FVe+JMXvyHhr5YVZeq6p/dzcnA5V777lLV/eclyso4X8a3OH8LWJIyYcWSlAkrbotpi4j8w338ey3HdBORnW7ra6+IdHHL7/Aq/6OINK7nch8And3XDnbXIdrnrtfTzC1/WirX8XrWLfudiDwiImNw5lF8071mlNsC6i0i00VkgVfMk0XkpZ8Y5za8JggVkVdEJEec9aN+75Y9hJMss0Uk2y0bIiLb3HpcKSIt6rmOMQFnScqEsiivrr4st+xb4EZVvRZIBRbV8rp7gRdVNRknSRx2p8FJBfq75WXAxHquPwLYJyLNgTQgVVW748zUMl1ELgVGAd1UtQfwpPeLVXUVkIPT4klW1VNeu1cBo722U4EVPzHOoTjTH501V1V7Az2A60Wkh6ouwpmfbZCqDnKnSHoM+JVblznAjHquY0zAhey0SMbgdvdVK2sKLHbvwZThzEVX3TZgroh0AN5R1c9FZDDQC9jlThsVhZPwavOmiJwCPDjLOXQFvlDVg+7+dOB+YDHOelV/EpH3AJ+XBlHVoyJyyJ077XP3Glvd8zYkzotxpgHyXl11rIjcjfP5bo+zwN/eaq/t65Zvda9zEU69GRNSLEmZcPMwcAToidMTUGNRQ1VdLiI7gGHA+yJyF87yBOmq+qgP15joPSGtiNS6Rpk7X9x1OJOWjgMeAG5owN+yAhgLfApkqaqKkzF8jhNnBdqngSXAaBHpCDwC9FHVYhFJw5lItToB1qvq+AbEa0zAWXefCTetgEJ3DaA7cVoRVYhIJ+CQ28W1BqfbayMwRkTaucdcKiJX+njNT4E4Eensbt8J/N29h9NKVf+CMyihthF2JTjLh9TmHeDXOOscrXDLGhSnqpbidNv1dbsKWwIngWMiEgvcXEcs24H+Z/8mEYkWkdpapcYElSUpE25eBiaJyHacrr6TtRyTCuSJSC4Qj7NU9n6cL/O/icheYD1OV1i9VPU0zkzRK0VkH1AOLMX5wl/rnu/vOK286tKApWcHTlQ7bzGwH7hSVXe6ZQ2O073XtRB4RFU/BvYAnwCv4XQhnrUM+KuIZKvqUZyRhxnudbbj1JUxIcVmQTfGGBOyrCVljDEmZFmSMsYYE7IsSRljjAlZlqSMMcaELEtSxhhjQpYlKWOMMSHLkpQxxpiQ9f8SWp8wfNYi6AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(randf, y_test, X_test, \"Decision Tree (Random Forest)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.94 0.88 6729\n", + " 1 0.64 0.37 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.74 0.65 0.68 8749\n", + "weighted avg 0.79 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, randf.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[1] = ([\"Random Forest\" , \n", + " classification_report(y_test, randf.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The random forest ensemble performs much better than the decision tree alone. The accuracy and AUROC are both superior to the decision tree alone." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Gradient Boosted Trees Classifier\n", + "\n", + "#### Theory\n", + "In this part we train a gradient boosted trees classifier using xgBoost. xgBoost is short for \"Extreme Gradient Boosting\". It is a boosting ensemble method for decision trees, which means that the trees are trained consecutively, where each new tree added is trained to correct the error from the previous tree.\n", + "\n", + "xgBoost uses the gradient descent algorithm that we learnt in BT2101 at each iteration to maximise the reduction in the error term. (More details? math?)\n", + " \n", + "#### Training\n", + "For consistency our xgBoost ensemble will use n_estimators = 300 as we have done for the random forest ensemble." + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n", + " learning_rate=0.1, loss='deviance', max_depth=4,\n", + " max_features=None, max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, n_estimators=300,\n", + " n_iter_no_change=None, presort='auto',\n", + " random_state=None, subsample=1.0, tol=0.0001,\n", + " validation_fraction=0.1, verbose=0,\n", + " warm_start=False)" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn.ensemble import GradientBoostingClassifier\n", + "xgb = GradientBoostingClassifier(n_estimators=300, max_depth = 4)\n", + "xgb.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.86 0.96 0.91 13475\n", + " 1 0.79 0.46 0.58 4021\n", + "\n", + " accuracy 0.85 17496\n", + " macro avg 0.82 0.71 0.74 17496\n", + "weighted avg 0.84 0.85 0.83 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, xgb.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that the xgBoost ensemble did not fully separate the data in the training set. (The default maximum depth is 3, so that might be a factor). Evaluating on the test set," + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (Gradient Boosted Trees) identified 750\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06339390
11270750
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6339 390\n", + "1 1270 750" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, xgb.predict(X_test), \"Decision Tree (Gradient Boosted Trees)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.26114231320864134\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KJySUFDohlEDoHaSISFFBEUVEUBEFP0UQO/aOiKLXdsFrFzsKWBAUBURBQWnSeye0QAjpCSn7+2Mm4RCScICcnJxkvc/DQ6avKWfWzJ49e8QYg1JKKXWhvNwdgFJKqbJBE4pSSqlioQlFKaVUsdCEopRSqlhoQlFKKVUsNKEopZQqFmU+oYjITSLyq7vjcDcRiRCRZBHxLsFlRoqIERGfklqmK4nIRhHpeR7TndcxKCIVRORHEUkQkRnnOn1Jcvb4Kqu/RxEJEJEtIlLN3bEUNxFpJyJLnBm3RBOKiOwRkTT7wDssItNEJMiVyzTGfGGMucyVyyiN7G3dJ7fbGLPPGBNkjMl2Z1zuYie2RhcyD2NMc2PM72dZzhlJ9AKOwcFAdSDUGHP9eUyfP7aeIpJj//6SRSRGRL4RkY4XOm9njy9X/B7tJJW7Tmn51jG5OJdVhLuABcaYWDumH0Tk7Xxx/igibzh01xKRD0XkoB3rThH5SESa2MMb2cdS7rocFpEprrxAExEfe5mRuf2MMauBNBHpd7bp3XGHMsAYEwS0AdoCj7khhgvmzqvusnLFfy7K6fauB2wzxmSd64RFxHzQ/v0FAxcBW4AlItL7/MN0LztJBdnr1Q97HR36ncZF+/NO4DOH7ruAoSLSw17mTUBz4Am7Oxz4G/ADumPtjw7AX0Afh/ngsB5tgB7AaBfEfzZfYK1j0YwxJfYP2AP0ceieDMx16PYHXgX2AUeAd4AKDsMHAmuARGAncIXdvzLwIXAIOAC8AHjbw24F/rT/fgd4NV9MPwAP2H/XAmYBR4HdwD0O4z0LzAQ+t5d/ewHrVxn41J5+L/Ak4OUQx1/Af4EErB9y73zTFrUOfwGvA8ftYQ2B34A44Ji9w6vY438G5ABpQDLwMBAJGMDHHud3YII93yTgVyDMIZ5b7HWIA57Kv+/yrXcF4D/2+AnAn3a/3GWOsPfpMeAJh+k6AcuAE/Z6TwH8HIYbYCywHdht93sT2G/vg1XAxQ7jewOP28dGkj28LrDYnleKvT1usMe/Cut4OgEsBVrlO1YfAdYBGYCP4zawY19px3EEeM3uv89eVrL9rwsOx6A9TnNgvr0vjwCPF7BNnwNOApn2fEZhXQA+aW/nWKxjrbI9fu62HmXHsLiAefYEYgroPwVY6dAd7RDfVmDIOezr3OPrVmCXvR92Azfl/z3a3V2BFfa8VgBdHYb9ThHHaCHHYmHrGAOMB9YDJ+1+dYDvOPV7H+swvhenjqVjwHSgaiHLbIB1bHnn6387sA3rwuAop5/7XsI6PqWIdWkEmHz9XgPezncs/YF1DK8HrnQYVgXrfHUU69h9LHd5QGOs30WCvX5f2v2Xcvpv5Tq7fz27n2+R29/ZZFAc/zj9B1nH3gBvOgx/A5gNhGBl7B+BSQ4/4ASgr72zawPR9rDvgXeBikA1YDlwZ/4DGCu773fYqFWxTrq17HmuAp7GumpogPWDuNwe91msH/c19rgVCli/T7ESVDDWD2wbMMohjizgfsAXuMFenxAn1yELGId1YqtgH2x9sZJwuH1wvFHQts53wnFMKDvtA6uC3f2SPayZfTB1t7fFq/a6F5ZQptrT18Y6qXe148pd5vv2MlpjnZyb2tO1x7pK9rHH3Qzc5zBfg3ViC8nd3sDNQKg9zYPAYSDAHpZ7wmgCiL28UId5NXKYdzusk3JnO+YR9jbzd9h+a7ASUoX82xQrEQ63/w4CLipoOxdwDAZjJc8HgQC7u3Mh2/VZ4HOH7pHADqxjMwj4Fvgs33I/xTqGCjo+e1LwybYX1gVIRfvffuA2exu3wzrhNHdyX/vY80gEmtjT1HSY3nFbhADxwHB7umF2d+4++51CjtEizjGFrWMM1u+7jj0vb3v/Po51jDey929ve/yHsBJZbXs/fZi7rQuY90BgbSHDFtjb78N8/VcCT55lXU5LKHYsG4Bb7G4/rET4MNY5pQ/W77aRPfxL+xgJto+ZHcAIe9gMrAsmL3v9utn9fez9GFlAPKlAsyJjPp/EcL7/7B2WjHW1YYCFnLqqFqwM2NBh/C6cujJ9F3i9gHlWxzpJOd7JDAMWFXAAC9bVWw+7+/+A3+y/OwP78s37MeBjhx/3GVd9DuN623E0c+h3J/C7QxwHcbgiwUoaw51ch32FLdse5xrg33zb+mwJ5UmH4WOAefbfTwNfOQwLxLpaPiOh2AdkGtC6gGG5y6yTb52HFrIO9wHfOXQboNdZ1js+d9lYV9MDCxkvf0L5HzAh3zhbgUsctt/IAo7f3ISyGOsuIizfOKdt5wKOwWGO++ks6/YspyeUhcAYh+4mWInex2G5DYqYX08KPtlG29PWxrrQWZJv+LvAM07u69yEcgK4jnyJLd+2GA4szzd8GXDr2Y7R81jHGOwTsd3dDdiVb5yngPftv7fnHgt2d12s36hXAfMegcNdVwH70ACXFnAs3e7QPcjeZknAT3a/Rva0J+x/BlgCBNnDL8UqzXA8p8zAuov1xboIbewwbCzWcx6wks3/gNr54ioqoRzB4Q6yoH/ueIZyjTEmGGvHRwNhdv9wrBPXKhE5ISIngHl2f7B26M4C5lcPa+MdcpjuXayr/NMYa6tMx/pRA9yIVVSUO59aufOw5/M41sk+1/4i1isM64phr0O/vVg/0lwH7Bgch9dych1OW7aIVBOR6SJyQEQSsW5twzg3hx3+TsW66sWOKW95xphUrKKvgoRhXeEUtG+KXI6INBaROfbDxkTgRc5ch/zr/aCIbLZrPp3AKirMnaawY6Qg9YAH8+3vuljrXuCy8xmFdeW8RURWiMhVTi73XGLMrxZnHl8+OH+MFqY2p05c9YDO+bbLTUANnNvXGGNSsBLTaKxjeq6IRDuxPnDmb6awY/R8OG6bekBEvvV8GGs9ASKAHx2GrcfaRgXV4orHugs4jb3O92GduF/L9+wmDuvODQBjzLfGmCpYd9l+jvMxxlSxh1XEKhb82R5UC+tCM/85pbYdpzeFn48exDrnrBSR9SIyooD1yi8Y6xgplNuqDRtj/gCmYRWngHVbmIZ1a1zF/lfZnHqoth/ruUF++7GuHMIcpqtkjGleyKK/AgaLSD2su5JZDvPZ7TCPKsaYYGNMf8ewi1ilY1hXi/Uc+kVgXUHkqi0ikm/4QSfXIf+yJ9n9WhljKmEVBUkR45+LQ1hFA4BVfRWrmKkgx4B0Ct43Z/M/rGdJUfY6PM7p6wAO6yEiF2Pdpg/BKs+uglVsmDtNYcdIQfYDE/Pt70BjzFcFLTs/Y8x2Y8wwrB/uy8BMEalY1DTnEWN+Bznz+MrCunLMC+085nstsNpOBPuBP/JtlyBjzF2cw742xvxijOmLddLcglXsebb1gTN/M8XJcdvsB7YX8HsfYA+PAfrmGx5gjDl85mxZBzR0rDItIl7AB1jnt3FY54aHHKZZCFyb73xQdPDWhd0nQHcRqYK1/eoWcE45gFWcm00h5yNjzCFjzO3GmJpYdy7viUh9Cjl+7PMlWHduhXL3eyhvAH1FpI0xJgfroHs9ty63iNQWkcvtcT8EbhOR3iLiZQ+LNsYcwnpY9x8RqWQPaygilxS0QGPMv1gPqT4AfjHG5Gbc5UCiiDwiVv1/bxFp4WyVSmNVl/wGmCgiwfYOeADrziFXNeAeEfEVkeuBpli3t+e0DrZgrOLDEyJSG+vKxtERrHLT8zETGCAiXUXED6top8AD395vH2FdgdWyt1sXEfF3YjnBWGXtyfbV3F1OjJ+Ftf98RORpoJLD8A+ACSISJZZWIpKbCPNvj/eB0SLS2R63oohcKSJnXGkWRERuFpFwe/1zj6FsO7YcCt/2c4AaInKfiPjbx0pnZ5aJdTF0v4jUF6u6/YvA1+b8aoGJ/Rt6Buvh8eMO8TUWkeH2ceorIh1FpKmz+1pEqovI1XaCzcA6TguqTvyTvawbxaquegPW87s557o+52EZcNK+4w2w16WliLS3h78DvCgiEZBXInB1QTMyxuzBKkpv79D7bqxj82X73DAKeExEouzhr2KdDz6196eISCWs534FsrfzcKySjtyKJFlYd9q+ItIL6A98Y4zJxPodvygiQXayuB/7fCQiQ+zzBpwqTsu2Y43jzOP3EqzisszC4gM3JxRjzFGsh4hP2b0ewXpw9LdYRSALsMqJMcYsx3pQ+DrWVekfnMq+t2DdJm7Cuv2cicPtZAG+wnqA9aVDLNnAAKyqebuxrsY+wCpScdY4rOdAu7Bqv3yJ9QPM9Q8QZc97IjDYGJNblHSu6/Ac1gPTBGAu1sM3R5OAJ+1b9ofyT1wUY8xGe12mY92tJGFd8WQUMslDWEUCK7BqBr2Mc8fWQ1jFjklYJ/ivzzL+L1i3+9uwbt/TOb0Y4zWspP4rVqL6EOsBLFhl2Z/Y22OIMWYl1jO0KVjbewdW+b6zrgA2ivWew5tYz4XS7avIicBf9rIucpzIGJOEVZliAFZxznassnBnfIRVg28x1jGajrWfzkUtO+ZkrP3VEuhpjPnVIb7LgKFYV8CHsfZnbtJwZl97YRWpHLTHuQTr+cdp7GP/KnvcOKwip6uMMcfOcZ3OmZ2E+2NV9tmD9Zt8l1MXKK9hFbkvFJEkrJN3UReX72Kd7BHrHY4JWBVyMu3lrQfeAt4XETHW+yoXYSWEZVi/gdVYRYpjHWcsp96nOYyVtAba88zAOo4G2vG/BdxojNlmTzoG69nnbqzz5SdY51uwSmdWiEgK1rljrDFmnz3sGeBL+/gdZPe7CSvJFim3tpNyMRG5FeshXHd3x3Ku7KvhE1hFU7vdHY9SpY2IBAD/Yj3Ij3V3PMVJRNoC/3Xm3OXuIi9VSonIABEJtIstXsW6Kt3j3qiUKp3su9OmZS2ZgPWYwNkLYU0oqjADsYosDmIV0w01ejurlCqCFnkppZQqFnqHopRSqlh4XCODYWFhJjIy0t1hKKWUR1m1atUxY0z42cc8fx6XUCIjI1m5cqW7w1BKKY8iIvlbJSh2WuSllFKqWGhCUUopVSw0oSillCoWmlCUUkoVC00oSimlioUmFKWUUsXCZQlFRD4SkVgR2VDIcBGRt0Rkh4isE5F2ropFKaWU67nyDmUaVhPfhemH1UZUFHAH1seWlFJKFaOcHEN6ZkGfoyl+Lnux0Riz2P4uQGEGAp/aDQ7+LSJVRKSm/bEppZRSBTDGsCM2mT+2HQUgK8eQlZ1DZrZhw4EEKvr7kJ6ZzYo9x6no70NMfFqJxebON+Vrc/rHkWLsfmckFBG5A+suhoiIiBIJTimlSlpWdg5bDieRmJbJpkOJeIlwIvUk244ks+lQIr7ews6jKWedT3SNYGoE+bN77wmStsYTElLhrNMUB3cmlII+KVtg08fGmPeA9wA6dOigzSMrpTzSmv0niE85ycnsHDKzc9gbl0pGZjabDiWxLuYEsUmFfRQVwoP98fH24fr2dfDz8aJ/y5q0qF0ZX2/Bx8sLHy/By8s6rRpj6NDhfWK2HuP55y/lnns64/vVUJevnzsTSgxQ16G7Dta3N5RSyuNkZeew73gq62ISmLfhMDnGsD8+jaNJ6YBwLLnwZBEc4EPTmpXoEFmVRuFBXNQwlIp+PtQLDSTA1xs/b6+8ZFGUpUv307JlNYKD/fnggwGEhQVSt+65fMX8wrgzocwG7haR6VjfN07Q5ydKKU9hjGHToURmrIzhWHIGc9adfvoSgdZ1qhBS0Y+mNStRNdAPEejXoiYV/a0k4evtRXiwPxX9L+xUHBeXyqOPLuCDD/7lmWcu4dlne9K2bc0Lmuf5cFlCEZGvgJ5AmIjEYH343hfAGPMO8BPQH9gBpAK3uSoWpZQ6X8YY/th2lDnrDpGdY1i59zj+Pt7siE0+bbxWdSrTuk4VujQMpWNkCOHB/iUS26efruWhh+YTH5/G+PFdGT++q8uXWxhX1vIadpbhBhjrquUrpdS5MsaQmW04kXaS71YfYM66Q2w4mIDjh22jawSTnpnNde2sZxm3X1yfhuFBbon3kUcW8MorS+natS7vvHMlLVtWd0scuTzueyhKKXW+jDHsO55K6slsMrJy2HY4iWxjmL3mIGtjTpB68sz3NRqGV6RH43BGdIkkMqyiG6I+XVpaJikpmYSFBTJqVFuiokIYNaqdU89YXE0TilKqTFq68xixiRkcTkxn1d54jqecZNXe+ELHD/D1YlyvRnkPwStX8KVfyxoEB/iWYNRFmzdvB2PH/kSbNjWYNWsITZqE0aRJmLvDyqMJRSnl8bYeTuKr5fvYcjiRhLQsNh9KPGMcEbg4KozKFXy5qlVN/H288fYSGoRXJDjAl8oVSk/iyO/gwSTuu28eM2ZsokmTUO6+u6O7QyqQJhSllMdKSM1k9toDPPXDxrx+wQE+dGsUSlS1YIZ1iqBSBR8qV/Al0M8zT3cLF+7i2mu/5uTJbCZMuJTx47vif4G1wlyldEallFIOsrJzOJ5ykq1Hkli4OZb1BxLOKL66t3cU9/WJQsT9zxKKQ2ZmNr6+3rRuXYP+/aN44YVeNGoU4u6wiqQJRSlV6mRl5/DblliW7oxjzrpDBb4U2K9FDSoF+NK6bhX6NqteItV0S0JiYgZPPfUb//xzgL/+GklYWCDTpw92d1hO0YSilHKrtJPZbDqUwOJtxziUkMbKvfHsytdeVeu6VejSIJT6YYG0rluFxtWCS0WtpuJkjGHmzE3ce+88Dh9OZsyYjmRkZBMY6DmfrdKEopQqUXHJGcTEp7F0ZxxLdx5jyfZjpw2vEuhL/5Y16BgZwmXNa1CrckCZKcYqzNGjKYwY8T0//7yDtm1r8MMPQ+nYsba7wzpnmlCUUi4TE5/Kz+sPs2pvPCv2HCcu5eQZ44QH+3NjpwgujgqjXUTVMnfn4YxKlfw5diyVN964nLFjO+Hj4zl3JY40oSilLtjOo8ks2xlHZnYOJ7NymLZ0D+mZ2cSnZuaN4+fjxcVRYUSGVqRdvSqEBfnTpUEoPt6eefK8UIsX72XixCXMmjWEoCA//v77do9PpppQlFLn7FhyBt+tPsCyXXFsPpTIoYT0M8YJ8PXi1q6RdIisyuXNa+BbThNHfseOpTJ+/HymTVtDZGQV9uw5QYsW1Tw+mYAmFKVUIRLTM9l0MJF/dh3neEoGWw4n4efjxY7Y5NMSSICvFz0ah3PLRfVoXrsSFXy98fX2uuAWdMsaYwwff7yG8ePnk5iYwWOPdefJJ3sQGFh6X6g8V7rHlVIALNsZx0/rDxETn8qWw0ln3HV4ewl1q1agflhFmtasxDVta3N58+r4+3i7KWLP8/nn62jWLJx33rmS5s2ruTucYqcJRalyIjvHkHIyi9jEDFbsOU5CWmbey4HzNx05Y/w+TavRu2l1omsE07pOlTJRJFPSUlMzefHFJYwe3YE6dSoxa9YQKlcOKLPbUhOKUmXYhgMJ/LT+EPM2HGbXscK/Rd42ogota1fmtm71qV8KWtQtC376aTtjx/7Enj0nqF07mLvu6kjVqiXzbXd30YSiVBmQnpnNvA2H2X0shX92x7H1cNJpNawAekVXIyIkkKjqQYRW9KNjZAghFf3K/DseJS0mJpH77pvHrFmbado0jD/+uJUePeq5O6wSoQlFKQ+VnpnN9/8e4LO/97Lx4KnWdQP9vPH38aJz/RBa1K7Mla1q0kaLrErMxImLmTt3Oy++2IsHH+yKn1/5ecYkxvFTZB6gQ4cOZuXKle4OQym3MMawet8J7v96DfuOp+b1jwwNZEjHugxqW4calQPcGGH5tHz5ASpU8KFly+rExaWSkJBBgwZV3R3WaURklTGmgyuXoXcoSpVyuQ/T35i/nV82HubAiTQAmteqRL8WNRjRNbJUfQSqPElISOfxxxfyv/+t5KqrGjN79jBCQwMJDQ10d2huoQlFqVLmSGI6nyzdw8LNscQmpZ/xLGTspQ1pVacKlzev4aYIlTGGr7/eyP33/0JsbArjxnViwoRe7g7L7TShKFUKGGP4ZuV+Hvt2PTl2KbQIRIQE0iEyhLYRVWgQVpE+TauX26ZKSpPPP1/HLbd8T4cOtZgzZxjt29dyd0ilgiYUpdzEGMPWI0n8sOYg0/7aQ1pmNgAd6lXltm71uax5dW2upBTJyMhi1654mjYNZ8iQ5mRl5XDLLa3x1n2URxOKUiXoUEIaz/+4iZ83HD5jWPdGYbw7vL02WVIKLVq0m7vumktqaibbt4/D39+H225r6+6wSh09cpVyoazsHFbujefDP3fz25ZYsnNO1ars2SScRuFBXNw4nB5RYfo+SCkUG5vCQw/9ymefraNBg6q8996AUvs999JAt4xSxSz3eciLP20hIe3UA3URGNSuNle1qkmv6OpujFA5Y8eO43Tq9D7JySd54omLeeKJi6lQQWvTFUUTilIXKD0zmx2xyfy54xjTl+9jT9yp90OiawRzfYe6XNI4nEbVgtwYpXJWYmIGlSr507BhVUaNasvIkW1p2jTc3WF5BE0oSp2n9TEJvDxvC0t3HsOhJIuqgb4M6VCXcb2jCNLiEY+RknKS55//g/ffX826dXdRp04lXnnlMneH5VH0aFfqHGw/ksQX/+xjXcwJVu87AUAFX28euaIJjaoF07F+VW3O3QP9+ONW7r77Z/btS2DUqLZl6hslJUkTilJnkZKRxed/72XSz1tO6z+4fR1u7RpJi9qV3RSZulBZWTkMGTKD777bQvPm4SxZchvdu0e4OyyPpQlFqUKs3hfPi3M3s9L+ZogI9G9Rk5s6R9A+Uu9EPJkxBhHBx8eLmjWDeOml3tx/f5dy1ZCjK2hCUcpBemY236zczw9rDuZ9fOriqDCubFmTq9vUItBPfzKe7u+/Yxg79ifef38A7drVZOrUK90dUpmhvw6lsBpgfG/xLl6ed6pYq07VCrw1rC3tIkpXq7Hq/MTHp/H44wt5991V1KoVTHx8mrtDKnNcmlBE5ArgTcAb+MAY81K+4RHAJ0AVe5xHjTE/uTImpfKbvfYg93z1b173fX2iGNm9PpW0Bd8y4+uvN3DPPfM4diyV++67iOee60lwsL+7wypzXJZQRMQbmAr0BWKAFSIy2xizyWG0J4FvjDH/E5FmwE9ApKtiUipXXHIGT3y3gXkbTzWBck2bWjwzoDlVK/q5MTLlClu2HCMysgrz5t1E27Y13R1OmeXKO5ROwA5jzC4AEZkODAQcE4oBKtl/VwYOujAeVc4ZY1i2K47xM9blfVME4Lp2dbimbS0ujtKX18qK9PQsXn75T9q1q8mAAU14/PGLefLJHtqQo4u5MqHUBvY7dMcAnfON8yzwq4iMAyoCfQqakYjcAdwBEBGhVfqU8w4npPPmwm1sOpjI2piE04Z9cEsHejetpm1olTELFuxizJi5bN9+nAcf7MKAAU3w9dXaWyXBlQmloF9p/u8NDwOmGWP+IyJdgM9EpIUxJue0iYx5D3gPrE8AuyRaVWYcSkjjqe83snpfPMdTTgIQ7O9DvdBA2tStwv19GhMZVtHNUariduRIMg888CtffrmeRo1C+PXXm+nbt6G7wypXXJlQYoC6Dt11OLNIaxRwBYAxZpmIBABhQKwL41JlTEpGFrNWx7B42zHWxZwgNikjb1i3RqHc2rU+fZtpY4xl3fz5u5g5cxNPP92Dxx67mIAArcRa0ly5xVcAUSJSHzgADAVuzDfOPqA3ME1EmgIBwFEXxqTKkIdmrOXn9YdIOZmd1y8iJJBOkSEMbFuLmzrXc2N0qiSsXXuY7duPM3hwM266qSXdutWlfn2t5u0uLksoxpgsEbkb+AWrSvBHxpiNIvI8sNIYMxt4EHhfRO7HKg671RijRVqqSKv2xvPYt+vYdiQZgN7R1ejdtDoD29TSj1OVE8nJJ3nmmUW8+eY/REZW4ZprovHx8dJk4mYu/fXZ75T8lK/f0w5/bwK6uTIGVXbMWhXD1EU72HUsBYAuDUL5+LaOBOgD13Ll+++3MG7cz8TEJHLHHe2YNKkPPj5ae6s00Ms55RG+XR3DgzPWAnBt29oMaldbq/mWQ+vXH+Haa7+mZctqfP31YLp2rXv2iVSJ0YSiSi1jDBPmbOajv3YDEOTvw5xx3bWGVjmTmZnNkiX76NWrPi1bVmfu3Bvp27eBVgUuhTShqFInKzuHB75Zy+y1pyoFDulQh0f7NSVE32IvV5Yu3c/o0XPYuPEoW7feTaNGIfTvH+XusFQhNKGoUuWblft5eOa6vO4nr2zKbd3q4+2lLx+WJ8ePp/Hoowt4//3V1K1biW+/HUKjRiHuDkudhSYU5XbGGL5ZuZ/X5m/jSKL1DsnANrV47urmVAnUO5LyJj09izZt3uHgwSQefLALzz7bk6AgPQ48gSYU5RZZ2TnExKex9UgSr/6yle2xVhXgiJBAXhncis4NQt0coSppMTGJ1KlTiYAAHyZMuJQ2bWrQunUNd4elzoEmFFWijDGs2BPPkHeXnda/Y2RVPry1ozYZXw6lpWUyadKfvPzyX8yceT0DBjRhxIg27g5LnQenEoqI+AERxpgdLo5HlVGpJ7OYsTKGZ2ZvzOvXum4VxvZsSOPqwVpzq5z69dedjBkzl50747n55lZ06lTb3SGpC3DWhCIiVwKvAX5AfRFpAzxjjLnW1cEpz7fzaDLP/7iJP7adalHnogYh3NmjIZdGV3NjZMrdxo37iSlTVhAVFcKCBcPp3buBu0NSF8iZO5TnsZqdXwRgjFkjIo1cGpUqEybO3cT7S3bndV/WrDovX9dKP2BVjmVnWw2Je3t7cdFFdQgLC+SRR7prQ45lhDN7MdMYcyLfNyO0vS1VqMT0TMZ+sZol248BcEePBjzev6mbo1Lutnr1IUaPnsPw4a0YN64zN93Uyt0hqWLmTELZLCJDAC+75eB7gb9dG5byRBlZ2by1cDtTF+3M6/fdmAQMw0MAACAASURBVK60jdAG+8qzpKQMnn56EW+9tZzw8EBq1gx2d0jKRZxJKHcDTwM5wLdYrQc/5sqglGfJys5h8i9beW/xrrx+t3evz0OXN9GGG8u5X3/dyciRP3DwYBKjR3fgxRd7U6VKgLvDUi7iTEK53BjzCPBIbg8RGYSVXFQ5lZNjWHcggU+W7mH22oNk5xgq+HpzV8+GDOsUQXiwv7tDVKWAn5831apVZNasIXTuXMfd4SgXk7N9fkREVhtj2uXrt8oY096lkRWiQ4cOZuXKle5YdLlmjGHN/hN8vWI/K/YcZ+fRlLxhvt7CkA51GX95E32zvZzLzMzmtdeWkZiYwcSJvQHr4sNLm85xO/u83cGVyyj0DkVELsf6PG9tEXnNYVAlrOIvVU488d16vvhnX1533ZAKNKkeTNdGoVzfvi7NalVyY3SqtPjzz315DTlef32zvESiyaT8KKrIKxbYAKQDGx36JwGPujIoVTqkZGRx7/R/WbA5FoA7ezSgT7PqdIzURvrUKXFxqTzyyAI+/PBfIiIq8+OPw7jqqsbuDku5QaEJxRjzL/CviHxhjEkvwZiUG207ksSdn60i9WRWXkONXgJ/jL+UuiGBbo5OlUZxcWlMn76Bhx/uytNPX0JFfc+o3HLmoXxtEZkINAPyqmcYY/QSpAyJS85g5CcrWbv/RF6/nk3Cubx5DYZ0qKvNx6vTbN58lG++2cgzz/SkceNQ9u27n5CQCu4OS7mZMwllGvAC8CrQD7gNfYZSJhhj+GXjYe6dvoaMLGuX+ngJM+/qSpu6VdwcnSqNUlMzmThxMa+8spSgID9GjWpHnTqVNJkowLmEEmiM+UVEXjXG7ASeFJElrg5MudbkeVv4bNlekjKyALi6dS16N63GwDbaOJ8q2Lx5OxgzZi67d59gxIjWvPJKX8LDtVFPdYozCSVDrHZXdorIaOAAoK36eaBlO+OYuSqG37fGEpdykkA/bx7rF83VbWpRs7JeYarCJSefZPjw7wgNrcCiRSPo2TPS3SGpUsiZhHI/EATcA0wEKgMjXRmUKl4nUk9yx2erWL77OAAhFf0Y2rEuzwxoTgU/fZNdFSw7O4evvtrAsGEtCAryY8GC4URHh+Hvrw05qoKd9cgwxvxj/5kEDAcQEX3l1QNkZucwfcV+Xpy7mbTMbIIDfPh+bDcahge5OzRVyq1adZA775zDqlWHqFDBh+uua6ZfT1RnVWRCEZGOQG3gT2PMMRFpjtUESy9Ak0opNnfdIcZ+uTqv++ErmnBnj4ZaW0sVKSEhnaeeWsTUqSuoVq0i06dfx6BB2lK0ck5Rb8pPAq4D1mI9iP8Oq6Xhl4HRJROeOh/rYxLyksk9vaO4s0cDKmoxhXLCddd9w2+/7Wbs2I688EIvKlfWhhyV84o6ywwEWhtj0kQkBDhod28tmdDUuTLGMH7mOmauigHg8f7R3NGjoZujUqXdrl3xhIcHEhzsz8SJvfDyEjp21Np+6twVlVDSjTFpAMaY4yKyRZNJ6WOMYd6Gw6yJOcG7f5xqPv7NoW20CrAq0smT2bz66lImTFjMPfd04uWX+2qLwOqCFJVQGohIbhP1AkQ6dGOMGeTSyFSREtIyeX3+NqYt3ZPXL6SiHx0jqzLx2paEBWnz8apwixfvZfToOWzefIzBg5txzz2d3R2SKgOKSijX5eue4spAlPP+3hXH0Pesj2YG+nkzrFME/3dxA2poebdywuuvL+OBB34lMrIKc+feSP/+Ue4OSZURRTUOubAkA1FFy84xvD5/Gws2H2HL4SQA2kVUYdZdXbHeO1WqcDk5hpSUkwQH+3PllY05ejSVJ5/sQWCgr7tDU2WIVv0p5XJyDO8s3snkeaceX7WuW4X/XN+KRtX029zq7DZujGX06Ll5X05s3DiUF1/s7e6wVBnk0oQiIlcAbwLewAfGmJcKGGcI8CxggLXGmBtdGZMn2X88lYsnL8rrvrNHA+7r01jfbldOSU3NZMKEP3j11WVUruzPyJFtMMboHa1yGacTioj4G2MyzmF8b2Aq0BeIAVaIyGxjzCaHcaKAx4Buxph4EdE2woC9cSk8+M1aVu6NB6BeaCDz7u2hiUQ57d9/DzFo0Dfs2XOC225rw+TJfQkL0+/ZKNc6a0IRkU7Ah1hteEWISGvgdmPMuLNM2gnYYYzZZc9nOta7LZscxvk/YKoxJh7AGBN77qtQtqzeF8+gt5cC0L5eVW7pUk+r/yqn5d6BRERUJiKiMp98cg09etRzd1iqnHDmDuUt4CrgewBjzFoRudSJ6WoD+x26Y4D8dRMbA4jIX1jFYs8aY+Y5Me8y6ZsV+3l41joARnWvz1NXNXNzRMpTZGXlMGXKcmbP3sr8+cMJDQ3kjz9udXdYqpzxcmYcY8zefP2ynZiuoIJak6/bB4gCegLDgA9E5IwvO4nIHSKyUkRWHj161IlFe56lO4/lJZPnrm6uyUQ5bfnyA3Tq9D733/8LAQE+JCY6XTKtVLFyJqHst4u9jIh4i8h9wDYnposB6jp018FqviX/OD8YYzKNMbuBrVgJ5jTGmPeMMR2MMR3Cw8OdWLRnmbfhEDe+bzXq/O7w9ozoGunegJRHSE4+ydixc7noog84ciSFGTOuZ+7cG6laVb9to9zDmSKvu7CKvSKAI8ACu9/ZrACiRKQ+1ke5hgL5a3B9j3VnMk1EwrCKwHZRTiSmZ3Lj+3+z4UAiAPf2juLy5tpEuHKOr68Xv/++l3HjOjFhQi8qVdLWEZR7OZNQsowxQ891xsaYLBG5G/gF6/nIR8aYjSLyPLDSGDPbHnaZiGzCKkYbb4yJO9dleaLYpHQ6TTz17ugPY7vRWr/jrs5ix47jPP/8H0yd2p/gYH9WrbqDgAB9nUyVDmJM/sca+UYQ2YlVFPU18K0xJqkkAitMhw4dzMqVK90ZwgVzrMk1tGNdJg1qqe8GqCJlZGQxefJfTJy4BD8/b+bOvZGLL9baW8p5IrLKGNPBlcs46zMUY0xD4AWgPbBeRL4XkXO+Y1GWHbFJeclkUNvavHRdK00mqkiLFu2mdet3ePrp37nmmmi2bLlbk4kqlZy6VzbGLAWWisizwBvAF8B0F8ZVJh1JTKfPa4sBmH9/D6Kqa9MpqmjGGCZOXEJmZg7z5t3E5Zc3cndIShXKmRcbg7BeSBwKNAV+ALq6OK4yJSfHMHN1DA/PtKoFV/Tz1mSiCpWTY/jww9VccUUj6tatzGefXUuVKgFUqKANOarSzZk7lA3Aj8BkY8wSF8dT5uyNS2HwO8s4mpRBtWB/xl/ehMHt9SNGqmDr1h1h9Og5LFsWw9NP9+C55y6lZk29+FCewZmE0sAYk+PySMqgJ79fz+d/7wMgukYw34/tRoCvtselzpScfJLnnvud11//m6pVKzBt2kBuuaW1u8NS6pwUmlBE5D/GmAeBWSJyRlUw/WJj4dJOZnPv9H/5ddMRACYPbsWQDnXPMpUqz5599nf+859l3H57W156qQ+hodqQo/I8Rd2hfG3/r19qPAcvzNnEB3/uzuv+9f4eNNbnJaoA+/cnkJKSSXR0GI8+2p1rromme/cId4el1HkrtNqwMWa5/WdTY8xCx39YD+dVPv9duD0vmUwe3IrtE/tpMlFnyMrK4bXXltG06VTuvHMOAGFhgZpMlMdz5hnKSM68SxlVQL9yKzE9kyteX8zBhHQAFjzQQ7+mqAr0998xjB49h7Vrj3DllVFMmdLf3SEpVWyKeoZyA1ZV4foi8q3DoGDghKsD8ySXv76YQwnphAX58cXtF2kyUQWaO3cbAwZ8Ra1awXz77RCuuSZaX2pVZUpRdyjLgTisVoKnOvRPAv51ZVCeZNLPmzmUkM5lzarz7vD2eoJQpzHGcPBgErVrV6JPnwY8//yl3HtvZ4KDtSFHVfYUmlDs5uR3Y7UurAoQm5TOu39YjSO/MbSNJhN1mm3b4hgzZi7btsWxadNYgoL8ePLJHu4OSymXKarI6w9jzCUiEs/pH8YSwBhjQlweXSn25/Zj3PqxVW/htSGtCfTTFl+VJT09i5de+pNJk/6kQgUfJk3qTYUKenyosq+oozz3M79hJRGIJ8nJMdz8ofVBrJHd6jOonb75riyHDyfTo8fHbN9+nGHDWvDaa5dTo0aQu8NSqkQUVeSV+3Z8XeCgMeakiHQHWgGfA4klEF+pk5GVTbeXFgHwaL9oRl/S0M0RqdIgMzMbX19vqlevSI8e9Zg6tT99++qxocoXZz4B/D3W538bAp9ivYPypUujKsWGf7CcY8kZdGsUyh0XN3B3OMrNcnIM77yzkoYN3yImJhER4YMPrtZkosolZxJKjjEmExgEvGGMGQfUdm1YpdOO2CSW7zlOzybhfHH7RXh56UP48mzt2sN07fohd901l6ioUDIzs90dklJu5dQngEXkemA4cI3dr1y2o/34txsAeGZAczdHotzJGMP48fN5442/CQmpwGefXctNN+lXN5Vy9k35MVjN1+8SkfrAV64Nq/T5YMkulu85zjVtalE/rKK7w1FuJCLEx6cxapTVkGPVqhXcHZJSpcJZvykPICI+QO6n4nYYY7JcGlUR3PFN+Y/+3M3zczZRvZI/P47rTrXggBJdvnK/vXtPcO+983j66Uto164mOTlGizyVRykV35QXkYuBHcCHwEfANhHp5sqgSpPsHMPzczYBMP+BSzSZlDOZmdlMnvwXzZq9zfz5u9i69RiAJhOlCuBMkdfrQH9jzCYAEWkKfAa4NNOVBiezcuj8otVQwO3d61MpoFw+Oiq3li7dz513zmHDhlgGDmzCW2/1IyKisrvDUqrUciah+OUmEwBjzGYR8XNhTKXG83M2Ep+aSVS1IB7vry32lzcLFuwiISGd77+/gYEDo90djlKl3lmfoYjINCAD664E4CYg0BgzwrWhFayknqGkZGTR/JlfANj5Yn+8tYijzDPG8Nln6wgPD6RfvygyMrLIzMwhKKhcXD+pMq5UPEMBRgM7gYeBR4BdwJ2uDKo0eH+J1ejjfX2iNJmUA1u2HKNXr08ZMeJ7Pv54DQD+/j6aTJQ6B0UWeYlIS6Ah8J0xZnLJhOR+xhjeWLCd2lUqcG/vKHeHo1woLS2TF19cwssv/0XFin68++5V3H57O3eHpZRHKvQORUQex2p25SZgvoiMLLGo3Oz7NQcAGNqxrr6sVsb9+OM2XnhhCTfc0IItW8Zyxx3ttQaXUuepqDuUm4BWxpgUEQkHfsKqNlymHU3K4P6v1xJS0Y8xlzY6+wTK4xw+nMyaNYe54opGXH99MyIjb6dTp3LZmpBSxaqoZygZxpgUAGPM0bOMW2Z0nGhVEx7ZLVKfnZQx2dk5vP32Cpo0mcLw4d+RlpaJiGgyUaqYFHWH0sDhW/ICNHT8trwxZpBLI3ODtJPZiEDlCr7c3UufnZQlq1cfYvToOaxYcZA+fRrw9tv9qVBB3ytSqjgVlVCuy9c9xZWBlAbTlu7BGHhlcGt3h6KK0e7d8XTq9D5hYYF8+eUghg5toc/GlHKBoj6wtbAkAykNvlq+j/phFenTtJq7Q1EXyBjD+vWxtGpVnfr1q/LxxwMZMKAJVapo0zlKuUq5eC7ijG1Hkth3PJXWdSrr1auH2707nquu+oq2bd9l3bojAAwf3lqTiVIu5tKEIiJXiMhWEdkhIo8WMd5gETEi4rb2wSbP20KArxcPX6FNbHiqkyezeemlP2ne/G3++GMPr77al2bNwt0dllLlhjNteQEgIv7GmIxzGN8bmAr0BWKAFSIy27FdMHu8YOAe4B9n513cfl5/iAWbY3nossbUqqLftvBE2dk5dO36IatWHWLQoKa88cbl1K2rDTkqVZKcab6+k4isB7bb3a1F5L9OzLsT1rdTdhljTgLTgYEFjDcBmAykOx928Xp53hYARnSNdFcI6jwlJlrXON7eXowc2ZYffxzGrFlDNJko5QbOFHm9BVwFxAEYY9YClzoxXW1gv0N3DPm+RS8ibYG6xpg5Rc1IRO4QkZUisvLo0aNOLNp58Skn2ROXytWtaxGszdN7DGMM06atoUGDN/nhB+uCYMyYjlx1VWM3R6ZU+eVMQvEyxuzN1y/biekKerKd17SxiHhhfWvlwbPNyBjznjGmgzGmQ3h48ZaJf/63tWpDO9Ut1vkq19m06Sg9e37Cbbf9QHR0GA0bhrg7JKUUzj1D2S8inQBjPxcZB2xzYroYwPEsXQc46NAdDLQAfrdrVdUAZovI1caYEvnGb1Z2Dl+vtG6iujQILYlFqgs0efJfPPHEb1Sq5M8HHwzgttvaattbSpUSziSUu7CKvSKAI8ACu9/ZrACiRKQ+cAAYCtyYO9AYkwCE5XaLyO/AQyWVTAAm/byFmPg07unVSKsKl3LGGESEGjWCuOmmlrzySl/Cwyu6OyyllIOzJhRjTCxWMjgnxpgsEbkb+AXwBj4yxmwUkeeBlcaY2eccbTH78M/dANx5SUM3R6IKc/BgEvfeO4+LL47gnns6c8strbnlFm3JQKnS6KwJRUTex+HZRy5jzB1nm9YY8xNWK8WO/Z4uZNyeZ5tfcZq/yXrhrUF4RSr6O117WpWQ3IYcn3jiNzIzc+jatY67Q1JKnYUzZ9IFDn8HANdyeu0tj5ORlc3/fWqVrH1zZxc3R6PyW7PmMLffPptVqw5x2WUNefvt/vrgXSkP4EyR19eO3SLyGTDfZRGVgM+WWTW7rmxZk7AgfzdHo/JLSEjn4MEkvv56MNdf30yfbynlIc6nrKc+UK+4AylJn9lVhV+/oY2bI1FgPXCfMWMT27fH8cQTPbjkkkh27bqXgAAtilTKkzjzpny8iBy3/53Aujt53PWhuca2I0nsjUulXUQV/Hy0bUx327nzOP37f8kNN8zkhx+2kplpveKkyUQpz1Pkr1assobWWNV+AXKMMWc8oPckuQ/jR2vNLrfKyMji1VeX8sILS/D19eLNN69gzJiO+GiSV8pjFZlQjDFGRL4zxrQvqYBc7Vc7ofRorK3QutP+/YlMmLCYAQOa8MYbl1O7diV3h6SUukDOXA4uF5F2Lo+kBCzaEsva/Sfo26w6Ab7e7g6n3Dl6NIUpU5YD0KhRCJs2jWXGjOs1mShVRhR6hyIiPsaYLKA78H8ishNIwWqjyxhjPC7JfPDnLgDu7a3fiy9JOTmGjz/+l4cfXkBSUgZ9+zagSZMwGjSo6u7QlFLFqKgir+VAO+CaEorFpYwx/LUjjs71Q2hRW5s2LykbNsRy111z+fPPfVx8cQTvvHMVTZqEnX1CpZTHKSqhCIAxZmcJxeJSE+duBqC2fkCrxJw8mc1ll33GyZPZfPTR1dx6axt9p0SpMqyohBIuIg8UNtAY85oL4nGJlIwsPrDb7Xr+mhZujqbs++233VxyST38/Lz55pvriY4OIyws0N1hKaVcrKiH8t5AEFYz8wX98xjf2E3Uj+vViCBtt8tlYmISue66b+jd+1M+/XQtAN27R2gyUaqcKOrsesgY83yJReJCa/efAODuXo3cHEnZlJWVw5Qpy3nqqUVkZ+cwaVJvbrqplbvDUkqVsLM+Q/F0iemZzF57kJa1K+Pvo1WFXWH48O+YPn0D/fo1YurU/tSvr7W3lCqPikoovUssCheatSqGHAM3dY5wdyhlyokT6fj4eBEU5MfYsR257rqmXHddU33orlQ5VugzFGPM8ZIMxBWycwxTftsBwNVtark5mrLBGMP06Rto2nQqTz31G2A9Jxk8WFsFVqq8K9MNJ206mEhcykmuaF6DQD99GH+hduw4zuWXf86wYbOoU6cSN9+sz0mUUqeU6bPsvI2HAHi0X7SbI/F8X365npEjf8Df34cpU/oxenQHvL3L9PWIUuoclemEsmBTLA3DKxIZVtHdoXiszMxsfH296dChFoMHN2Py5L7UquVRtcaVUiWkzF5iZmRls/VIEo2qBbk7FI8UG5vC8OHfccMNMwFo3DiUzz8fpMlEKVWoMptQYuLTAGhRS9vtOhc5OYb33ltFkyZT+PrrDTRvHk52do67w1JKeYAyW+SV+yGt6JraNLqzdu2K5+abv2XZshh69ozkf/+7kuhobchRKeWcMptQft14GIBe0dXcHInnqFzZnxMn0vnkk2sYPryVVgNWSp2TMlnklZKRxep9J+jZJBxvLz0pFmX27K0MGvQ12dk5hIYGsmHDGG65pbUmE6XUOSuTCeWTZXsA6Now1K1xlGb79iVwzTXTGThwOtu2xXHoUDIAXpqAlVLnqcwVeWVm5/DG/O0EB/hwa9f67g6n1MnKyuGNN/7mmWd+xxjDyy/34f77L8JXP4mslLpAZS6h/Lj2ICezc3ikXzR+PmXyBuyCZGfn8MEHq+nVqz7//W8/IiOruDskpVQZUebOuG8u3A7ow3hH8fFpPPLIfJKSMvD39+Gvv0Yye/ZQTSZKqWJV5hLK3rhU6oZUoL6+HY8xhi++WEd09FT+859lLFq0B4DQ0EB96K6UKnZlqshr//FUAPq1qOnmSNxv27Y4xoyZy8KFu+nUqTa//HIzbdrUcHdYSqkyrEwllNikDADaRWhRzn33zWPlyoO8/XZ/7rijvTbkqJRyuTKVUP7dFw9ASEV/N0fiHvPn7yQ6Ooy6dSvzv/9dib+/DzVqaFtmSqmS4dLLVhG5QkS2isgOEXm0gOEPiMgmEVknIgtFpN6FLO9Xu7mVVnXKV/tdhw8nc+ONs7jsss95+eW/AKhXr4omE6VUiXJZQhERb2Aq0A9oBgwTkWb5RvsX6GCMaQXMBCZfyDKX7z5OvdBAAsrJOxU5OYZ33llJdPQUZs3azDPPXMKrr17m7rCUUuWUK+9QOgE7jDG7jDEngenAQMcRjDGLjDGpduffQJ3zXdi+OGs2PaLCz3cWHmfSpCXcdddc2revxbp1o3n22Z4EBJSpUkyllAdx5dmnNrDfoTsG6FzE+KOAnwsaICJ3AHcAREREFDjx7rgUADrWDzn3SD1IUlIGx46lUr9+VUaP7kD9+lUZNqyFVgNWSrmdK+9QCjrDmQJHFLkZ6AC8UtBwY8x7xpgOxpgO4eEF34HsiLXaompQRt8/Mcbw3XebadbsbW64YSbGGEJDA7nxxpaaTJRSpYIrE0oMUNehuw5wMP9IItIHeAK42hiTcb4LO3TC+qBWWfxC4969J7j66ukMGvQNISEVeOutfppElFKljiuLvFYAUSJSHzgADAVudBxBRNoC7wJXGGNiL2Rh62ISiCyDD+SXLdtPnz6fAfDqq325996L8NE2ypRSpZDLEooxJktE7gZ+AbyBj4wxG0XkeWClMWY2VhFXEDDDvuLeZ4y5+lyXdSw5g+V7jnNr18jiWwE3S0zMoFIlf9q1q8nIkW0YP74bERHlqzq0UsqzuLRKkDHmJ+CnfP2edvi7T3Es5+sV1rP/nk08v4ZXXFwqjz66gF9/3cXGjWMICvLjv//t7+6wlFLqrMpEHdN5G6zP/Xby4Bpexhg++2wdDz74K/HxaTzwQBf0MYlSypOUiYSy/kACXRqEEujnmauTkJDONdd8ze+/76FLlzq8885VtGpV3d1hKaXUOfHMM7CDRVusZ/kNq3ledWFjDCJCpUr+hIUF8t57VzFqVDv9DK9SyiN5fHWhuesPATCmZyM3R3JufvllB+3avUdMTCIiwowZ1/N//9dek4lSymN5fEJJSMvEx0uoVaWCu0NxyqFDSQwdOpMrrviC1NRMYmNT3B2SUkoVC48v8pq/6QhtPeT7J1OnLufxx38jIyOL557rySOPdMPf3+N3gVJKAR6eUDYfSgQgyENOyqtWHaJz59pMndqfqKhQd4ejlFLFyjPOxIWY9tceAF4Z3Nq9gRQiMTGDp59exPDhrWjfvhZvv30l/v7e2myKUqpM8uiEciQpHYAalQPcHMnpjDHMmrWZe++dx6FDSUREVKZ9+1ratLxSqkzz6DPcupgEIkIC3R3GaXbvjufuu3/mp5+206ZNDb79dgidO5/3Z16UUspjeHRCSU7PonYpq931xRfrWbx4L6+/fjl3391JG3JUSpUbHptQEtIyOZmdQ++m1dwdCkuW7CUjI5s+fRowfnxXbr21DXXqVHJ3WEopVaI89vL5y3/2AVAv1H1FXseOpTJy5A/06DGN55//AwB/fx9NJkqpcslj71D2HbdeCLy0ScnfoRhjmDZtDePHzychIYNHHunGU0/1KPE4lGfIzMwkJiaG9PR0d4eiyoGAgADq1KmDr69viS/bYxPK0STr445VAv1KfNk//bSdkSNn061bXd555ypatHB/sZsqvWJiYggODiYyMlKrjCuXMsYQFxdHTEwM9evXL/Hle2yR18ItsVSuUHIZODU1k7/+sorZ+veP4ocfhrJ48W2aTNRZpaenExoaqslEuZyIEBoa6ra7YY9MKHuOpWAMJZZQfv55Oy1avE2/fl9w4kQ6IsLVVzfRhhyV0zSZqJLizmPNIxPKz/YHtV6+rpVLl3PgQCLXXz+D/v2/xN/fhx9/HEaVKqXrJUqllCotPDKhbDuSBED7elVdtozY2BSaNXubOXO28cILl7J27WguuSTSZctTylW8vb1p06YNLVq0YMCAAZw4cSJv2MaNG+nVqxeNGzcmKiqKCRMmYIzJG/7zzz/ToUMHmjZtSnR0NA899JA7VqFI//77L7fffvtp/QYOHEiXLl1O63frrbcyc+bM0/oFBQXl/b1t2zb69+9Po0aNaNq0KUOGDOHIkSMXFNvx48fp27cvUVFR9O3bl/j4+DPGWbRoEW3atMn7FxAQwPfffw/AwoULadeuHW3atKF79+7s2LEDgClTpvDxxx9fUGwuYYzxqH/t27c3V721xLR9/lfjCjExCXl/v/nm32bHjjiXLEeVH5s2bXLr8itWrJj39y233GJeeOEFY4wxqamppkGDBuaXX34xxhiTkpJirrjiTQ1HVAAAEXRJREFUCjNlyhRjjDHr1683DRo0MJs3bzbGGJOZmWmmTp1arLFlZmZe8DwGDx5s1qxZk9cdHx9v6tSpY6Kjo82uXbvy+o8YMcLMmDHjtGlzt01aWppp1KiRmT17dt6w3377zaxfv/6CYhs/fryZNGmSMcaYSZMmmYcffrjI8ePi4kzVqlVNSkqKMcaYqKiovONn6tSpZsSIEcYYa1+1adOm0PkUdMwBK42Lz88eWcsrx+EKqrgkJKTz5JO/8e67q/j779tp164m99zTudiXo8q3537cyKaDicU6z2a1KvHMgOZOjdulSxfWrVsHwJdffkm3bt247LLLAAgMDGTKlCn07NmTsWPHMnnyZJ544gmio6MB8PHxYcyYMWfMMzk5mXHjxrFy5UpEhGeeeYbrrruOoKAgkpOTAZg5cyZz5sxh2rRp3HrrrYSEhPDvv//Spk0bvvvuO9asWUOVKtZnKBo1asRff/2Fl5cXo0ePZt8+qzLMG2+8Qbdu3U5bdlJSEuvWraN161MNxM6aNYsBAwZQvXp1pk+fzmOPPXbW7fLll1/SpUsXBgwYkNfv0ksvdWqbFuWHH37g999/B2DEiBH07NmTl19+udDxZ86cSb9+/QgMtN6vExESE63jJSEhgVq1agHWvoqMjGT58uV06tTpguMsLh6ZUDYeTOTy5sXzzXVjDDNmbOK+++Zx+HAyd9/diYYNXVeUppS7ZGdns3DhQkaNGgVYxV3t27c/bZyGDRuSnJxMYmIiGzZs4MEHHzzrfCdMmEDlypVZv349QIHFOvlt27aNBQsW4O3tTU5ODt999x233XYb//zzD5GR/9/e3UfXfOcJHH9/FiUIq1Q2RitUVR4ammYGHZqWKR3TZTkq9Th6dNtiaiqlu3vsObW10+kMw66lq9b2eDiD0KKKsnR1aEmJFQ8VYzykRjeVUEuqNCSf/eP3y81NciM33Ifc+LzOuefc3/P3fs6993N/39/vfr5xxMTEMHLkSKZMmULv3r05c+YMAwYMIDc3t8J+srOzSUpKqjBv5cqVvP7668TExDBs2DC/EsqRI0eqxMKXoqIi+vTp43PZihUrSEhIqDDv3LlzxMbGAhAbG0tBQcFN979q1SoyMjI804sXL2bgwIFERUXRokULsrKyPMtSU1PZtWuXJZTbcb3EOTsRbv9OBlVl6NDVrF9/jJSUWDZsGEFqarvb3q8x1fH3TCKQrl69Svfu3cnLy+ORRx7hySefBJz3f3V3BNXmTqHt27ezatUqz3SrVjX/IHvmmWdo0KABAOnp6bzxxhs899xzrFq1ivT0dM9+jx496tnm8uXLFBUVER0d7ZmXn5/PPffc45k+d+4cJ06coHfv3ogIDRs25MiRIyQlJfl8TbW9Iyo6OpqcnJxabeOv/Px8Dh8+zIABAzzz5s6dy+bNm+nRowezZs0iIyODxYsXA9C2bVuOHTsWlLbcqoi7KH+9pBTgtmp4Xb9eAjhvpt6972XevKfYu/d5SyamXoqKiiInJ4cvv/yS4uJiFixYAEBiYiLZ2dkV1j116hTNmzcnOjqaxMRE9u/fX+P+q0tM3vMq/y+iWbNmnue9evXixIkTFBYWsn79eoYOHQpAaWkpe/bsIScnh5ycHL766qsKyaTstXnvOzMzk4sXL9KxY0fi4uLIy8vzJLvWrVtXOHv65ptvaNOmjScW/rzWoqKiChfQvR/eya9MTEwM+fn5gJMw2rat/ntr9erVDBkyxPMP98LCQg4ePEiPHk7Xe3p6Ort37/asf+3aNaKi6lZx3IhLKN/fcJLBrZat/+STPJKTF/LBB05mf/XVR3n55R40aBBxoTCmVlq2bMm8efOYPXs2169fZ9SoUXz66ads374dcM5kJk+ezGuvvQbAtGnTePPNNzl+/DjgfMHPmTOnyn779+/P/PnzPdNlX9oxMTHk5uZ6urSqIyIMGTKEjIwM4uPjad26tc/9+joziI+P99z5BE5315YtW8jLyyMvL4/9+/d7Esrjjz9OZmYmxcXFACxZssRznWTkyJHs3r2bTZs2efa1ZcsWTzdembIzFF+Pyt1dAIMGDWLp0qUALF26lMGDB1cbh5UrVzJixAjPdKtWrbh06ZIn/tu2bSM+Pt6z/Pjx41W6+8Iu2Ff9A/34q/sTtMPfbdQzF65Ue4eDLwUF3+rYsesUZmjHjv+iH398quaNjAmAunSXl6rq008/rcuWLVNV1UOHDmlaWpp26dJF77//fp0xY4aWlpZ61v3www81JSVFu3btqvHx8Tp16tQq+y8qKtKxY8dqYmKiJicn6/vvv6+qqmvWrNFOnTppWlqaTpo0yXOHkq+7rfbt26eALlmyxDOvsLBQhw8frg899JDGx8friy++6PP1JSUl6eXLl/X06dParl27Cu1XVX344Yc1KytLVVVnzJihSUlJ2q1bNx06dKgWFBR41svNzdUBAwZo586dNT4+XtPT0/Xrr7++aWxrcv78ee3bt6927txZ+/btqxcuXPC83vHjx3vWK2t7SUlJhe3Xrl2rSUlJmpycrGlpaXry5MkKr6uwsNDnccN1l5doEO6YCqa7O8RrixGzyXvrZ35vs3LlYSZN2sy33xYzbdqjTJ/+GE2bhr5wmrkz5ebmVvhlaQJr7ty5REdHV/kvSn124MAB5syZw/Lly30u9/WeE5H9qpoazHZFXD9PSakS1ahBrba5caOUpKS25OS8xK9+1c+SiTH1yIQJE2jcuHG4mxFS58+fZ+bMmeFuRhURd5fXtRsl9O168wvyV64UM3PmTu67ryUTJ/6Q0aOTGT062eopGVMPNWnShDFjxoS7GSFVdqdeXRNxZygAbVtU/2tk48bjJCa+zW9+8xnHj18AnIt+lkxMOEVa17KJXOF8r0XcGQpAQmzVERHPnr3M5MkfsW7dMRIS7mHnznH06dMhDK0zpqImTZpw4cIFK2Fvgk7VGQ+lSZPwFLGNyITSoXWzKvNOnbrI1q0n+fWv+5GR0Yu77qrddRZjgqV9+/acPXuWwsLCcDfF3AHKRmwMh4hMKE3dZLF371fs2fNnfvnLnjz2WAfOnHmF1mEcY94YXxo1ahSW0fOMCbWgXkMRkadE5I8ickJE/t7H8sYikuku/1xE4vzZ73dXipk4cRM9ey5mzpwsrlxx/qhkycQYY8InaAlFRBoAC4CfAgnACBGp/FfS8cBFVe0MzAWqL8PpZVDf5bzzzn4mT+7B4cMTaNYs9OPKG2OMqSiYXV4/Ak6o6ikAEVkFDAa8C94MBma4z98D5ouI6M1uU1C4NzaazRtGkJISG5SGG2OMqb1gJpQfAH/2mj4LVB5gxLOOqt4QkUtAa+C890oi8gLwgjv5ffa5F474UWn6TtCGSrG6g1ksylksylksyj0Y7AMEM6H4uj+y8pmHP+ugqouARQAikh3s8gGRwmJRzmJRzmJRzmJRTkSya17r9gTzovxZ4F6v6fbA/1a3jog0BFoC3wSxTcYYY4IkmAllH/CAiHQUkbuAZ4ENldbZAPzcfT4M+O+bXj8xxhhTZwWty8u9JvILYCvQAHhXVb8QkTdwyihvAP4TWC4iJ3DOTJ71Y9eLgtXmCGSxKGexKGexKGexKBf0WERc+XpjjDF1U0QWhzTGGFP3WEIxxhgTEHU2oQSrbEsk8iMWGSJyVEQOicjHIlJvyyzXFAuv9YaJiIpIvb1l1J9YiMhw973xhYisCHUbQ8WPz8h9IrJDRA64n5OB4WhnsInIuyJSICJHqlkuIjLPjdMhEUkJaAOCPcbwrTxwLuKfBDoBdwEHgYRK60wEFrrPnwUyw93uMMbiCaCp+3zCnRwLd71oYCeQBaSGu91hfF88ABwAWrnTbcPd7jDGYhEwwX2eAOSFu91BisVjQApwpJrlA4GPcP4D2BP4PJDHr6tnKJ6yLapaDJSVbfE2GFjqPn8P6Cf1c7CJGmOhqjtU9Tt3MgvnPz/1kT/vC4CZwG+Ba6FsXIj5E4u/BRao6kUAVS0IcRtDxZ9YKFA2kFJLqv4nrl5Q1Z3c/L98g4Fl6sgC/lJEAlbDqq4mFF9lW35Q3TqqegMoK9tS3/gTC2/jcX6B1Ec1xkJEHgbuVdWNoWxYGPjzvugCdBGRz0QkS0SeClnrQsufWMwARovIWWAz8HJomlbn1Pb7pFbq6ngoASvbUg/4/TpFZDSQCqQFtUXhc9NYiMhf4FStHheqBoWRP++LhjjdXo/jnLXuEpEkVf2/ILct1PyJxQhgiar+TkR64fz/LUlVS4PfvDolqN+bdfUMxcq2lPMnFojIT4DpwCBV/T5EbQu1mmIRDSQBn4hIHk4f8YZ6emHe38/IB6p6XVVPA3/ESTD1jT+xGA+sBlDVPUATnMKRdxq/vk9uVV1NKFa2pVyNsXC7ed7BSSb1tZ8caoiFql5S1TaqGqeqcTjXkwapatCL4oWBP5+R9Tg3bCAibXC6wE6FtJWh4U8szgD9AEQkHieh3IljMm8Axrp3e/UELqlqfqB2Xie7vDR4ZVsijp+xmAU0B9a49yWcUdVBYWt0kPgZizuCn7HYCvQXkaNACTBNVS+Er9XB4WcsXgX+Q0Sm4HTxjKuPP0BFZCVOF2cb93rR60AjAFVdiHP9aCBwAvgOeC6gx6+HMTXGGBMGdbXLyxhjTISxhGKMMSYgLKEYY4wJCEsoxhhjAsISijHGmICwhGLqHBEpEZEcr0fcTdaNq66yai2P+YlbrfagW6rkwVvYx0siMtZ9Pk5E2nktWywiCQFu5z4R6e7HNq+ISNPbPbYxNbGEYuqiq6ra3euRF6LjjlLVbjhFR2fVdmNVXaiqy9zJcUA7r2XPq+rRgLSyvJ1v4187XwEsoZigs4RiIoJ7JrJLRP7HfTzqY51EEdnrntUcEpEH3Pmjvea/IyINajjcTqCzu20/dwyNw+5YE43d+W9J+Rg0s915M0RkqogMw6mp9nv3mFHumUWqiEwQkd96tXmciPzbLbZzD16F/UTk30UkW5yxT/7JnTcZJ7HtEJEd7rz+IrLHjeMaEWlew3GM8YslFFMXRXl1d61z5xUAT6pqCpAOzPOx3UvAv6pqd5wv9LNumY104Mfu/BJgVA3H/2vgsIg0AZYA6ar6EE5liQkicjcwBEhU1WTgn703VtX3gGycM4nuqnrVa/F7wFCv6XQg8xbb+RROeZUy01U1FUgG0kQkWVXn4dRqekJVn3BLsPwj8BM3ltlARg3HMcYvdbL0irnjXXW/VL01Aua71wxKcOpSVbYHmC4i7YG1qvonEekHPALsc8vSROEkJ19+LyJXgTyc8uYPAqdV9bi7fCkwCZiPM9bKYhHZBPhdKl9VC0XklFtH6U/uMT5z91ubdjbDKTPiPeLecBF5AedzHYszkNShStv2dOd/5h7nLpy4GXPbLKGYSDEFOAd0wzmzrjJ4lqquEJHPgZ8BW0XkeZxy3UtV9R/8OMYo70KSIuJzfB23dtSPcIoNPgv8Auhbi9eSCQwHjgHrVFXF+Xb3u504oxK+BSwAhopIR2Aq8ENVvSgiS3AKIFYmwDZVHVGL9hrjF+vyMpGiJZDvjl8xBufXeQUi0gk45XbzbMDp+vkYGCYibd117haRDn4e8xgQJyKd3ekxwB/caw4tVXUzzgVvX3daFeGU0/dlLfA3OGN0ZLrzatVOVb2O03XV0+0uawFcAS6JSAzw02rakgX8uOw1iUhTEfF1tmdMrVlCMZHibeDnIpKF0911xcc66cAREckBuuIMdXoU54v3v0TkELANpzuoRqp6Daca6xoROQyUAgtxvpw3uvv7A87ZU2VLgIVlF+Ur7fcicBTooKp73Xm1bqd7beZ3wFRVPYgzfvwXwLs43WhlFgEficgOVS3EuQNtpXucLJxYGXPbrNqwMcaYgLAzFGOMMQFhCcUYY0xAWEIxxhgTEJZQjDHGBIQlFGOMMQFhCcUYY0xAWEIxxhgTEP8PCzsv1jsat28AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(xgb, y_test, X_test, \"Decision Tree (XGBoost)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.94 0.88 6729\n", + " 1 0.66 0.37 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.66 0.68 8749\n", + "weighted avg 0.79 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, xgb.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[2] = ([\"Gradient Boosted\" , \n", + " classification_report(y_test, xgb.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From the accuracy and AUROC, we observe that the XGBoost performs similarly to the random forest ensemble. It has a slight bump in AUROC at 0.76, but the accuracy is the same." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ModelRecall-1AUROC
0Decision Tree0.4099010.606997
1Random Forest0.3707920.768455
2Gradient Boosted0.3712870.778407
\n", + "
" + ], + "text/plain": [ + " Model Recall-1 AUROC\n", + "0 Decision Tree 0.409901 0.606997\n", + "1 Random Forest 0.370792 0.768455\n", + "2 Gradient Boosted 0.371287 0.778407" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Logistic Regression\n", + "\n", + "#### Theory\n", + "Logistic regression is a regression technnique used to predict binary target variables. It works on the same principles as a linear regression model. \n", + "\n", + "Our binary target (default vs non-default) can be expressed in terms of odds of defaulting, which is the ratio of the probability of default and probability of non-default. \n", + "\n", + "In the logistic regression model, we log the odds (log-odds) and equate it to a weighted sum of regressors.\n", + "\n", + "![image.png](https://wikimedia.org/api/rest_v1/media/math/render/svg/4a5e86f014eb1f0744e280eb0d68485cb8c0a6c3)\n", + "\n", + "We then find weights for the regressors that best fits the data. Since the binary target (default or not) follows a bernoulli distribution, each data point has the following probability distribution function:\n", + "\n", + "![image.png](https://wikimedia.org/api/rest_v1/media/math/render/svg/614e0c64d59f0ff2e926deafcb2de6e502394fac)\n", + "\n", + "We would like to update p for each data point such that the log product (joint probability) of the above function for all data points is maximised. In other words, we are maximising the log-likelihood function.\n", + "\n", + "The logistic regression equation produces a \"squashed\" curve like the one below. We then pick a cutoff value for the y axis to classify a data point as 0 (non-default) or 1 (default).\n", + "\n", + "![image.png](https://upload.wikimedia.org/wikipedia/commons/thumb/8/88/Logistic-curve.svg/1280px-Logistic-curve.svg.png)\n", + "\n", + "\n", + "#### Training\n", + "We will adopt a top-down approach for training our logistic regression model, i.e. include all regressors first and then remove the most insignificant ones at each iteration to achieve the best fit." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": {}, + "outputs": [], + "source": [ + "import statsmodels.api as sm" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization terminated successfully.\n", + " Current function value: 0.445436\n", + " Iterations 7\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
Logit Regression Results
Dep. Variable: Y No. Observations: 17496
Model: Logit Df Residuals: 17452
Method: MLE Df Model: 43
Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1737
Time: 19:35:44 Log-Likelihood: -7793.4
converged: True LL-Null: -9431.5
Covariance Type: nonrobust LLR p-value: 0.000
\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
coef std err z P>|z| [0.025 0.975]
LIMIT_BAL -0.8354 0.114 -7.302 0.000 -1.060 -0.611
SEX -0.1196 0.041 -2.920 0.004 -0.200 -0.039
AGE 0.0615 0.100 0.614 0.539 -0.135 0.258
PAY_0 0.6546 0.059 11.081 0.000 0.539 0.770
PAY_2 -0.5622 0.098 -5.715 0.000 -0.755 -0.369
PAY_3 -0.1059 0.113 -0.941 0.347 -0.327 0.115
PAY_4 -0.2041 0.159 -1.287 0.198 -0.515 0.107
PAY_5 -0.0925 0.178 -0.521 0.603 -0.441 0.256
PAY_6 0.3289 0.147 2.244 0.025 0.042 0.616
BILL_AMT1 -1.0372 0.528 -1.964 0.049 -2.072 -0.002
BILL_AMT2 0.6000 0.765 0.785 0.433 -0.899 2.099
BILL_AMT3 1.5425 0.732 2.107 0.035 0.107 2.978
BILL_AMT4 0.2846 0.715 0.398 0.690 -1.116 1.686
BILL_AMT5 -1.9241 0.913 -2.107 0.035 -3.714 -0.134
BILL_AMT6 1.3621 0.839 1.623 0.105 -0.283 3.007
PAY_AMT1 -1.1622 0.306 -3.794 0.000 -1.763 -0.562
PAY_AMT2 -2.0159 0.390 -5.172 0.000 -2.780 -1.252
PAY_AMT3 -0.5874 0.310 -1.898 0.058 -1.194 0.019
PAY_AMT4 -0.0638 0.281 -0.227 0.820 -0.614 0.487
PAY_AMT5 -1.0410 0.291 -3.576 0.000 -1.612 -0.470
PAY_AMT6 -0.3858 0.256 -1.507 0.132 -0.887 0.116
GRAD 1.3893 0.230 6.030 0.000 0.938 1.841
UNI 1.3690 0.229 5.975 0.000 0.920 1.818
HS 1.3654 0.233 5.863 0.000 0.909 1.822
MARRIED 0.0176 0.158 0.111 0.911 -0.292 0.328
SINGLE -0.0968 0.159 -0.609 0.543 -0.408 0.215
PAY_0_No_Transactions 0.0316 0.124 0.255 0.799 -0.211 0.274
PAY_0_Pay_Duly 0.2120 0.122 1.734 0.083 -0.028 0.452
PAY_0_Revolving_Credit -0.6866 0.137 -5.022 0.000 -0.955 -0.419
PAY_2_No_Transactions -1.4519 0.236 -6.151 0.000 -1.914 -0.989
PAY_2_Pay_Duly -1.3960 0.225 -6.201 0.000 -1.837 -0.955
PAY_2_Revolving_Credit -0.9936 0.230 -4.318 0.000 -1.445 -0.543
PAY_3_No_Transactions -0.5672 0.275 -2.059 0.039 -1.107 -0.027
PAY_3_Pay_Duly -0.5717 0.250 -2.284 0.022 -1.062 -0.081
PAY_3_Revolving_Credit -0.5379 0.239 -2.249 0.025 -1.007 -0.069
PAY_4_No_Transactions -0.7044 0.356 -1.980 0.048 -1.402 -0.007
PAY_4_Pay_Duly -0.8759 0.338 -2.593 0.010 -1.538 -0.214
PAY_4_Revolving_Credit -0.7888 0.328 -2.404 0.016 -1.432 -0.146
PAY_5_No_Transactions -0.4367 0.393 -1.110 0.267 -1.208 0.334
PAY_5_Pay_Duly -0.4715 0.378 -1.247 0.212 -1.213 0.270
PAY_5_Revolving_Credit -0.3428 0.369 -0.930 0.352 -1.065 0.379
PAY_6_No_Transactions 0.4826 0.323 1.492 0.136 -0.151 1.117
PAY_6_Pay_Duly 0.4427 0.316 1.401 0.161 -0.177 1.062
PAY_6_Revolving_Credit 0.2143 0.308 0.695 0.487 -0.390 0.818
" + ], + "text/plain": [ + "\n", + "\"\"\"\n", + " Logit Regression Results \n", + "==============================================================================\n", + "Dep. Variable: Y No. Observations: 17496\n", + "Model: Logit Df Residuals: 17452\n", + "Method: MLE Df Model: 43\n", + "Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1737\n", + "Time: 19:35:44 Log-Likelihood: -7793.4\n", + "converged: True LL-Null: -9431.5\n", + "Covariance Type: nonrobust LLR p-value: 0.000\n", + "==========================================================================================\n", + " coef std err z P>|z| [0.025 0.975]\n", + "------------------------------------------------------------------------------------------\n", + "LIMIT_BAL -0.8354 0.114 -7.302 0.000 -1.060 -0.611\n", + "SEX -0.1196 0.041 -2.920 0.004 -0.200 -0.039\n", + "AGE 0.0615 0.100 0.614 0.539 -0.135 0.258\n", + "PAY_0 0.6546 0.059 11.081 0.000 0.539 0.770\n", + "PAY_2 -0.5622 0.098 -5.715 0.000 -0.755 -0.369\n", + "PAY_3 -0.1059 0.113 -0.941 0.347 -0.327 0.115\n", + "PAY_4 -0.2041 0.159 -1.287 0.198 -0.515 0.107\n", + "PAY_5 -0.0925 0.178 -0.521 0.603 -0.441 0.256\n", + "PAY_6 0.3289 0.147 2.244 0.025 0.042 0.616\n", + "BILL_AMT1 -1.0372 0.528 -1.964 0.049 -2.072 -0.002\n", + "BILL_AMT2 0.6000 0.765 0.785 0.433 -0.899 2.099\n", + "BILL_AMT3 1.5425 0.732 2.107 0.035 0.107 2.978\n", + "BILL_AMT4 0.2846 0.715 0.398 0.690 -1.116 1.686\n", + "BILL_AMT5 -1.9241 0.913 -2.107 0.035 -3.714 -0.134\n", + "BILL_AMT6 1.3621 0.839 1.623 0.105 -0.283 3.007\n", + "PAY_AMT1 -1.1622 0.306 -3.794 0.000 -1.763 -0.562\n", + "PAY_AMT2 -2.0159 0.390 -5.172 0.000 -2.780 -1.252\n", + "PAY_AMT3 -0.5874 0.310 -1.898 0.058 -1.194 0.019\n", + "PAY_AMT4 -0.0638 0.281 -0.227 0.820 -0.614 0.487\n", + "PAY_AMT5 -1.0410 0.291 -3.576 0.000 -1.612 -0.470\n", + "PAY_AMT6 -0.3858 0.256 -1.507 0.132 -0.887 0.116\n", + "GRAD 1.3893 0.230 6.030 0.000 0.938 1.841\n", + "UNI 1.3690 0.229 5.975 0.000 0.920 1.818\n", + "HS 1.3654 0.233 5.863 0.000 0.909 1.822\n", + "MARRIED 0.0176 0.158 0.111 0.911 -0.292 0.328\n", + "SINGLE -0.0968 0.159 -0.609 0.543 -0.408 0.215\n", + "PAY_0_No_Transactions 0.0316 0.124 0.255 0.799 -0.211 0.274\n", + "PAY_0_Pay_Duly 0.2120 0.122 1.734 0.083 -0.028 0.452\n", + "PAY_0_Revolving_Credit -0.6866 0.137 -5.022 0.000 -0.955 -0.419\n", + "PAY_2_No_Transactions -1.4519 0.236 -6.151 0.000 -1.914 -0.989\n", + "PAY_2_Pay_Duly -1.3960 0.225 -6.201 0.000 -1.837 -0.955\n", + "PAY_2_Revolving_Credit -0.9936 0.230 -4.318 0.000 -1.445 -0.543\n", + "PAY_3_No_Transactions -0.5672 0.275 -2.059 0.039 -1.107 -0.027\n", + "PAY_3_Pay_Duly -0.5717 0.250 -2.284 0.022 -1.062 -0.081\n", + "PAY_3_Revolving_Credit -0.5379 0.239 -2.249 0.025 -1.007 -0.069\n", + "PAY_4_No_Transactions -0.7044 0.356 -1.980 0.048 -1.402 -0.007\n", + "PAY_4_Pay_Duly -0.8759 0.338 -2.593 0.010 -1.538 -0.214\n", + "PAY_4_Revolving_Credit -0.7888 0.328 -2.404 0.016 -1.432 -0.146\n", + "PAY_5_No_Transactions -0.4367 0.393 -1.110 0.267 -1.208 0.334\n", + "PAY_5_Pay_Duly -0.4715 0.378 -1.247 0.212 -1.213 0.270\n", + "PAY_5_Revolving_Credit -0.3428 0.369 -0.930 0.352 -1.065 0.379\n", + "PAY_6_No_Transactions 0.4826 0.323 1.492 0.136 -0.151 1.117\n", + "PAY_6_Pay_Duly 0.4427 0.316 1.401 0.161 -0.177 1.062\n", + "PAY_6_Revolving_Credit 0.2143 0.308 0.695 0.487 -0.390 0.818\n", + "==========================================================================================\n", + "\"\"\"" + ] + }, + "execution_count": 57, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "glm = sm.Logit(y_train,X_train).fit()\n", + "glm.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 13475\n", + " 1 0.67 0.35 0.46 4021\n", + "\n", + " accuracy 0.81 17496\n", + " macro avg 0.75 0.65 0.67 17496\n", + "weighted avg 0.79 0.81 0.79 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train,list(glm.predict(X_train)>0.5)))" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.69 0.35 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm.predict(X_test)>0.5)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The logisitc model with all features performs quite well on both the train and test set with an accuracy of about 0.8. We will now try removing all the insignificant features to see how that affects the model performance." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization terminated successfully.\n", + " Current function value: 0.446914\n", + " Iterations 7\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
Logit Regression Results
Dep. Variable: Y No. Observations: 17496
Model: Logit Df Residuals: 17472
Method: MLE Df Model: 23
Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1709
Time: 19:35:44 Log-Likelihood: -7819.2
converged: True LL-Null: -9431.5
Covariance Type: nonrobust LLR p-value: 0.000
\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
coef std err z P>|z| [0.025 0.975]
LIMIT_BAL -0.7989 0.111 -7.183 0.000 -1.017 -0.581
SEX -0.1204 0.040 -2.986 0.003 -0.199 -0.041
PAY_0 0.5793 0.037 15.471 0.000 0.506 0.653
PAY_2 -0.6376 0.077 -8.284 0.000 -0.788 -0.487
PAY_6 0.2012 0.030 6.656 0.000 0.142 0.260
BILL_AMT1 -0.8123 0.359 -2.261 0.024 -1.516 -0.108
BILL_AMT3 2.2505 0.535 4.206 0.000 1.202 3.299
BILL_AMT5 -0.9943 0.358 -2.779 0.005 -1.696 -0.293
PAY_AMT1 -1.3333 0.287 -4.642 0.000 -1.896 -0.770
PAY_AMT2 -2.3238 0.373 -6.232 0.000 -3.055 -1.593
PAY_AMT5 -0.9218 0.252 -3.658 0.000 -1.416 -0.428
GRAD 1.2429 0.179 6.961 0.000 0.893 1.593
UNI 1.2478 0.177 7.052 0.000 0.901 1.595
HS 1.2675 0.181 7.014 0.000 0.913 1.622
PAY_0_Revolving_Credit -0.8716 0.092 -9.431 0.000 -1.053 -0.690
PAY_2_No_Transactions -1.6757 0.200 -8.387 0.000 -2.067 -1.284
PAY_2_Pay_Duly -1.4967 0.179 -8.379 0.000 -1.847 -1.147
PAY_2_Revolving_Credit -1.0909 0.182 -6.007 0.000 -1.447 -0.735
PAY_3_No_Transactions -0.3503 0.159 -2.199 0.028 -0.663 -0.038
PAY_3_Pay_Duly -0.3319 0.111 -3.003 0.003 -0.548 -0.115
PAY_3_Revolving_Credit -0.3494 0.081 -4.332 0.000 -0.507 -0.191
PAY_4_No_Transactions -0.2852 0.130 -2.193 0.028 -0.540 -0.030
PAY_4_Pay_Duly -0.5573 0.103 -5.433 0.000 -0.758 -0.356
PAY_4_Revolving_Credit -0.4761 0.075 -6.332 0.000 -0.623 -0.329
" + ], + "text/plain": [ + "\n", + "\"\"\"\n", + " Logit Regression Results \n", + "==============================================================================\n", + "Dep. Variable: Y No. Observations: 17496\n", + "Model: Logit Df Residuals: 17472\n", + "Method: MLE Df Model: 23\n", + "Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1709\n", + "Time: 19:35:44 Log-Likelihood: -7819.2\n", + "converged: True LL-Null: -9431.5\n", + "Covariance Type: nonrobust LLR p-value: 0.000\n", + "==========================================================================================\n", + " coef std err z P>|z| [0.025 0.975]\n", + "------------------------------------------------------------------------------------------\n", + "LIMIT_BAL -0.7989 0.111 -7.183 0.000 -1.017 -0.581\n", + "SEX -0.1204 0.040 -2.986 0.003 -0.199 -0.041\n", + "PAY_0 0.5793 0.037 15.471 0.000 0.506 0.653\n", + "PAY_2 -0.6376 0.077 -8.284 0.000 -0.788 -0.487\n", + "PAY_6 0.2012 0.030 6.656 0.000 0.142 0.260\n", + "BILL_AMT1 -0.8123 0.359 -2.261 0.024 -1.516 -0.108\n", + "BILL_AMT3 2.2505 0.535 4.206 0.000 1.202 3.299\n", + "BILL_AMT5 -0.9943 0.358 -2.779 0.005 -1.696 -0.293\n", + "PAY_AMT1 -1.3333 0.287 -4.642 0.000 -1.896 -0.770\n", + "PAY_AMT2 -2.3238 0.373 -6.232 0.000 -3.055 -1.593\n", + "PAY_AMT5 -0.9218 0.252 -3.658 0.000 -1.416 -0.428\n", + "GRAD 1.2429 0.179 6.961 0.000 0.893 1.593\n", + "UNI 1.2478 0.177 7.052 0.000 0.901 1.595\n", + "HS 1.2675 0.181 7.014 0.000 0.913 1.622\n", + "PAY_0_Revolving_Credit -0.8716 0.092 -9.431 0.000 -1.053 -0.690\n", + "PAY_2_No_Transactions -1.6757 0.200 -8.387 0.000 -2.067 -1.284\n", + "PAY_2_Pay_Duly -1.4967 0.179 -8.379 0.000 -1.847 -1.147\n", + "PAY_2_Revolving_Credit -1.0909 0.182 -6.007 0.000 -1.447 -0.735\n", + "PAY_3_No_Transactions -0.3503 0.159 -2.199 0.028 -0.663 -0.038\n", + "PAY_3_Pay_Duly -0.3319 0.111 -3.003 0.003 -0.548 -0.115\n", + "PAY_3_Revolving_Credit -0.3494 0.081 -4.332 0.000 -0.507 -0.191\n", + "PAY_4_No_Transactions -0.2852 0.130 -2.193 0.028 -0.540 -0.030\n", + "PAY_4_Pay_Duly -0.5573 0.103 -5.433 0.000 -0.758 -0.356\n", + "PAY_4_Revolving_Credit -0.4761 0.075 -6.332 0.000 -0.623 -0.329\n", + "==========================================================================================\n", + "\"\"\"" + ] + }, + "execution_count": 60, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#remove all insignificant attributes\n", + "sig = glm.pvalues[glm.pvalues < 0.05]\n", + "glm_2 = sm.Logit(y_train,X_train[sig.index]).fit()\n", + "glm_2.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 13475\n", + " 1 0.66 0.36 0.46 4021\n", + "\n", + " accuracy 0.81 17496\n", + " macro avg 0.75 0.65 0.67 17496\n", + "weighted avg 0.79 0.81 0.79 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train,list(glm_2.predict(X_train[sig.index])>0.5)))" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.35 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.5)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since there is not much change to the model performance on both the train and test set when we reduce the features, we will use the reduced logistic regression model from this point onwards (Principle of Parsimony). \n", + "\n", + "We now Calculate the AUROC for the train set." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.22554888390620112\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e8lvUEgCTUJSWihtwhSRRARFEGaomIBfopgR8SOioi9vWDB3kFABAEREBBEWhCklxBaSGjpve19/5hNXEIIC2Sz2c35PA8P2ZnZmTNl58zcO3Ov0lojhBBCXEg1ewcghBCicpNEIYQQokySKIQQQpRJEoUQQogySaIQQghRJkkUQgghyiSJogxKqTuUUsvtHYe9KaVClVIZSimXClxmmFJKK6VcK2qZtqSU2q2U6nUZ37usY1Ap5aWU+lUplaqUmnup368oSqnflFJ3X8b3eiil9tsipsrMHr9FAOUo71EopY4AdYBCIANYBjyotc6wZ1zOyLytx2qtV9oxhjDgMOCmtS6wVxzmWDTQRGsdY+PlhFFO66yUGgU8BHQtj+1nTnLfaa2Dr3Rel7n8ctkHSqkXgWeBXKAA2ANM1FpvuOIgnZij3VEM1Fr7Au2A9sDTdo7nstjzKtlZrtAvRRXd3g2BA5eTJKrAMTLHfB4JBFYDNrnjcqrtqLV2iH/AEeA6i89vAEssPnsAbwHHgFPAx4CXxfhBwHYgDTgE3GAeXgP4HEgATgCvAC7mcfcAf5n//hh4q0RMC4HHzX/XB+YDZzCuCh+2mO5FYB7wnXn5Y0tZvxrAN+bvHwWeA6pZxLEe+D8gFdgH9Cnx3bLWYT3wLpBkHtcIWAUkAmeB7wF/8/TfAiYgG+PO7UkgDNCAq3maNcBU83zTgeVAoEU8d5nXIRF4vuS+K7HeXsDb5ulTgb/Mw4qWebd5n54FnrX4XidgA5BiXu8ZgLvFeA1MAA4Ch83D3geOm/fBVqCHxfQuwDPmYyPdPD4EWGueV6Z5e9xqnv4mjOMpBfgbaFPiWJ0M7MC4cnW13Abm2KPNcZwC3jEPP2ZeVob5XxcsjkHzNC2BFeZ9eQp4ppRt+hKQB+Sb5zMG46LwOfN2Po1xrNUwT1+0rceYY1hbyjx7AXEX2IdlHbsu5v17FuN38SDnH0tjzX83Bv40HwdnMU7olLYPSsZj3lc/m2NIBGZcINYXMe6Mij63MM87yGJYWfu2A7DNfIzMBeYAr1huI/O+Pwl8a8X8JmP8ZtOB/Zh/12UcI0X7qmj71QcWmY+HGOB/Jdb1J/O+SQd2A1GXdf6t6BP+5f7j3B9aMLATeN9i/HvmDVYL8AN+BaZbbPRUoC/GD6YBEGke9wvwCeAD1AY2A/dbnGSLEkVPjJNMUXFdTYyTaX3zPLcCLwDuQAQQC/Sz2GH5wGDztF6lrN83GInHz3wwHADGWMRRADwGuGH8UFKBWlauQwFGMYQrxkm4sXlbeABBGD/E90rb1hc4ONdgnFCbmue3BnjN4oeXAXQ3b4u3zOt+oUQx0/z9Bhgnla7muIqW+al5GW0xTrrNzd/rCFxtXqcwYC/wqMV8NcYJtVbR9gbuBALM35mI8WP2NI+bhHFMNQOUeXkBFvNqXOJkcRrobI75bvM287DYftsxTl5eJbcpRoIbZf7bF7i6tO1cyjHoh5EUJwKe5s+drTwhjsY4kUSYl/kz/53Iipb7DcYxVNrx2YsLJ4qyjt1xGMU7wRi/mZVcOFH8iFEsVM28ft1L7M/GpcVj3gf/YlwM+ZT87oW2C8bx+RpGUiqK54L71jz9UeARjN/hEIyEbJkoCoDXzdN7XWR+zTDOKfUt9kOjSzlGMBLrh+Z1boeRKPtYrGsOMMC87OnAxss6/9o7AVgdqLFxMzAyowb+4L+rYIVxtdHIYvou/Hcl+QnwbinzrINx8rG88xgJrC7lR6owrrZ6mj//D1hl/rszcKzEvJ8GvrTYYeddpVlM62KOo4XFsPuBNRZxxGNOUuZhm4FRVq7DsQst2zzNYGBbiW19sUTxnMX48cAy898vAD9ajPPG+DGdlygwTgjZQNtSxhUtM7jEOt92gXV4FFhg8VkDvS+y3slFy8a4mht0gelKnqQ+AqaWmGY/cI3F9htdyvFblCjWYlz1B5aY5pztXMoxONJyP11k3V7k3ETxBzDe4nMzjATuarHciDLm14tSEoUVx+4qzBct5s/XlXIsFSWKb4BZlvu8jH1QHA/Gb/2M5Xa7yHbJw7i6L8S4++hlzb7FuFg8wbm/w784N1HkYb74sGJ+jTGSyHUY9VKW01z0GMG4ECkE/CzGTwe+sljXlRbjWgDZ1hw/Jf85Wh3FYK21H8YOicQoYwTjqtgb2KqUSlFKpWBUdgeZx4dgXAGX1BDjyiDB4nufYFyVn0MbW3o2xo8V4HaMIpui+dQvmod5Ps9gnMSLHC9jvQL572qlyFGMq+wiJ8wxWI6vb+U6nLNspVRtpdRspdQJpVQaRpFYIJfmpMXfWRhXPZhjKl6e1joL48dYmkCMK6HS9k2Zy1FKNVVKLVZKnTSvw6ucvw4l13uiUmqv+UmgFIwik6LvXOgYKU1DYGKJ/R2Cse6lLruEMRh3Y/uUUluUUjdZudxLibGk+px/fLli/TF6IRc7ds85Hi6yjCcxLsg2m58SG21lDCHAUW19fcxPWmt/jHXfhXF3WqSsfVuf83+HJdfnjNY6x5r5aaNi/lGME/pp82+y6Biy5hipDyRprdMthpU8b5T8/XheTt2JoyUKALTWfwJfYRRrgHHrmA201Fr7m//V0EaFFRg7s1EpszqOcTUUaPG96lrrlhdY9I/AMKVUQ4y7iPkW8zlsMQ9/rbWf1nqAZdhlrNJZjKu7hhbDQjGuXoo0UEqpEuPjrVyHksuebh7WRmtdHaNIRpUx/aVIwChmAIzHNDGKe0pzFuPWuLR9czEfYdTVNDGvwzOcuw5gsR5KqR4Y5cEjgJrmE0WqxXcudIyU5jgwrcT+9tZa/1jaskvSWh/UWo/ESOavA/OUUj5lfecyYiwpnvOPrwKM8u/i0C5jvhc7ds85HjBOkqXSWp/UWv9Pa10f467kQ6VUYytiOA6EXuoJUGt91rycF5VS9SzmdaF9m8D5v8OS61NyG5Z5rGitf9Bad8fYfhrjeCjrGLEUD9RSSvlZDCt53igXDpkozN4D+iql2mmtTRhl2e8qpWoDKKUaKKX6maf9HLhXKdVHKVXNPC5Sa52AURH7tlKqunlcI6XUNaUtUGu9DeMW9zPgd611innUZiBNKTXZ/Py6i1KqlVLqKmtWRGtdiFHpNE0p5WdORI9jXOkXqQ08rJRyU0oNB5oDSy91Hcz8MIrxUpRSDTDK5y2dwijLvhzzgIFKqa5KKXeM2+eSJ3AAzPvtC+AdpVR983bropTysGI5fhgVfRlKqUjgASumL8BcRKGUegGobjH+M2CqUqqJMrRRShUluJLb41NgnFKqs3laH6XUjSV+sBeklLpTKRVkXv+iY6jQHJuJC2/7xUBdpdSjSikP87HS2ZplYlzkPKaUCldK+WLcgc25hKvwotg9Lf+Z4y3r2P0JeMT8m/PHSNYXmvdwpVRRUknGOHEWmj+XdUxuxjiJv2beF55KqW7WrI/Weh/wO8bdDJS9bzeY43lQKeWqlBqEUf9ZlgvOTynVTCnV23y852Bc7Baat8WFjhHL2I9jVI5PN69zG4w7ke8pZw6bKLTWZzDKNJ83D5qMUVm3URlFESsxymHRWm8G7sWo7ErFqAAqugK6C+PWeQ/GwTkPKLq6KM2PGGWKP1jEUggMxKhMOoxxlfUZRtGGtR7CqGeJxSj3/AHjJFpkE9DEPO9pwDCtdVGRzqWuw0sYlWypwBKMik1L04HnzLfKT1zCOqC13m1el9kYP950jHLY3At85QmMSuQtGE9uvI51x+UTGMV/6Rg/xjkXmf534DeMitajGD9My2KDdzBOassxEtDnGJWRYBQNfG3eHiO01tEYdVQzMLZ3DEZdgrVuAHYrpTIwnsS6TWudYy6mmwasNy/rassvmYsY+mIcaycxnui61splfoHxRNtajGM0B2M/XYoGGCczy3+NKPvY/RRjm+7AeFpoKUbCPuekZ3YVsMm8XRYBj2itD5vHvYjFPrD8ksXvrzFGPWIcxgMf1noTuE8pVbusfau1zsOowB6DcfK+EyN5X+jY5iLHigf/VaafxLgYfMY8rtRjpJRFjMSot4gHFgBTtNYrLmHdreIwL9xVZUqpezAq/LrbO5ZLZb56TcEoIjp8semFc1NK9Qc+1lo3vOjEDkAptQljfb60dyy25LB3FKLyUkoNVEp5m8tU38K4Yzhi36iEPZiLYgeYi2oaAFMwrnwdklLqGqVUXfP63A20wXhwxqlJohC2MAjjVjgeo7jsNi23rlWVwijqTMYoetqL8Qi1o2qG8c5GKsb7LMPM9YROTYqehBBClEnuKIQQQpTJ4RqtCgwM1GFhYfYOQwghHMrWrVvPaq2DLj7l+RwuUYSFhREdHW3vMIQQwqEopY5efKrSSdGTEEKIMkmiEEIIUSZJFEIIIcokiUIIIUSZJFEIIYQokyQKIYQQZbJZolBKfaGUOq2U2nWB8Uop9YFSKkYptUMp1cFWsQghhLh8tryj+AqjqdwL6Y/RDlAT4D6MjmiEEEKUo0KTJie/tFbdrWezF+601muVUmFlTDII+MbcWNxGpZS/UqpeVWhgSwghykNOfiFn0nPZHZ/KkcQs9iWkYdKQnV/IgVPpHE3MKpfl2PPN7Aac23FMnHnYeYlCKXUfxl0HoaGhFRKcEEJUBlpr4pKzOZ6cReyZTJbuTCD2TCYn00rrx8gQWdcPP1cX/NLyidt1htoB3lcUgz0TRWndY5balK3WehYwCyAqKkqauxVCOB2tNUcSs9h8OJHTabnsik8l5nQGp9NzSc85t8faWj7uDG5XHy93F1rUq46fpxttQ/ypV8MTTzcXtNZERX3K4f1nefnla3n44c64ud152bHZM1HEcW7H5MEY/RcIIYTTybEoDkrKzCM+NZvEjDyOnM0k5kwGKVn5533H3bUaLetX57rmdahXw5OGAd40CvLF39u91GX8/fdxWreujZ+fB599NpDAQG9CQi6lR+bS2TNRLMLopHw20BlIlfoJIYQzyMkvZMuRJDYcSmTLkSS2HEkudTpPt2o0reNH0zp+BPl6ULeGJ9e3qEOj2r5U93TD3dW6540SE7N46qmVfPbZNqZMuYYXX+xF+/b1ym19bJYolFI/Ar2AQKVUHEYXiG4AWuuPMTpZH4DR2XgWcK+tYhFCiPJUaNIcPJ3O7hNp7IpPZV9COh5u1UjOzGPfyXRyC0znTB9Sy4smtf0Y2LYedat7EejrTn1/L3w8ruwUrLXmm2/+5YknVpCcnM2kSV2ZNKnrFc2zNLZ86mnkRcZrYIKtli+EEFfCZNLsjk/jwKl04pKzSck2iom2Hk0mJ99EXuG5ySDQ152IIF/ah/rTKMiXhgHe9G1Rl/BAH5vFOHnySt5882+6dg3h449vpHXrOjZZjsP1RyGEEOVBa01SZh6n0nKJS85i14lU1h48S0pWHqfTc8nKO//dg9Ba3rQN8celmqJX0yAaBvrQqn4Ngvw8Kizu7Ox8MjPzCQz0ZsyY9jRpUosxYzpQrVppzweVD0kUQginlpSZx9ajyUQfSSL6aDKHz2biWk1xOj231Okj6/oxsE19CrUmPNCHDqE1aRTkg7+3u9V1BraybFkMEyYspV27usyfP4JmzQJp1izQ5suVRCGEcBp5BSb2JqSxIy6F33efYtPhRPIL/3ui3tfDleCaXjTw9yKkljduLorgmsaTRHVreNK4tq8do7+w+Ph0Hn10GXPn7qFZswAefPCqCl2+JAohhEMqNGkOn83g138T2B2fyoFTGRxLOvdNZKXguua16deyLldHBBBS68pePLOHP/6I5ZZb5pCXV8jUqdcyaVJXPK6wEvxSSaIQQlR6h85k8FP0cRIz8th1IpW45Gwycs99CS0iyIc7OocSWsubNsH+tA2pgbe7457i8vMLcXNzoW3bugwY0IRXXulN48a17BKL425FIYRTOZ2ew+bDSWw7lkJWXiEr957CRanzmqpoUtsoJmpmfv/gqrCadI4IwMWGlbkVKS0tl+efX8WmTSdYv340gYHezJ49zK4xSaIQQlQIrTX7T6Wz5UgyKZl57D2Zxl8Hz1KtmiItOx9TicZ5wgN9yM4r5N5uYXi6udC3RR06hNa0T/AVQGvNvHl7eOSRZZw8mcH48VeRm1uIt7f9uw2SRCGEsImCQhMbY5P4ZsMRdsSlltqInadbNer7enHrVSH4e7kTEeRDZF0/Qmt5o5Rz3CFY48yZTO6++xd++y2G9u3rsnDhbVx1VQN7h1VMEoUQ4ooVmjQbYxOZtzWOo4mZHDiVcV4dQo8mgbRqUIPrW9ShVYMauLnY/0q5sqhe3YOzZ7N4771+TJjQCVc7P4ZbkiQKIcRlMZk0W44kMXPNIdYeOHPOuIhAH66NrE1ITS+GdgymUVDlfOzUntauPcq0aeuYP38Evr7ubNw41qYvzV0JSRRCCKvk5BeyYs8pVu49xYnkbKKP/tfQXXigD/1a1uWOzqEE1/SqUsVGl+rs2SwmTVrBV19tJyzMnyNHUmjVqnalTRIgiUIIUYZTaTl8/tdh/j50ll0n0s4Z1ym8Ft0bB3LrVSHUqe5ppwgdh9aaL7/czqRJK0hLy+Xpp7vz3HM98fZ2s3doFyWJQgiB1poz6bnEnM5gT0Ia/xxLZuvRZE6l/dfMRddGAfRtUYcb29QjyNdD7houw3ff7aBFiyA+/vhGWrasbe9wrCaJQogq6GRqDvP/iWPfyXQ2H048JyEU8XSrRo8mgYzuFs61kY5zUqtMsrLyefXVdYwbF0VwcHXmzx9BjRqelbqYqTSSKISoIkwmzdcbjjBzdQxnM/KKh/t6uNIh1J9WDWrQukENmtTxI7KuH55uLvYL1gksXXqQCROWcuRICg0a+PHAA1dRs6aXvcO6LJIohHBChSZNXHIWexPS2XnCaCAv5nRG8fgbW9djSIcG9GpW22neaK4s4uLSePTRZcyfv5fmzQP588976Nmzob3DuiKSKIRwEkmZeXy74SibDify96HEc8YF1/Sia6MAosJqMe6aCIduA6mymzZtLUuWHOTVV3szcWJX3N0d/85MGR3NOY6oqCgdHR1t7zCEsLtdJ1KZG32cmDMZbIpNosCiDYz+repSr4YXV0fUom2IvzyVZGObN5/Ay8uV1q3rkJiYRWpqLhERlau5EaXUVq111OV8Vy4rhKjktNYcT8pm78k0ElKyWb7nFNFHks/pirNvizqE1vKmZ9MgejQOdLjKUkeVmprDM8/8wUcfRXPTTU1ZtGgkAQHeBAQ4XnPmZZFEIUQllJVXwJIdCfy6I+G8t5693Fxo1aA63RoH0q9lXVo1qGGnKKsurTVz5uzmscd+5/TpTB56qBNTp/a2d1g2I4lCiEqg0Nwcxj/HktlwKJF1B88Wj2tWx48ODWvSo0kgobW8aVbXT9pJsrPvvtvBXXf9QlRUfRYvHknHjvXtHZJNSaIQwo72nUxj6Y4EPlgVUzzMy82FyLp+DO0QzO2dQ/Gp4N7MROlycwuIjU2mefMgRoxoSUGBibvuaotLFUjacgQKYQfxKdk8s2Ana/YbxUrhgT70bBLIhN6Nqe0nFc+VzerVh3nggSVkZeVz8OBDeHi4cu+97e0dVoWRRCFEBUpIzebdFQeY/88JCk2a8EAfPr6zI83q+tk7NFGK06czeeKJ5Xz77Q4iImoya9bACu+vujKoemssRAUzmTSztxznu41H2ZNgNKzXKMiH929rLxXRlVhMTBKdOn1KRkYezz7bg2ef7YGXV+VvwM8WJFEIYQO5BYX8dfAsmw4nMWttbPHwPpG1ebhPE9qG+NsxOlGWtLRcqlf3oFGjmowZ057Ro9vTvHmQvcOyK0kUQpST3IJCth5NZunOBL7beKx4eGRdP3o1q83YHuEE+nrYMUJRlszMPF5++U8+/fQfdux4gODg6rz55vX2DqtSkEQhxBXKKzCxYFscr/22j+Ss/OLh93QNY3S3cEKd7OUrZ/Trr/t58MHfOHYslTFj2jtEHxEVSRKFEJdpb0Iav+1MYMbqGIpaz3i4TxOGdwyWXt4cREGBiREj5rJgwT5atgxi3bp76d491N5hVTqSKIS4BIUmzbytx3l92X6SMo2mumt4uXF3l4bcd00jfKvgEzGOSGuNUgpX12rUq+fLa6/14bHHujhFA362IEe1EFaIOZ3O138fZfmek5xKy8XDtRpD2jfg9s6hdAitKW0rOZCNG+OYMGEpn346kA4d6jFz5o32DqnSk0QhRBlSsvJ4/Kd/WbXvdPGwx/s2Zdw1jXB3df43cp1JcnI2zzzzB598spX69f1ITs62d0gOw6aJQil1A/A+4AJ8prV+rcT4UOBrwN88zVNa66W2jEkIa/xzLJn/++Mgaw6cQWsY0r4Bo7uH07J+dal7cEBz5uzi4YeXcfZsFo8+ejUvvdQLPz95As1aNksUSikXYCbQF4gDtiilFmmt91hM9hzwk9b6I6VUC2ApEGarmIQoy9mMXD7/6zCfro0t7tuhgb8XT97QjEHtGtg5OnEl9u07S1iYP8uW3UH79vXsHY7DseUdRScgRmsdC6CUmg0MAiwThQaqm/+uAcTbMB4hzrMzLpVftp8gITWbpTtPFg+/tlkQr9zSmgb+jtnHcVWXk1PA66//RYcO9Rg4sBnPPNOD557rWSUa8LMFWyaKBsBxi89xQOcS07wILFdKPQT4ANeVNiOl1H3AfQChofLomrgyqVn5fLDqIMv3nOR40n/l1L0jazO2ezhXRwRI5bQDW7kylvHjl3DwYBITJ3Zh4MBmuLnJ00xXwpaJorRfWsl+V0cCX2mt31ZKdQG+VUq10lqbzvmS1rOAWWB0hWqTaIXTO5maw6tL97Lo3/9uXO+8OpT7ezYipJa8FOfoTp3K4PHHl/PDDztp3LgWy5ffSd++jewdllOwZaKIA0IsPgdzftHSGOAGAK31BqWUJxAInEaIcpCQms3M1TEs3B5Pek4BAO4u1Xj31nbc0KouLnLn4DRWrIhl3rw9vPBCT55+ugeenvJQZ3mx5ZbcAjRRSoUDJ4DbgNtLTHMM6AN8pZRqDngCZxDiCu06kcqHa2KK6x183F24q0tDekfWpkeTIEkQTuLff09y8GASw4a14I47WtOtWwjh4TXtHZbTsVmi0FoXKKUeBH7HePT1C631bqXUy0C01noRMBH4VCn1GEax1D1aaylaEpdFa82fB87w7Yaj/GF+7yGyrh8P92nCgNbypIszycjIY8qU1bz//ibCwvwZPDgSV9dqkiRsxKb3ZuZ3IpaWGPaCxd97gG62jEE4N601u+PTeGXJHjbGJhUPb1bHj2m3tCIqrJYdoxO28Msv+3jood+Ii0vjvvs6MH36dbjKy482JYV4wiElZebx/C+7WLIzoXhYSC0vhrQPZsRVIfJYq5PaufMUt9wyh9atazNnzjC6dg25+JfEFZNEIRzKvpNpzFgVw5r9Z8jILSAiyIdrm9Vm1NUNCQv0sXd4wgby8wtZt+4YvXuH07p1HZYsuZ2+fSPkkdcKJIlCOISkzDx+2XaClxcb72tGNazJpH7N6BwRYOfIhC39/fdxxo1bzO7dZ9i//0EaN67FgAFN7B1WlSOJQlRqJpNm0rwdzP8nDgA/D1e+vPcqqXtwcklJ2Tz11Eo+/fQfQkKq8/PPI2jcWPa5vUiiEJXWyj2neGfFAfYkpNEoyIexPSK4qU09/Dyl9zFnlpNTQLt2HxMfn87EiV148cVe+Pq62zusKk0ShaiUvtlwhBcW7gbggV6NeLJfM2m11cnFxaURHFwdT09Xpk69lnbt6tK2bV17hyWQRCEqmdX7TvPG7/vZm5BG49q+zL2/CzV95GrSmWVn5zN9+l+8/vp65s0bzsCBzbj77nb2DktYsCpRKKXcgVCtdYyN4xFVkNaa33ad5OM/D7EjLhUwKqs/GdVRkoSTW778EOPHL+HQoWTuvLMNnTpJc+6V0UUThVLqRuAdwB0IV0q1A6ZorW+xdXDCuZ1Ky2HR9nim/7YXc/cPdG0UwKu3tJZHXauAhx5ayowZW2jSpBYrV46iT58Ie4ckLsCaO4qXMZoHXw2gtd6ulGps06iEU9t1IpVP18WycLvRRqSPuwt3dmnI7Z1CaRggCcKZFRYaDUO7uFTj6quDCQz0ZvLk7tKAXyVnzd7J11qnlKhIlPaYxCVbe+AMby3fX1y81DbEn4d7N6ZXs9rSSF8V8M8/CYwbt5hRo9rw0EOdueOONvYOSVjJmkSxVyk1Aqhmbgn2EWCjbcMSzuazdbG8smQvHq7VGNK+AQ/1aUK4FC9VCenpubzwwmo++GAzQUHe1KvnZ++QxCWyJlE8CLwAmICfMVqDfdqWQQnnUNSa64uLdnMkMYu2If58c28nanjLexBVxfLlhxg9eiHx8emMGxfFq6/2wd/f095hiUtkTaLop7WeDEwuGqCUGoKRNIQ4R2ZuAV/8dZgF204QezazeHin8Fp8OipKkkQV4+7uQu3aPsyfP4LOnYPtHY64TOpi3T8opf7RWncoMWyr1rqjTSO7gKioKB0dHW2PRYsypOfk8/byA3y38SgF5keYmterTqewmoztESFdjVYR+fmFvPPOBtLScpk2rQ9gNMMifZDbn/m8HXU5373gHYVSqh9GN6UNlFLvWIyqjlEMJQRZeQV8tOYQ/7fKeMUmItCHx/o2pX+ruri6SB8BVclffx0rbsBv+PAWxQlCkoTjK6vo6TSwC8gBdlsMTweesmVQonIzmTQbDyfy1u/7+edYSvHwt4a3ZWiHBtLURhWTmJjF5Mkr+fzzbYSG1uDXX0dy001N7R2WKEcXTBRa623ANqXU91rrnAqMSVRS62PO8tm6WFbv/69bc9dqionXN2Nsj3Dc5A6iSkpMzGb27F08+WRXXnjhGnzkbXqnY01ldgOl1DSgBVD8uILWWi4ZqohCk6DMJ6IAACAASURBVOb+b7eycu8pANoG16Bb40CGdQwmIsjXztEJe9i79ww//bSbKVN60bRpAMeOPUatWtKroLOyJlF8BbwCvAX0B+5F6iiqhNTsfN5beYAfNx8jJ9/Y5euevFYqpquwrKx8pk1by5tv/o2vrztjxnQgOLi6JAknZ02i8NZa/66UektrfQh4Tim1ztaBCfvacCiRkZ8a71X6uLvw0s0tGXV1Q6mYrMKWLYth/PglHD6cwt13t+XNN/sSFCQvTVYF1iSKXGXUTh5SSo0DTgC1bRuWsIe8AhMv/bqb7zcdKx42vlcjJklfEFVeRkYeo0YtICDAi9Wr76ZXrzB7hyQqkDWJ4jHAF3gYmAbUAEbbMihR8fbEpzH++60cScwCYEiHBgzrGEzXRoF2jkzYS2GhiR9/3MXIka3w9XVn5cpRREYG4uEhDfhVNRfd41rrTeY/04FRAEopecXSSeQWFDJjVUzxexATrm3EQ72b4OnmYufIhD1t3RrP/fcvZuvWBLy8XBk6tIX0NleFlZkolFJXAQ2Av7TWZ5VSLTGa8ugNSLJwcOtjzvK/b6LJyisEYN64LkSFSQf2VVlqag7PP7+amTO3ULu2D7NnD2XIkOb2DkvYWVlvZk8HhgL/YlRgL8BoOfZ1YFzFhCfKm8mkWbIzgQ/XHGJvQhoA9/WM4PG+TeUuQjB06E+sWnWYCROu4pVXelOjhjTgJ8q+oxgEtNVaZyulagHx5s/7KyY0UZ4KTZr3/zjI4h3xxJ4xGusb3jGYR/s2pYG/PNpYlcXGJhMU5I2fnwfTpvWmWjXFVVdJl6TiP2UlihytdTaA1jpJKbVPkoRjWrXvFM/8vIuTacYL9vd0DeOp/pFyB1HF5eUV8tZbfzN16loefrgTr7/eV1p4FaUqK1FEKKWKmhJXQJjFZ7TWQ2wambhi6Tn5jP/+H9YdPAvAyE4hTBvcWt6FEKxde5Rx4xazd+9Zhg1rwcMPd7Z3SKISKytRDC3xeYYtAxHlp6DQxEu/7mH2lmPkF2qubRbEW8PbEuDrYe/QRCXw7rsbePzx5YSF+bNkye0MGNDE3iGJSq6sRgH/qMhAxJXLLzTx/sqDzFoXS16BCZdqindGtGVIBylOqOpMJk1mZh5+fh7ceGNTzpzJ4rnneuItHUkJK8ibM05i7YEzPPvLTo4nZRPg485zQ5tzS3tJEAJ27z7NuHFLinuaa9o0gFdf7WPvsIQDsWmiUErdALwPuACfaa1fK2WaEcCLgAb+1VrfbsuYnE1SZh5PzP2XVftOAzC2ezjP3dTCzlGJyiArK5+pU//krbc2UKOGB6NHt0NrLc2xiEtmdaJQSnlorXMvYXoXYCbQF4gDtiilFmmt91hM0wR4GuimtU5WSkkbUlZKycpj2pK9zN0aVzzslwndaBfib8eoRGWxbVsCQ4b8xJEjKdx7bzveeKMvgYHS6q+4PBdNFEqpTsDnGG08hSql2gJjtdYPXeSrnYAYrXWseT6zMd7N2GMxzf+AmVrrZACt9elLX4WqJTO3gGcW7GTh9vjiYW8Oa8PwqBA7RiUqi6I7htDQGoSG1uDrrwfTs2dDe4clHJw1dxQfADcBvwBorf9VSl1rxfcaAMctPscBJZ/BawqglFqPUTz1otZ6mRXzrpJOpubQ7721pGbnA/DqLa257aoQedxVUFBgYsaMzSxatJ8VK0YREODNn3/eY++whJOwJlFU01ofLVGuWWjF90o7e+lSlt8E6IXRdtQ6pVQrrXWK5URKqfuA+wBCQ0OtWLRz0VqzYNsJJs/fQX6hZsrAFtzdJUwShABg8+YTjBu3mG3bTtK/f2PS0nKpWVPethflx5pEcdxc/KTN9Q4PAQes+F4cYFkeEozRDEjJaTZqrfOBw0qp/RiJY4vlRFrrWcAsgKioqJLJxqnlFhQy9uvo4pfmZo3qyPUtpRVPYfQRMXnyCj76KJp69fyYO3c4Q4c2l8pqUe6sSRQPYBQ/hQKngJXmYRezBWiilArH6OzoNqDkE02/ACOBr5RSgRhFUbHWhe78Fm4/waR5O8grMNG9cSDv3NqW2n7SSJswuLlVY82aozz0UCemTu1N9eryQqWwDWsSRYHW+rZLnbHWukAp9SDwO0b9wxda691KqZeBaK31IvO465VSezCKsyZprRMvdVnOptCkmTx/B/O2xhHo686rt7SWuwgBQExMEi+//CczZw7Az8+DrVvvw9NTXocStqW0LrskRyl1CNgPzAF+1lqnV0RgFxIVFaWjo6PtGYJNbYxN5PE524lPzaFzeC3eHtGW4JryWGNVl5tbwBtvrGfatHW4u7uwZMnt9OghTzMJ6ymltmqtoy7nu9b0cNdIKdUVo+joJaXUdmC21nr25SxQXNgnfx5i+m/7AHiqfyT394yQ8mbB6tWHeeCBJezfn8itt7bknXf6Ub++n73DElWIVfesWuu/gb+VUi8C7wHfA5IoyklBoYmnft7JvK1xhAV48+EdHWlRv7q9wxKVgNaaadPWkZ9vYtmyO+jXr7G9QxJVkDUv3PlivCh3G9AcWAh0tXFcVYbWmjs/38TG2CRa1KvOoge74epSzd5hCTsymTSff/4PN9zQmJCQGnz77S34+3vi5SUN+An7sOaMtAu4GnhDa91Yaz1Ra73JxnFVGe+uPMjG2CQGtq3Pkoe7S5Ko4nbsOEX37l9w332L+eyzfwCoV89PkoSwK2uKniK01iabR1IFTf9tL5/8GUuXiAA+uK2d1EdUYRkZebz00hrefXcjNWt68dVXg7jrrrb2DksIoIxEoZR6W2s9EZivlDrv0Sjp4e7yJWfmMWnev6zce5rwQB8+urODJIkq7sUX1/D22xsYO7Y9r712HQEB8qSbqDzKuqOYY/5ferYrRwu2xfHYnH8BaNWgOvPGdZW+q6uo48dTyczMJzIykKee6s7gwZF07171mqgRlV9ZPdxtNv/ZXGt9TrIwv0gnPeBdonHfbmXZ7pMAvH9bOwa1a2DniIQ9FBSY+OCDTbzwwmo6dqzPn3/eQ2CgtyQJUWlZU3M6upRhY8o7EGf37IKdLNt9kjrVPdj54vWSJKqojRvjiIqaxcSJy+nVK4yvvx5s75CEuKiy6ihuxXgkNlwp9bPFKD8gpfRviZIycwt48Id/WL3/DEF+Hvw1uTdu8mRTlbRkyQEGDvyR+vX9+PnnEQweHCl1U8IhlFVHsRlIxGj1dabF8HRgmy2DchapWflc+/YakjLzGNSuPm8PbyuPv1YxWmvi49Np0KA6110XwcsvX8sjj3TGz08a8BOO46JtPVU2jtLW0yd/HmLW2liSsvJ4fUgbRlwlPdBVNQcOJDJ+/BIOHEhkz54J+Pq62zskUYXZpK0npdSfWutrlFLJnNvhkAK01rrW5SywKnhx0W6++vsI9Wt48vndUfSOrGPvkEQFyskp4LXX/mL69L/w8nJl+vQ+eHlJC6/CcZV19BZ1dxpYEYE4g0KTZsQnG9h6NBmANZOuxd1VipqqkpMnM+jZ80sOHkxi5MhWvPNOP+rW9bV3WEJckbIejy16GzsEiNda5ymlugNtgO+AtAqIz2Hk5Bcy4P11xJ7NBGD9U70lSVQh+fmFuLm5UKeODz17NmTmzAH07dvI3mEJUS6sOZP9gtENaiPgG4yGAX+waVQOZlNsIm1eWk7s2Ux6Ng3i8PQBNPCXPourApNJ8/HH0TRq9AFxcWkopfjss5slSQinYk3BqUlrna+UGgK8p7X+QCklTz2ZnUzN4c7PN5FfqJnYtykP9Wli75BEBfn335Pcf/9iNm06Qe/e4eTnF9o7JCFswqquUJVSw4FRQNHbQdKUJZBfaOL+b6MpMGkWTuhG2xB/e4ckKoDWmkmTVvDeexupVcuLb7+9hTvuaC3vRAinZU2iGA2Mx2hmPFYpFQ78aNuwKjetNRtiE5mycDcHT2fw3I3NJUlUIUopkpOzGTPGaMCvZk0pZhTOzar3KJRSrkBR11oxWusCm0ZVBnu/R5GdV0jzF5YB4OfhyvM3tZB3JKqAo0dTeOSRZbzwwjV06FAPk0lTrZrcQQjHcSXvUVy0Mlsp1QOIAT4HvgAOKKW6Xc7CHF2hSXPtW2sAqKZg/dO9JUk4ufz8Qt54Yz0tWnzIihWx7N9/FkCShKhSrCl6ehcYoLXeA6CUag58C1xWZnJkH62J4WRaDgPb1peOhqqAv/8+zv33L2bXrtMMGtSMDz7oT2hoDXuHJUSFsyZRuBclCQCt9V6lVJVriyA1O5/3/zgIwJvD2kiSqAJWrowlNTWHX365lUGDIu0djhB2Y02i+Ecp9QnGXQTAHVSxRgGz8wq5atpK8gs1s0Z1lI6GnJTWmm+/3UFQkDf9+zdh8uRuPP54F2mjSVR51rxwNw44BDwJTAZigfttGVRlorXRLEdegYmb2tTj+pZ17R2SsIF9+87Su/c33H33L3z55XYAPDxcJUkIwUXuKJRSrYFGwAKt9RsVE1LlobWm//vr2HcynTuvDuWVwa3tHZIoZ9nZ+bz66jpef309Pj7ufPLJTYwd28HeYQlRqVzwjkIp9QxG8x13ACuUUqX1dOe0tNa8unQv+06mE+Djzgs3tbR3SMIGfv31AK+8so5bb23Fvn0TuO++jvJEkxAllHVHcQfQRmudqZQKApZiPB5bJUz/bR+frjuMm4uSBv6czMmTGWzffpIbbmjM8OEtCAsbS6dO0jWtEBdS1tkvV2udCaC1PnORaZ3K34fOMmttLF0iAtg/tb9UXjuJwkITH364hWbNZjBq1AKys/NRSkmSEOIiyrqjiLDoK1sBjSz7ztZaD7FpZHYSn5LN6K+2UKe6B28ObyPFEE7in38SGDduMVu2xHPddRF8+OEAvLykyTIhrFFWohha4vMMWwZSWUz/bR85+SY+u6sdwTW97R2OKAeHDyfTqdOnBAZ688MPQ7jttlbyHowQl6Csjov+qMhAKoN9J9P49d94+reqS/cm0rGfI9Nas3Pnadq0qUN4eE2+/HIQAwc2w9/f096hCeFwqky9gzXGfm00Nvh0/+Z2jkRcicOHk7npph9p3/4Tduw4BcCoUW0lSQhxmWyaKJRSNyil9iulYpRST5Ux3TCllFZK2a39qJmrY4hLzkYpCA2QIidHlJdXyGuv/UXLlh/y559HeOutvrRoEWTvsIRweNY04QGAUspDa517CdO7ADOBvkAcsEUptciy3SjzdH7Aw8Ama+dd3uJTspmxKoaa3m6smtjLXmGIK1BYaKJr18/ZujWBIUOa8957/QgJkQb8hCgP1jQz3kkptRM4aP7cVin1f1bMuxNG3xWxWus8YDYwqJTppgJvADnWh12+3l5+gOz8Qj4ZFUVNH2mywZGkpRnXLi4u1Rg9uj2//jqS+fNHSJIQohxZU/T0AXATkAigtf4XuNaK7zUAjlt8jjMPK6aUag+EaK0XlzUjpdR9SqlopVT0mTNnrFi09fafTGf+P3H0aBJIp/Ba5TpvYTtaa776ajsREe+zcOE+AMaPv4qbbmpq58iEcD7WJIpqWuujJYZZ04t8ac8fFnenp5SqhtHXxcSLzUhrPUtrHaW1jgoKKr8y54JCE+O/34q7SzWmDGxRbvMVtrVnzxl69fqae+9dSGRkII0aSYIXwpasqaM4rpTqBGhzvcNDwAErvhcHWHb/FgzEW3z2A1oBa8zPtNcFFimlbtZaV0hfp7/vPsWhM5k8d2NzGtf2q4hFiiv0xhvrefbZVVSv7sFnnw3k3nvby0uRQtiYNYniAYzip1DgFLDSPOxitgBNlFLhwAngNuD2opFa61Sg+GUFpdQa4ImKShIArywx6tXv6NywohYpLpPWGqUUdev6cscdrXnzzb4EBfnYOywhqoSLJgqt9WmMk/wl0VoXKKUeBH4HXIAvtNa7lVIvA9Fa60WXHG05WnvgDAmpOdzYuh5e7tKWU2UVH5/OI48so0ePUB5+uDN33dWWu+5qa++whKhSLpoolFKfYlG3UERrfd/Fvqu1XorR6qzlsBcuMG2vi82vPP22KwGAqYNbVeRihZWKGvB79tlV5Oeb6No12N4hCVFlWVP0tNLib0/gFs59mskhrdp3mqiGNaklj8NWOtu3n2Ts2EVs3ZrA9dc34sMPB0iFtRB2ZE3R0xzLz0qpb4EVNouoAmw/nsKptFzulLqJSik1NYf4+HTmzBnG8OEtpAE/IezM6jezLYQDDn2G/Wq90SHRyM6h9g5FYFRUz527h4MHE3n22Z5cc00YsbGP4Ol5OYenEKK8WfNmdrJSKsn8LwXjbuIZ24dmG2fSc/llezzXNqtNoK+HvcOp8g4dSmLAgB+49dZ5LFy4n/x84xUdSRJCVB5l/hqVcc/fFuPxVgCT1vq8im1H8uGaGAAe6NXIzpFUbbm5Bbz11t+88so63Nyq8f77NzB+/FW4SpezQlQ6ZSYKrbVWSi3QWnesqIBsSWvNl+uP0DbEn/ahNe0dTpV2/HgaU6euZeDAZrz3Xj8aNKhu75CEEBdgzeXbZqVUB5tHUgG2H08B4Gpp08kuzpzJZMaMzQA0blyLPXsmMHfucEkSQlRyF7yjUEq5aq0LgO7A/5RSh4BMjDactNba4ZLHy4v34O5aTYqdKpjJpPnyy208+eRK0tNz6ds3gmbNAomIkLs6IRxBWUVPm4EOwOAKisWmUrLy2HYshQ6h/vh7y7sTFWXXrtM88MAS/vrrGD16hPLxxzfRrJl0MyuEIykrUSgArfWhCorFpuZGxwHwWF9phrqi5OUVcv3135KXV8gXX9zMPfe0k3cihHBAZSWKIKXU4xcaqbV+xwbx2Mymw4n4e7vRo4l0jWlrq1Yd5pprGuLu7sJPPw0nMjKQwEDpXlYIR1VWZbYL4IvRHHhp/xxGRm4BK/eepnez2vYOxanFxaUxdOhP9OnzDd988y8A3buHSpIQwsGVdUeRoLV+ucIisaHZm48B0KOplI3bQkGBiRkzNvP886spLDQxfXof7rijjb3DEkKUk4vWUTiDFXtO4VJNcXPbBhefWFyyUaMWMHv2Lvr3b8zMmQMID5enmYRwJmUlij4VFoUNbT+ewqbDSYzpHo6L9IRWblJScnB1rYavrzsTJlzF0KHNGTq0uVRWC+GELlhHobVOqshAbGXOFqNF9Hu6htk3ECehtWb27F00bz6T559fBRj1EMOGSSuvQjgrp29YZ/7WOLo3DiSkllSoXqmYmCT69fuOkSPnExxcnTvvlHoIIaoCp26i89/jKeQVmmgYIEniSv3ww05Gj16Ih4crM2b0Z9y4KFxcnP46QwiBkyeKFXtOATCme7idI3Fc+fmFuLm5EBVVn2HDWvDGG32pX9+hno4WQlwhp04UX6w/TEgtLyKCfO0disM5fTqTiROXk5mZx88/30rTpgF8990Qe4clhLADpy07yC80kZVXiLebU+fCcmcyaWbN2kqzZjOYM2cXLVsGUVhosndYQgg7ctqzaFGx04O9G9s5EscRG5vMnXf+zIYNcfTqFcZHH91IZKS8pChEVee0iWLLEePp3v6t6to5EsdRo4YHKSk5fP31YEaNaiOPuwohACcuevo7JpHIun64ypM5ZVq0aD9DhsyhsNBEQIA3u3aN56672kqSEEIUc8qzaE5+IftPpVOnuqe9Q6m0jh1LZfDg2QwaNJsDBxJJSMgAoJq8vS6EKMEpi55mrY0FpNipNAUFJt57byNTpqxBa83rr1/HY49djZubi71DE0JUUk6ZKDYfNuonhnQItnMklU9hoYnPPvuH3r3D+b//609YmL+9QxJCVHJOV/SkteavmLN0Dq+Fu6vTrd5lSU7OZvLkFaSn5+Lh4cr69aNZtOg2SRJCCKs43Zl09f7TAPSOlE6KtNZ8//0OIiNn8vbbG1i9+ggAAQHeUlkthLCa0xU9fbbuMAB3dQmzbyB2duBAIuPHL+GPPw7TqVMDfv/9Ttq1kzobIcSlc6pEkZSZx9+HEukcXgsv96pdOfvoo8uIjo7nww8HcN99HaUBPyHEZXOqRLFw+wkAxvaIsHMk9rFixSEiIwMJCanBRx/diIeHK3XrSjtXQogrY9PLTKXUDUqp/UqpGKXUU6WMf1wptUcptUMp9YdSquGVLG/5bqPZjqpWP3HyZAa33z6f66//jtdfXw9Aw4b+kiSEEOXCZolCKeUCzAT6Ay2AkUqpFiUm2wZEaa3bAPOANy53eVprNsQm0sDfq8p0eWoyaT7+OJrIyBnMn7+XKVOu4a23rrd3WEIIJ2PLO4pOQIzWOlZrnQfMBgZZTqC1Xq21zjJ/3Ahc9osP+06mA3Bjm3qXOwuHM336Oh54YAkdO9Znx45xvPhiLzw9nao0UQhRCdjyrNIAOG7xOQ7oXMb0Y4DfShuhlLoPuA8gNDS01C8fTcwE4JqmQZceqQNJT8/l7NkswsNrMm5cFOHhNRk5spU87iqEsBlb3lGUdubSpU6o1J1AFPBmaeO11rO01lFa66igoNITwa4TaSgF7UKc8yUyrTULFuylRYsPufXWeWitCQjw5vbbW0uSEELYlC0TRRwQYvE5GIgvOZFS6jrgWeBmrXXu5S5s85EkGgX54uPhfEUvR4+mcPPNsxky5Cdq1fLigw/6S3IQQlQYW55VtwBNlFLhwAngNuB2ywmUUu2BT4AbtNanL3dB+YUmNh9O4ua29a8k3kppw4bjXHfdtwC89VZfHnnkalylaRIhRAWyWaLQWhcopR4EfgdcgC+01ruVUi8D0VrrRRhFTb7AXPMV8jGt9c2XuqyY00YT2c5U7JSWlkv16h506FCP0aPbMWlSN0JDa9g7LCFEFWTTchqt9VJgaYlhL1j8fV15LGffyTQAOoXXKo/Z2VViYhZPPbWS5ctj2b17PL6+7vzf/w2wd1hCiCrMKQr0txxJxsO1Gk3r+Nk7lMumtebbb3cwceJykpOzefzxLkg1hBCiMnCKRLE3IY02wTUctlnx1NQcBg+ew5o1R+jSJZiPP76JNm3q2DssIYQAnKCZ8YJCE3sT0qjv72XvUC6Z1sbTwtWrexAY6M2sWTfx11+jJUkIISoVh08Uh85kkpNvIqphTXuHckl+/z2GDh1mEReXhlKKuXOH87//dZQ+q4UQlY7DJ4ptx5IBCK7pbedIrJOQkM5tt83jhhu+Jysrn9OnM+0dkhBClMnh6yg2HzH6x27rAI/Gzpy5mWeeWUVubgEvvdSLyZO74eGELwgKIZyLw5+l4pKyAajl427nSC5u69YEOnduwMyZA2jSJMDe4QghhFUcPlEcScystC/apaXl8sILqxk1qg0dO9bnww9vxMPDRZrfEEI4FIdOFNFHkjidnsv/KlmPdlpr5s/fyyOPLCMhIZ3Q0Bp07FhfmgAXQjgkhz5zLd6RAMANreraOZL/HD6czIMP/sbSpQdp164uP/88gs6dL7ubDSGEsDuHThRFTXcE16w871B8//1O1q49yrvv9uPBBztJA35CCIfn0IniREo2kXX97F7mv27dUXJzC7nuuggmTerKPfe0Izi4ul1jEkKI8uKwl7vJmXkcT8rm2sjadovh7NksRo9eSM+eX/Hyy38C4OHhKklCCOFUHPaOYntcCgBNavtW+LK11nz11XYmTVpBamoukyd34/nne1Z4HKLyy8/PJy4ujpycHHuHIqoIT09PgoODcXNzK7d5Omyi2BNv1E90axxY4cteuvQgo0cvolu3ED7++CZatbLfXY2o3OLi4vDz8yMsLMzuRaTC+WmtSUxMJC4ujvDw8HKbr8MWPc3ZchwvNxfqVPeskOVlZeWzfv0xAAYMaMLChbexdu29kiREmXJycggICJAkISqEUoqAgIByv4N12ERxLCmL7PzCClnWb78dpFWrD+nf/3tSUnJQSnHzzc2kAT9hFUkSoiLZ4nhz2ETh4+5Cfxu/P3HiRBrDh89lwIAf8PBw5ddfR+LvXzF3MEIIUVk4ZKJIy8knM6+QlvVt93TR6dOZtGjxIYsXH+CVV67l33/Hcc01YTZbnhC24uLiQrt27WjVqhUDBw4kJSWleNzu3bvp3bs3TZs2pUmTJkydOrW4nxSA3377jaioKJo3b05kZCRPPPGEPVahTNu2bWPs2LHnDBs0aBBdunQ5Z9g999zDvHnzzhnm6/vfwzAHDhxgwIABNG7cmObNmzNixAhOnTp1RbElJSXRt29fmjRpQt++fUlOTj5vmtWrV9OuXbvif56envzyyy8A9OjRo3h4/fr1GTx4MACLFy9mypQpVxTbJdFaO9S/jh076mOJmbrh5MV6zpZjurzFxaUW//3++xt1TExiuS9DVB179uyxdwjax8en+O+77rpLv/LKK1prrbOysnRERIT+/ffftdZaZ2Zm6htuuEHPmDFDa631zp07dUREhN67d6/WWuv8/Hw9c+bMco0tPz//iucxbNgwvX379uLPycnJOjg4WEdGRurY2Nji4XfffbeeO3fuOd8t2jbZ2dm6cePGetGiRcXjVq1apXfu3HlFsU2aNElPnz5da6319OnT9ZNPPlnm9ImJibpmzZo6MzPzvHFDhgzRX3/9tdZaa5PJpNu1a1fqdFqXftwB0foyz7sO+dTTbvMTT55uLuU2z9TUHJ57bhWffLKVjRvH0qFDPR5+uHO5zV+Il37dXfy0XnlpUb86Uwa2tHr6Ll26sGPHDgB++OEHunXrxvXXXw+At7c3M2bMoFevXkyYMIE33niDZ599lsjISABcXV0ZP378efPMyMjgoYceIjo6GqUUU6ZMYejQofj6+pKRkQHAvHnzWLx4MV999RX33HMPtWrVYtu2bbRr144FCxawfft2/P2Nxj0bN27M+vXrqVatGuPGjePYMeMhkvfee49u3bqds+z09HR27NhB27Zti4fNnz+fgQMHUqdOHWbPns3TTz990e3yww8/0KVLFwYOHFg87Nprr7V6F8/i4QAAD45JREFUu17IwoULWbNmDQB33303vXr14vXXX7/g9PPmzaN///54e5/bv056ejqrVq3iyy+/BIx6iF69erF48WJGjBhxxXFejEMmiuNJWQC0C77yVmO11sydu4dHH13GyZMZPPhgJxo1cqze8oSwRmFhIX/88QdjxowBjGKnjh07njNNo0aNyMjIIC0tjV27djFx4sSLznfq1KnUqFGDnTt3ApRavFLSgQMHWLlyJS4uLphMJhYsWMC9997Lpk2bCAsLo06dOtx+++089thjdO/enWPHjtGvXz/27t17znyio6Np1arVOcN+/PFHpkyZQp06dRg2bJhViWLXrl3nbYvSpKen06NHj1LH/fDDD7Ro0eKcYadOnaJevXoA1KtXj9OnT5c5/9mzZ/P444+fN3zBggX06dOH6tX/K26Piopi3bp1kiguZOvRZNxdqhEacGW92mmtGTLkJ375ZR8dOtRj0aKRREXVL6cohTjXpVz5l6fs7GzatWvHkSNH6NixI3379gWM4/9CT8hcypMzK1euZPbs2cWfa9a8+IXW8OHDcXExSgRuvfVWXn75Ze69915mz57NrbfeWjzfPXv2FH8nLS2N9PR0/Pz8ioclJCQQFBRU/PnUqVPExMTQvXt3lFK4urqya9cuWrVqVeo6XeoTQn5+fmzfvv2SvmOthIQEdu7cSb9+/c4b9+OPP55XD1O7dm3i4+NtEktJDlmZnV9oIq/QdPnfNz9Wq5Sie/cQPvjgBjZvHitJQjglLy8vtm/fztGjR8nLy2PmzJkAtGzZkujo6HOmjY2NxdfXFz8/P1q2bMnWrVsvOv8LJRzLYSWf6/fx8Sn+u0uX/2/v7oOrqu88jr8/S4UEi9k1LIwt2sQxhcQQkIcW7VBEWsYKhYVxRJ6UHbuOINIWxWEHZ9ZdH0pbQRfRReo6SWsbIk4FBhAW2SgVeRC2ERRsmtIM4DCaxixLUR4C3/3jnCSXPNwcIvcmN/m+Zu7Mveeeh+/9zr3nd8/vnPP93UhFRQVVVVWsWbOGyZMnA3D+/Hl27NhBWVkZZWVlfPTRRxc0EnWfLXbdJSUl1NTUkJ2dTVZWFpWVlfWNWGZm5gVHO59++im9e/euz0WUz3rixIkLTjzHPmIbtTp9+/bl2LGgyvWxY8fo06fl+65eeeUVJk2a1OSO6urqanbv3s24ceMumH7q1CnS05NTEDUlG4q3yqsYmdO2O7LffLOSgoIVrF37IQAPPngTDzzwTbp1S8lUOBdZRkYGy5Yt46mnnuLs2bNMnz6dt99+mzfeeAMIjjzmzZvHww8/DMCCBQt48sknKS8vB4Id99KlS5usd+zYsSxfvrz+dd3OuG/fvhw8eLC+a6klkpg0aRLz588nNzeXzMzMZtfb3D/53NxcKioq6l8XFxezadMmKisrqaysZO/evfUNxc0330xJSQlnzpwBoLCwsP48xLRp03jnnXfYsGFD/bo2bdpU351Wp+6IorlH424ngAkTJlBUVARAUVEREydObDEPxcXFTJ06tcn01atXM378eNLSLrw0v7y8vEm3W6Kk3N6x9rxRe97Iyry89ZljVFWd5O671zB6dBGnT9fSq1ePBEXoXMd1ww03MGjQIFatWkV6ejpr167l8ccfp3///gwcOJDhw4czd+5cAAoKCnjmmWeYOnUqubm55Ofn1/87jvXII49QU1NDfn4+gwYNorS0FIDFixczfvx4brnllvp++pZMmTKFl19+ub7bCWDZsmXs2bOHgoIC8vLyWLFiRZPlBgwYwPHjxzlx4gSVlZUcPnyYESNG1L+fnZ3NFVdcwa5duxg/fjwjR45k6NChDB48mO3bt9efWE5PT2f9+vU8++yz5OTkkJeXR2FhYdwjgCgWLlzIli1byMnJYcuWLSxcuBAIzq3EdiVVVlZy5MgRRo0a1WQdq1atarYBKS0tbXKUkSiymGumU8GAgYPt1Lgn+MVdw/huXt9IyxQX7+f++zfy17+eYcGCm1i06Nv07HnpCmY515KDBw+Sm5vb3mF0ak8//TS9evVq0offmX388cdMmzaNrVu3Nvt+c987SXvNbFhbtpdyRxRnaoNzE1dlRL9Durb2PPn5fSgru48nnhjjjYRzncjs2bPp0aNr9RAcPnyYJUuWJG17KXfV04lTtWT36hH3ruyTJ8/w2GPbuOaaDObMGc6MGQXMmFHgNXec64TS0tKYOXNme4eRVMOHD0/q9lLuiOLk6VqGZ1/Z4k5//fpyrr/+eX760+2Ul1cDwckybyRce0m17l2X2hLxfUu5I4pzZlxzZdP7J44e/T/mzXud1177kLy8v2fbtlmMHPm1dojQuQZpaWlUV1d7qXGXFBaOR9H4CqkvKuUaCoBeaU3DPnSohs2b/8RPfjKG+fNvpHv3S1few7m26tevH0ePHqWqqqq9Q3FdRN0Id5dSyl311OOqHHtr+05GXJvJ7t0fsWPHEX74w+ByuOrqz8j8gndrO+dcZ9Rhr3qSdKukP0iqkLSwmfd7SCoJ398lKSvKevv0uIw5czYwYsSLLF26k5MngxtovJFwzrlLL2ENhaRuwHPA94A8YKqkxrcu3gPUmNl1wNNAy2UVY9w0dCUvvLCXefO+yf79s7n88u6XMnTnnHMxEnmO4htAhZkdApC0CpgIxBZEmQg8Gj5/FVguSRanP8xqz3N1VgYbN05nyJD4d3s655z74hLZUHwVOBLz+ijQeICH+nnMrFbScSAT+EvsTJLuBe4NX57e85d7349QEbgr6E2jXHVhnosGnosGnosG/du6YCIbiuauBWx8pBBlHsxsJbASQNKetp6Q6Ww8Fw08Fw08Fw08Fw0k7Wl9ruYl8mT2UeDqmNf9gMbF0+vnkfQlIAP4NIExOeecu0iJbCjeBXIkZUvqDtwJrGs0zzrg7vD57cB/xzs/4ZxzLvkS1vUUnnOYC2wGugEvmdkHkv6NYJDvdcB/Ar+SVEFwJHFnhFWvTFTMKchz0cBz0cBz0cBz0aDNuUi5G+6cc84lV8oVBXTOOZdc3lA455yLq8M2FIkq/5GKIuRivqQDkvZJ2iqp05bNbS0XMfPdLskkddpLI6PkQtId4XfjA0m/SXaMyRLhN3KNpFJJvw9/J7e1R5yJJuklSZ9Ier+F9yVpWZinfZKGRFqxmXW4B8HJ7z8B1wLdgfeAvEbzzAFWhM/vBEraO+52zMVooGf4fHZXzkU4Xy9gG7ATGNbecbfj9yIH+D3wd+HrPu0ddzvmYiUwO3yeB1S2d9wJysW3gSHA+y28fxvwOsE9bCOAXVHW21GPKOrLf5jZGaCu/EesiUBR+PxVYIw6Z8H/VnNhZqVm9ln4cifBPSudUZTvBcBjwM+AU8kMLsmi5OKfgOfMrAbAzD5JcozJEiUXBtQNi5lB03u6OgUz20b8e9EmAr+0wE7gbyW1WgupozYUzZX/+GpL85hZLVBX/qOziZKLWPcQ/GPojFrNhaQbgKvNbH0yA2sHUb4XXwe+Lmm7pJ2Sbk1adMkVJRePAjMkHQU2Ag8kJ7QO52L3J0DHHbjokpX/6AQif05JM4BhwKiERtR+4uZC0t8QVCGelayA2lGU78WXCLqfbiY4yvydpHwz+98Ex5ZsUXIxFSg0syWSbiS4fyvfzM4nPrwOpU37zY56ROHlPxpEyQWSvgMsAiaY2ekkxZZsreWiF5APvCmpkqAPdl0nPaEd9Tey1szOmtmfgT8QNBydTZRc3AO8AmBmO4A0goKBXU2k/UljHbWh8PIfDVrNRdjd8gJBI9FZ+6GhlVyY2XEz621mWWaWRXC+ZoKZtbkYWgcW5TeyhuBCByT1JuiKOpTUKJMjSi4OA2MAJOUSNBRdcXzadcBd4dVPI4DjZnastYU6ZNeTJa78R8qJmIufA18GVofn8w+b2YR2CzpBIuaiS4iYi83AWEkHgHPAAjOrbr+oEyNiLh4EfiHpxwRdLbM64x9LScUEXY29w/Mx/wJcBmBmKwjOz9wGVACfAf8Yab2dMFfOOecuoY7a9eScc66D8IbCOedcXN5QOOeci8sbCuecc3F5Q+Gccy4ubyhchyPpnKSymEdWnHmzWqqUeZHbfDOsPvpeWPKifxvWcZ+ku8LnsyR9Jea9FyXlXeI435U0OMIyP5LU84tu23Vd3lC4juhzMxsc86hM0nanm9kggmKTP7/Yhc1shZn9Mnw5C/hKzHs/MLMDlyTKhjifJ1qcPwK8oXBt5g2FSwnhkcPvJP1P+LipmXmul7Q7PArZJyknnD4jZvoLkrq1srltwHXhsmPCMQz2h7X+e4TTF6thDJCnwmmPSnpI0u0ENbd+HW4zPTwSGCZptqSfxcQ8S9KzbYxzBzEF3ST9h6Q9Csae+Ndw2jyCBqtUUmk4baykHWEeV0v6civbcV2cNxSuI0qP6XZ6LZz2CfBdMxsCTAGWNbPcfcC/m9lggh310bBcwxTgW+H0c8D0Vrb/fWC/pDSgEJhiZgMJKhnMlnQlMAm43swKgMdjFzazV4E9BP/8B5vZ5zFvvwpMjnk9BShpY5y3EpTpqLPIzIYBBcAoSQVmtoygls9oMxsdlvJ4BPhOmMs9wPxWtuO6uA5ZwsN1eZ+HO8tYlwHLwz75cwR1ixrbASyS1A/4rZn9UdIYYCjwbljeJJ2g0WnOryV9DlQSlKHuD/zZzMrD94uA+4HlBGNdvChpAxC5pLmZVUk6FNbZ+WO4je3hei8mzssJylXEjlB2h6R7CX7XVxEM0LOv0bIjwunbw+10J8ibcy3yhsKlih8DHwODCI6EmwxKZGa/kbQLGAdslvQDgrLKRWb2zxG2MT22gKCkZsc3CWsLfYOgyNydwFzglov4LCXAHcCHwGtmZgr22pHjJBjFbTHwHDBZUjbwEDDczGokFRIUvmtMwBYzm3oR8bouzrueXKrIAI6F4wfMJPg3fQFJ1wKHwu6WdQRdMFuB2yX1Cee5UtHHFP8QyJJ0Xfh6JvBW2KefYWYbCU4UN3fl0QmCsufN+S3wDwRjJJSE0y4qTjM7S9CFNCLstroCOAkcl9QX+F4LsewEvlX3mST1lNTc0Zlz9byhcKnieeBuSTsJup1ONjPPFOB9SWXAAIIhHw8Q7FD/S9I+YAtBt0yrzOwUQXXN1ZL2A+eBFQQ73fXh+t4iONpprBBYUXcyu9F6a4ADwNfMbHc47aLjDM99LAEeMrP3CMbH/gB4iaA7q85K4HVJpWZWRXBFVnG4nZ0EuXKuRV491jnnXFx+ROGccy4ubyicc87F5Q2Fc865uLyhcM45F5c3FM455+LyhsI551xc3lA455yL6/8BCJegqS+KdRoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7698058845975142" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_roc(glm_2, y_train, X_train[sig.index], \"Logistic Regression\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since the optimal cut off was found to be 0.2697615225249289 using the training data, we will use that as our cut off for our evaluation of the test set." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.86 0.86 0.86 6729\n", + " 1 0.54 0.55 0.54 2020\n", + "\n", + " accuracy 0.79 8749\n", + " macro avg 0.70 0.70 0.70 8749\n", + "weighted avg 0.79 0.79 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.2697615225249289)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Unfortunately, the training accuracy has gone down when we used the optimal cutoff. However, accuracy may be misleading in a dataset like ours where most of the targets are non-defaults. \n", + "\n", + "The recall here is more important - detecting defaulters is more useful than detecting non-defaulters. With a higher recall, our model with lower cutoff is able to correctly catch more defaulters.\n", + "\n", + "\n", + "Calculate the confusion matrices for both cut offs to better compare their performance." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Logistic Regression identified 1104\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PredictedFalseTrue
Actual
05790939
19161104
\n", + "
" + ], + "text/plain": [ + "Predicted False True \n", + "Actual \n", + "0 5790 939\n", + "1 916 1104" + ] + }, + "execution_count": 65, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test,glm_2.predict(X_test[sig.index])>0.2697615225249289, \"Logistic Regression\")" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Logistic Regression identified 716\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PredictedFalseTrue
Actual
06391338
11304716
\n", + "
" + ], + "text/plain": [ + "Predicted False True \n", + "Actual \n", + "0 6391 338\n", + "1 1304 716" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test,glm_2.predict(X_test[sig.index])>0.50, \"Logistic Regression\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is evident that the lower cutoff is better able to detect defualts." + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.21841387811546378\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e9NT0gCKXQICTX0AAGkiihSBEGkiIoKWBDsyA8rKKjYRV9Q7NhBQBBRQaoCghTpvUMIEAjpPdn7/jGbuAkpC2Sz2c35PA8P2ZnZmTNl58zce+eO0lojhBBCFMXF3gEIIYQo3yRRCCGEKJYkCiGEEMWSRCGEEKJYkiiEEEIUSxKFEEKIYkmiKIZS6i6l1B/2jsPelFIhSqlkpZRrGS4zVCmllVJuZbVMW1JK7VVK9biK713VMaiU8lZK/aKUSlBKzb/S75cVpdTvSql7r+J73ZRSB20RU3lmj98igHKU5yiUUieA6kAOkAwsAx7RWifbMy5nZN7W92utV9oxhlDgOOCutc62VxzmWDTQSGt9xMbLCaWU1lkpNRJ4FOhcGtvPnOS+1VrXudZ5XeXyS2UfKKVeAp4HMoBsYB8wQWu98ZqDdGKOdkcxQGvtC0QAbYBn7RzPVbHnVbKzXKFfiQq6vesBh64mSVSAY2Se+TwSDKwBbHLH5VTbUWvtEP+AE8BNFp/fBH61+OwJvA2cAs4DswFvi/EDgR1AInAU6GMeXhn4HDgLnAFeAVzN4+4D1pv/ng28XSCmn4GnzH/XAhYCFzCuCh+zmO4lYAHwrXn59xeyfpWBr83fPwm8ALhYxLEB+D8gATgA3Fjgu8WtwwbgPeCSeVwDYDUQC1wEvgOqmKf/BjABaRh3bv8DQgENuJmnWQtMM883CfgDCLaI5x7zOsQCLxbcdwXW2xt4xzx9ArDePCx3mfea9+lF4HmL73UANgLx5vWeCXhYjNfAeOAwcNw87H3gtHkfbAO6WUzvCjxnPjaSzOPrAn+Z55Vi3h7DzdP3xzie4oG/gVYFjtVJwC6MK1c3y21gjn2rOY7zwLvm4afMy0o2/+uExTFonqY5sMK8L88DzxWyTV8GMoEs83zGYFwUvmDezjEYx1pl8/S523qMOYa/CplnDyCqiH1Y3LHrat6/FzF+F49w+bF0v/nvhsCf5uPgIsYJncL2QcF4zPvqJ3MMscDMImJ9CePOKPdzM/O8q1oMK27ftgW2m4+R+cA84BXLbWTe9+eAb6yY3ySM32wScBDz77qYYyR3X+Vuv1rAEvPxcAR4oMC6/mjeN0nAXiDyqs6/ZX3Cv9p/5P+h1QF2A+9bjJ9h3mCBgB/wCzDdYqMnAL0wfjC1gXDzuMXAx0AloBqwGXjI4iSbmyi6Y5xkcovrAjBOprXM89wGTAY8gPrAMaC3xQ7LAgaZp/UuZP2+xkg8fuaD4RAwxiKObOBJwB3jh5IABFq5DtkYxRBuGCfhhuZt4QlUxfghzihsWxdxcK7FOKE2Ns9vLfC6xQ8vGehq3hZvm9e9qEQxy/z92hgnlc7muHKX+al5Ga0xTrpNzd9rB1xnXqdQYD/whMV8NcYJNTB3ewN3A0Hm70zA+DF7mcdNxDimmgDKvLwgi3k1LHCyiAE6mmO+17zNPC223w6Mk5d3wW2KkeBGmv/2Ba4rbDsXcgz6YSTFCYCX+XNHK0+IozFOJPXNy/yJ/05kucv9GuMYKuz47EHRiaK4Y3csRvFOHYzfzEqKThQ/YBQLuZjXr2uB/dmwsHjM+2AnxsVQpYLfLWq7YByfr2Mkpdx4ity35ulPAo9j/A4HYyRky0SRDbxhnt67hPk1wTin1LLYDw2u5BjBSKwfmtc5AiNR3mixrulAP/OypwObrur8a+8EYHWgxsZNxsiMGljFf1fBCuNqo4HF9J3470ryY+C9QuZZHePkY3nnMQJYU8iPVGFcbXU3f34AWG3+uyNwqsC8nwW+tNhhl12lWUzrao6jmcWwh4C1FnFEY05S5mGbgZFWrsOpopZtnmYQsL3Ati4pUbxgMX4csMz892TgB4txPhg/pssSBcYJIQ1oXci43GXWKbDOdxSxDk8Aiyw+a6BnCesdl7tsjKu5gUVMV/Ak9REwrcA0B4HrLbbf6EKO39xE8RfGVX9wgWnybedCjsERlvuphHV7ifyJYhUwzuJzE4wE7max3PrFzK8HhSQKK47d1ZgvWsyfbyrkWMpNFF8Dn1ju82L2QV48GL/1C5bbrYTtkolxdZ+DcffRw5p9i3GxeIb8v8P15E8UmZgvPqyYX0OMJHITRr2U5TQlHiMYFyI5gJ/F+OnAHIt1XWkxrhmQZs3xU/Cfo9VRDNJa+2HskHCMMkYwrop9gG1KqXilVDxGZXdV8/i6GFfABdXDuDI4a/G9jzGuyvPRxpaei/FjBbgTo8gmdz61cudhns9zGCfxXKeLWa9g/rtayXUS4yo71xlzDJbja1m5DvmWrZSqppSaq5Q6o5RKxCgSC+bKnLP4OxXjqgdzTHnL01qnYvwYCxOMcSVU2L4pdjlKqcZKqaVKqXPmdXiNy9eh4HpPUErtN7cEiscoMsn9TlHHSGHqARMK7O+6GOte6LILGINxN3ZAKbVFKdXfyuVeSYwF1eLy48sN64/RopR07OY7HkpYxv8wLsg2m1uJjbYyhrrASW19fcyPWusqGOu+B+PuNFdx+7YWl/8OC67PBa11ujXz00bF/BMYJ/QY828y9xiy5hipBVzSWidZDCt43ij4+/G6mroTR0sUAGit/wTmYBRrgHHrmAY011pXMf+rrI0KKzB2ZoNCZnUa42oo2OJ7/lrr5kUs+gdgiFKqHsZdxEKL+Ry3mEcVrbWf1rqfZdjFrNJFjKu7ehbDQjCuXnLVVkqpAuOjrVyHgsuebh7WSmvtj1Eko4qZ/kqcxShmAIxmmhjFPYW5iHFrXNi+KclHGHU1jczr8Bz51wEs1kMp1Q2jPHgYEGA+USRYfKeoY6Qwp4FXC+xvH631D4UtuyCt9WGt9QiMZP4GsEApVam471xFjAVFc/nxlY1R/p0X2lXMt6RjN9/xgHGSLJTW+pzW+gGtdS2Mu5IPlVINrYjhNBBypSdArfVF83JeUkrVtJhXUfv2LJf/DguuT8FtWOyxorX+XmvdFWP7aYzjobhjxFI0EKiU8rMYVvC8USocMlGYzQB6KaUitNYmjLLs95RS1QCUUrWVUr3N034OjFJK3aiUcjGPC9dan8WoiH1HKeVvHtdAKXV9YQvUWm/HuMX9DFiutY43j9oMJCqlJpnbr7sqpVoopdpbsyJa6xyMSqdXlVJ+5kT0FMaVfq5qwGNKKXel1FCgKfDbla6DmR9GMV68Uqo2Rvm8pfMYZdlXYwEwQCnVWSnlgXH7XPAEDoB5v30BvKuUqmXebp2UUp5WLMcPo6IvWSkVDjxsxfTZmIsolFKTAX+L8Z8B05RSjZShlVIqN8EV3B6fAmOVUh3N01ZSSt1S4AdbJKXU3Uqpqub1zz2GcsyxmSh62y8FaiilnlBKeZqPlY7WLBPjIudJpVSYUsoX4w5s3hVchefG7mX5zxxvccfuj8Dj5t9cFYxkXdS8hyqlcpNKHMaJM8f8ubhjcjPGSfx1877wUkp1sWZ9tNYHgOUYdzNQ/L7daI7nEaWUm1JqIEb9Z3GKnJ9SqolSqqf5eE/HuNjNMW+Loo4Ry9hPY1SOTzevcyuMO5HvKGUOmyi01hcwyjRfNA+ahFFZt0kZRRErMcph0VpvBkZhVHYlYFQA5V4B3YNx67wP4+BcAOReXRTmB4wyxe8tYskBBmBUJh3HuMr6DKNow1qPYtSzHMMo9/we4ySa6x+gkXnerwJDtNa5RTpXug4vY1SyJQC/YlRsWpoOvGC+VX76CtYBrfVe87rMxfjxJmGUw2YU8ZWnMSqRt2C03HgD647LpzGK/5IwfozzSph+OfA7RkXrSYwfpmWxwbsYJ7U/MBLQ5xiVkWAUDXxl3h7DtNZbMeqoZmJs7yMYdQnW6gPsVUolY7TEukNrnW4upnsV2GBe1nWWXzIXMfTCONbOYbTousHKZX6B0aLtL4xjNB1jP12J2hgnM8t/DSj+2P0UY5vuwmgt9BtGws530jNrD/xj3i5LgMe11sfN417CYh9Yfsni99cQox4xCqPBh7XeAh5USlUrbt9qrTMxKrDHYJy878ZI3kUd25RwrHjyX2X6OYyLwefM4wo9RgpZxAiMeotoYBEwRWu94grW3SoO88BdRaaUug+jwq+rvWO5Uuar13iMIqLjJU0vnJtSqi8wW2tdr8SJHYBS6h+M9fnS3rHYksPeUYjySyk1QCnlYy5TfRvjjuGEfaMS9mAuiu1nLqqpDUzBuPJ1SEqp65VSNczrcy/QCqPhjFOTRCFsYSDGrXA0RnHZHVpuXSsqhVHUGYdR9LQfowm1o2qC8cxGAsbzLEPM9YROTYqehBBCFEvuKIQQQhTL4TqtCg4O1qGhofYOQwghHMq2bdsuaq2rljzl5RwuUYSGhrJ161Z7hyGEEA5FKXWy5KkKJ0VPQgghiiWJQgghRLEkUQghhCiWJAohhBDFkkQhhBCiWJIohBBCFMtmiUIp9YVSKkYptaeI8Uop9YFS6ohSapdSqq2tYhFCCHH1bHlHMQejq9yi9MXoB6gR8CDGi2iEEEKUsvSswnp1t57NHrjTWv+llAotZpKBwNfmzuI2KaWqKKVqVoQOtoQQ4lrlmDRRcakkZ2RzJCYZk9bEJGZwNsF4bcWxiykkp2fx76n4EuZUMns+mV2b/C+OiTIPuyxRKKUexLjrICQkpEyCE0IIe0pIy2Ln6XjSsnLIyjGRmW3i0PlkMrJz2HYyjl1RCUV+18PVBR93VxJSM0k7kYifvzUvjSyaPRNFYa/HLLQrW631J8AnAJGRkdLdrRDCKVxMzuB8YjoZ2SbOJaTz079RHDqfzLnEdDKzTUV+z8/TjaY1/Wlaw4+bmlXH3dWFhtV88XJ3obK3O97urkRGfsqFgxeZOvUGHnusI+6fX32c9kwUUeR/MXkdjPcXCCGEw4tNzmDD0VjSM3NIz85hx+l4An08SMnMYd3hC0TFpRX6PT8vN5pU9yMyNIAGVX1pWtMPfy933F1dcHdzobqfJ26uhVcv//33aVq2rIbycOOzzwYQHOxD3bpX8kbmwtkzUSzBeEn5XKAjkCD1E0IIR2Qyaf7Yd56ouFS+3HCCc4np5JgKL/zwcHXBpDUtavvTqk4Vmtb0p26AN55urtSvWonq/l5XvPzY2FSeeWYln322nSlTruell3rQpk3Na12tPDZLFEqpH4AeQLBSKgrjFYjuAFrr2RgvWe+H8bLxVGCUrWIRQojSciklk+T0bBLTszgRm8LWE3Es3nGG+NSsvGmq+LhzfeOqRNYLoGfT6ni6ueDj4YqPR+mecrXWfP31Tp5+egVxcWlMnNiZiRM7l+oywLatnkaUMF4D4221fCGEuFrZOUbF8Q+bT3EmPo1Tl1JxVYrzSen5EoKlYZF1eLhHQ4J8PfD3ci+TOCdNWslbb/1N5851mT37Flq2rG6T5Tjc+yiEEKK0RMensf9sIusOX+TguSTOJqRxIjb1sumq+nnSqJov4TX98HRzoX1oIF7urni6udCslj+1q3ijVGHtc0pfWloWKSlZBAf7MGZMGxo1CmTMmLa4uNhu+ZIohBBOa/3hi8QkpbPzdDzuri5kZJvYfSaBmMR0os3PG+RyUeDn5c4NTapSv6ovQb4eXFc/iLYhAXaK/nLLlh1h/PjfiIiowcKFw2jSJJgmTYJtvlxJFEIIh6e15mRsKltPxvHlhuOcT8zgYnLGZdMFVvLAzUWRmWPiuvqBRNQNoHWdyjSq7kfDar52iNw60dFJPPHEMubP30eTJkE88kj7Ml2+JAohhMOKSUrnm40n+ervEySmZ+cbN6pLKFrDbW1qUzvAm2Dfa3vozF5WrTrGbbfNIzMzh2nTbmDixM54epbtqVsShRDCYWitOXQ+mT8PxfDxn8eITcnMGzekXR16NatO+9BAAit52DHK0pGVlYO7uyutW9egX79GvPJKTxo2DLRLLJIohBDlgtaa05fSOHQ+iZTMbM4nprP6QAzHLqTg4eZCVo6J84n5i5PqV63E4zc2onfzGni5u9op8tKVmJjBiy+u5p9/zrBhw2iCg32YO3eIXWOSRCGEKDMmkyYj28T5xHQWbT9DamY2W07EsTc6gaycwh9QC6zkQaMgH+pU8cHT3QV3Vxe6NQqmfWgglcq4CMaWtNYsWLCPxx9fxrlzyYwb156MjBx8fOz/2iDn2cpCiHIlPSuHj9YeJSvHxNYTcZxLTOfUpcubngb4uBPg40G7egGEBVciom4ValT2ItjXE39vd3ydKBkU5cKFFO69dzG//36ENm1q8PPPd9C+fW17h5XH+feAEMLmsnNMHLuYwpcbTrDvbCI7T+fv2trHwxV3Vxeuqx/I9Y2r4e3uQt1AH3qGVyuz5w/KM39/Ty5eTGXGjN6MH98BNzf730VYkkQhhLgi6Vk57D+byPurDrPtRBxpWTlkF+jXqENoINX8PWldpwqjuoQW2YldRfbXXyd59dV1LFw4DF9fDzZtut+mD81dC0kUQogiRcWlcjYhndOXUpm/NYqNx2Lzjfdyd6FDWCAdwgJxVYoWtSvTtl4Alb3LpgsLR3TxYioTJ65gzpwdhIZW4cSJeFq0qFZukwRIohBCFGAyaX7eeYapv+wjrkC/Ro2r+xJew5/2YYG0rlOZVnWq2ClKx6O15ssvdzBx4goSEzN49tmuvPBCd3x8yn9SlUQhhCA9K4e/j15k9tpjbD5xKW94rcpeTOobTu0q3tSs4k3tKt52jNLxffvtLpo1q8rs2bfQvHk1e4djNUkUQlRg20/FMWnhLg6dT84b1qS6H2HBlXh3eOtS7xa7oklNzeK119Yxdmwkder4s3DhMCpX9irXxUyFkaNAiAokx6Q5fjGFd/44yD/HL3HJ/GRzJQ9XHr2xEbe2rkUtuWsoFb/9dpjx43/jxIl4atf24+GH2xMQ4JjbVhKFEE4sIzuHVftjWLAtiiMxyZc9x9ClYRCP3NCITg2C7BSh84mKSuSJJ5axcOF+mjYN5s8/76N793r2DuuaSKIQwkmYTJpjF5M5diGFv4/GsjMqnu2n/nuewcvdhXb1AuhUP4j2YYF0bRiMq4MVgTiCV1/9i19/Pcxrr/VkwoTOeHg4ftciynjRnOOIjIzUW7dutXcYQthVQmoW+88lcjI2hUPnjTuFFfvOXzZd7Sre3NKqJv1b1ZQWSja0efMZvL3daNmyOrGxqSQkZFC/fvl5jwWAUmqb1jryar4rdxRCOIi0zByW7T3Lk/N2Fjq+WU1/QoN9GBZZl9pVvGlU3a+MI6x4EhLSee65VXz00Vb692/MkiUjCAryISjIx96hlSpJFEKUYyaT5vc955gwfwfpWaa84bWrePNwjwa0qF2ZmpW9qObnKV1hlCGtNfPm7eXJJ5cTE5PCo492YNq0nvYOy2YkUQhRTiSmZ3H6Uip/H4nlwLkkNh2L5UJyBpnZRoLo36om7UMDubFpNeoEONcVq6P59ttd3HPPYiIja7F06Qjatatl75BsShKFEHaUmW1i9JwtrD9ysdDxbUOqcGvrWvRpUZMalb3KODphKSMjm2PH4mjatCrDhjUnO9vEPfe0xrUC9GMliUKIMpadY2JPdCILt0XxzaaTAPh6utGnRQ06hAVSzc+TLg2Dca8AJyBHsWbNcR5++FdSU7M4fPhRPD3dGDWqjb3DKjOSKISwsZSMbFYdiGHx9jOsP3yRzJz/6hpa1q7MwIha3N+tvh0jFEWJiUnh6af/4JtvdlG/fgCffDKgzN9XXR5UvDUWogzEpWSyeMcZZq05ysXk/17fWTfQ6C/p5mY1aFcvgNZ1pclqeXXkyCU6dPiU5ORMnn++G88/3w3vCtorriQKIUrRin3n2Xk6nplrjuQNCwn0YVSXUHo3ryHdYziAxMQM/P09adAggDFj2jB6dBuaNq1q77DsShKFEFchM9vE+iMXOHYhhc3HL+Hh5sLSXWfzTTOuRwMe6dlQOtZzECkpmUyd+ieffvovu3Y9TJ06/rz11s32DqtckCNYiCuQmJ7Fy0v2sfDfqHzD3VwULWtXprK3O9MGtaBmZS+83B2/64aK4pdfDvLII79z6lQCY8a0cYh3RJQlSRRCWCHHpHnnj4N8uPZo3rC7rwvhvs5hhAVXkj6THFR2tolhw+azaNEBmjevyrp1o+jaNcTeYZU7kiiEKMbh80nc+dk/XEj6r0L6xf7NGNM1zI5RiWultUYphZubCzVr+vL66zfy5JOdnKIDP1uQRCFEAVprtpyI44fNp1i0/Uze8D7Na/DBiDZ4uMnzDY5s06Yoxo//jU8/HUDbtjWZNesWe4dU7kmiEMLC3ugEbvlgfd7nWpW9mDygGX1a1LRjVKI0xMWl8dxzq/j4423UquVHXFyavUNyGDZNFEqpPsD7gCvwmdb69QLjQ4CvgCrmaZ7RWv9my5iEsJSVY2LV/hg+W3eMvdGJpGXlADCiQ13G39BQ+lRyEvPm7eGxx5Zx8WIqTzxxHS+/3AM/P097h+UwbJYolFKuwCygFxAFbFFKLdFa77OY7AXgR631R0qpZsBvQKitYhIiV3R8Gl+sP85n64/nDatV2YvudYIZGFGbfi3lDsKZHDhwkdDQKixbdhdt2si+vVK2vKPoABzRWh8DUErNBQYClolCA/7mvysD0TaMRwi01tz35Rb+PHQhb9iQdnV4uEcDGlT1tWNkojSlp2fzxhvradu2JgMGNOG557rxwgvdK0QHfrZgy0RRGzht8TkK6FhgmpeAP5RSjwKVgJsKm5FS6kHgQYCQEGm6Jq5Odo6JUXO2sO6w0VPrZ/dE0r1xVamcdjIrVx5j3LhfOXz4EhMmdGLAgCa4yzMt18SWiaKwhuUF37s6ApijtX5HKdUJ+EYp1UJrbcr3Ja0/AT4B41WoNolWOB2tNecTM1i0/QzfbjrJmXij8jIk0Ie1T/fARZ59cCrnzyfz1FN/8P33u2nYMJA//ribXr0a2Dssp2DLRBEF1LX4XIfLi5bGAH0AtNYblVJeQDAQY8O4RAWw50wC9325JV+HfO1DA7g1ojZ3dwyRt8E5oRUrjrFgwT4mT+7Os892w8tLGnWWFltuyS1AI6VUGHAGuAO4s8A0p4AbgTlKqaaAF3ABIa6C1prle88xa81Rdp9JAKBvixr0bVmTXk2r4y0PUzmdnTvPcfjwJYYMacZdd7WkS5e6hIUF2Dssp2OzRKG1zlZKPQIsx2j6+oXWeq9SaiqwVWu9BJgAfKqUehKjWOo+rbUULYkrcjI2hTeWHWD7qXjOJqQDRvHStEEtuL5xxe7101klJ2cyZcoa3n//H0JDqzBoUDhubi6SJGzEpvdm5mcifiswbLLF3/uALraMQTgnk0nz4s972H0mgV1RCXnD+7Wswf3d6tM2RE4Yzmrx4gM8+ujvREUl8uCDbZk+/SbcpEGCTUkhnnA48amZRExdkff5lpY1ub5xVYa1r1vMt4Qz2L37PLfdNo+WLasxb94QOneWfV4WJFEIh3ExOYNnFu5i5f7/2jocfa2f9Nzq5LKycli37hQ9e4bRsmV1fv31Tnr1qi9NXsuQJApRrqVn5fD8oj3sjU7gwLmkvOGv3taCEe1DpImrk/v779OMHbuUvXsvcPDgIzRsGEi/fo3sHVaFI4lClFspGdm0mbaCzGzjsZoRHULo1iiYPs1rSIJwcpcupfHMMyv59NN/qVvXn59+GkbDhoH2DqvCkkQhypXo+DT+PhrL4u1nWH/EeIK6dd0qLB7XWZ59qCDS07OJiJhNdHQSEyZ04qWXeuDr62HvsCo0SRSi3FiyM5rHftieb9gLtzRldJcwSRIVQFRUInXq+OPl5ca0aTcQEVGD1q1r2DssgSQKUQ4cOJdInxnr8j5/dFdbrqsfREAluYqsCNLSspg+fT1vvLGBBQuGMmBAE+69N8LeYQkLViUKpZQHEKK1PmLjeEQFsv1UHP9bsIvDMckAeLu7snh8F5rU8LNzZKKs/PHHUcaN+5WjR+O4++5WdOhQ294hiUKUmCiUUrcA7wIeQJhSKgKYorW+zdbBCec19Zd9fLHBeBeEm4viq9Ed6NIw2M5RibL06KO/MXPmFho1CmTlypHceGN9e4ckimDNHcVUjO7B1wBorXcopRraNCrhtLJzTHyw+khekvhzYg/qBVWyc1SirOTkGC3YXF1duO66OgQH+zBpUlfpwK+cs2bvZGmt4wtUJkp/TKJERid95zkTn8a2k5eIjk9nx+n4vPE/jessSaIC+fffs4wdu5SRI1vx6KMdueuuVvYOSVjJmkSxXyk1DHAx9wT7OLDJtmEJR6W15uiFZOZvi+LjP49dNj7Ax51hkXV5uncT3OVtYxVCUlIGkyev4YMPNlO1qg81a0odlKOxJlE8AkwGTMBPGL3BPmvLoIRjGvXlZtYczN9L/OguYYzuGkp1fy9JDBXQH38cZfTon4mOTmLs2Ehee+1GqlTxsndY4gpZkyh6a60nAZNyByilBmMkDVHBmUyaqUv3sf10PDvNxUqT+zejbb0Awmv44SX98VRoHh6uVKtWiYULh9GxYx17hyOukirp9Q9KqX+11m0LDNumtW5n08iKEBkZqbdu3WqPRQsLsckZvLHsAD9ujcobNqRdHR7sXp/G1aVooaLKysrh3Xc3kpiYwauv3ggYFxPS5Yr9mc/bkVfz3SLvKJRSvTFeU1pbKfWuxSh/jGIoUcEcvZDMvC2nmbPhBJk5/x0CD3QLY8LNTeTuoYJbv/5UXgd+Q4c2y0sQkiQcX3FFTzHAHiAd2GsxPAl4xpZBifJl7uZTPPPT7nzDBkXU4obwatzSsiZuUvdQocXGpjJp0ko+/3w7ISGV+eWXEfTv39jeYYlSVGSi0FpvB7Yrpb7TWqeXYUyinIhJSuf33eeYssS4TqhV2YsX+zfjhvBqcvcg8sTGpjF37h7+97/OTJ58PZWk6xWnY01ldm2l1KtAMyCvuYLWWi4ZnNT5xHQG/BilNlYAACAASURBVN96YpIy8oYtfbQrLWpXtmNUojzZv/8CP/64lylTetC4cRCnTj1JYKC3vcMSNmJNopgDvAK8DfQFRiF1FE4nOSObmauPsHzvOY5fTAGgbqA3z/VtSrvQAKr5SZNGAampWbz66l+89dbf+Pp6MGZMW+rU8Zck4eSsSRQ+WuvlSqm3tdZHgReUUutK/JZwKN3fXMOllEwA2tUL4IFu9enTQrp4Fv9ZtuwI48b9yvHj8dx7b2veeqsXVavKk/UVgTWJIkMZ/XccVUqNBc4A1WwbligLWmv+PRXHPZ9vJiUzh2BfT1Y+1Z0qPlLGLPJLTs5k5MhFBAV5s2bNvfToEWrvkEQZsiZRPAn4Ao8BrwKVgdG2DErYXkpGNoNmbcjr4tvDzYW1E3vg6ymdswlDTo6JH37Yw4gRLfD19WDlypGEhwfjKcdIhVPiHtda/2P+MwkYCaCUkkcsHVRqZjZPz9/Jb7vPAVAvyIeZI9rSora/vEVO5Nm2LZqHHlrKtm1n8fZ24/bbm8nb5iqwYhOFUqo9UBtYr7W+qJRqjtGVR09AkoUDWXf4Ai//so8j5juIekE+PH5jI25rU1sShMiTkJDOiy+uYdasLVSrVom5c29n8OCm9g5L2FlxT2ZPB24HdmJUYC/C6Dn2DWBs2YQnrkVCahbzt53ml53R7IxKyBs+dWBz7ukUar/ARLl1++0/snr1ccaPb88rr/SkcmVp7SaKv6MYCLTWWqcppQKBaPPng2UTmrhaWmveWn6QD9cezRsWXsOPh3s0YGCEvGpS5HfsWBxVq/rg5+fJq6/2xMVF0b69HCfiP8UlinStdRqA1vqSUuqAJIny7+3lB5m55r9Xmz/XL5yh7eoSIE/LigIyM3N4++2/mTbtLx57rANvvNFLengVhSouUdRXSuV2Ja6AUIvPaK0H2zQycUVikzN4+Zd9LNkZDcDI6+ox4ebG0tRVFOqvv04yduxS9u+/yJAhzXjssY72DkmUY8UlitsLfJ5py0DE1fv4z6NM//0AAN0aBfPO0NZU85eyZVG4997byFNP/UFoaBV+/fVO+vVrZO+QRDlXXKeAq8oyEHFlMrNNfL3xBHP+PkFUXBoAz/drygPd69s3MFEumUyalJRM/Pw8ueWWxly4kMoLL3THx8fd3qEJByBPzjiYw+eTeOePQyzbey5vWI8mVZl9dzvp0VUUau/eGMaO/TXvTXONGwfx2ms32jss4UBsmiiUUn2A9wFX4DOt9euFTDMMeAnQwE6t9Z22jMmRrT98kbs/N55/rF+1Ej2bVGPCzU3w9pAEIS6XmprFtGl/8vbbG6lc2ZPRoyPQWstzM+KKWZ0olFKeWuuMkqfMm94VmAX0AqKALUqpJVrrfRbTNAKeBbporeOUUtKHVCFMJs3EBbtY+K/x2tEXbmnK/d2kiEkUbfv2swwe/CMnTsQzalQEb77Zi+BgH3uHJRxUiYlCKdUB+Byjj6cQpVRr4H6t9aMlfLUDcERrfcw8n7kYz2bss5jmAWCW1joOQGsdc+Wr4NwOnU/i5vf+AsBFwey723Fzc+lKQRQu944hJKQyISGV+eqrQXTvXs/eYQkHZ80dxQdAf2AxgNZ6p1LqBiu+Vxs4bfE5CijYBq8xgFJqA0bx1Eta62VWzNvpFXxorn+rmswYHiGvHRWFys42MXPmZpYsOciKFSMJCvLhzz/vs3dYwklYkyhctNYnC5Rr5ljxvcIKQnUhy28E9MDoO2qdUqqF1jo+34yUehB4ECAkJMSKRTs2rTWDP/qb7aeMzSBFTaI4mzefYezYpWzffo6+fRuSmJhBQIC8SEiUHmsSxWlz8ZM21zs8Chyy4ntRQF2Lz3UwugEpOM0mrXUWcFwpdRAjcWyxnEhr/QnwCUBkZGTBZONU3l1xiA9WHQYgom4VvhrVgcrShFEUIjk5k0mTVvDRR1upWdOP+fOHcvvtTaWyWpQ6a8oxHgaeAkKA88B15mEl2QI0UkqFKaU8gDuAJQWmWQzcAKCUCsYoijpmXejO590/DvLBqsMoBfd1DmX+2E6SJESR3N1dWLv2JI8+2oH9+8czZEgzSRLCJqy5o8jWWt9xpTPWWmcrpR4BlmPUP3yhtd6rlJoKbNVaLzGPu1kptQ+jOGui1jr2SpflDP46dIEPVht9NB2c1hcPN6mLEJc7cuQSU6f+yaxZ/fDz82Tbtgfx8pLHoYRtKa2LL8lRSh0FDgLzgJ+01kllEVhRIiMj9datW+0ZQqnbcyaB/v+3HoCFD3emXb0AO0ckypuMjGzefHMDr766Dg8PV3799U66dZPWTMJ6SqltWuvIq/muNW+4a6CU6oxRdPSyUmoHMFdrPfdqFij+o7XmzeUH+cjcsmnG8AhJEuIya9Yc5+GHf+XgwViGD2/Ou+/2plYtP3uHJSoQq+5ZtdZ/A38rpV4CZgDfAZIorlJ2jokftpzm7eUHSUjLAmDawOYMaiPvABD5aa159dV1ZGWZWLbsLnr3bmjvkEQFZM0Dd74YD8rdATQFfgY62zgup3XwXBK9Z/yV97l9aAA/PHCdPB8h8phMms8//5c+fRpSt25lvvnmNqpU8cLbWxo2CPuw5o5iD/AL8KbWep2N43FqWmteWLwbgBEdQni2Xzj+XvLjF//Ztes8Y8cuZePGKCZP7s7LL99AzZpSzCTsy5pEUV9rbbJ5JE5u8/FLPPXjDqLi0ujbogbTB7e0d0iiHElOzuTll9fy3nubCAjwZs6cgdxzT2t7hyUEUEyiUEq9o7WeACxUSl3WNErecGe9hNQshn28Me/za7dJkhD5vfTSWt55ZyP339+G11+/iaAg6cBPlB/F3VHMM/8vb7a7Blk5Jnq8vQaAt4a0Ymhk3RK+ISqK06cTSEnJIjw8mGee6cqgQeF07er8XdQIx1NkDarWerP5z6Za61WW/zAqtYUVRs/ZQlxqFpH1AiRJCMDowO/ddzfStOksHnpoKQDBwT6SJES5ZU1Tm9GFDBtT2oE4o9d/P8C6wxcBmPdQJztHI8qDTZuiiIz8hAkT/qBHj1C++mqQvUMSokTF1VEMx2gSG6aU+slilB8QX/i3RK7JP+/h640nAVj3vxtwdZE+eCq6X389xIABP1Crlh8//TSMQYPCpW8m4RCKq6PYDMRi9Po6y2J4ErDdlkE5Mq01o+ZsYe3BC4CRJOoGSsVkRaW1Jjo6idq1/bnppvpMnXoDjz/eET8/T3uHJoTVSuzrqbwpz3097Y1OYOjsjaRmGq/rWD3heupX9bVzVMJeDh2KZdy4Xzl0KJZ9+8bj6+th75BEBWaTvp6UUn9qra9XSsWR/4VDCtBa68CrWaCzSs3M5pYPjI79Rl5Xjxf7N5MeYCuo9PRsXn99PdOnr8fb243p02/E21t6eBWOq7ijN/d1p8FlEYgjy84xMXqO8a6lW1rVZNqgFnaOSNjLuXPJdO/+JYcPX2LEiBa8+25vatSQu0rh2IpMFBZPY9cForXWmUqprkAr4FsgsQzicwiTl+xl07FLvHxrc+7tHGrvcIQdZGXl4O7uSvXqlejevR6zZvWjV68G9g5LiFJhTdnIYozXoDYAvsZ4huJ7m0blILTWTFu6j+//OYWbi+K2ttL7a0VjMmlmz95KgwYfEBWViFKKzz67VZKEcCrWFJyatNZZSqnBwAyt9QdKqQrf6klrzVM/7mTR9jNU9fNkzdM98PWUcuiKZOfOczz00FL++ecMPXuGkZWVY++QhLAJq16FqpQaCowEcp8OqtBdnn6z6SRv/H6A5IxsHugWxrN9m+Iiz0lUGFprJk5cwYwZmwgM9Oabb27jrrtayjMRwmlZkyhGA+Mwuhk/ppQKA36wbVjl1w+bT/Hi4j0APHR9fSbe3ESSRAWjlCIuLo0xY4wO/AICvO0dkhA2ZdVzFEopNyD31VpHtNbZNo2qGPZ6jkJrzew/j/HGsgMAbH+xFwGVpF18RXHyZDyPP76MyZOvp23bmphMWi4QhEO5lucoSqzMVkp1A44AnwNfAIeUUl2uZmGObN3hi3lJYsqAZpIkKoisrBzefHMDzZp9yIoVxzh40Oi7S5KEqEisKXp6D+intd4HoJRqCnwDXFVmclTvrDgEwKoJ19NAnrauEP7++zQPPbSUPXtiGDiwCR980JeQkMr2DkuIMmdNovDITRIAWuv9SqkKdTm9bM9Zdp6OZ0DrWpIkKpCVK4+RkJDO4sXDGTgw3N7hCGE3JdZRKKXmABkYdxEAdwE+Wut7bRta4exRR9Hh1ZXEJGXwz3M3Ut3fq0yXLcqO1ppvvtlF1ao+9O3biIyMbLKyTNJHk3AKNq2jAMYCR4H/AZOAY8BDV7MwR/TzjjPEJGXQtWGwJAknduDARXr2/Jp7713Ml1/uAMDT002ShBCUUPSklGoJNAAWaa3fLJuQyo8tJy7x+FzjpCH9NzmntLQsXnttHW+8sYFKlTz4+OP+3H9/W3uHJUS5UuQdhVLqOYzuO+4CViilCnvTndP6+8hFhs7eCMAtLWsSFlzJzhEJW/jll0O88so6hg9vwYED43nwwXbSokmIAoq7o7gLaKW1TlFKVQV+w2ge6/Q++esor/1mNIX98K629GtZ084RidJ07lwyO3aco0+fhgwd2ozQ0Pvp0EH66RKiKMXVUWRorVMAtNYXSpjWKeSYNO+tOJSXJN6/I0KShBPJyTHx4YdbaNJkJiNHLiItLQullCQJIUpQ3B1FfYt3ZSuggeW7s7XWg20aWRnTWtNiynLSzB27yStMncu//55l7NilbNkSzU031efDD/vh7V2huywTwmrFJYrbC3yeactA7O2FxXtIy8ohvIYfn94TKUnCiRw/HkeHDp8SHOzD998P5o47WkgHfkJcgeJeXLSqLAOxp+2n4vjun1MALB7fBS93VztHJK6V1prdu2No1ao6YWEBfPnlQAYMaEKVKtLEWYgr5fT1Dtb4ZtNJAGbf3VaShBM4fjyO/v1/oE2bj9m16zwAI0e2liQhxFWyaaJQSvVRSh1USh1RSj1TzHRDlFJaKVXm/UeZTJrVB2IA6NNCKq4dWWZmDq+/vp7mzT/kzz9P8PbbvWjWrKq9wxLC4Vn9SjallKfWOuMKpncFZgG9gChgi1JqiWW/Uebp/IDHgH+snXdp+mn7GeJTsxjSro49Fi9KSU6Oic6dP2fbtrMMHtyUGTN6U7eudOAnRGmwppvxDkqp3cBh8+fWSqn/s2LeHTDeXXFMa50JzAUGFjLdNOBNIN36sEuH1pqn5+8EYFIf6fTNESUmGtcurq4ujB7dhl9+GcHChcMkSQhRiqwpevoA6A/EAmitdwI3WPG92sBpi89R5mF5lFJtgLpa66XFzUgp9aBSaqtSauuFCxesWLR13l91GIA+zWtQ1c+z1OYrbE9rzZw5O6hf/31+/tl47mXcuPb079/YzpEJ4XysSRQuWuuTBYZZ8xb5wtof5nVVq5RywXjXxYSSZqS1/kRrHam1jqxatXTKnDOyc5ix8jCuLoqP7pa+fRzJvn0X6NHjK0aN+pnw8GAaNAi0d0hCODVr6ihOK6U6ANpc7/AocMiK70UBdS0+1wGiLT77AS2AteY27TWAJUqpW7XWNu9HfM+ZBADu6hgibeodyJtvbuD551fj7+/JZ58NYNSoNtI3kxA2Zk2ieBij+CkEOA+sNA8ryRagkVIqDDgD3AHcmTtSa50ABOd+VkqtBZ4uiyQB8N0m47mJcT0aljClKA+01iilqFHDl7vuaslbb/WialXpqFGIslBiotBax2Cc5K+I1jpbKfUIsBxwBb7QWu9VSk0Ftmqtl1xxtKUkPSuHn7afIaiSBzUqS9v68iw6OonHH19Gt24hPPZYR+65pzX33NPa3mEJUaGUmCiUUp9iUbeQS2v9YEnf1Vr/htHrrOWwyUVM26Ok+ZWWKT/vBWBQG+kMrrzK7cDv+edXk5VlonNnab4shL1YU/S00uJvL+A28rdmcihaa+ZtNcJ/vl9TO0cjCrNjxznuv38J27ad5eabG/Dhh/2kwloIO7Km6Gme5Wel1DfACptFZGNLdhr16S/c0lQqQcuphIR0oqOTmDdvCEOHNpPGBkLYmdVPZlsIA+qVdiBl5ZuNRkvfgRFS7FReaK2ZP38fhw/H8vzz3bn++lCOHXscL6+rOTyFEKXNmiez45RSl8z/4jHuJp6zfWil70x8GltPxgEQ7Oth52gEwNGjl+jX73uGD1/Azz8fJMv8PhBJEkKUH8X+GpVxz98ao3krgElrfVnFtqOYvHgPADOGR0hxhp1lZGTz9tt/88or63B3d+H99/swblx73NykQ2MhyptiE4XWWiulFmmt25VVQLZ0OCYZTzcXae1UDpw+nci0aX8xYEATZszoTe3a/vYOSQhRBGsu3zYrpZyij4tTl1K5qVl1e4dRYV24kMLMmZsBaNgwkH37xjN//lBJEkKUc0XeUSil3LTW2UBX4AGl1FEgBaMPJ621dqjkcT7R6Jy2nrzitMyZTJovv9zO//63kqSkDHr1qk+TJsHUrx9g79CEEFYoruhpM9AWGFRGsdjUgm1RAHSsH2TnSCqWPXtiePjhX1m//hTduoUwe3Z/mjQJLvmLQohyo7hEoQC01kfLKBabemv5QQBa15H3FJSVzMwcbr75GzIzc/jii1u57z5pRCCEIyouUVRVSj1V1Eit9bs2iMcmYpKMYicPVxeq+EizWFtbvfo4119fDw8PV378cSjh4cEEB0uRnxCOqrjKbFfAF6M78ML+OYxtJ4xnJ166tbmdI3FuUVGJ3H77j9x449d8/bXx5sCuXUMkSQjh4Iq7ozirtZ5aZpHY0Mw1RwDo1kjKxm0hO9vEzJmbefHFNeTkmJg+/UbuuquVvcMSQpSSEusonMHe6ERcFNSVFk82MXLkIubO3UPfvg2ZNasfYWHSmkkIZ1JcorixzKKwodz6ia6NSucVqsIQH5+Om5sLvr4ejB/fnttvb8rttzeVymohnFCRdRRa60tlGYitHIlJBmBAq5p2jsQ5aK2ZO3cPTZvO4sUXVwNGPcSQIdLLqxDOyuk71slNFA2q+do5Esd35Mglevf+lhEjFlKnjj933y31EEJUBE7fReemY7EAhAXJ+5Wvxfff72b06J/x9HRj5sy+jB0biaur019nCCGoAIli9YEYQoN8CKgkz09cjaysHNzdXYmMrMWQIc14881e1KrlUK2jhRDXyKkTxelLqaRnmejSUJrFXqmYmBQmTPiDlJRMfvppOI0bB/Htt4PtHZYQwg6cuuzg+82nAOjcQBKFtUwmzSefbKNJk5nMm7eH5s2rkpNjsndYQgg7cto7ihyT5vP1xwG4IVyaxlrj2LE47r77JzZujKJHj1A++ugWwsMlyQpR0TltovjnWCyZ2Sbu7xqGj4fTrmapqlzZk/j4dL76ahAjR7aS5q5CCMCJi56W7T0HwIiOIXaOpHxbsuQggwfPIyfHRFCQD3v2jOOee1pLkhBC5HHaRHHgbBIADarK8xOFOXUqgUGD5jJw4FwOHYrl7FnjeRMXF0kQQoj8nLZMZm90ArUqe9k7jHInO9vEjBmbmDJlLVpr3njjJp588jrc3V3tHZoQopxyykSRlWMiJTOHNiHSOV1BOTkmPvvsX3r2DOP//q8voaFV7B2SEKKcc8qip7UHLwDQqYG89hQgLi6NSZNWkJSUgaenGxs2jGbJkjskSQghrOKUieKt5QcAGNKujp0jsS+tNd99t4vw8Fm8885G1qw5AUBQkI9UVgshrOZ0RU8mk+bQeaNitrp/xa2jOHQolnHjfmXVquN06FCb5cvvJiKihr3DEkI4IKdLFO+sOAjAhF6N7RyJfT3xxDK2bo3mww/78eCD7aQDPyHEVXO6RDF382mq+nnySM+G9g6lzK1YcZTw8GDq1q3MRx/dgqenGzVqSPNgIcS1sellplKqj1LqoFLqiFLqmULGP6WU2qeU2qWUWqWUqncty9NaE5uSSWS9gApVBn/uXDJ33rmQm2/+ljfe2ABAvXpVJEkIIUqFzRKFUsoVmAX0BZoBI5RSzQpMth2I1Fq3AhYAb17LMtOycgBoXsv/WmbjMEwmzezZWwkPn8nChfuZMuV63n77ZnuHJYRwMra8o+gAHNFaH9NaZwJzgYGWE2it12itU80fNwHX1Expx6l4ALwqyMNj06ev4+GHf6Vdu1rs2jWWl17qgZeX05UmCiHszJZnldrAaYvPUUDHYqYfA/xe2Ail1IPAgwAhIUX33fTeykOAcz8/kZSUwcWLqYSFBTB2bCRhYQGMGNGiQhW1CSHKli3vKAo7c+lCJ1TqbiASeKuw8VrrT7TWkVrryKpVC+8y/PjFFLaciKOanyfNa1W+2pjLLa01ixbtp1mzDxk+fAFaa4KCfLjzzpaSJIQQNmXLRBEF1LX4XAeILjiRUuom4HngVq11xtUu7NQlowRr8oCC1SCO7+TJeG69dS6DB/9IYKA3H3zQV5KDEKLM2LLoaQvQSCkVBpwB7gDutJxAKdUG+Bjoo7WOuZaFpWVmAxAS6HMtsyl3Nm48zU03fQPA22/34vHHr8PNTZ6JEEKUHZslCq11tlLqEWA54Ap8obXeq5SaCmzVWi/BKGryBeabr5BPaa1vvZrlHThndCtew0l6jE1MzMDf35O2bWsyenQEEyd2ISTE+YrUhBDln02byGitfwN+KzBsssXfN5XWsr7ddIqG1Xyp5ufYiSI2NpVnnlnJH38cY+/ecfj6evB//9fP3mEJISowpyjD+GDVYS4mZ9CspuM+P6G15uuvdxIePosvv9zB8OHNkWoIIUR54BSN7t9dYTSLfeW2FnaO5OokJKQzaNA81q49QadOdZg9uz+tWlW3d1hCCAE4QaKIT80EoENYIP5e7naO5sporVFK4e/vSXCwD5980p8xY9rK60iFEOWKwxc9bTwaC8BQB3v3xPLlR2jb9hOiohJRSjF//lAeeKCdJAkhRLnj8Ini9z3nAOOOwhGcPZvEHXcsoE+f70hNzSImJsXeIQkhRLEcuugpIzuHJTujqeHvRb2gSvYOp0SzZm3muedWk5GRzcsv92DSpC54ejr0LhBCVAAOfZY6ftG4Gu/ZtJqdI7HOtm1n6dixNrNm9aNRI+ftj0oI4VwcOlGcjDW67bi5WflsIZSYmMHkyWsYObIV7drV4sMPb8HT01W63xBCOBSHThQnzHcUgZU87BxJflprFi7cz+OPL+Ps2SRCQirTrl0t6QJcCOGQHPrM9c/xSwCEBZef+onjx+N45JHf+e23w0RE1OCnn4bRsaNjtcgSQghLDp0o4szPUPiVo+cnvvtuN3/9dZL33uvNI490kA78hBAOz6ETxdGYZOoGets7DNatO0lGRg433VSfiRM7c999EdSp47jdiQghhCWHvdxNzcwmMT2bpjXsd0K+eDGV0aN/pnv3OUyd+icAnp5ukiSEEE7FYe8oVuw7D0DXRsFlvmytNXPm7GDixBUkJGQwaVIXXnyxe5nHIcq/rKwsoqKiSE9Pt3coooLw8vKiTp06uLuXXpG8wyaKrSfiAOjfqlaZL/u33w4zevQSunSpy+zZ/WnRwjGe4xBlLyoqCj8/P0JDQ6VZtLA5rTWxsbFERUURFhZWavN12KKnZXuNrjvKqmlsamoWGzacAqBfv0b8/PMd/PXXKEkSoljp6ekEBQVJkhBlQilFUFBQqd/BOmSi0FpzISmDYN+ySRK//36YFi0+pG/f74iPT0cpxa23NpEO/IRVJEmIsmSL480hE8WlFKNZ7J0dQmy6nDNnEhk6dD79+n2Pp6cbv/wygipVHPsNekIIcaUcMlGcvGR03VGziu2axsbEpNCs2YcsXXqIV165gZ07x3L99aE2W54QtuLq6kpERAQtWrRgwIABxMfH543bu3cvPXv2pHHjxjRq1Ihp06ahtc4b//vvvxMZGUnTpk0JDw/n6aeftscqFGv79u3cf//9+YYNHDiQTp065Rt23333sWDBgnzDfH198/4+dOgQ/fr1o2HDhjRt2pRhw4Zx/vz5a4rt0qVL9OrVi0aNGtGrVy/i4uIum2bNmjVERETk/fPy8mLx4sUAdOvWLW94rVq1GDRoEABLly5lypQp1xTbFdFaO9S/du3a6UX/Rul6k5bq3VHxurRFRSXk/f3++5v0kSOxpb4MUXHs27fP3iHoSpUq5f19zz336FdeeUVrrXVqaqquX7++Xr58udZa65SUFN2nTx89c+ZMrbXWu3fv1vXr19f79+/XWmudlZWlZ82aVaqxZWVlXfM8hgwZonfs2JH3OS4uTtepU0eHh4frY8eO5Q2/99579fz58/N9N3fbpKWl6YYNG+olS5bkjVu9erXevXv3NcU2ceJEPX36dK211tOnT9f/+9//ip0+NjZWBwQE6JSUlMvGDR48WH/11Vdaa61NJpOOiIgodDqtCz/ugK36Ks+7Dtnq6cC5JACq+5deMVBCQjovvLCajz/exqZN99O2bU0ee6xjqc1fiJd/2cu+6MRSnWezWv5MGdDc6uk7derErl27APj+++/p0qULN998MwA+Pj7MnDmTHj16MH78eN58802ef/55wsPDAXBzc2PcuHGXzTM5OZlHH32UrVu3opRiypQp3H777fj6+pKcnAzAggULWLp0KXPmzOG+++4jMDCQ7du3ExERwaJFi9ixYwdVqlQBoGHDhmzYsAEXFxfGjh3LqVNGI5IZM2bQpUuXfMtOSkpi165dtG7dOm/YwoULGTBgANWrV2fu3Lk8++yzJW6X77//nk6dOjFgwIC8YTfccIPV27UoP//8M2vXrgXg3nvvpUePHrzxxhtFTr9gwQL69u2Lj49PvuFJSUmsXr2aL7/8EjDqIXr06MHSpUsZNmzYNcdZEodMFBuPXgQolcpsrTXz5+/jiSeWce5cMo880oEGDQKutktM5AAADxJJREFUeb5ClDc5OTmsWrWKMWPGAEaxU7t27fJN06BBA5KTk0lMTGTPnj1MmDChxPlOmzaNypUrs3v3boBCi1cKOnToECtXrsTV1RWTycSiRYsYNWoU//zzD6GhoVSvXp0777yTJ598kq5du3Lq1Cl69+7N/v37881n69attGjRIt+wH374gSlTplC9enWGDBliVaLYs2fPZduiMElJSXTr1q3Qcd9//z3NmjXLN+z8+fPUrFkTgJo1axITE1Ps/OfOnctTTz112fBFixZx44034u//38O8kZGRrFu3ThJFUUwa3F3VNdfua60ZPPhHFi8+QNu2NVmyZASRkWX/XIaoGK7kyr80paWlERERwYkTJ2jXrh29evUC/ntne2Gu5Le1cuVK5s6dm/c5IKDkC62hQ4fi6uoKwPDhw5k6dSqjRo1i7ty5DB8+PG+++/bty/tOYmIiSUlJ+Pn55Q07e/YsVatWzft8/vx5jhw5QteuXVFK4ebmxp49e2jRokWh63Sl5xA/Pz927NhxRd+x1tmzZ9m9eze9e/e+bNwPP/xwWT1MtWrViI6OtkksBTlkZXZ0fNo1PWiXlZUDGAdJ1651+eCDPmzefL8kCeGUvL292bFjBydPniQzM5NZs2YB0Lx5c7Zu3Zpv2mPHjuHr64ufnx/Nmzdn27ZtJc6/qIRjOaxgu/5Klf7r8blTp04cOXKECxcusHjxYgYPHgyAyWRi48aN7Nixgx07dnDmzJl8SSJ33SznPW/ePOLi4ggLCyM0NJQTJ07kJbGgoKB8dzuXLl0iODg4b1tYs65JSUn5Kp4t/1kmtVzVq1fn7NmzgJEI/r+9uw+uurrzOP7+LAIJFdIWF3YrKrSmkBjCg9CmdhGRLmM1C6XjiOGh0rHrCCK1KA47OLPuai1tBS2iRdY6SWsbEKcWBhAW2Vgq8iC0Cgo2pTQT2OkozbKsUnn+7h+/X5JrcnPzS+Q+5vuauTP393zynZvfueec3/2efv3a/t3V888/z+TJk1v9orqhoYFdu3Zx0003fWT9yZMnyc9PTa67rKsozpvRcOJ0p1OLv/JKHaWly1mz5h0A7r33Gu6++4t065Z1oXCuQwoKCli6dCmPPvooZ86cYdq0abz66qu8/PLLQNDymDt3Lvfffz8A8+fP55FHHqG2thYIbtxLlixpdd4JEyawbNmypuXGm3H//v05cOBAU9dSWyQxefJk5s2bR1FREX379o173njf5IuKijh48GDTcnV1NRs3bqSuro66ujr27NnTVFFcd911rFq1itOng8frKysrm8Yhpk6dymuvvcb69eubzrVx48am7rRGjS2KeK+W3U4AEydOpKqqCoCqqiomTZrUZhyqq6upqKhotX716tWUl5eTl/fRMdna2tpW3W7JknV3x1NnzwPQp4OTAB09eoLbbvsV48ZVcerUWXr37pmM4jmX0UaMGMGwYcNYuXIl+fn5rFmzhocffpjBgwczdOhQRo8ezZw5cwAoLS3l8ccfp6KigqKiIkpKSpq+Hcd64IEHOHbsGCUlJQwbNoyamhoAFi1aRHl5Oddff31TP31bpkyZwnPPPdfU7QSwdOlSdu/eTWlpKcXFxSxfvrzVcUOGDOH48eO8//771NXVUV9fT1lZWdP2QYMG0adPH3bu3El5eTljxozh6quvZvjw4Wzbtq1pYDk/P59169bxxBNPUFhYSHFxMZWVlQlbAFEsWLCAzZs3U1hYyObNm1mwYAEQjK3EdiXV1dVx+PBhxo4d2+ocK1eujFuB1NTUtGplJIss5pnpbDBwyFDja4tYd/c/UHJpQaRjqqv3cdddG/jgg9PMn38NCxdeS69emTOHhctdBw4coKioKN3FyGmPPfYYvXv3btWHn8veffddpk6dypYtW+Juj/e5k7THzEZ15npZ16L44NRZAAr7X9zOns3Onj1PSUk/3njjTr773fFeSTiXQ2bNmkXPnl2rh6C+vp7Fixen7HpZ99TTiVPn+FyfnvS8qFvb+5w4zUMPbeXyywuYPXs006eXMn16qefccS4H5eXlMWPGjHQXI6VGjx6d0utlXYvivBkTiv+uze3r1tVy1VVP8f3vb6O2tgEIBsu8knDpkm3duy67JePzlnUtCoDxRa0HmI4c+T/mzn2JF198h+Liv2Xr1pmMGXNFGkrnXLO8vDwaGho81bhLCQvno2j5hNTHlZUVRUF+6zGGQ4eOsWnTH/ne98Yzb96X6NGj7a4p51JlwIABHDlyhKNHj6a7KK6LaJzh7kLKyooir3tQCeza9d9s336Yb3+7jGuvvYL6+nvo27dXO0c7lzrdu3e/oDONOZcOSR2jkHSDpN9LOihpQZztPSWtCrfvlDQwynk/eVE3Zs9eT1nZMyxZsoMT4fwUXkk459yFl7SKQlI34Engq0AxUCGp5U8XbweOmdmVwGNA22kVY4wY+mOefnoPc+d+kX37ZvGJFE2H6pxzXVEyu56+ABw0s0MAklYCk4DYhCiTgAfD9y8AyyTJEgzb29nzXDawgA0bpjFyZOJfezrnnPv4kllRXAocjlk+ArSc4KFpHzM7K+k40Bf4S+xOku4A7ggXT+3+yx1vRcgI3BVcQotYdWEei2Yei2Yei2aDO3tgMiuKeM8CtmwpRNkHM1sBrACQtLuzP0PPNR6LZh6LZh6LZh6LZpJ2t79XfMkczD4CXBazPABomTy9aR9JFwEFwP8ksUzOOec6KJkVxetAoaRBknoAtwJrW+yzFrgtfH8z8F+Jxiecc86lXtK6nsIxhznAJqAb8KyZvS3p3wkm+V4L/AT4maSDBC2JWyOcekWyypyFPBbNPBbNPBbNPBbNOh2LrEsz7pxzLrWyLimgc8651PKKwjnnXEIZW1EkK/1HNooQi3mS9kvaK2mLpJxNm9teLGL2u1mSScrZRyOjxELSLeFn421Jv0h1GVMlwv/I5ZJqJP0u/D+5MR3lTDZJz0p6T9JbbWyXpKVhnPZKGhnpxGaWcS+Cwe8/Ap8FegBvAsUt9pkNLA/f3wqsSne50xiLcUCv8P2srhyLcL/ewFZgBzAq3eVO4+eiEPgd8KlwuV+6y53GWKwAZoXvi4G6dJc7SbG4FhgJvNXG9huBlwh+w1YG7Ixy3kxtUTSl/zCz00Bj+o9Yk4Cq8P0LwHjlZsL/dmNhZjVm9tdwcQfBb1ZyUZTPBcBDwA+Ak6ksXIpFicU/A0+a2TEAM3svxWVMlSixMKBP+L6A1r/pyglmtpXEv0WbBPzUAjuAT0pqNxdSplYU8dJ/XNrWPmZ2FmhM/5FrosQi1u0E3xhyUbuxkDQCuMzM1qWyYGkQ5XPxeeDzkrZJ2iHphpSVLrWixOJBYLqkI8AG4O7UFC3jdPR+AmTufBQXLP1HDoj8d0qaDowCxia1ROmTMBaS/oYgC/HMVBUojaJ8Li4i6H66jqCV+RtJJWb2v0kuW6pFiUUFUGlmiyV9ieD3WyVmdj75xcsonbpvZmqLwtN/NIsSCyR9BVgITDSzUykqW6q1F4veQAnwiqQ6gj7YtTk6oB31f2SNmZ0xsz8BvyeoOHJNlFjcDjwPYGbbgTyChIFdTaT7SUuZWlF4+o9m7cYi7G55mqCSyNV+aGgnFmZ23MwuMbOBZjaQYLxmopl1OhlaBovyP/IrggcdkHQJQVfUoZSWMjWixKIeGA8gqYigouiK89OuBb4RPv1UBhw3sz+3d1BGdj1Z8tJ/ZJ2IsfghcDGwOhzPrzeziWkrdJJEjEWXEDEWm4AJkvYD54D5ZtaQvlInR8RY3Av8h6TvEHS1zMzFL5aSqgm6Gi8Jx2P+FegOYGbLCcZnbgQOAn8FvhnpvDkYK+eccxdQpnY9OeecyxBeUTjnnEvIKwrnnHMJeUXhnHMuIa8onHPOJeQVhcs4ks5JeiPmNTDBvgPbypTZwWu+EmYffTNMeTG4E+e4U9I3wvczJX0mZtszkoovcDlflzQ8wjH3SOr1ca/tui6vKFwm+tDMhse86lJ03WlmNowg2eQPO3qwmS03s5+GizOBz8Rs+5aZ7b8gpWwu51NEK+c9gFcUrtO8onBZIWw5/EbSb8PXNXH2uUrSrrAVsldSYbh+esz6pyV1a+dyW4Erw2PHh3MY7Atz/fcM1y9S8xwgj4brHpR0n6SbCXJu/Ty8Zn7YEhglaZakH8SUeaakJzpZzu3EJHST9GNJuxXMPfFv4bq5BBVWjaSacN0ESdvDOK6WdHE713FdnFcULhPlx3Q7vRiuew/4RzMbCUwBlsY57k7gR2Y2nOBGfSRM1zAF+HK4/hwwrZ3r/xOwT1IeUAlMMbOhBJkMZkn6NDAZuMrMSoGHYw82sxeA3QTf/Ieb2Ycxm18Avh6zPAVY1cly3kCQpqPRQjMbBZQCYyWVmtlSglw+48xsXJjK4wHgK2EsdwPz2rmO6+IyMoWH6/I+DG+WsboDy8I++XMEeYta2g4slDQA+KWZ/UHSeOBq4PUwvUk+QaUTz88lfQjUEaShHgz8ycxqw+1VwF3AMoK5Lp6RtB6InNLczI5KOhTm2flDeI1t4Xk7Us5PEKSriJ2h7BZJdxD8X/89wQQ9e1scWxau3xZepwdB3Jxrk1cULlt8B3gXGEbQEm41KZGZ/ULSTuAmYJOkbxGkVa4ys3+JcI1psQkEJcWd3yTMLfQFgiRztwJzgOs78LesAm4B3gFeNDNTcNeOXE6CWdwWAU8CX5c0CLgPGG1mxyRVEiS+a0nAZjOr6EB5XRfnXU8uWxQAfw7nD5hB8G36IyR9FjgUdresJeiC2QLcLKlfuM+nFX1O8XeAgZKuDJdnAL8O+/QLzGwDwUBxvCeP3idIex7PL4GvEcyRsCpc16FymtkZgi6ksrDbqg9wAjguqT/w1TbKsgP4cuPfJKmXpHitM+eaeEXhssVTwG2SdhB0O52Is88U4C1JbwBDCKZ83E9wQ/1PSXuBzQTdMu0ys5ME2TVXS9oHnAeWE9x014Xn+zVBa6elSmB542B2i/MeA/YDV5jZrnBdh8sZjn0sBu4zszcJ5sd+G3iWoDur0QrgJUk1ZnaU4Ims6vA6Owhi5VybPHusc865hLxF4ZxzLiGvKJxzziXkFYVzzrmEvKJwzjmXkFcUzjnnEvKKwjnnXEJeUTjnnEvo/wHwZD1sgbEwAgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(glm_2, y_test, X_test[sig.index], \"Logistic Regression\")" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[3] = [\"Logistic Regression\" , classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.2697615225249289), output_dict = True)[\"1\"][\"recall\"],auroc]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "iCxBcin11EI8" + }, + "source": [ + "### Support Vector Machine\n", + "#### Theory\n", + "Support vector machines attempt to find an optimal hyperplane that is able to separate the two classes in n-dimensional space. This is done by finding the hyperplane that maximises the distance between itself and support vectors (data points that lie closest to the decision boundary).\n", + "\n", + "SVM is computationally expensive for a dataset with a lot of features. Therefore, it is neccessary at this stage to do some data reduction." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### PCA\n", + "We would like to reduce the dimensionality of our dataset before training an SVM on it. This can be done through Principle Component Analysis (PCA). The idea would be to reduce the number of features without loss of information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 2 Component PCA\n", + "First, we will visualize the information retained after performing a 2 component pca." + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": {}, + "outputs": [], + "source": [ + "#perform pca\n", + "from sklearn.decomposition import PCA\n", + "pca = PCA(n_components=2)\n", + "principalComponents = pca.fit_transform(X_train)\n", + "principalDf = pd.DataFrame(data = principalComponents\n", + " , columns = ['principal component 1', 'principal component 2'])" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Explained variation per principal component: [0.2966725486 0.1736524263]\n" + ] + } + ], + "source": [ + "#amount of information each principal component holds after projecting the data to a lower dimensional subspace.\n", + "print('Explained variation per principal component: {}'.format(pca.explained_variance_ratio_))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows that the information of principal component 1 retained is 28.4% and principal component 2 retained is 17.8%, both of which is quite low" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAH6CAYAAAAeHtXEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOyde5xT1b3ovyuZJMMkQWQQ0AEUnBFlqFh81Tq1WlHq2NqqaNrSFk+xtkdry6PnQnv7sKf3Fg9VW1q0py3tFR9HYlFLKwMoVbSjtr6xgEDwUaCDoANKJjB5rvvHToZMZifZCclMZub35ZPPJnuvvfZvJ5n9W+u3fg+ltUYQBEEQhP6Pra8FEARBEAShNIhSFwRBEIQBgih1QRAEQRggiFIXBEEQhAGCKHVBEARBGCCIUhcEQRCEAYIodUEQBEEYIIhSFwYsSqlapdT1SqlHlFI7lFKHlVIfKKValVKzlVLy+x9gKKUuVEpppdQtRZz7dvLc1CuhlHpfKfWsUuompVRVlvPGKqVuVUq9pJQ6oJSKKqX2KaXWK6W+pZQ6Jsc1Z6Zd79JCZRaETEx/pIIwQLgG+BWwB3gS2AmMAq4ClgGXKaWu0ZKBSejOEuB9wA6Mx/i9nAdcnPx/F0qp64GlgAvYCDwAHABqgSbg58D3gRFZrnUDoAGV/P9jpb0VYbAhSl0YyGwHrgBWa60TqZ1Kqe8CzwNXYzykH+ob8YQK5eda67dTb5RSjcALwJVKqY9rrZ9K7v8C8FsMJX611np1ZkdKqfOBO80uopSaCFwArAeGA1copUZprfeW+H6EQYSYH4UBi9b6Ca31n9MVenL/O8B/J99eWEifSqlTlVK/T5pqw0kz61+VUv9u0vZipdRapdR+pVSnUmp70kzbwxyrlNqQNME6lFI/UEq9kTxnq1Lqq2ntvq6U+kdyKWG3UupHmcsISqmTkn3dnZT3j0kZQsmlB1Mzr1LKpZRaqJR6TSl1SCl1MHlv15q0Tb/GSUqpFUqp95Iyv6iU+lSOz/DzSqknk6bqTqXU60qp7ymlXCZtdfKzGaGU+o1Sak/yc9+slPq3jLZ3Y1hkAH6YYUq/MJs8+dBabwY2JN+ek7yWF/hlct/nzBR68txngHOzdJ36Xv8fcDfgAK4rVk5BAJmpC4OXaHIbs3qCUupy4A8Ypta1GKbWYcAU4H9hmPpTbb+WfB9KnrMPYwCxAPi0Uup8rfX7JpdZgaEEWpIyzgB+o5SKAqcDs4BHgb9gWCF+ABwC/sukr/HAc8Am4NfA8YAPWKOU+oLW2p8mrxNYB3wc2Ioxu6xJXt+vlDpDa/1dk2uciGH1eBO4F2PG6QNWKaWmaa2fTG+slPod8BVgN/Awhpn7I8CPgYuVUpdorTO/k2HAM0AEWAlUJ+X6vVIqobVenmz3x+R2FvAURxQxwNsmsheCSm5TSzUzMO71b1rrnCZzrXW4R2fG5z0LOAg8gvFZ3wZcr5RaLEtCQtForeUlr0H1whjM/gPjAT3d4jkjgA8wFMvHTY6PSfv/iUAY44F9aka7u5LX/U3G/g3J/S8Aw9L2T0he8wDwFlCXdmwY8B7wLlCVtv+kZF8a+GnGdc7CGCwcAIam7f9Osn1LRl8jMRSiBj6a5Ro/zLjG9FRfGfuvS+5/GBiSceyW5LFvZexPXWMZYE/bPwljQLYlo/2Fyfa3FPG7SN3nSRn7GzEGThr4WHLf75Lv/0+Rv8HPJc//ddq+h5L7Lu7rvxF59d9XnwsgL3n19gtjRqQx1tqtnjM/ec4SC23/d7LtT0yOHZtU9ocBV9r+Ddke6MATyWNfMTn2/5LHTkzbl1K47wNek3PuTh6flbYvACTIGIQkj81Otv+9yTXeTle2acf/CbyXse8VjAHFMJP2dowByvMZ+zWGtWOoyTlPJY970/aVQqn/PDnI+DFwX5pCfzitbUty39eL/A2mvtPz0vZ9OrlvRV/+fcirf7/E/C4MKpRS38RQ0FuBLxVw6keS2zUW2k5Nbp/IPKC1PqCUegXDQepUDI/pdF406a8tuX3J5Ni/ktsxGIo0nZe11kGTczZgmH4/DCxPrg/XA//SWm81aZ+6jw+bHHtVax032b8Lw2McAKVUDcYyxXvAHKWUySmEgdNM9ge01gezXAMMi4XZfRbLt5JbDXQAr2Eo9/9Oa5NpjreMUqoeY/CxTWv9XNqhNcBeDIe8EVrr9wrtWxBEqQuDBqXUTRjhSlswZsT7Czh9WHL7r5ytDFKOcHuyHE/tH5Z5QGv9gUn71BpzrmMOk2PZvKjfSW6PydgWLC+GNcCMGN0dcY/FUITHAT/Mck42cl0DjFl+KRmv07zfs5AaaI0pov+vYnwWd6fv1FrHlFL3YQw6r8OwKAlCQYj3uzAoUErNwYgn3gRcpA0P+EJIKZY6C21Tynd0luPHZ7QrF6Oy7E/J9UHGtpzyps59RWutcr2O4hq9SWtye3EhJyml0j3cF2V46GsMhQ5HPOMFoSBEqQsDHqXUAuBnwKsYCn1fEd38Lbm9zELbV5LbC01kGQacAXQCrxchRyFMTZrWM0nJ9QpA0kT/BlCnlGowaX9RcvtysYJorTuAzUCjUmp4sf1YILUUUOrZeyYrgf3AeUqpabkaZoTqfQbD+XAbhrOd2etN4BSl1MfLILcwwBGlLgxolFLfB27FWI+++CjWKZdjOLj9u1LqApPrpJth78NwCLs5uX6azo+BocB92iTUqcQcgxHy1oVS6ixgJsbM+ZG0Q7/HMAn/VCllT2s/AiMjWqrN0XAH4MQIRethyldKHauUmtrztIJoT27HHWU/OUkOhL6ZfOtXSk03a6eU+ghGWGGKG5LbH2itrzd7AT/JaCsIlpE1dWHAopSaBfwnxuztr8A3TRy03tZa352vL631e8rIILYSeFIptQbDgWooRvz4WIy4cLTWbyfN/XcCLyulHsQIO/s4hvPYVox49XLzNEbc87kYcd6pOHUb8LUM57PbMKwQnwE2KqVaMGKnr8GYWS7WWrdyFGitf6+UOhO4EXhDKbUOI3XvcIzP7gIMb/6vH8VltmH4PXxOKRVJ9q+Be7XWmY6ER4XW+n6l1BCMZZ21SqlXgWc5kib2PI44B6KUGg9MS77/o2mnBiswLEtXK6VuLtD3QxjkiFIXBjLjk1s7MCdLm6fIcFjKhtZ6dXKmuwBjLfVSjAf4VmBRRtu7lFI7gG9jpKOtwfDW/ilGqFs2569S8haGgrw1uXVhmND/U2u9LkPeiFLqEmAe8AXgZgxHtI3AHK31A6UQSGt9U3JA9HUMBTcMw4y9E+Ozue8o+48rpa7EuOdrAS+GBaKVntEBR43WellycPIN4BIMK4gbwwdjEzCXIxaO65Oy3Ku1juToM6SUWoGxrj4LQ8ELgiWU1pK4SBAGEkqpkzAU+nKt9XV9KowgCL2KrKkLgiAIwgBBlLogCIIgDBBEqQuCIAjCAEHW1AVBEARhgCAzdUEQBEEYIPT7kLYRI0bok046qa/FKBuhUAi3293XYpSdwXKfMHjudbDcJwyeex0s9wmVfa8vvfTSe1rr48yO9XulftJJJ/Hii2aFrQYGGzZs4MILL+xrMcrOYLlPGDz3OljuEwbPvQ6W+4TKvlelVNacC2J+FwRBEIQBgih1QRAEQRggiFIXBEEQhAFCv19TNyMajbJ79246Ozv7WpSsVFdXM2bMGBwOR1+LIgiCIAwQBqRS3717N16vl5NOOgmTqlx9jtaa9vZ2du/ezfjx4/OfIAiCIAgWGJDm987OTmpraytSoQMopaitra1oS4IgCILQ/xiQSh2oWIWeotLlEwRBEPofA1ap9zVf+cpXGDlyJJMnT+5rUQRBEIRBgih1gGAQli2DBQuMbTB41F1ed911rF27tgTCCYIgCII1BqSjXEG0tkJzMyQSEAqB2w3z5kFLCzQ1Fd3tBRdcwNtvv106OQVBEAQhD4N7ph4MGgo9GDQUOhjb1P6Ojr6VTxAEQRAKYHArdb/fmKGbkUgYxwVBEAShnzC4lXogcGSGnkkoBDt29K48giAIgnAUDG6l3tBgrKGb4XZDfX3vyiMIgiAIR8HgVuo+H9iyfAQ2m3G8SD7/+c9z3nnnsW3bNsaMGcPvfve7ovsSBEEQBCsMbu93r9fwcs/0frfZjP0eT9FdP/DAAyUUVBAEQRDyM7iVOhhha21thlPcjh2Gyd3nOyqFLgiCIAh9gSh1MBT47Nl9LYUgCIJQYQTDQfyb/QTaAzTUNuBr9OF1eftarKyIUhcEQRAEE1p3ttJ8fzMJnSAUDeF2uJm3bh4tM1toGld8crJyMrgd5QRBEATBhIRO0Hx/M8FIkFDUCH0ORUMEI0Ga72+mI1KZyclEqQuCIAhCBvsP7yehzZOTJXQC/6bKTE4mSl0QBEEQMgjHw10z9ExC0RA79ldmcjJR6mVk7dq1TJw4kfr6em699da+FkcQBEGwiMvuwu0wT07mdripH16ZycnEUY7yeDfG43FuuukmHn/8ccaMGcPZZ5/NFVdcwaRJk0oktSAIglAuhg8Zjk2Zz3ttyoZvcvHJycrJoFfq5fJufP7556mvr2fChAkAfO5zn2PVqlWi1AVBEPoBNmWjZWZLD/2Q2u9xVmYuk0Gt1IPhYJd3Y4rUGkrz/c20zW8r+ov717/+xdixY7vejxkzhr///e9HJ7AgCILQazSNa6Jtfhv+TX527N9B/fB6fJN9FavQYZArdf9mf17vxtlTi0tKo7XusU8pVVRfgiAIQt/gcXqK1gN9waB2lAu0B8rm3ThmzBh27drV9X737t2ccMIJRfcnCIIgCPkY1Eq9obahbN6NZ599NoFAgLfeeotIJMKKFSu44ooriu5PEARBEPIxqJW6r9FXNu/Gqqoqli5dyvTp0znttNO49tpraWxsLLo/QRAEQcjHoF5T97q8ZfVubG5uprm5uUTSCoIgCEJuBrVSh/7p3SgIgiCUDrNcJd0bBI3y3IEANDQY5bm9lVmpbdArdeh/3o2CIAhCaciWq+R/zvyfZINWaG6GRAJCIXC7Yd48aGmBpsqr1CZKXRAEQRiU5MpVEtgfoKN9D57mZmOm3tUgGTHV3AxtbeCpLKvugHWUM4sTryQqXT5BEISBTq5cJQD++xYaM3QzEgnDJF9hDMiZenV1Ne3t7dTW1lZkwhetNe3t7VRXV/e1KIIgCIOWXLlKEjrBjne3QShE0An+yRAYDg37wbcJvKEQ7Ki8Sm0DUqmPGTOG3bt38+677/a1KFmprq5mzJgxfS2GIAjCoCWVq8RMsduUjfrjJtJ6ykaar+4kAYRc4A7DvOnQ8lA1TfWVV6ltQCp1h8PB+PHj+1oMQRAEoYLxNfqYt25e1uPN136PiXvvIeg6si+U/H/z1Z20XXk53VbUK8BLfsCuqQuCIAhCLlK5SrxOb1d2UbfDjdfppWF4A6v3PEViiPkyaWJINf63Vx/Z0doKdXUwZw4sXmxs6+qM/b3IgJypC4IgCEI+guEgW9/byvVTr6f9UDu1Q2ppHNmIb7KPF5990VhzT3SanhtKdB6pDxIMGt7wFeAlL0pdEARBGHSYxadnZhNtqG2guqqazlhPxV5dVX2kPojfn99Lfnbv5EIR87sgCIIwqEiPT085yYWiIYIRY39HpAOA5vpmU4UO0Bnr5PJTLjfeBAJdM/OgE5ZNhQXTjG0w2rte8jJTFwRBEAYVueLTEzqBf5OfkzmZlh0tOG1OIolIj3ZOm5PV21cb2UgbGsDtprU2RPNMenrKj9H0Vu45makLgiAIg4pc8emhaKhrrXzTvk2mCh0gkoiw5d0txhufj6BL0TwTgq4jHvIhl/G++f07u2b/5UaUuiAIgjCoSMWnm+F2uLvWyg8cPpCzn/bD7cZ/vF78d91IIkuuswQa/6beyT5XUUpdKTVWKfWkUup1pdRmpdS3+lomQRAEYWDha/RhU+bqz6Zs+CYbVdqGDxmes5/a/UfW2wPDIeQ0b5c++y83FaXUgRgwX2t9GvAR4Cal1KQ+lkkQBEEYQHhdXlo+dCveMLiT1nV3BLxhaPnQrV3e740jG6m2m8epV0dh0rI/QodhVrc6+y83FaXUtdZ7tNYvJ/8fBF4H6vpWKkEQBGFAEQzS9IWFbPsFXL0Fzt1lbLf9Apq+sLBLUfsafdiyhKrZNPhet3cVdbE6+y83qlKrhSmlTgKeBiZrrQ9mHLsBuAFg1KhRZ65YsaLX5estOjo68FRYab9yMFjuEwbPvQ6W+4TBc68D5j7fe4+OvTsJDDP0X8IGtqTubnhf4Rk1jo7qamrcNWx851US9NSTNg1T3gHbqNFG5jigI9JBYH/A6FMnupR8w/CGrtl/Kbjooote0lqfZXasIpW6UsoDPAX8X631w7nannXWWfrFF1/sHcH6gA0bNnDhhRf2tRhlZ7DcJwyeex0s9wmD514Hyn0GF8yhzr6kW073FN4wtOl5vDj90+wYuoM5f/4GIcI92rnDsORJJ7Ovv6tbYpmOSAf+TX527N9B/fB6fJN9JVXoAEqprEq94uLUlVIO4CHg/nwKXRAEQRAKxX/CARL7zI8lFPhHt3MyydA3E4UORrjajmO1UbQlDY/TY8Su9xEVtaaujOLnvwNe11rf0dfyCIIgCAOPQP3w7J7qTtjRUAsYzm81VTWm7dwRqL9uXq/ldLdKpc3Uzwe+BPxDKfVqct93tdYtfSiTIAiCMIBoOL4Rt63atFiL21ZN/ahJcBDGDR3Hodgh0z5sbg++z37vyI5gkOADy/G//SiBWmg471P4zpyF1zWIS69qrVu11kprfbrW+ozkSxS6IAiCUDJ8jT5sVQ7TY7YqB77JPhI6wWf9n83ax0rfQ0fWyltbaT1nFHVv38wc1rG4Yx1zVt9M3U9H07qzd0uvVpRSFwRBEIRyk6uOeqpKW/vhdg7HDmftI9BueLkTDBL8zGU0X3m4e4pYJwTjh2i+77JeSxELlWd+FwRBEISy0zSuibb5bVk91d/vfD/n+asDq7npnJvA78d/SoQshVdJxKL4N/l7zXlOlLogCIIwKDkqT/VUNHggQMAT6ZqhZxLS4V5LEQtifhcEQRCEHgyrHpbz+OUTk7XUGxpo6HDiNo98w61cvZYiFkSpC4IgCEIPaofUUuMwD2ercdQwa8os443Ph2+7M6syTTne9Rai1AVBEAQhA5uy8dNLfmp67KeX/PSI57vXi3fVGloeGWIUiEnO2N0R8NpraPnimpJnlMuFrKkLgiAIQgYJnWDh+oWmxxauX8iXp3z5iLJuaqLphX20PbAc/z9Xs8MD9R+9HN+Zs3pVoYPM1AVBEAShB/sP7yehzX3aEzqBf5O/+06PB/3lL6OvuorE6R9Cu1z0RW0VmakLgiAI/YZgOIh/s59Ae4CG2gZ8jb6yZG0Lx8OEoiHTY6FoqIdHe+vOVprvbyahE4SiIdwON/PWzaNlZgtN45pKLl82RKkLgiAI/YLeVJxOe5bk8EnGHDOm6//BcJDm+5sJRoJd+1IDgub7m2mb39ZrZngxvwuCIAgVT7riTCnMUDREMGLs782sbQDpJdb9m/2FmerLiCh1QRAEoeLpbcUZiUdyHt99cHfX/wPtgYJM9eVElLogCIJQ8fS24nTZXV154TNxO9zdEso01DZYbltuRKkLgiAIJScYDrLs5WUseHwBy15eRjAczH9SDnpbcQ4fMhybMleRNmXrllDG1+iz3LbciFIXBEEQSkrrzlbq7qhjzto5LH52MXPWzqHujrqjKkPa24rTpmx5K7mlsFL1rbcQ73dBEAShZJTLEzylODO931PKtxyKM18lt2LblhNR6oIgCELJ8G/2E01ETY9FE1GWv7ocV5WrqDjzvlCchVRyO6qqbyVClLogCIJQMjbv20xnrNP0WGesk7lr5+KschYdZ14JirOSkTV1QRAEoWTsP7w/5/GojlZGnPkARZS6IAiCUDKOHXJswef0doKWgYyY3wVBEATL5Mu9PnloPc4YRArQLr2doGUgI0pdEARBsERHpIO6O+py5l5vfq2TiD1LBxpQPXf3doKWgYyY3wVBEIS8BMNBAvsDeXOvP/zW6oL77u0ELQMZUeqCIAhCXvybs695p6+JP6q3m87GAVBg1/R5gpaBjJjfBUEQhLwE2gOM1CNNj3VbE3fkViufiI3Dd8UP+jRBy0BGlLogCIKQl4baBoLt5vnb09fEP3Xc+ax7f6f5bF3DZ0Y0SZx5GRHzuyAIgpAXX2P2Ne/0NfGrfLfk7Ofqz/1nKcUSMhClLgiCIOTF6/IyZugY02O3Tru1y4Tesudpqm1O03bVNier2zaUSUIBRKkLgiAIFgiGg+w+uNv02ML1C7u83wPtATp1xLRdp45IPHqZEaUuCIIg5MWq9/vYY8bm7GfMMeazfaE0iFIXBEEQ8hJoD5DQCdNjBWWE0yUUSuiBKHVBEAQhLw21DdiUucpI935/btdzOfvJZsIXSoModUEQBCEvVrzfg+EgD299OGu7mqoaSQdbZkSpC4IgCHnxurw0DG/A4/DgtBve7U67E4/D05URzr/Zj83cQg9AXMclHWyZEaUuCIIgWEeBSmaWUcabLgJ7NnMo0Zn11BkNn5XscWVGlLogCIKQl1RBl45IB+F4GIBwPExHpKOroEvDjv24zaPZcEfgor3VvSjx4ESUuiAIgpAXKyFtvrZjsWXxbrdp8O2pLZN0QgpR6oIgCEJe8oW0/d+//l+WH7+PlatceMPgNibzuMPgDUPLQ9V46if1osSDEynoIgiCIOQlV0EXgLfef4ubeYshn4U/roBdx8CO4VC/H3ybweNygE+c5MqNzNQFQRCEvOQKaUvnsBOu8sHhGicJBdrlRHs80NICHnGSKzcyU89D28E2vvOX77D1va2cOuJUFl28iBOGntDXYgmCIPQqqZA2r9NLJB7pcpYzI+SCb1+SIAy4lWLep6FlHDT1nriDFpmp5+CO5+6g7md13PPaPTzf9jz3vHYPdT+r464X7upr0QRBEHodj9ND2/w2Ljrporxtw8QACOkwwegRD3mhvIhSz8IfX/8j8x+bb3rsppabeKfjnV6WSBAEoe/xOD1cPelqXHZXQeelF30RyocodROC4SBXPnhl9gYa5q/+Zu8JJAiCUEH4Gn047I6Czimo6ItQNKLUTfj+ugW5KwkpeGLrul6TRxAEoZLwurysmbmGGkdNQedJ2dXyI0rdhDtf+XW31Idm7E0c7B1hBEEQKpCmcU3s/fZebrvkNiaPnMz4YeM5c/SZuU+SsqtlR7zfTYjpRF6lrvMcFwRBGOi8+s6r/OipH5HQCULRELvUrpztpexq+RGlLgiCIBRMMByk+f5mgpEjCWliOpa1fXrNdaF8iFLvS4JB8PshEICGBiPbktfb11IJgiDkxb/ZnzVtrBmpmutCeRGl3le0tkJzMyQSEAqB2w3z5hlZl5okRYMgCJVNoD1AKBrKetxhcxBNRHE73NiUravmulBeRKn3BcGgodCDaXmUQ8k/juZmaGuTdIpC3yDWI8EiY48Zm/P4NZOuYdwx46gfXo9vsk8Uei8hSr0v8PuNGboZiYRxfPbs3pUpG/KQHzyI9ahfEQwH8W/2E2gP0FDbgK/Rh9dVOX+bHx33UW46+6a+FmPQIUq9LwgEjszMMwmFYEeFJGiQh/zgQaxH/YrWna0039/c5XXudriZt24eLTNbaBrXO3+buz7I4+n+gXi69wUSp94XNDQYCtIMtxvqK8BDNP0hn3q4h0JH9ndIDucBhRXrkVARpHudp9a0Q9EQwUgwb371YDjIspeXseDxBSx7eRnBcPZSqvloqG3A7TB/jomne98hSr0v8PnAluWjt9kqo+awPOQHF/3FeiTk9DrPlV+9dWcrdXfUMWftHBY/u5g5a+dQd0cdrTtbi5LD1+jDpsyfY+Lp3neIUu8LvF6Cf1rJ0o85+eSXbXxyJiw930lwuIe2h+9h1uM3ce5vz2XWI7OIJqJ9I6M85AcX/cF6JAC5vc6z5Vc/mtl9NrwuLy0zW/A6vV0zdrfDjdfpFU/3PkTW1DM4GnOUVVp3tnLpM5/l8MWRrn3rGiLMm54g+syRQjLPtz3P6aecTuCFADeefWPZ5epG6iFvptjlIT/w8PkMfwkzKsV6JABHzN5mij2b2dvK7H721MKdc5vGNdE2vw3/Jj879u8QT/cKQGbqGSzfuLys/QfDQS677zIOxw73OBZNmGdj6pNSr/1hiUAoHV6v4QDp9R6ZsbvdR/aLk1zFUIzZu5jZvVU8Tg+zp85m0bRFzJ46WxR6HyNKPYNHtz+aN+/70eDf7CcS6yz4vIXrF5ZBmhzIQ37w0dRkeLkvWQILFxrbtjaJdKgwijF7i1Pb4EHM75nE4pabvtPxDqM9owvqPrBnM5Ec+ZGzse29bQWfc9SkHvJ+v7GGXl9vzNBFoQ9cPJ7KyZHQC1R6rHc2CjV7+xp9zFtnvrwiTm0DC1HqGZyzW7NOY2m2fs39V/HXrz1bUP9jA3uN8oOFWAM0TDxmQkHXKRmD7CEvDB5ad7Zy2f2XEY1HCcfDuOwu5q6by5qZa3ot1vtoSJm9rZCa3WfGtkv61oGHKPUMVry3AYZZaKig9Z3nCr/Avn3gLPy0W/fnqVMsCIJlguEgl957aTfflnA8TDgeZvp909n77b0VreiKsTCIU9vgQJR6BoFj4tZn0brw/nfVVkGB0SOjDsLo4L8Kv5ggCKb84MkfmDqrAhyKHmL5xuUVm+L0aLLJFTK7F/on4iiXSRmd5AAazvsUQyL526XjjAPt7WWRRxAGG8FwkF8+/8ucbVZvW91L0hRGOeLNhYGFKPVexnfmLBIuR0Hn7BoGHbWV77zTXyhlqkyh/+HfbCEbYhGD+974XRWbTU4YPIj5vZfxurxMPq6Rl/a9av0kBcvH7acyjYH9i0oohCH0LYH2AHGdO8rl8obLC+qzI9JB3R11Zf9dlTPeXBgYyEy9D+jcv6+wExSsPvbd8ggziBDTpQBGzHaNoybrcYfNwawzZlnuLxgOEtgf6JXflcSbC/kQpd4H1HRaj4VPESvzWv9gQEyXAhgx23Zlz3r80S88WpBHeC5zfup3VSrTvCZc8xIAACAASURBVBRREfIhSr0PeLMqWLDnvMNW2Dq80BMxXQpgnpHNYXNQXVXNui+u49KTL7XUT0pR//al32YdLIaiIX770m854Y4TSlIdTYqoCPmQNfU+4BDRgh1xnv7n0+URZhBRTCEMYWBytDHbmb4Z155ybda2f2/7e7f3qd9f8/3NtM1vK1gRN41rYts3trFw/UK2tW9jYu1Ebp12K8d7jy+oH2FgIko9A1cMwlWUNbStKpHsv4BryHrv0ZMzVWYsju/E5l6WSOgNsiVqKTZmO90342gotjpa5oBi075NPLL1EXH2FABR6uaUUaEHw0GCtiJqpGvgjTfg5JNLLlM6/TUXthW6pcqMRQklOnFHwKah5SHw3DbRKFbTnwuYBINGrv5AwCif6/MZRXgGKeWIdsjlm1EIxSz5mA0ojnbmLwwsRKlnEMnuP1MSLMXImqHgnes/x+gnXziq6+dS2lYfgP1Z8TeNa6Lthm34Pz2eHTVQvx98m8ET6QQ6obnZKGLTH4vWtLYa8icSEAoZVfXmzev/A5UiKZcCzOWbUQjFLPmUqy66MHAQpZ5BlYYi5tGWCbQHij73mye8xoNHce1cSnvKqCmWHoADIc7b88hqZr9WBaFwz4OJhDHT7W9FbIJBQ6EH00zCoaTi6c8DlaPAqgIMvr0d/23XETj4Fg1Dx+P79t14Tzola7+5fDNcdhcaTSSeP21kMd7q4uwp5EO83zPwJMo7zmmobSgqZzwKnhxT/HAjV4z2ZfddxvKNy/M+AAdMnHcgcEThZRIKGWVm+xt+vzEgMSM1UBlkWFGArUvmUfebicwZ+hyLT36HOUOfo+43E2ldYu57AbnDyhx2B05b7opN1VXVRXurS5y6kA9R6hlEKDyGvBCa64t3xuq0FTMaMMg1a+mIdnDfxvvyPgAHTJx3Q4NhmjbD7Tbqxvc3BuJA5SjJpwDHaA/Ne39G0AUhl7E/5IKgC5r3/oyOXW+YnmsWVmZTNrxOL2tmrmHNF9fkVNZaa7bfvL0oy5bEqQv5EKWewSF78YrTCi07Woo7UUNHlgmAlcQW+dYBn297HluWn0NqBjBgTH8+H9iy/PRtNuN4f2MgDlSOknwKUK/6E9EsTrFRBf7/+nLWvlMhcUs+uYSF5y9k7NCxtM1vo2lcE03jmlh08SKcdvM/2CpbFau3F1cwRuLUhXzImnovE2gPFOddn+Wc1Bp3XMc5FD2Ew+bg5jU3s8q3ikvrL+1yatu4dyN2Zc+a81on/5mRmgHc/erdOUUcM3RM/3Ci83oN57FMpzKbzdhfgrXnXv8cfD7DKc6M/jpQOUq6RTuk+YDYlI2WmS2s/P7VdB5nfm6nE7YE38zZf3pI3IYNG7op1F0f7Mq6rn60A2Cpiy7kIqdSV0p9CpgLjAS2AHdqrZ/OaHMu8KzWuiR+40qp3wOfAvZprSeXos9CsEFZDfBda+olCJsz8+6NJqJEE1Gm3z+d2y+9nVs23NL1QCuUGkcNdmW3PAN448AbvVLUoiQ0NRnOY36/YZqurzcUXwkUep84E/bCQKU/kksB/vY4D+h95n+LGtqPS7N8FBgqWO5ER1IXXchGVqWulLoEWAX8DXgKOA94Uin1c+DbWuty2anvBpYC95Sp/5xUazshq2q9CMXsO7GZrxZ+min+zX6iiezOc/Mfm1985xo+NHwSX5w6iz9v+zNb39vKjndez3nK0ueXdpOn4uNnPZ6SebmnZuab9m3i1y/9ms5YZ9exzM+hbJRxoNKfyaYAh5/3CdiUZTauoPb8acb/iwgVzJnoSNa+hTKSa6b+Q+AerfW/pXYopb4C/AKYoJT6vNa6M+vZRaK1flopdVKp+7VKvpKMXWgYVj2s8As89HDh56SzdCnMmgVeL5v3be6mPEqKghf2vsSm9a8bs01bNfFIJ04gYvKrcdgcKGU+yhno8bOZM/NspD6HkyljAqGMgUowHMT/8rLKXg7pIxrHn0v1puV0mgSxVuNg0olnFx0qmM/0X3EDXGHAkEupT8ZQ7F1orX+vlNoIPAo8kTTPDyhiVhsquOq0qwru3//2o5A74iUn7/xgHqO/+11oaWH/4f3Fd5QPDQmlj4SuJTqNX0sW+0wui0G/cqIrkEJShqY+h5OrypsVMMVAyClQTlKz6c5Iz9+uw1ltzKbvWZE/VDCLtUfWvoW+QGWzoiul3gFu0Fr/yeTYBGAdxvLzLcD9pVpTT/Z/EvBotjV1pdQNwA0Ao0aNOnPFihWlujQvtb1k2ax+4jEnMqJmREH9/2tvgHcSBy23H+Maw+7w7q73ww7DyQcAm41dE0aw71CBtdnLhEp+aGbOdjZlY+zQsTk/q46ODjz90Ez83qH32HVwl6W0oanPoTpRXfZ7TegEG/duNJXLpmxMGTUlq2d4qegP3+m7h95l5wc7e+wfd8w4jqs5Dv71L3jnnewdjB4NdXX94l5LwWC5T6jse73ooote0lqfZXYsl1J/DHhDa/3vWY6PBFowZvSO3lTq6Zx11ln6xRdfLNWlsf1Qoa086zQEvxsseNS99OnbuPmJ/7A8cLjtlNv49vZvd733dELwVsDtZtlPruarB/rE9cAUh81hOmP3Or1s+8Y2VgdWZzUDb9iwgQsvvLAXpS0NCx5fwOJnF1tq63V6aZvfxovPvlj2e1328jJuXnOz6fJMdVU1Sy9bWvxyiEWnsVJ/p6WOKAiGg9TdUWdqZUl9V557VsCcOeY5ANxuWLIEZs/ut7/fQhks9wmVfa9KqaxKPZf5/SHgu0qp4VrrHnZerfU+pdTHgUeAaaURte/xRCBYbbFtEWa094/SDSGeGnCEQvj21HJDtcoailYWcnjuX9t4LX/a9ifiiTiHYoeoUlXYbXZuufAWJi6d2Kdm4HKFmOXyck5htpZa7pC3XP4WnbFOtry3pbiO+yi/fG8XZunyA5FQQaGfkVWpa61/Dfw618la6xBwaSkFUko9AFwIjFBK7QZ+qLX+XSmvkYuvvH0sSyYeKFultvs33lt83xoaU9Z2txtvfSP/7v0yd21ZnjUsp+T3kaO/VdtWcd+V9/G5hz7XNWt3KmcPL/xSe8XnU5DlXFv2NfqYu26u6TGX3cUFJ16AXdm5/JTLmTJqCmCU0S136F8+f4v2Q+2Fd9pH+eWtFmYpdKBkKZmShAoK/YyKSz6jtf58X14/cOoI0AfK1v++A7vzN8rBipXJ/yRnCad95zdQm6VxOQYmOWrBJ3SCa/9wLZHEkaQbh2KHsndVAq/4fAq73KUqN+7dSDzRM2KiSlURS8R46p9PEYlH+OvOv/Ldv3yXldeuJLA/UPbSmccOOTbn8doh2X40ObCSX74MhXCszKgnjphY8MDNciy5hAoK/QhJE5vBBt4uaz31rA9FS+fCyVH3kdmDx8MbB/9ZXnkzsZH1eoeih7op9HwcrVe8lQIz5cxXn7r+4djhHsdiOkZcx7uyiqXkuuKBK7L2V8r8+fmSm0wYPqHwTvsov3y+GfUr77xSVKGhgvKop0IFFy0ytqLQhQpFlHoGEW05qK0oQraj6F9hOOa0tXWtX+49bkj2qm+9uNReDO4I1LuN1LLvHXovZ+56M6wo7HLmq891/WzEE/Gs55Qy9C9f/oJw1KTsbD7KkF/eSt2CXIVZAH790q8Jx8zvJxqPZh0oSR51YSBSceb3PkcpyqkNoxzFTB16mDf3NBwP77xt3rY3Z/BFoICxL71B3YY6fjT+Ryzethi3w83ctXO58ZwbQZNzbTSfwt68bzPth9uxYSNh8rlX26uPKl1nviI5ZsR0rCv8L5MaRw1tHW0seHzBUTvPrX9zfe7jb61n3kezlxc1pcROY1Z9HXJlZwOIJbIPlDvjnWx5N7tToMSSCwMNUeoZuKqqieVYB04n2N6Gt/aEgvp32B2E40XMksBUSbdFinB4KjFVMUgoSBQS1Kjh+hdhxnl3ElSRrtlrSkkufsYIE8u1NtpQ24DbVm0kxsmgWjn51Uu/QmttqtDBeOBffsrlBQjdHSue72Zki1Y4FD3Eys0rORQ7VP4IgWLGrSV0GivE1yE1o77k3kuKyqDYfjj334jkURcGEpbM70qpJ5RSp2Y5dopS6onSitV3NB432fIDb+TP63hs1R0F9e9xFD8DcKqeWvP9YryYS4UGewJiVVClKUxRKNg2ykbCltuc0LU2evcldLTv6Xas+fgLiEWyhG0lInTGOvMOoB56/SHT/VbMwrnWZK1QU1VjbB01XftSjoVW1oRz8alTcid7vHxikYOZlNPYkiWwcGGP5SCr5KpbEE30NJk3jWvi62d+vSiRt7y7paBlHUHoz1h9Il0IDM1ybChwQUmkqQDuvepeaw0VdFbB9Ffm89g/Vlnu32UrMkeshlPVqB67O4PvF9dfKVBH4uYjVRRm7tcQtUNIW7NaRKKdXDNvLMv8CwiGg7TubKXhtx8inDnO0eCIQbVF14XV23rWtW7d2UrdHXXMWTuHxc8uZs7aOdTdUUfrztZu7czWZK3idri5pvEaFp6/kBmTZjDEPsS0XbHOc6fUnpL1WI2jhllTZhXcZxclcBorJo6+cWRjwZ8zwAttL2T9DgVhoFHINKPHPEwp5QQ+AeTIo9i/OGXEKQXPOK946GrLsyn74c7iTJ8KPhLtXvy5dWcrB6vKWSg2C4XOyrMwcsxEyw/pcBWsnRBnzmuLOeG247n03ks5pCM9BxLKGCx0OiwKkXG+FY/6dFJrsks+uYSF5y9kwUcX4HF68t5XKBrieM/xLJq2iFg8xuF4Tw/6VLtCneeC4SAzHpyR9fgjvkf6fM24mDh6X6Mva9GgfByt5SOTTEtOoQ6TglAucpVe/SHwg+RbDfwtxx/UT0ssV99S4HMjnojjf2k5s8+7KW/bYDxUXMyBhvDQI4oipXz6xBmuFNdUEIvHCjZfh5xALM8adsrX0YKclzd0N0NbyjKWsf6auSb7vY9/D/8mPyu3rOSJt5/oCmtLJxUHHQwHeXhr9sp9NVU1BTvz5bqHmqoadn2wq6D+ykG+OPrN+zaz7OVl3ZwFs+UEcNldaLTp55xJ5ndYTGY/Mwe/H0/4Mc6dTimUI/Q5uRzlWoD3MB6NvwBuB97OaBMBtmqt/1oW6foIpUEXoLhidtjx7GqwoNTDiVhxSl2B95QPdb0tJpyqIPIpxWzHC8hit8oW4E8n/4QZW/8TWx+MTuzKzqwzupuhSxECl1Ly1zZeS90ddabKJhUHvWLTipwDm7iOF1x7O9c9HIodqoiKeZNHTqbaXk1n3NwE/+KeF3l97etdzoJTRk3JmhOgEMfT9O+wdWcrl91/GdF4lHA8jMvuYu66uayZuSarcs7m4JfQiZImDxKEYsmVJvYF4AUApVQQWK21fq+3BOtL6sIOdldHLSsnWwLqLSahy+aJnRcNwbS88cWEU5WUbJ9NAbpZK9h163dpe3MIaxdpFv6tCq0Ud54DOho1ZuWlli+NKe6TezyALWcZI/8sz0pN7UB7gEPR7NEWMybNKFhJFHIPfUVX2dMsSh2OeMNPu2caHxv3seKjRtJI3X/bwTYuvufibgOucDxMOB5m+n3T2fvtvaafezGWHEHoTSzNGbXWyweLQgdYNXpOQe0V4DvRmjexo9gZqeqe2rOhtgGX3VVcXxavV24iVbBjOHg+OMyIQ7BobYxb10TZ8wsn33jFgT1O0Yl1qlT+aM2Xg9u56Y83dPOMtpplzKozXeaa+5JPLmHKqCldM8FciVXcDjcXnXRR3vvIpKBMaX2E1+Vl5TUr8zfEULbr31pvybyeD5uyMfaYsUz4xYSs/R2KHmL5xuWmx8qZzEgQSoHVkDaHUurbSqlnlVI7lVL7Ml/lFrQ3mTr7+1z7us2aQ5iGb75ox/N5a97EIxzHFudkpmHScZO63voafWQrm9tfsCXg1dGwbKoR555Ca81d5yjidrIOLoZEjVe2z9Jus3eFjOXiro2/7aaQrWQZy+VMN+2eaewJdg+9S5njF01bxOyps7sp3HIo4P6SKW3nwZ2WvqNS4LQ78Tq9rLx2JTMenJF31m8WFQH5B2GVYAURBjdWk8/8DPga8CjwJMZa+sDF68V/81Oc+qML+c+PxvPOWj/5pvVF8mmNn2bZJvNZQD7SE6V4XV7+/ex/Z8nfllR85rhsJBSsbYC/joMfjwLnOGjaCf4Jh0gk7GCWzEaDKwY3Pw8RG2waCU+cDEop4mjcOFEOJ7OnzuZXL/wqtwDJzy0z6Um+LGO5TLDheJgJv5jA41963JLTlBUTfTGUJFOaxbrplrtLLlds3reZ/Yf3s3nf5pwFf0qF0+7kjkvvYNYZs1ixaYU1X5Qsf1O5sttVihVEGNxYVerXAAu11reXU5iKoqmJzrnfhBd+lrudgk/NiLL/geV4vprfUe7c8U38v38sJ16EIl69fXW39bqyzgrKUbY1k5RSdUHCBs0zYdsvYOVpELJnCdVTxmDgzrON84ZEQCXAE9E4EjD53QjPnxDhzr8vJUa8oHtJXxPNlWUsnz9DZ6yzIKepcqUqPapMaUXWTU/3Mzg7ejbBcBCvy9vlMR5NRIvKCmdGTVUNdpudW6fdysL1C4nGo93W6KurqnHYHN2y8ln1RcmMikhRrkGYIJQKq0pdAa+VU5BKpGH0pPyNMOKif/lPP98hv1L3NfqY9+jN3ZzeLKHosV73xv43yqZ47XGw221EVO/F30ZtMH4OxHIZPjREq4wXwOGkM90HydwtG2pIfiZpgwKLn1EoGmLHkyvBvz3nzNRKethCnaYqKlVpMEjwM5fhb+ggMBwa9oNvUwhvhB5104PhIMs3LufR7Y9y4PABNu7diF3ZORQ7xB0T76Dujrouk3e6x/jRYk/ANcddwNLr/oDH6eHLU76Mf5OfLe9uof1wO7U1tUwaManH4MjKdzekakiPqIh0zAZh4z8YL+FsQkVgVan/Fvg88HgZZak4fI0+vvqnr+ZXCgqWOl/jOxb69Lq8tFy7io+tmF5wBrbMmfne0N7yzKg1fG6bnVWnKSIkes28bzlhTC6OQlZ7HNr+9jjBP63F68g+M22ub+YbsRtz9tWfnaZa7/k/NH+tgwSGNcQdhnnToeV+aGo/Uje9dWcrl957qWmYGRgDm1S52SpbnkdNgb/juA2O/+Nf8HwBcFofFOUrDuOucrP2S2vzzrgzr7dhwwarogtCWbGq1PcCM5VST2Io9szcpFprnWcBs//hdXkZFob3q/O3LWTm3XTqpUUp48z1ul0f7CqPwlXwh4lxIr1dmNfKZ1LGAUbcDg+dEufh+XDjCyHCdjjwi4sYfuBrNNadga/Rx8a9G7n0nksJJ3KHPPZXp6lgOEjzuz8nmBZYEUr+v3kmtN0ewrNjB8FwkMvuuyyrQk8nnojnD0cr8Ht1h6H+gOoaYFjFzHyeSl4z99y5fO/j3xMTutCvsarUf57cjgM+bnJcAwNOqQMom51uplwzNDgKdER32uxE8vWbxlnHTenxsHnzwJuFXdQizmiyAm1vUwEOf6nY+MXnk8xMF4OX76T6tWrmrptLLBajM9GZV9b+6jTl3+wnkeXLjwP+qU5m19fj3+wnkrDmLxvTMRw2R9YCLmA4P2oNEYvWGhvgezkCOwq3hki5VWEgY0mpa617e85WMSinC8jjoauMIiKFdWwHbV2pX3fOV3vsq7KXp3KuVskCLb1FiXLJlxTVfWvVucsZBVeVo986TQXaA4Qwn1UfcsGTY+PM9vkIPPdjy3HjNVU1JEjkVOqO5J+CaY8a3FFjwOUOGwq95X7wONxQX5w1pKJ8GAShhEg99Tx0qKglhXPAXliUX7WjmkjE2jkuu8u0qtb5Y89n5/s7SzvD1XDNFlg18YjZtdwcfxCO7aS8HvfaUBzRMv/iT2mHx/6gOH4ahl0rSTav8EqjobaBIfYhWQvMPPQhO//tNNo57U5Lij2mY/z5839mxoMzjni/J7/r6ohh5Wq532jbPJNua/k2YKUfdh1jJCqq3w++zeCJAF6b4cwoCEIXlh9xSqmRwHzgLGAscKXWerNS6lvA81rr58okY58SybN22tVOFTbVPH3U6bTuslYG0m4zC9iGWz5+Cw/844GCrpsXBX+a2IuWcG3EpldNojwXTX0tKtl9InmdMt3g9lqYeEOElq9eStML+8Dj6VEAJOUVnh5qVSrSvdHBqKs+a8osywOIcUPHZVXoYHid+2+5Bt9JlzNXOYhYSFkRiUc4fdTpXSbvLf96lfZ7/pvaD2JMei9NSQNttxsm/h1DY9S/b8P3aqzrWDeqqw0nxiLKvgrCQMZqRrlzgABwNUZRl5OB1DzueAxlPzApk1m4kPVWhTKtqf2n1x4spUhdaAXXv2RkfCu7WVzB2IMYddHLca00BR6pwvjFl3HEEnFA0AXNVx6m44HlptnnUl7hpSoDmqJ1Zyujbx/NzWtuZt0b61j3xjpuXnMzo24bZamOeDAcZMYfspdsBQgRYcfza/HOW8ia+xIMsVkz59y4+kZWbFrB9vbtnFY3hV/etJrb/+Zl9na3obRrasDlwnPNTGZffxeLGr/F7OezKHSAG2/MGS8vCIOVQjLKPQlchfFY/Le0Y88DXyixXJVDAQqgI9JheR31qlOv4uY1N1tqmy086vfPLC2Lggo54a5zwRWHw6UIM8uFhhOChiKsBEe5UhEH/P9cjd7s6pUCIMFwkMvuv8y0OMzh2GEuu+8ytt+8ndWB1VkL0Fip/OeKJVP7Tgzh2wT7fu7hNyt/wH88/f2cxYoe2foIj73xWFeylnnKRsvzK2l6Zpfh7FZfb5jSUzPvZcuMhDchk3hytxsmWcshIQiDDatKfSrwGa11QvUsqt4OjCytWBVEAeu8VmuqA7TsaKG6qtqSA1a28Kh94XYok9KN2DFP01oGXjoBzuydSxVNdRQ6q7D8Wzjkgh0eSPRSARD/Zj/ReHZHtM5YJxN+MQG7sh9RrMmypoVkWwvbj6T2nTcdbn06wi1P/chQ6Hk+m/Q8+QDNj8zInnXP5zPyBJhhk7V0QciGVa/2D4DjshybgBHHPrhRsOWZVZabB9oDlj2qs4ZHJSrNZbwIFDxxEhU7S//omI8yb+o3WPqEi3X3gjdsOHDloyoO9R+9vNcKgATaAzljwWM6Rmess0cBmvQlgFyypvsmgOHIFnTBTdMiBFWkqO8vEo9wzYPXdKuS14XXa6yZe73GzByMbWq/rKULgilWlfoq4EdKqQlp+7RSagTwbeDhkktWKVh9WGlo/2BP/nZJcj5Ak+TLKd3RS97pZUXD3qF9LUR2HvI9xO2f/iWzF6/n0ne9tP2qhiVr4ZI3FdnS0wPYqxwctms27dtEPEvoYilj2YstxZtaAoDcFePKQTgeZu0ba7OWraWpyUhJu2QJLFxobNvaZC1dEHJg9S94IXAQ2AI8ndz338A24DDwg9KL1ve0HWyz3lhB7THHW26e6wHqsruYd968HrW3M0n04gO4bFToDB0MB8XV25MlOJMKxnPbL5j4yZk8d7LTKA2bBZu9ioXrF7Lk70u6ZrnVVUZqQpuylbwMqq/Rh8Ne+FpMKBpiy7tbgOwlW502R/bvycr3l8egZGY16MLjMTLGLVpkbGWGLgg5sZp85oBS6iPAl4CLgRCwH1gG3KO1tmCQ7H985y9Wsrkn0TDp/M9Ybp6v2lNKkT/x5BMse3mZqXPTCdrNP/X7Fa0YS0Kp4tc12JIKJmFhPKTR3de8PR6CX7yW5jvm0hEx/8k7bU7sNnu39KmpymGJRIJLJlyCx+nhJxf/hCmjphR9K9A99r2htoGHrnmIKx+8soeznMPmwG6zZ13uufOFO7nytCtpGtdkmm3tcOwwC9f9B6GEyfklTO1bSsdBQRisWI5T11pHgN8lX4OCze9uLkiZ+M7MXtnJjHzpKlt3trJx70a+//R3CBHGjYt5a+fS8sU1NI1rwrdJsbixoEuak3wwpwYVqVKWHeEguhIGDCWUIWHDcuic0+7sseadz0P8gpMu4Lld5ikbIokIT771JNMbpvPD9T/ku3/5btGx6pmx76nv7hHfIwT2B1i9bTUoo4ToVaddxcSlE+nEXKmH42E+sfwT3DH9jq6Y9nTFGgwH+V9rs0StWpypVyXAVeUilGP835+L4AhCpVBwfi2lVBXgzNyvtc6TS7X/4bQ5rc8SNUWZUrOlqwyGgzQvv5Qf1v+4K21niDBEwzTfM522/7WX5z0HSqLwqhLwzXO+waQTzugaVHx5ype54KeTeCVapqIxhVCqmXpG6lcrZK555/MQX//m+pz9xbSRTzjVxyX3XsKb33yT473Wl27SY99TpPqb8aDhUX7T2d2jMFpmtnDJvZdkna1HE1Hmrp3LvHXzmHveXL73se8dCXd74i+ocMTkrx6csWSkRK7PVMHXn4czzvoEK0/VPPH2E6aZ6PprERxBqCSsJp8ZqpRaqpRqAzqBoMlrwFHvPanPFJr/wR8QCZtn9op2HsL/0nK21ubpJM+MVCXAHYEnz7mT2z/1S2ZPnd01MDnYeZBXYhWg0KHPZJh77tweA7Wxx4wt6TU6Y52MXzK+m5NYMBxk2cvLWPD4AlPP8FzWgnTHt3SaxjXx9TO/nlOWmI4RTURZ/MxiTrjjBP74+h+Z9eAXOH3dlYbiNkEBKk8JA3sMDroV24/VXH7K5TjtJqMD+m8RHEGoJKzO1H8NfApjDX0LWeouDDR2vPF367PEo1Q83dJ7xuLs/dt6wlkmb50OI3wunO/byyGTQ9v4gucjLJ39EJ7ho3scL8ifoJykir30sk9gFTa+9/Hv9cq1wvEwzfc30za/jVffebWbWd1pd3Lj6hu7zZ5zWQtymbBPHn6yZZk6Ih1c+eCVxmd/bA7ZLTxB4nZYearmUMda3Ov/SkInqHHUoFA9fEn6YxEcQagkrCr16cBcrfWycgpTaUQ6Q9YSsGgYUkQ4UYrWna1MYD8oTgAAIABJREFUv296dwennnq22/XaP9iDyp38K+eApNrlZum31mV9iG59b2ueznuPqgTEelOpa5gWP9H0s9n1wS7L3diw5cyylk5CJ1j+6nK+85fvdDOrp8zUi59ZzF0v3MWamWu6wiHNFHvJTdj5BqsWHeUOJSfnKZk9Dg+3XnIruz/YLaVPBaGEWH1UhoDd5RSkEmn0TLDmVKWgOlFc+rWs6T3zrFF6jxnZ9aDM1W5qfCTVVdU4bEa4k9vhthROdeoxJ1dEOVRnrJcVOoCCSceYK8aG2gbD18ICk46b1PW55yMUDfHo9kdzOuF1RDpovr+Zyxsuz9ouHAtzOHa4ZzIXyOrA1xdoNNX2ahZNW9Rt2UcQhKPD6uPyduBGpQZCYLR1Fn3pbsttPyjSTzBfek9TNARPHE3cwrdRO2QE7/7Hu/zq8l+x8PyFLPnkEtrmt+X1uF60f2phMhWDBpeZ91UaEQe9vqbuimYPT/Q1+tDa2mjnug9f1xWbno9UXHjeNK2xMA+//jA9szUbxHSMeevmUbu4lgXrF3Qp92B7Gw9vKl0BIHfYGHAV+92Ip7sglAerSroOmAJsU0r9Rim1OOP1X2WUsc84wXM8NWEszViLzcSVL72nKQpqh46mrmp4btk0HDdhcpeHfSGzohPeepfb11HW2brS8Lkpn+eiEy/K07B8MpgRc9jYfPANUyc1r8vLh0edbulzqbZX0zKzhSpb/lWueCLOtAnT8raLJCI89PpDqBwfSjQR7ebw1vroXfivmIAtEssvtAVGdMCS9XZuP/GrDCnSu0Y83QWhPFjVRDMwKlFXAZcA15i8Bh7LlxsmbgvrivXkc0U3p6G2AZdFc246k46bxD3X/E/uRgpGDxuTu00waFTEWrDA2AaTSqyhgaE4cebxbD4atA2Od9Uy2pvLgaD3iZPgZ3/7mWn60mA4yMvvvGxpoLH74G6axjXx6VM+nbdtggQfdH5gSb6t727NO6NP0RHpoPnZm9jsCedfrkmRZ6D4nWdh9utDmFU/g4Sr8N8uiKe7IJQLS0pdaz0+z2tC/l76H8E/r7Q8S3QVqfx8B8dR1Vn4dMc32cfGYCCnfHZlZ9JxOUpUtrZCXR3MmQOLFxvbujpjv89H4JiYUYO8TNREoD7Qzvb27eW7yFFglr7Uv9lP3KL5fcwxYwiGg0dSzeYgkUjwu1es5XWKJCJ56wZ061vBXg/WrC552tjjcMOLQEcH3itmcPXEz1qSoUoZPySrPh2CIBTHoFojL5Rf17xuraGCt+xFhOoHg3g/fTVN/6QoM/eqrXmqwumeyVPSr01zs7FN1awOhY7sV4qG8WdZqkhWLHYNvj21lh3P+or02O9N+zZZzrIXjobxb+4ZM25GTMdMndvMGOkeiY5bN6WHnPCO1TGAIudA8aw94EmNQRMJLtpbbakw0czTZxbk0yEIQnFYVupKqQlKqV8ppf6hlPpXcntXRuW2AcVdJ1vP2KacRYS0+f0QiXCgmoLXjZe/upw9Hbmrwo0bNi77bMjvh0SCoBOWTYUF04xt0AkkEuD34/vwl8o26rPFoeWhajz1kzjj+DPKdJXSkO7UdeDwAcvnrX9rPYH2AJFEfktMTVUNNq3yD+40fOmUa9Cd1kdb7gjsHkpJfBOGp+dDCoXw7anN609iUzaWNi8VT3dB6AWsZpQ7E3gVuBp4Abgnub0aeEUp1Quu0r3PzhrrXulTRucuzmGaJSwQgEiEkw9Q8Ex91bZVjKgZkbPN+GHjsx8MBHhsVIiR/wE3Xg6Lm+Bbn4S6+dBaG4IdO/B+fhYtjwzBURr/qiNomLoHmvY4wOdj6rGTej18zqpXeooxQw3fhIIUkk6GwGXJoJaO3WYnFOvIq3irlA3v5kBBg60EsCP3T+UIqWQ/WY5NezPtvduNt76Rlun34E1UUR3rLrxSSkztgtDLWF0xvQ14BbgsPce7UqoGaEke/0TpxetbrISMAaDh2snXZj3curOV5vsuIxGNdi/MMu5GmpxOJu0rfE19T8cePnTch3K2Ge3J7oD22KgOpn+Jbkok5Ug17Utw47AXmRTw0/zL/+Gzv5/BHxripVusUZCwK2hpAY+H5tc6eeCY3nNx/7cp/8b/bMrjZJiOhvChgwA9S4Pm4PKJl+Nr9DF3zRwi2ZIwavA43LTMbOGC338sr1L3nXYNu157g1AhKxbVLnTC2szeoRVRlX2E5Uof4Nls8MEHNJ15PW1O8DfClhHQXgO1H/0E44aOo21+myh0QehFrD6mzwEWZxZtSb6/DTi31IJVAjZrycBQwKwp5hXaguEgzfdMJxjt6FaYJRjtoHn/L+lwO2g9kYJNo8d7jmeUZ1TONtm8yoMbHuOK/XdlvWa4Cn4WWs/NLTdT98yV/GFiCRU6gIbqqecYNcqBlo1/6NWZ+ro31hUWRqhg/d+MQcCxQ3LkTE1jSBRmnXw1XpeXNcNuMkK/0u9RG05nC/5exZ6R/0XTuCbs+cIiNUw9bgoNx52K2+I40KltJCz8tlQCPGH4eLs3Z+303ccAbjd4vXDPPTDfqN7micDsV+D2x+HuVXD7gicY4RwmCl0Qehmrj+rDkDVmazhkqenYzxkdxNIa51mR47I+vPwvLSfaaZ6YJho+jH/JDWArfJb6mYmfoXFkY1YzcnVVNZNGmHi+B4P4//cVxPNkrIMjdcBLjoKpY87ukmfTP1/o1Vj0to62wk86ZHyHk0dOpspCet5vvlSF5xHD671pJ/yfJ44cSxG3wbj2GJ43jWSNDguphp/5+x/wfXFRV134fERUIv8ARsOMLfDyMhuttdktETURqK+bDEuWQFsbPPJI7n53D7oklILQ51hV6quBW5VS3VxWk+8XAX8utWCVwMXv5pi1pFDwuutg1sObnn6IziyZQjsdsKXtVT51/eKCZqp2ZWfWGbPwNfqypiF12Bzmnu9+P4GhcWJlDFXLh9Pm5IzRZ3TJs2lknhNKPIvPlbgl2/VPHWr4g/oafdjyna8gGo/BDsO5rq3Oy/zpdPcsT/7/pmZ454ShBMNBDsfNq/Kl97uh83W8tSfw6w//wNLn4rQ5cWUbLKTWzxW0NMCHbkiAI3ta27gC388eh9mzweOBrXnqA3QOyLG+IFQ0VpX6POBN4Cml1DtKqY1KqT3AU8n988slYF+iTj3F0oMzSuxI0pYMtra9ltPxqP2fW5l17tcKmqlOHT0Vj9OD12U4IXmd3q6worxxwIEAQRXr07zuripX14AjGNjEhnH5asSW9vq60JtX8Nvwc3REOvC6vMw5cGreBC3tQ+1Qb2RM+3bojzm7n9/xsOXQt/2qk7teuIsvv/4Tq9LjsGdR1GmDjJALwg7ozDGrnzHm0u4V/U49NfeFqwtzRhQE4eixmnymXWvdBFwO3Ak8A9yF4Tj3Ma11exll7DOGn36upWxytnD8SNKWNILhIE8M259zjbL2/Qh0BC0Vg0sxwn3ElblpXBNt89tY8sklXXHA276xja3vbTWtx/3Y2Ai/OodeMXdX26sZUjUk54DDf8IBy3HfR0VyVjptwjRqHDWFnx6PdcWqTxzVmLuxgtpOG/iMgctznTmSBCn4W2eAQHvAsiw3tdxETMcsfYdTj5/Kmplrug38XDFyDjTNcEfgoilXdN+5aFHui4/Jk81QEISSU5ARVmu9FlhbJlkqjsYxHzZ8/nOhjIxd/5+9M4+Pqrz3//uZNcnMgCQoSwJFTIASFPeljb0gihq8XQSdurS0pZugLUt7icu9rW2tXOtSboX+WrmtWK2MYi0VAijWLeoVl6IFBBJcgAZEEoXJJJn1+f1xZiYzmXPOnAmJBnjevvJKMufMOc8Mcb7Pd/t806Itzc1aaBJNfUwbvGHwSSk1LyrwxdGIC7D8rzGtalrW7yltd9Aq7cfeOzY9j9vj9DB//Xzqr6ln4pCJfOnj3/apQbcJGw6bg3nnzEvPIw9sDtDU2qQ7YrOxspTEa323nkxePm4Bhz43lYsfvLjg54ZkmKbWJoLhIN+J5lEalOCccWX676DIWQJx4xRNEXZGDBxhbYxpgQwqHpTe+AU2B2h6ZiWb3nySdScZFAYY3N8mwb8ZOC/jweHDYckSmDMn9wlLloDjU8zxKBTHKAX9XyeEmIpWCT8M2Au8IqV8qi8W1h+oray1dJ4z9fmYFG1hlmZgG1saiZhVNAm4d8Q+ajuxnON22pzMPNWk0v6h2qx53CmN8NqHavnllF8SNxntebjYsPG1U77GvbX3Zhnu1IZDj6ph1ZSIns+it0pRDF4/Yxg3PjKjR8934aCytJLAlkD+anIBS5pXcUukDa/Ly4TR57Btxyp9gymhevvHMKVv8s/jyrQQeXrjF9jBsi3reKFCC7l3xx0BIcCe0I57wlo4r/4h8F6tU/g2ezZcfjnU1cH27QTHjSbw9dNpjL3PWe0nEAwH8bl9ffLaFApFLlbFZ4YLIV5B89KvB85Pfl8vhNgohCjvwzV+atQ31Vuqfr/g3eTPoVC6OAqSwiN5AutROzyeJzWZdX4iatgrHdgSMJyznZAJ1uxYQzRR4JhXtBGbwsJewOPy5Bj0fPir/didfS8T2+mE1e9vMJ1X7jAR2YkQY9KoSZbD5BKZDtefMLDcNPw+JCTY/erTfRJBue+N+7L/Xqqq8L9TYvg/ftgJf3wcpm+Fc3Zr37f/BmpaPOkagRyGDoX776ch8CvKxzzB3Jd/wh0v3cHuQ7tzBuIoFIq+xWqh3O/RvPMaKeVQKeUpUsqhaMZ9KPC7vlrgp0lji/nAlBT/9n7yB0/2B5+/2p/XM444IFFgD3jdhjrdxxtbGg2nd4WiIQSiR/lkewLWrXTjE/o63x6nB6/Ty3VnXcfPn/u57shSI3xuH4suXFTwmrrjsDmwmfw5pyrAzaabHec+znQT9+Mnf0xVWRVFtvyRhSxp2c6PTHPYq06K4/vI2tS1QonGI+nNBQB+P764nZUB9Nck4Kor4LFqeGUEPDYext4ADSMS6RoBPTKjRKn3OCETOQNxFApF32LVnFwA/IeU8qXMB6WULwJ1QJ6B2Ecm2gd4ngpeAR+m7JzNlvXB53P7qLDnESuRhRv17Qe26z4+YuAI0+dNOWkKdlFISZ5GhxPoDNPcMTtdkHdv7b3ce+m91H2+juvPul4LOW9cwh0v3aE7stSIYDhouEkphDOGnkEC4w2URDJtzDTD4SMep4dDcXOZ1id3PqlFFmL5dXMz54Xv3r3F1FPffRyssG0trCPB4rnhRCS9uQA00Zj6enYNdeMxCtoI0op1ITcE3VB7jaDNJKCSL0qUtbFQKBR9htWc+gdoAjR6dAAHemc5/Qt/tZ/56+fTGTHOd7qiUBGygc+Tlj3NxOE7Dg62Gj5fAM4YBY04HTt4rO7jnTHzvKxAUH9NPZPun0RcFjArVsAXr4IDnxmdkx8PhoOU312e5Yll5vG3X7+dNY1raGxppKqsCn+1tukJbAnQ2NLI7oO7CccOfxTcoOJBeJweQ0983jnzmDlxJjc9fZPucRuCeDyGWbYkkojw5gdvEpHxvEVt6XnhwSA79rwFedLKW0VLYeF3qyOBY6Q3F2lqami8eTahV++xfLuETRDYHND99w9sCXDf6/eZRomyNhYKhaLPsGpKfgn8TAjxupQyXS0jhKgAfgLc1heL+7RJ9YFf+qdLaIuGdD9IIw6ou8TJxD+upGZc7jjJUExfTS7FQOkmEg8XZNT/8wv/qfv4hnc2mD5vXeM6BrgHUDOyhufef876DdHUzwIToHvJm5mHFo1HOXGxNlQmHA/jtrv5wdofAJrRMwuFF4LH6eGyMZfx4u4XdY97nV5u+bdb8Lq81F9TT+1DtVndATZhY6X3W1wcXmx6n9LiUmofqiWaJ9jhlKKrbe+BZRzM166dGs7WB9XvTmy6IkRVQ8ebboK6o2eYG3Y1ZL2XRmRGLRQKRd9iNfA7FU0mdqcQ4mUhxCohxMvAzuTjFwohHkl+HVVxtpqRNez98T4WjroGZ5zcsKeANsLUrvyKbt7QbaD4lqLIM5CYXee6Btiw8ex7z+oey+epb3h3A3PXzS3YoAPE7NAUyq1+Nsvjd8Y7CcfDaZnScDxMR6yDjlhHrxl00KaBzTx1pqEQz9pr16aL9/T6+psXNLNr77auLgY9JHxuxOeIxM1F110xuOf829LzwoONmwlb3bD1skG3x2Ft7Z91Cxf91f68I1Mz6W6Y9XLoRqSjFgqFos+x+nEzGGhMfgEMQNN7T+XYj+/ldfUrvC4vi77xIBUbxjPv+Zs1I9yNzkg7y1/5HXPOzxbX+1xsGO/L3YbtTAc6DhQk2ZogYRjK7Iiay4xKZI+Nqcvm0vW2qsqqCvL4eh0Jc86cg9flze7HNuiLh+y+/hSNZRA1qeUSQhuik09HPe6wMfP8G9K/B4Z/BFZHsBfiqec7NwGL28+n5jx9Y5qKQmV62iWOEtoNIkvdDbNZhCbzOWr0qkLxyWLJnEgpj8pCuEJ5/tVHdQ06aK1p857+MRM/c07aSwP4acvJPFy80fCaTmkjZqVfLIkNm2Eos6A8eaEIdL2tVN3Bp4YgKfCjoWewLTHus/DqekND6bQ5tSr2PMRJ0JbsTwdNXAcr4jqS3p2EJ+C50Q7cbyzDX+3X7RXX2wQdDB9kwZO5qs+LLlyULRpkEqEBOLf8XEYMGKFGryoUnzC9+TFyVBMMB/lLeJPpOVEhqX3w0qww/PPDo7gMbK1NQofIX0md9RyTUGb18XnkSw+DeefM0/1wNtKf70mVfU9wYz/sfG0wHGTpW8tMPV+n3cWHoQ8tXS+zmr9qWDUOqzq4BVS/l4uB5ucLePTDZ5i79oemnQipTdDtF97OldVX8tNnf6p7Xt2Guqy/61SERg+P08O3T/82g0sGK4OuUHzCWDbqSQGa7wkhfiaEuKP7V18usj+w/M3lWPGDo9HOrPadxspSwyK4hA3DKWtGLDhvgeEH5c3n31zQtaySKjQzQi9P/dNJP+2TtXTH6Sy2nK8NhoMse2NZjiZ+YEsAKc0taigawmlzWspDZ7Ycjhw4kpgVa22joJx6yUCjScjZhGLtlnvFC2lLM8vJqxy6QvHpYSn8LoT4KrAc7WPnQ6B7tZAE/qN3l9a/WL1jtaUP3U5ibP1wa/r3qmHVeGxFhBK5RWwlwk3CJiyrvOUzrk++86Sl62SRmZtN2p6iKHS6tCEeNncR9RmFZoaXkRKJJCETSKThnPfUfVzxwtr49K5RYndnFcGZ0b1SO1MTP18oGTTvc9qYaax/Z33ee6VaDoPhIDMemdF7efIMmg6+W9AmIGWUzVIT+cSLMms59HLyqU4ClUNXKD49rH6s3gY8BnxfSmk8mUKhjd08tC/9azrnrNPrbne6ePzKlcx4ZEbWB6NEIqVMe0JWPyxXb12Vd3mporYSZwntkfZsw5Ax63v+izD+APgvno13ZG6rHnT1KD/z7jP8ZdtfsAkb7dF2PE4P0bjJRkXAv70Hz55kI1pAPUEKOwKHw8XjV/0tq37BCCua+PmK/WzCxuWfvZwb1t5geE6KlEKe5vlarHMosJ2t0PGxVnrFzYoe9drSrBYmKhSKTw6rRr0M+N9j2aBfNuYy1jcZF1KlEVD2flfuNZ9HY/TBCNp0s6J/FbH4ksXWPiz37TU9fJpjBHMu+QlNrU00tzWz8s0VtOcEXTRZ2PEHYNYWF8wZr3utlOcbT8RzKqbzer1hGNZpx2VzEJWFC8/EkcTjYWY8MsNSIVa+sLJAmBrJVH/7msY1CCFMQ/XjysYx1KvNHNc8X3Odgiz6oE89hZVecbOiR6OQeo8LExUKRZ9g1aj/BZgEPN13S+nfzJw4kxvq83tpSHB+nL33yefRGH0wzjp9Fs8eepZJp0+ytMbL3nOxfiiG7XNbo82MHTyWWafPYuFTC3UNOmjSoE2lgMulq/et5/kWgg1t3nioBwY9EyshZcgfVt7ZutPQUDuFk8YbGhnqHcoT25/I28Z1/TnXp3+uKqvCE7cTsvduV4Jd2LFLiFiq8tCwkudWIXWF4sjHaqHc9UClEGKZEOJqIURt96++XGR/wPL4SAFL7K9nFSUFw0FWbF7BjpYdnFR6EldWX9n7H5DBIDP/9BYlJnreYRFPF0yZ6sRLqOhwwNq1ObK3YK1HOYUTB57k3sETBl8Y6h8vZvy18wyrp1O47W5OPO5Ew+NW5UfzVWq3dLQYFn25wlHW1P86fR2zgTgu7MwcfXn6d3+1H5GnAK9gJKy8ciUuab1xJRVpsPI3ZyTOYyXNoVAoPn2sfjKMQZuj/i3gQWB1t68n+mR1RygyGiPw+nJAC1OX313O3HVzCx52UhCBAL5OyfoHwR3FsN0p5d12Rk3U5wSEb/sZ1Oh/kFspLEtx6wU/Y/qpV3OO80Sm2yew/TN3UvPqfmov/SGxhHk7n5SScyvONTSkVuVH81Vql5WUGXvyLmi6/25oa9OGuRi16kl44hEH3hPHQoP2b+tz+5jtvcBaq5rV0LuEL4/7Mo+Vzc57XYdwsPDzC9m7YG9BRjmzzW3W6bOUh65QHEFYNep/BA4B04CxwIndvkb3yer6GxadrpAbml5aoyulGYqG+mYcZWMjJBLU7ILrXsXQQISiIbbs38IfNv3B9HIb9rxgeGzEwBHpUab5uPW5W3mscRWvRN/lseJ3Gdt6K0vffoCqe6vyqrNFEhFWbVtFu0Fe2mrrlFEvfUrtbPzx4/FIg9ZCqW0uCASyr+PQNhrOuNYtsP5PMHVrGIJBqK2FtuS/7YQJloy14cS07iSvteuMSuPnSLhk1EV8VPdRjmiMQqE4urGaUx8DXC6lzN/PczSTwNo2SMLxH3Va6vs1ygenKsuLg8UsM1EFS1NVBS4XQSK8fTyGnl8RDpa8tiSvhrmRIWrY1cCNG27Ma5BTZGq/pzY2c+rnWHoukFWEl6rM7kme16yuYaJnNPMiUdDbpwhYclqMW5q24s28zm/n0LTuz1R+EMO/BbyZb2ciAYEAzJpF1bBqrX3PTItHwtToCB53GcgJd1sPQGNod3o8qt45p5afoYy5QnEMYtWobwRG9uVC+jvBcNB0LGd3Xh/QTkUBfb+ZZPZU33rirfxk3U/SPdWGYVS/n4Y7f8Cll6PNvTYwDp0yhpX6qmlV03IeS0Ue2sxE0vuIEkcJM8bPYJh3WI9bp4wKEn2P1zP7DTt3nBvXfd+idlheto/UVsTr8jKreSisM0gfhELQpP3b+qv9zP7rd/KubcJpl/D4jvvynuckv1iRmoqmUBy7WA2/zweuF0Jcm1SWK+n+1ZeL7A8sf3O59ZMFNA6I5i3Q0vvg7WnIPuiC2msFbW6MvT1pbV9S7Chm5qkzcx4vpECut2mPtTPMO6xv8ryNjRDXN+gAYQfMO/Rodh1EVRV4DAr9PB6o1P5tfW4fX953XN7UzeP/DFgK0w8sGajJ2r621PAcgVCKbgrFMYpVo/46cDKaqtxuIKjzdVSzeuuqgpTBnM6iHklp5gvZL9+03FDqNGHLs0ABcQuv4c6pd+oazUIK5AActsORjMump96nkTRsFlVVVLW58JhkFKIymr2p8vvBZvC/j82WbgUMtjTz2NCPzf92BAQ7D1mq2ShxlOSVtZ1z9hwVelcojlGsfup+i4LGTRx9xPbmzhI3RMDJQ07uUd9vvp7q+U/Ox2lz9kjq1BWDhNNOLI/KWd2GOr4+8es56ytkzGqxoxi7sNOW6J1QfU/0xM2kYbPSGH4//oXzmD/ZvM4gqw7C54P6eq0oLpHQQu4ej2bQ6+vTrYC/W/5DEvm2zhKGtgveH5j/f7FhvmF5/61FXynYKBSKfo/V0av39/E6+j2DQnHt3bLYdtQZ01rGCpXSzGc4I/FIusitUKlTl6sY7Pa8OXGjIj4rY1bddjdOu5O116xNr637hmbRhYv48VM/zqlqL7YXc+fFd1K3oe6wxU/yScNmKdH5fPhWraX+O1OZckWHYVFbTh1ETQ00NxN8eDmB99fQOEhSdd40/GdMJFXS+PuPn86f8xAwsc3HK+QXbDyr/CzTv0FPBCpbj+n9t0JxTFNQfFQIMRw4DygFWoGXpZTNfbGw/saBEgEFtB35RFcpdSFSmv5qP/PWzi1obSmp03z57seu/islzhIufvBiwzYxMC7iM4o8CATfOeM7vP3h2wBMGzONiUMm4nP7DDc0laWV/PvD/040Hk1LtNpsNk4Zckqv6IkX3HlQU0PNq/u563+/y4KPHiEicqMZeimAhtZN1B64kURJglBbCM+zLzD/uZvS0YA9dguZqQR0jq+C6Ot5Tx1XNi5rtGt3hAT/dUug9hZd4SCFQnF0Y3VKmx34DfAdsv2OuBDi98ANUn5KFVSfEB/67No2xgoSWre/2aP7vPnUn0h0hLKjAnmESULREO989A5xk7B6sb2Y3Qd3M+v0WXzwow/43hPfY8XmFSTI/Wczy1/rRR5GDByRNZTmhV0vcNPTXYat+4YmGA7y5RVfzmmrC0VDTP3TVPb/eP9h64kXMnEsjdfLzO//jpvuXk1ERwK3ewrASjQgJrCUuCo75Vx4I79Rl8n/DA4yZyN4wzLdUqdQKI4trBbK3YqWV78JGAUUJ7/flHz8p72/tP7FoKJB1k8W8GHbBwXfI9jSTO1Lc2h3oj89zQCP08OBjgOmxVMd8Y60IfO6vPy/y/4fHpd+9Xa+/HXmmNWOWAeXBy4vqFp/+ZvL6Yh16K8z1sHyTQV0Ghhg1nkAxlPO8gnVZEYMAlsChmNzo4kogc0B8qrEShgeL2Z8+anmxl/CmbET2H1wt3GKRST/VDJa6hQKxbGFVaP+deAWKeWvpJS7pJTh5PdfAf8JfKPPVthPaNu/u6BSQUdJ4aHPwIM3ErUu6Z3GFotTWlRqOpfdIRxZ3nchxiuT7rK389fPNzQycRknsDmQ8/jqHatNX8+axjWmx63gr/YjhPEyba2wAAAgAElEQVRuaMnGJYbtgVb1z7fs35KunehOZ6yTrQe2IkWePxoBQ4uOT3ZKmO/ehnuGJjcrBh2kMvknWlKSbqlTKBTHFlZNyAnAWwbH3koeP6p5r7O5oJa2aZ//ZsH32PzhFjrza4tk3QcJi8Lnc1LpSaan2m32HO+70OEdej30ZhuJ9mg7z7z3TAEvKEkv1Hn53D5mnznb5BZSd8ORaoH7+XM/RyK56fybDPviWzvM8zEt7S04RJ4Ml4STx9bgc/vwOE02ggJeFHu0zUrUIM0iYMnZ0GaL6U7XUygURz9WjfoO4KsGx74KbO+d5fRTgkHa8rSBZSFg5jnfLfg2Hw1wFWbQhPZV536BcMxctnX2mbN1DVMhwzt6Ij6zcvOjOR7xZWMuM33OtLG5ana9jV5evdDhO4OKzVMyZcVlTDhhQt61LBqnjfTVq2/IJByPaJuV6ETDvxMJBL52miqSUyiOUawa9V8A3xBCbBBCfF8I8RUhxPeEEBuAmcnjvYIQ4hIhxHYhRJMQwrjM95MkEKCASZeUCk+PxD9Kx51mPRqQQcJhZ8M7G0zPcTusDWAxo1DxGQB7OJKeWJdi5sSZhpPXSpwlzJyYq2bXEwpR9CtEyS/lzb+8+2XDexcJF6MHjebSykuNN2oSrt0EQ7+/AIBh3mGmr2e4b7j2w6BBxgN73NB0UqnpdRQKxdGLJVMlpXwEuATwAIuBx4D/AUqAS6SUj/bGYpJV9kuAS4HxwFVCiPG9ce3DorERu1UHVcINkYk9uk11xWk9el7KCBXZi3SPF9ndhtXsRopreo/nKz7To92lTazLxOf2sf7a9Xid3nR4WiDwOr3a472khlaIop+VFjjo8uZ/uPaHbGzeaHzzcIS6DXX86uVfGW/UBAROhrY97wBw6uBq0w3AqWWa11913mXpGfXd8USg8nN9H+lQKBT9E8v+p5TySSnleWiV70OBYinl56SUT/Xies4GmqSU70gpI8AK4Eu9eP2eUVWF12RGeXfcpUN6dBt/dc/yoB7h5sLRF9IZNyraCjPto8E5jxuFm5e+ulT38ZEDRxoaSSNKItD88W5DmVabkdRqL1BIMaCVFrhMbz5zelwOEhI2aIu25Z1mF7XD8vOKAThuT4vpBmDQvw4A4D9jJrZi/UiHrbgE/xm9E+lQKBRHHqafqEKIk4UQFZmPSSkTUsr9UsqEEKJcCHFyL66nHE1bPsWe5GOfLn4/w4JYC40L+LOrZyUGvgic/AEFF4rZ4nHcDrexpx6DNf/p75rxjXm4eU79HN3HZwSms/LKlVlGMt9a252w0rEja3Pw17f/ykV/uoi2aFu6V10iaYu29eqc+WA4yLYD2/j26d9m+menM++ceYbFgFZC9ZZrCgR5suPZ566Zog1A3BJ8x9RT/2dQqwHwuX3Uf309PqcXT1LkyCPc+Jxe6r/ee5EOhUJx5CGMepuFEFOBVcBZUsrNBudMQBvLepWUctVhL0aIK4CLpZTfTv7+NeBsKeUN3c77LvBdgCFDhpyxYsWKw711XrZ9sJlQwtoMcZfdxcknFL7XSRz4kE2RXciMzUOFu4I9YXPd+ZEHBZHhJ7AvZNwbP7RNUD5oJAzWPPYD7QfYfWh3QYVvNgkjiodSetwwWttbCO/djZSS/Z6kLcpYt5BkvY58pF6nTdgYMWAEg0tyIwuF0BZpo7G1EdDC56kIQ1Vpla7RS8gEb37wpu77YRM2Jg6ZyN62vexr25f33vlee/d/04HugVSWVvL2vn/SLo315z3CxbihXX9XCZmgtaOVcDyM2+6mtLi04EhKX9LW1ob3GCnYO1Ze67HyOqF/v9bJkye/LqU8U++YWb/NXOCPRgYdQEq5WQjxv8D30TYAh8seYETG7xVAjgytlPL3wO8BzjzzTDlp0qReuLU5P7zxK7xV9LGlc88Zfg7/d+X/FXyPZTddwn841hPP0Oy7c8yd/GjHj0yf5wvDLzuv5icfP0pI5LaYecKweB1MurgObr8dgIVPLeSO7XcUvMa6V5zc/kgrrFhBw69upnZ6JwkBIRfYkvbw9GYY87Hgr2Mk7S5r1818nXWfr+P2SbcXvLYUwXCQ8rvLs5TeUvhcvmzd9wxcu1yGw3dqRtaw7I1l/GTdT/IWCxZFtfei3aA2sfu/6b219zLprEn88ZHf88DWh/UjQhJmVl/N9yfdoHOwf/Lss8/ySfy/2R84Vl7rsfI64ch9rWbb+nMBKyog69By4b3Bq0CVEOJEIYQLrV3ub7107R4TDAf5p8uaQQeYODR/oZxeIdrm0liWQbdKAhD79mKz6T/ZBvi3QKa8WU+K3jxhqGwVMGcOwT/8P2qndxJ0awYdtDxywgavlcOj460b9Kx7WBmxGgzCsmWwcKH2PZhtvK0WvXUnX9++WeEdaEVqvjCsetja3HrQBuCkqv1vv+RO05z6okvvsnhVhUJxrGLmqZeAhbFR2jkGEleFIaWMCSGuB9ajfS7+QUq5pTeufTgEtgQKCiXnK45q2NVA7YOXkohGCRHGg5v56+bxueHnQhsFt7WF3LDn4w+p/8Yqah+4mETyMU9YM+j1D4E3AixZArdogz5MJ64ZaM3bAP8/IvD6nwmcEjPOGwuI9nD6Z94Rqw0NueNO58/Xxp3WaMa3R7rvScyG7+gNtCkJQ9wGM7bC5Pe0zZM3or3ntd90kXA5TT37zE3C8AHDWVK7hDn1c3LOW1K7hKHeoYbXUSgUCjA36nuAzwIv5LnGeOBfvbUgKWU9UN9b1+sNGlsaCzK0ZSVlhseC4SC1D1xMMN5VPR0iDNEwf9/1bI/61D1hqHQPo2bcVJrjcwk89WuaSqGytcvIAJqnnhz0oWeg3HY3UkpmuE/lbwc3kkBrSXPGwZ6AlYHUtWI0lmobh96kyFFkPmI1GNQMeqZnHkoazNpaaG4Gr9d0fK2lSIAJWQNtnllJ5bq/438j0vUep85r8dBc/t8Ezihi5daV/P29v+cMsAHNqGdOjJt91mwu/+zl1G2oY/uB7YwdPJZFFy5SBl2hUFjCzKivBhYIIR6SUuq6GkIILzAPeKIvFtdfqCqryjspLY2E0ceNNjy8/JXf0Rlp14/PxmI4HA5iMlbQ+gTgP0nr/PPiYtY/DE7sNugjZaB+8fwvuPvlu5FIIokIq2KbSQiICXDEIOoAZwxm+DUPtGYXVLVqmwkrhr3EUUJ7rJ0SZ4nhyFeB4N0fvmtuvAIBzUPXI5FIb1jMohB5IwEWSHvzVVfCreUQ0Slus9nwXjWTWV4vO1p2sG7nOt1r6UUOhnqHcv+X7z+sNSoUimMTs5z6LwEv8JIQolaIrgHhQgiXEOJSNC/eC/S8qukIoLaytiAP2ij83rCrgfnPLCRqkHCN2kEUOsFWwpxNTrxXJXuTq6q0kLQeHk/OoA8pJUtfXUo0EU17kqFoOx1OzZjHktu+djcE3VB7DbS5wL/ZmsiBy+7iiuorqPt8Hf9zyf+w/tr1un3jY8rG5PdGGxu7PPPuZGxYejqsRg8jcR7tRj6or6d5iIeZM+ycMwtmzrDTPMSjpQOSlbOFKNspFArF4WDoqUsp9wshLgAeQvPaY0KID9F81uMBJ/A6cIGUcv8nsdhPi/qmemzY8mpzAyBgwzsbmH9etqeY6guPml1DwpT4Z3iuaJ/haFK9+4lJk7u0vv1+LccMBF0QmACNpZpn7X9P4Os26KNQPfcEEKiGWf/QvPZLk0beaNMTiUcY5h3G7Rd27fu6z2P3T/Dz2kuv5b95asOiZ9i7bVj05r77J/gLMugNuxpyquHnr5+froYHWOp+iznXhdKRnI0VcR6YEGKJ+y1m01Vg15eRA4VCoUhhOkJKSrkdOFMI8QXgC3QJwfwLeFZKqT/p4iijsaXRmkFP0tqeO71LM555hsIIGOmrwB7/0PriJFQ88SzMbtMMe9J7bPjOVGq/0tFVNBeB+cUJ6ls3UePtEl4pVM895IampLR4zS7Yexdc8VU760brvzY9T9SsGM2UjA1LDjZbzmSyQu7TfKiZG5++kW0HtjFu8DhuPv/mtDhPitT7VPtQLc0LmjnUeairqC21qUl+n1M/h8s/ezlDvUNz6heArHY5JRajUCh6izxzITWklM8Dz/fxWvotZoVXemz6YBNtkTYdGVITaVEACW/4gkRajAVI9AiLrnwyQPCsidR+zU4wo2U95ALi7WmDlF5bgYV5njBUHrQBCfB48NpsPPLNlQx/ebquElyveqLJDUtO9bvNlhXuLpSlry7Nqjjf2LyRB956AJddvycv1Rb3fNPTxrUWEurWLuD+Kx4CsiMHRf8qYvEliwuOHCgUCkU++o/8VD8mX39yd4QQOb3QVWVVeGSeYekCXtu/SbdK2uw5q0fFsgrgAlsCRBP6xXbReDS9tmA4yNKNS63fi2Rb24SvQl0dLF4Mzc34Jk1l7TVreyWHnZeaGq3KffHirDWk2tkKpflQs24LGWD475Aqbnt1xzOmfeXbG7MFiFKRg3Jfed4xtwqFQtETLHnqxzqp8OmFD1yozS3P4912xjrZemBr1mP+aj/zVl9fsK67FbYPJiufvHn/ZuPhLvFOtn6orS2wJYDMt6CkJ5rueX+8GO+rv8vximsGTaS57JcE3ltDUxlUnncZ/jNn9o3h8nrTUYnD5canbyz4OR6nh4oBFeyIfWD8tyBhbEevyDcoFAqFZZRRt0jNyBr+ed0/GfObMZbOb2lvyfrd5/Yx+4zruOPVX/eoF90MV4KsfPJHh/abhoVb3ngRPhe0lE93x+GCd2D6Oy78O1zIxx7j3q33s3rHagAuG3MZM9vH4PviDLyJBLPSIfEXoX5ijz1oQ4JBLdXQ2KgVzvn9Wli+h2w7sK3g59iEjY5YB3FhviFaVPHNni5LoVAoeoQy6gXw3PvPYRc24haK5jqiOtXrLlevG3QkfK5qcpbnXLrrQ9OwcPPbGwmOGk7V0tl5awXCDthWOZCOs6bx0pQr+MoTX8nqNV+/cz3/EYEnB2mFc4CuIEyvYEFNrlDGDR5nPhddh5VXruTul+82PWf8ARj6o+/2aE0KhULRU1ROvQC27N9iyaADPL71LzmFY1VlVYbFV4fDz666L+v36lY7RblzXdI8/xlJ+ffaGPnz32CzsMt413aQGw7+mYv/8hVd8ZgOV0ZrWyYpQZjeIFNNLrVpCIW6Hm/r2bjW26cUJrFQ4ihh98Hdec/7zNhzem8zo1AoFBYxNOpJwRnLX5/koj8tWjtyW9WMiMoYy1/5fdZj/mo/LlvvGnWHsPHse89m32fUZThN9h5hhyYkM+OLHawc+B2K7cWHvY6ITetfz6Kbgt1hYUVNrgek9NazMImqt8faaWpt4rIxl5led9o5X+vRehQKheJwMPPUV6PJv6628HVUy8SmGFQ8yPrJAh5+Pruy3Of2sfaU/6Y4QrbhkFBic3PX1MKncMVI5MiM+q6aSf3jxfjCYDNRnE0A/9y7qaAefCMizq7+9TQ6CnY9xqKaXE84ZcgpeBwe7EKT+hMJDA27RzqpLK1kTKlxbUWxo5iZp87s8XoUCoWip5jl1E/8xFZxhDDhhAkU2YsMK8u782J0Jw27GtLqYwSD1Fxdx/4wLD8F1iTtwrRGmLnTyf3/1jPjWjGwIvsBn4+a+55k/s8mc+vnjK16yA0LeZZ4/PBL8l1RbYBMFjqCMD2mADU5XQwK7FJKf6FY13WlydxUm81ObVUtY+8da3jOX7/6V9WuplAoPhXMZGLf/yQXciSQkvu0atQRZIu9JEPIEnAn4OT9SfnWzeB1Sla/dH+P1hWO5WrNN58ymls/n2cwjCRvBbdVXAnwv1sCtPeKIEwOBarJZWFSYBco2WYskyu16v+wI6Ol7+urWNO4xvA5HqfHUs5doVAo+oKCqt+FEA5gJFDU/ZiUcmvuM44uMuU+O6IdlqappdTHZp0+CxobaSgLUXsNWTPP518M9Q+FoN1gelseNuzM1Zr/0VM/yv/EXqrEL3YUs9b/V7wn7dbC4JWVmpHtBYMeDAcJbAnQ2NJI1X3X4Z+9FF9YWleTyzOutXHFt42r/4XWzjex1UHlQTv+X/4NeeJ5/OLRu3s8r12hUCj6EktGXQjhBP4HmAkYDdvsgTk68kjJfU763/N5/YNNeQ1j5od88KQRWpV4xjuYGl1aew3816AzWX/o3YLXpLe5eHH3iwVfxwpFjiI6Y53p/PMPzv4BP7vgZ1okYlzv3kt3oMoCQf1x11OzW1jbPOQpsKtqajGevS7cTD/xAmZdMB38fhpaN1F7d7mp4p+auqZQKD5NrLa0/RdwGTALzYxdD3wTeBp4D/j3vlhcf8Xr8vLx/j2WPd2KAVrO+xflO3PbvpIkBBSfeQ7FjsIr0fcc2pMzFjQWL2wme16SUfrOmJZ6iMs4cRln2T+W9e59kqRy3cFIMG1wQ9EQwWgbF7b+mvmTwiw7TRJ05kkf5Cmw8+8tM5QAtjld+P/rEZg1i6BTptdjNFoX1NQ1hULx6WLVqF8J/BR4JPn7RinlA1LKqUAD8KU+WFu/Zl/iY+snC81I/fqNpYYbgZAL9oQP8OTXnqTInpPdMOXtA28zd91cyu8up2GXNjjvBM8JBV2jp6TSC72N2UjYcDzMPf93T85r1iXPfHlfZbWl2ev5RtS67K6+0bpXKBSKArBq1EcAO6SUcaATyOztegiY3tsL6+/YbdazDXsO7tF01hPGo1ddMaj0VFAzsoZfTf1VwSI1oWiIYETzbvcG9/L2gbfzP0mCA1vamJnKwBttRvooh2xFwjbzNetNiAO08LzN4M88WWCXSqksvmQxdZ+vY/Eli2le0NzVtWBhPVNGTcl5jkKhUHzSWC2U2wscl/z5XbTZ6huSv5/U24s6EojGIloVQZ4QvAM7laWVbN7zBlGMjboUWhU858HO1p2FTWrLICET1G2o0zYdeca3I+D7sVM59d9n89jjt/GUeJeY0V7FQEs+lUPOKmgrq8Jf7cfn7rkmeyHjbrOKEbtjcVxrvtnrZuvxOD1MHz9deegKheJTx6pRfxY4H01k5j7gTiFEJRAG/MDDfbK6fkxYSEs59ZiMM234JJ7f8L+mQ1ZObwZvdA8Nuxr47Wu/7fG6QtEQ21u268q5dkck4NTjT2bW6bPY8Zffs9ZReJGeTdgYMXAE5XeXZxe0rZ9P/TX1PfZcU+2DVjCLFgTDQQIl22hc8W2qmlrw7y3DV1ldcHW+2XpUHl2hUPQXrBr1m4HBAFLKXwshBDADKAZ+A/ysb5bXf3FIsOpLP7biv/Ac6gCjceoCXh8GeysGU/tQrWkhFhJIaPeP6fzreZwexpaNZdO+TebXAWwJ8F+7CIJBqv7+Jp5JXdX43e9pFzbcziIEIm24bcLGyitWMuORGQQjXUV6KW82q0e/QDLbB1ObBSOMKs51q+cH2Ki/qJ4aE4NuFHXovp7Ue6Dy6AqFor9gyahLKfcB+zJ+vwe4p68WdSQw+pCDbYNi+b11AWta/4/jBwyBdkynp9WVvkGixVxVzomN2wdN59aOtQSjuXlkIQQnDzmZB//5YN7XMNF3Et7SoQR/fy+dtgRRo9C7gDgJbNhYdNEi9hzcQ2VpJf4JflZsXmFYQNYR7WDOmjncW3tvj0LxqVx3YHOALfu38NvXf5uuvs9Ez1POrJ5Pkd5s3H8RzXPewVs2LOdauhuBjKhDaj1NrU3p90AZdIVC0V8oaEqbEOI4IUSNEOIKIcTnhRDH5X/W0UmnE+viLSUllI47zfT8mB22H9yZN4ccFQke9e3iurNn43V6syq2S5wlxBNxfvr0f2qGNk+31xsdO1m1bRXlzQuo+0KUiAPtOQbPk0iK7EXcfuHtzDp9Fl6X17SALCZjPLz54dwK9WAQli2DhQu175nCMN3wurxcWX0l408Yz+XjLqfIUUSJsyT9mo0qzk2r56OdXDG/gmWBhek2wGA4yL0b72XK8im5bXQZxXip3Hvme6BQKBT9BaviMw7gNmAOUJJxqF0IsRS4WUppMuzz6ONfxRb7wCWcfdq/U3H8aBz/sBEzGJ7isrsYWzaWzfs35zXsr/zrFTbv34wQguvPvB4hBBUDKqjbUEdbLMN7t7DpmPHoDGIi1iUpZPIcvdx1voK2aCJKNBLtCsVv3KQv2frnP+s+v7vnXOLQNi7XnHwNk0dNNvSUG/duMVxTxAHrRid44a07mL9zKYsu+m/qNtQRjoeJJPSTKqbFeAqFQtFPsOqp3w38EPglMB4tvz4euB34AVD4eLEjnJjVd07AmvefYuSAkYYGHcBlc7HowkWGQijdCUVDtEXaWPraUm7+ws24HW5kPtdch7hJm1139HLX/mq/pTUnZILAa8uNZ6I3NubMRNcToGmPtROOh/nb9r8Zh74bGqj65W9NZ8qDpg0QjLYxp34OwUjQtONAyb8qFIojAaum6WvATVLKX0opt0kpW5Pfb0Mrojv2hkcXoJu+o2UHMx6dYXi8yF7E2mvXMsw3LC2EYlWAJuVBWunr1qOQjYBe7jpVQOZz+XAI48BPKBqi6eXVxpKtkDMT3SyEbih6k9R6r30rTGdBkw3MUfKvCoXiSMCqUU8AWwyObSZv9vboIhgOFvyCjYyTy+7izovvTLd+pYqx7q29l4tGX4TIs3tIeZCpMHihlDhKDI+l9N1ddhdep9ewyju15qtPvhqnTb/E3+P0UNmCsWRrIpEzE91so2LoOSe13uvHgMt6ECIvqm1NoVAcCVg16n8Cvm1w7DtA/lLro4jAloD1bYwEp81pnN+NR9hzcE/WY6lirNOGnpbXk3bb3Wzat4nOWGfeDUB37pp6l6kyXuqYQOSNTHhdXu6tvZcih36EwSZs+EdNM5ZstdlyZqKbbVQMPeek1vuWwVru/HBx291K/lWhUBwxWDXq7wPnCiG2CCFuF0LMS37fCpwDvCOEmJ38uq7vlts/aGxptP7OCTgUOVS4cQJGDByR9/LheJh1O9excMNCEiQMPeVMLjrxIvYu2Mv88+bn6J5neu6pHHM4HqYt0mYux0pXKN7r8uK2a5V3brsbryvp5V8101iyFXJmopvl6w0956TW+wdeDjt+5LQ5ueviu5T8q0KhOGKw6sukCuHKgc/qHL8742cJ9FwS7QigqqzKWB1Oh3gijs1RoHHKR7f7pxTkrBj1l/e8nPY6u/deNwebeeztx3QjC5YrwGVXrl5qv2iYSbZWVeUovFkRfMkRivlKLW/e9QMeHU/efx9PGGxFbhZdejd1G+py7rHyipXsOrSLnz/3816RvlUoFIq+xqr4TEH97Ec7tcO+UND5bpu7R2pkuw/u5gQMpq2ZGCy7sBOXxgnlzlhnlnHO1D1f+NTCwvPYSVLV6m0ZojiReIRIPMKlD13K7VNuZ3fHbqrW/RL/FoHvnT1dM9Ffe033mmaCL3pCMfOYR+LqBDGTejx7HKbuhOlvg//iOXjPms3XJ3496x4jBo5gxiMzelX6VqFQKPqaXqwPPnb4S+Cn1k+WMKh4UI/UyKrKqjjUcqigtUUT0by59ZiMsfXDrVmPpTzeNz94E7fdrSsxm68C3KxavS3SxoInFxCJRzQDKWzUX2fNQGZuOoLhICs2r9AU5l77bdY6rVb/x23wh1UwFA/MGa97j/K7y3td+lahUCj6GkOjLoQYD+yUUoaTP5sipdya75yjhVUtL1nfDgnojGvSpvkmgXWntrKWh7cXPitn7OCxbDuwzfSclo6W9M/dPV4j8qUK8rXVpXL0OQYyLOHAAU1hrqpK89x9uWFuq+vMhz0BCy+Ee18Q+Py5r8dKK50SoVEoFP0Rs7D6ZmBixs//NPhKHTtm+JfbfFBKd2xmxWEm1G/+S4+KvYZ7h+Owme86ykrKAH2Bl+6YybFmUmhbXUImCPz1F1BeDrt3wx13wNy52u8NDVnnWlmnVeJ2ePgUKJ+XoKF1U87xHrXSKRQKRT/AzNpMBrZm/HyBwVfq2DGDLGBkJxK+MMI8B998qJmZj8/knPvOYebjM2k+1AzA5pf+WpDITQq7zZ43BO8UWkGdmVfqsru49KRLWXzJYksV4FbV5VKEoiGa7r9bE4xJidKkFOZqa7MU5szWmYOFjVDUDsF4u25Ff49a6RQKhaIfYPgJLKV8TkrZlvGz6dcnt+RPn6HthXneZw4+2fDY0leXUn5POQ+89QAbmzfywFsPUH5POUtfXcpHH+8zfJ4Zx5ccn7cKfslrS2iLtJl6pZF4hIlDJ1oeXJKpLpcyiqnWNj08uKn8yGDzkUhkKcwVophXHAWvML5v1m10lOl61EqnUCgU/QBL1kkIMUUI8Q2DY98QQkzu1VX1c0rao9bD4gJeeO0x3UPNh5qZUz9H99ic+jm4BwwqeG0CwQmeE2iPtZueJ6UksDnQ615pqiBw8SWLqft8HXdNvQuvU39DYJMS/xsGeuuhUJbCnGloP/lv4QmDLwxPPgh7O+aw8PMLcdqc2Ez+zPXC6XqbE6spCIVCofg0sepy3gYMMTg2GG3QyzHDK879hYXF2/QFW258+kbTp20+LoooMKee6g/Pl9tOGTMzr1Qg6Ih1sPCphSx7Y1l6TGk+MseTzjl7DmuvXatvII+fi9donR5PlsKc2TqLYjD/RVi8DprvgpoWD97K8Sy6cBGtC1u59pRr05K3Oc+1F+luXLpvTqymIBQKheLTxGoNdzXa4BY9/gH8Z+8s58igFXMvOAsJ0w7q74fyVajHRAIhCk+q//a13+YNv6e8cCOBl4RMkCBB3Ya6w+7TNmznC0u40UCnyGbLUpjTXWcEbBLqH4KaXRnPdXc91+vycvuU23ngrQd0b9MZ72TamGm6xwrtVlAoFIpPG6tGPQaUGhwr66W1HDGIAmabCWCm6yzdY+MSg9hookxXNWAUMt6zcarXnXkdv37l18brEiKdG+5udCsGVFD3dF1WAdnh9mnrGkgXXQpzqRb/yq4AACAASURBVA6BlMJcfX2WwlwwHGTbgW3MOm0WrZ2t+Fw+grt3Urp6A9uG25i4L4LPqf/c+qZ6ihxFdMY6c9ZV5ChizY41yngrFIqjAqtGvQH4sRBilZQynQQVQriABcALfbG4/oonZifosDACTEJlC3hPPTX3WDDI7T9p4IHvGz/99GZpnPQwISZjbD+wHY/TY1hcNufMOVmGOdPoLntjGfKT6tOuqYHmZli3DurquhTmMoxy9/70InsRnfFOzVCfGcOTcDD/Qhv1H9RQc+sfYdiwrFs0tjTqGnTQ1PVUi5pCoThasJpTvxkYBzQJIX4lhJgvhPgV0AiMBer6aoH9keqQxV5sobVOdR9UAkAgwPCDcZbUoxV6pRzy5M9LjruG/S27kD1pacOORJpWi5uF9RvffIZQVD/F0Cd92l4vDB4Mt98Os2bleOjd+9NTYj4pQx2yxQg6EtQOXk/bhDE5Pe6qRU2hUBwrWDLqUsq3gLOAF4GvAf+d/N4AnC2l3NxnK+yHVAwebbn63WsvzhlUAsDmzdDZyexXYe+dMHMTnLtL+773Tpi9awhVx48ruFAOwGF3MG3MtJ4ZsmCQqmWP4THQ1/mkjWAh/ekJIPCZtqwe92A4SGesk2hcv8petagpFIqjCcva71LK7cBVfbiWI4aPhx4Huy2cKOGbrSP1j330UfrHoSG4f1W34y0t1P7oZh5+9omC1xeYEeCCEy/gpqdv0j1uasgCAfxb7cz/N/3Dtlj8EzWChfSnh9zQVEq6x73horFa2D4WJZLIaEMU4IyD3eli5RUrVYuaQqE4alDT13qA3WVN2ATg6uqv6h8o7ao7DLpg2emaHvmy07XfKSujfu/zeZXh9NhzaE/Pe60bG/F93E79Q1rPd8pjT/WA13dO/0SNYCHSs54wVLYCoRDBpi1dYftEMp8ukl9SS4vYOiPMeGQ6DbsaTK6qUCgURw6WPXUhxAzgcqACKOp+XEp5di+uq19z2ZjLWL9zvaVzvxl5hOlvVOTO4q6uhqIiGk7opPYaLXQccmuGaf7FUF/hpLGl0Xj0qglrGtcw5+w5PZoMR1UVeDzU7ArRfBcEqjXvt+IQSJeTJ77SwrY3ln1is8X91X7mr59v6Vwb4N8CeDwEhrWQaDMI2yf3Se0uINqmJq8pFIqjBquKcj8FHgE+ixZ43qLzdcwwc+JMax60gHXRt5m7bi7ld5dne4R+P8ESB7XXQNCtGXTQvgfdUPvxEo73HN+zBWbk4TOFYCzJvfr96fYybwRm/QOmNcKNF0Ldv0W5o22d/uvpI3xuH/VfWYlPFOGRWu+9K9WD3624cNEGbc3YbDRWllkO2+tJxSoUCsWRiNXw+yxgkZTyNCnlNVLKb3b/6stF9jd8bh+1lbWWzw9FQwQjwezhIT4fgaWzSRhJnyN5Y+8bPVrftLH6YiqW8Pm0Pm+fDzwegi66Nh4u7RTd19NXNDRQc/YMmu+xsfhvUeZttGOLRLVjqfcuGVafPxXmX+Zk2X3XMWLwaDzOEku3UJPXFArF0YJVo+4Dnu7LhRxplDkHFDwWtbtH2FjaZSi7E4qGeOejdwpeV7GjmJkTZ5qeEwwHWfbGMmP511Tv+OLFBH58KYki/RqCPvdwU9PagkG8H7cz6x8wfl8cu4FEQNgJ95wZZe7OJdy44UZiUQNd+W6otjaFQnG0YDWnvgK4BGXY05Tu+rDgsajdPcKqsipKHCW6w1c8Tg+jB40Ga3YpzQ/O/oFpiL27kIuh/KvXC7Nm0fjUDkIvrbX0enqLYDhIYEuAxmdWUlUdxv8G+CJaAeHKz3alKoxIh91N1PoyUW1tCoXiaMGqUX8a+G8hxGDgKeDj7idIKet7c2H9nepWO3YnxAvoH+juEY4cMNJwmloiHuOJ7U9wxolnFLQuM1GZTCGXFEbyr8FwkOVvLueJ7eYtdRUDKgpaXz5yNh2TYf5kLV9edyFE9Oey6GP0ViSNvUe4sTldavKaQqE4arBq1FMx1lGAXmxXAoV83B7x+Eddxuw967EgFpsm0yMMhoN8ccUX9U+UQDhMm9NAAcYAFw7TMLKZkEum/GvDrgYufvBi2g1U5bLogeJd2hNvaaSqrAp/ddd7krPpSHrlc2p7di9dBJz7L8G3v3kX/jNmKoOuUCiOGqwa9RP7dBVHIL6rZjLvq/O546xofmMjwRWD+jMWpQ3I7177HeG4sdGO9MCAuRxuwzByMBxk5daVhhXhqVB6MBzk0ocutWbQgT0H9xS0xu6euMvuYvaa2fzx1D+y+c3NltXjDgdPBL594X8w6zz9WfYKhUJxpGJVJvb9fF99vdB+h8/HLT9ejSdq4VwBEQe8cPfctHzpb179jen5ccsKAkkk3HrBz3S9zoZdDZTfXc6z7z1r+PRUaiCwJUA4Zj1CIKX1akE9HfdIPEI0EWVf2z7mr59v3IbWW146YPN48X/5lt67oEKhUPQTDI26EKIk8+d8X5/McvsXvklTWfe19ZQIV/5KeAE3fSHKqvvrCIaD7Dq4y/z8QjXfBdz09E05LWaZhtQsMpBKDWzZv4VowspORWPJa0sst7Xl03Ev5L5muOwufC4fS2qX6CvqXbtWhdwVCsVRiZk/GBRCnCel3Ai0kd/MHFM59RQ146byQV0L5beVcUjkKVUXMOPAUu55c1z+C/fAM40lYjljUfMZUpfdhdvu1orFwpLWt16xXDUOmqce2Bzgyuorc/Lk3RXnCtFxPxymjJrCI1c+gtfl5esTv16Yop5CoVAcwZgZ9W8BO5M/H1PiMoXidXm5ovhM/rfzpbzGMC4ka3as6ZN1xGU8p8UsnyFNG8CNm2BCOYO+0K7N47NIKBrimfeeYd76ebptchOHTEwb+31t+yhxlljO1/cEj9PD9PFd+vSZc+IVCoXiaMfQqEsplwMIIZxAE/CulLL5k1rYkcZjFqfPak6wwIaNBL1bFOayu3Kq36vKqihyFKVnj2dS5CjSDGBYpkVeJuyFogh0GojidKfEWcLKrSuzQvupTcSUB6aQSCSw2WxE4hHDnvzeRPWcKxSKYxkrhXJx4O9ouu8KAz4OH7IUsi62FzNtzDRkwUnz/AhEjkGrrazVNegAnbFOpo2ZBoGANq4U8G8GZwFLS8gEdqGfeYnEIsRkjEhylnlfGnRLE+gUCoXiKCevUZdSJoBGYEjfL+fIJNhiPYBhEzYOhg/2iVGfMX5GjkGrb6qnyJ4zVC/Nr1/+NTQ2Qkjzrn0RdMeuFke0tjxnsjG/JAI+p5fLx11ubKwNNjkljhIuOekSLZ9vsxgSMGH0caNZfMlimhc0Z6viKRQKxTGGVT20m4H/EkKc3JeLOVIJPHij5XPPLj+bm/9+c97z7AXWHZY4Spg8anLO440tjXTG9T11gHteuYe2k0aAp2tmec0uaL4LFq+Duhfg+o1gl+CIa3PIHTGIC1g58DtMPnGy5XnnKdpj7Zw69FRa/qOF27xfKrzSvxsneE6wNoFOoVAojnKsGvVbgDJgkxBilxDiVSHExsyvPlxjv6fxw22Wq8Vfb37NkhETQuCyW/di7Ta7bi65qqzK9DoCQWCCSI9bDbpg2enw8y+AFHDDK7D0bGhzQ3tS3S3m0IanzPhwCdOqpmETBWjlonn5lZ4KpJQ81vpiQc/V4/Xm1/t+WpxCoVAcAVj9NN4CrAYeQNOB38wxPE+9OyMGn2TZ2zwUDVraAFz+2cstCbvkyyX7q/2m1wnHwzS174H6ehrGFVO+AOZeDHfUwNxLYPQPIWrwV5KwCeob66m/pl7rB5fd5pwbEBcw4vWdlN9dziuOvebvRwLceYQPozLK8k3LzW+qUCgUxwCWdMuklN/o43Uc2Zx+Ovz9YWvnWuwBn3rSVEYNGgUf6h932V1MGTWF6eOnm/Ze+9w+5p03jztevMPwOpWllQSrJ1L7NTvBDP0Xo7Gw6eMyTFNrE7NOn0XzgmYCv53D1if/zD1nxpAmr3HaDviS43/oFHmU8yWM/xC2DomZnwesaVzDnLOV7KtCoTi2MfXUhRDFQojpQogFQoirhRCqWE6H3eECxrAK8nqyNgn+CX5uOf8Ww9C22+7mkSsfsZRLnnWqcZ92JB5h0qhJBLYEiCYMjKfBejOnznldXmZ9615++n9FOE069dxR+NtY8hv0JN/a4sAj8sxaBdi2DZYt02awKxQKxTGKmUzsaLSw+qPAr4AHge1CiKmf0NqOGHwun/ViLwvnpbxcn9tHVWmVvtRpKtweDGrGbOFCQ6N22wu3md7v58//nM37NxsX1BlsWBIy0ZXHDwYhECBw5XgcJkY97NBy8qZI7WtJPXx3cxE2hzPv+VM2vAtz50J5OTQ05LmBQqFQHJ2YfbzeASSA84HX0Sa1LQV+h5ralsUTb/+1oPNPKTmRtzreNTwuEGm5V6/Lq4W29aROGxo00ZhEQmtJ83hg/nyor4eartaubR+Ylzxs37+VMccfhgxBxjoazwvRbtZVlieiYUvAubvhsdXFDO10QH09i9xvMafePLT+08lwTnOIml1oa2luBq+qhlcoFMcWZkb9PGCBlDJVnvy2EOJ7ye/DpJR7+355RwbvfGC9+t0pbDgHloKJUU8gs+RedaVOg8G0ClyaZK95d6M27pCLjUa5fAljDzoZNLLU2gvIXKdMEHhtObNqb0yvo6pV62sPWYiY61Fkd7F+yPV4bx8Pfj9Bp6Tu7lrzJwmtOr/2Gq0Vz5tIaII6s5Q8rEKhOLYwy6kPA97p9thONNMwtM9WdATSmYhYC79L8OLiww6D6rcMKgZWmJ+QoQKXQ8qoJbm95VTTSy1qPZ3qE6opchiL1OgRjodpenl11jr8m623VOgh7Xa49VbNIHu9eQfSZJIAAtVom5umpnynKxQKxVFHvs/f3pc9Owpx2ZzWPHUBsrOTEmeeSbUS6Mwz0zxDBS6HbkZteNXpLHnKmc5Vp++RzFsPlR781X4corAh7g5sVLaQtY6UIp2th7L2NmEjsLlrQ1LIZLeQG5pK0dIQlZV5z1coFIqjjXxGfb0QYn/qC0iF3J/OfDx57JhlgCiyvP2JCJhw/ATzkwTs2bjB/JyqqrQKXEowZuGF2vfgcSXZRm3kSGa/FGXvnTBzE5y7S/u+906Y/SqwZAlvvvsy8ajFiENqmRL8o6ZlqdEBTNwHdmvF7TmEoiEe2/oYwXAynF9WZVmxzhOGylY0IR2/GuqiUCiOPcxcs1s/sVUc4ZwQdfGeRSfXF7dTMSBPaF3C8R8ZS7sCmtGaP5+GkVouOYHmqXrCMJ926j8/ghrQct0zZgAwNAT3r8q9VNCRoPaRL9FBxHprnoT5A6bivWom/OimrEOBCZj2qefjqXeeYvhdw1l77VpqK2u5Qd5g6Xk2wP++VysUVEVyCoXiGMRs9Koy6hbZ42y37OGOLS6n+oRq7AjiJk96fUCeiWY+H8G/raT2yYsJZhSlpQrUah+fQfOCZrxmufckgdHtJBIWUwhJXDH44dW/AZ9PM6IZVfhbhjmIOfILxhgRkzHaom1MeWCKJnHb7W1y2V1E4hGK7EV0xjvxCDe2hKS+Yh7e929RBl2hUByzFJZEVegiEwlrBlHC0KAm3fq9J75rfJ6AZ2zvs+yNZYyWow1PCwzYRcLrAZ2cc0ImtLa4ZO496NI86MZSrULdv1nLfwM0DnEQEtGca3Rfe1o4R4DN7WLsH0+n/pp6ampqtGr7QACammgtfQ7aX877duQjEo+kx7ZmEk/EuW3ybQwsGsieQ3uy2/wUCoXiGEYZ9V6gvbMNishv2AXsDbfic/u46KSprN+53vDUvaF9zF03l5+P/jmuXa6skaLBcJDAlgD3vX6fYRFZKBrS2uKqqmgYU0Tt9M7sEP3FWkFbzS6oOmjH43TrXssdg8/vETz/GUGMrs1LZyJCZyRC7UO1WkTA6023kA1aNxdeOXyjbkRcxrn1+Vtx293apkKNW1UoFArg8LqPFEkOubEcut5xnOYR/+GLf8h7bigaIiET1D5Um55C1rCrgfK7y5m7bi4bm82H41UMqCD4lVpqp3cSdHeF5kNuCLrh0mugbZAH/21/M5605nBgO38SNpv+8UQsmlWtDjDhhAnGM9wzK/APg0g8QjASzHpvFAqF4lhHGfVeoJCisLakMPrwAcNZUrvE0nNSofRgWDNiwUjQUptXOB4m8H49iWJ9A9vmgl88+F18k6Z2TVpLVpqnjLJwONiw+xkiUj9HHkp00vTB1qzH/NV+HDb9IFBJbjT9sEi9NwqFQqHoR0ZdCHGFEGKLECIhhDjz015PIRTSvlUU6XJTZ581m70L9jJz4kyGeYcZPicVSi9EiAVgw84NWp93wljT/Z43ltAWaaNmZA3NC5pZfMli5p0zryvMHjOvwrfHYfD23V0PBIO8uewXJCKd2R65hOIIrH9I643vLY89nWZQKBQKRf8x6mgz2i8Hnv+0F1IoJ3QIy4pyXzg4KOuhod6h3P/l+/nZ5J8Z9mOnpqEVIsQCWhV5VVmVVkFuQEpnHrrkaMefMB67sFu6R9wGP/r4UZa+uhQaGgiOGk7tv+6gXcSyUxIC7BJO3af1xu9dUsRnIwMs3cNMFCdzUpxCoVAc6/Qboy6lfFtKuf3TXkdPEALLOfWaUefrPu6v9mNL6O8MbAmJf4K/ICEWJBzn8OKv9iOl8Y4jHA/neLoFbR6Sr33Omjns++IFBEa1YRS4kCRlXIGhBzo5qzFkaTN02rDTcNr0J7XZhK1rUpxCoVAc4/Qbo34kc8hpMY4s4MPqUbqHfBFYtD6WK+WK9rg3kjT8RgVtOvfasfNVfG4f886bZ3ianqdrunkweal1X4jyzChoNxjmEnJD0/FdEYBBwbilzdBbH7zF6qtXm4+gVSgUCgXCzIvr9ZsJsQH9YTA3SylXJc95FviRlPI1k+t8F/guwJAhQ85YsWJFH6zWOq83v27ZU68YUMEQz5CcxxMHPuTN8C4S3Wx2hbuC5o49THSPxDb4eNoibTS2NiJlwlLE/7ShpwHw5gdv6ubjbcLGxCETszYLCZkwPN8MZxxiNuPCQVsCRhyCwZ02SCQ4UAK7BmrnV7gr2BPeo/88YWPEgBGUFpfS2tFKOB7GbXdTWlxqfZPTj2hra9NaAI9yjpXXCcfOaz1WXif079c6efLk16WUurVnn2ifupTywl66zu+B3wOceeaZctKkSb1x2R4z+SeTrcU8JFx98tU8NO2hnEP/v707j4+qvvc//vpMNsiCGpAdpZKIAi0uXNyoWxEwemvrlrZaqZdqW6kF0Vuxdr0/Wy2CV+pS2/Jr1SvVqVpXUBSEtthqtV6rgEuiIigICBYmCWSb7/3jnEAYksmZMJOZTN7PxyOPZM58z5nPmQQ+890XfG8K37cl1MV0f889fC4/Wn0N80NnMu2GxQBsjGzkmkUz+P2bD3b4YWLByAVMO2Ya+evyqVhYQdRFqW2spSiviJCF2p3nnb8un4r7ziTa2Egt9RSQQ4Pz18BrZwvX1ovTtMWiMGeZMfklR0mDt2b9kKu96XVzD5/LNW9f0+59zD5pNjeedmP8m+0mVqxYQbr/ZrtCT7lP6Dn32lPuE7rvvWrxmWQIOqXN4N1PYnez9Sw/8BPqdrZ9Wm0BVPsfGFeuW0nFwgoamxs6TKLA7v7yltHt4VVhqrdVd7gK24R13t7kN4xv5pZx4GjGxftrsZjvbXAh+M+Jjh+dAkvu815j8UJv7XqL0+ygwXAiIsFkTFI3sy8CtwEHA4vM7FXn3OQ0h5VcDkb2G7nP4Uh9hIcb/tnuaUUNUHbiWXvNU98tThLNceyVDFtGt3coEoGKClx9DXceDY3J/CsxqMv3Fr7ZOG/Ph4en5nvryTe08VoaDCciEkzGdEg65x5xzg11zhU45wZ0l4S+YePbCZW/aeJN+xwLrw4TipOdm0NQOfK8hOep51hO55KhvwlMeIy3+1sqNOZA+Bivr6G4AQ6uhWX3Qkm99yEGNBhORCRRGVNT766u+5+vBSvovJrowAee9LZNLSnZ/dTy95azs7n9RV4++75R/Mgiqg5JYKqZg2EHHtK5ZOhvAlNVumdp2WSrz4Xq8z8Hp46FoUOhf38mfGU2Gz41lPAYqK7VRi0iIolSUt9Pb9a8B0HWaTFoyAFmzoRZs7ztSidMIFIf4Q+r/hD31BWHOmrWvEr50WMpyisKltgN3t+xvuNybSkvh6IioLbDPvvOKsgpoOy086ClO2DFCrjxRoqBAB0EIiLShoxpfu+ujij+VGLLndbW7u6zpqaG8OowTcTfe7wxBOH6V6gcXZlQ83uiU9J2q6wkUmDc+W+kJKED5OXkqZ9cRCTJlNT3041fvbtzJ0ajEA5TtbWq47IhqO7j7e5mFjzLHlx08F6PI/URFryygGufvZYFrywgUh9p+8SSEsJ3XpHQRjW7dbSmu//8zyf+XM3qIiJJpub3/VRSOogQwQaU7bVDWW0tVFdTfuSwjpu4HZQNHEV4dRhLoOo8Y/yM3T+3TIVrPU991pJZ7c5TryqF2vaXjG+TReFzvY9kaf0bcQp532Yvnc0lYy9RYhcRSSLV1PdTeHWYUChAp7qDqMG1E2HBMRA5sBDKyqhcBbkd7PKW1wyVF9+U8IYulx5zKUCbW7bWNtbG3Y+8vG85vWKXt+vg/r77Yg6VJ19BUV5hh8W1ZaqISPIpqe+nqq1VNLkAe68a7MqHORNg5mQY8q06Vp40jJJ31jP9JeI2WX/bjaO4dCDlfcspDJAwAfIsl0VvLwKIOxWuOdrcZnKt2NaPXZZYn/xlAyqoPHYqoQA7vGnLVBGR5FNS30/lfcspyAk478tveq4t8JZGrXjkfGpGDOtwl7eWfvTK0ZU0ReMPqmvR6JpYs2UNEH/XtbqmOn7zyA+I/Pp2bwAfQCTC4h99iV7BXmq3H47eTEmBN6+8JL+EglD77fdaJU5EJPmU1PdT5ehK8nLa3ha0I1EXJTzG2FQcfz/2j0r23mwlEAdbd24FYNgBw+IWfTFnI4PXXsnK8QNg5UpvAN8BzexK5LYMXsjbBOxZknbe5Fu0ZaqISBdSUt9PJQUlPHzBw506t7axluq6D1h//Ki4NfX1pd54xvDqMDkBmrYBMOhb2Nf7edeu+CPSDWoKoOKLO6k550xYs4byTU3kBOhVaK2w155Bb8X5xUwfP53npj6nLVNFRLqIRr8nwbod6yjMLaSuqS6h81qaoO/lk7jl3tnxHuA1o9c31we6tgGj+o0CYP1LSwPNN28MQfjwBqZt3UrF+t40h9rZYaYdl469dJ9jiW4kIyIinaekngRVW6sSTugATdEmKsdUMnvx1XHLNTZ7c9TL+5YHXlHOOTj5kJO987ZCfggaOmhO35UHa0oaoG9f/nhEYoPkQg4uH3d5m88F3khGRET2i5rfk6Al2SbqqIFH4SI72Nq4PW65LXVbqGmooaKsIvBAOQwOv+Nw7nzpTiqHn01ekFXvHGwtCcHo0Tx5wdiEVpM7Nm+4at8iImmmpJ4ElaMrCVnib2Vpr1Lu+Z9rOl65zcEPnvsBZbeVBW5+bzF98XRq/30KX38tt+PlbA36RqJw1llQ0ieh19l5QDtT7SIRWLAArr3W+x5pZxU7ERHZb0rqSdB6GlfrAWEdTXU7a+RZPLb1+Y5rxAa3vngrO5sS6+NucemyK/llgHXcezXAqO25sGgRR35MQmvaDzpgyL4HV66EIUO8TWzmzPG+DxniHRcRkaRTn3qStDUgrKK8ghG/GNFmMi7MK2Tq2KnMfeJ7KY/tmXefCTQVLs9B5atNRKpXs6DgTwk1v08ZMWXvAy2b1rSumdf6YwEqKmDDBihWc72ISDIpqSdR7ICwO/9yi5fQY9Z2L8gpYMnFS3CRHay3HSmPK0SIaLzV6R0UN8DihVCcV8SCQVuJfpLYbi4FeTGtEuGwt2lNW/zNbJimwXMiIsmk5vcU2bD0UaYv80e1x+TH+uZ6ykrLCN93HTmd3B01EU0uzuA6BxOr4e3bYMI6IBRi+YBd1NHQ/jlt+Nv6v+19oKpqT808lr+ZjYiIJJeSeipEIlx31/ntP+9g9lNXU7XlTRpS3FZywagL4q8Xb7C0DA6ZCc8cBpHwvTxc9VjCr/PwGw/vvTFMeTkUtTMjoKgIyrRErIhIsimpp8I99/DEYc3t90kbvFX1AsP6jUhoMFqici2XUw49peP+dIOmXJj8VfjhmjuCL0XbSshCTF88fc9e7V+sgFA7f16hEFRqiVgRkWRTUk+BV5bexyfxNlNz4HbWwahRga6X66xTU+aaXBMf7PiAc484N9gJBrdFltEYbUz4teoa6/j9a79nzl/nMPPpmQz59RGs/P1NUFKyp8ZeVOQ9XrxYg+RERFJAA+VS4LyRr8YvYPBizkeM++dzgUaY//yAC5hvL7Bu+7qE4mhZhnbE1iiPNkBd+5um7daMwzBcJ5oQWvruW1a8q3h9NhvWvk3xI4u8PvSyMq+GroQuIpISSuop8EGv+kDJ+qUdb0AOHZa9fOp8Xl4+K+GkHrIQlYdW4M6YyVXfCHiS0amE3paoixJeu4hpGuUuItIl1PyeZJH6CE1BZoMZfJQXYHU4B+G1i5h7xtyE4ijIKfB2QntkESX1jiteIqX9922pbayleptGuYuIdBUl9SQLrw4HLttU2CvQanLV26oZ3GcwVzWM8xJzS3KOk6R/eMoPmXDIhL2nliU29Xy/tTT/i4hI11Dze5JVba0KnDxHHnwEWyIf0RhqPzsbUFZaRqQ+woKCVXsn8jiv84PlP2D7ru18f8QwSoqKKN9WS2HAfvVkCVmIyjEa5S4i0lVUU0+y8r7lwZq5HXx22GfJyYufZR3wzUe/zuCfltIQb1W4GFEXZc5f5zB4y2xWDotSuQqaOvptd7Z53kEBuXute1+S762Hr53bRES6jpJ6klWODl4zXfLWkzz2pcfbL+AnG+gL7AAAF0pJREFU2aYcqAk1Ue8SW+UNoKaploqLjI/6F3bc19/J5vnCRli765vMnzKf2SfNZv6U+Wy4eoPX/C8iIl1Gze9JVlJQss9a720yeP+dV5j06UKWXLyEz9//eZqb6r3E6/aUaV2+sxotyqjLmtpdir1FXjOEcnIhJyfQFq8WhaJGeOrhXgz87lF7rXsvIiJdTzX1FMgP0kruoKARqKhg0oAT+fhb73HXQZcwu/E48puTG8+upl3eHPIOPhgUNMPab7zJvEnzCHXwpxGKwiX/hI3zYMLGPK0QJyKSAZTUU6B3c4BqtcH6A2HloEa44QaKPzWSadc/zPdufpGGjuau+zX5Xrm9khHu7mtOH34hAweOYPr46Tx18VNxixc2wu1/KqS4QCvEiYhkCiX1FMhvDDbiLBqCivN2UXPbPG/f8dpaLp/S8XkAs0+azdwz5lKU286mKYkysKFDdz+cNGISSy5eQl4ob5+ihZbPU1xE8dxfePuiT1DfuYhIJlCfegok0v0dNQiPdkx7yXv8wFEdXyDk4MaJN3L732+nIRpg8FyQPn4H619/Hs7cc2jSiElsu3Yb97x6D4uqFoGDs0aexdSxUzWqXUQkAympp0BjTvCytflQ3adVJ3qATwQHRQtYuW4lVz19Vfy90hO4JgYL615kbs1HDCweCHir44VXh1m3fR3nHnkulaMrvYGAIiKSkZTUU6Cpdz4QbPpZoculrDYUuDxAA02ced+ZwRI6gAMzKKSAWurbr7kbzF46m7u/cDcr162kYmEFUReltrGWorwiZi2ZxeKLFmuqmohIhlKfepJt2LGBSAIJus6aGFaT2K8hEmpmV/Ou4CcYDOFA5p8xj0ER4tbc3/r4LSL1ESoWVhBpiOzeca22sZZIg3e8pqEmoXhFRKRrKKkn2YynZyS8MtsXLnTUlBZ7+40HOdegKRqwlu5rijYy7cTpnFE+Oe5rjOw3kvDqMFHX9ry8qIsSXhV8fXsREek6SupJ9nTVUwkvFLMzWs89j/4E5s9PTVAAOV5H/40jvhm32E0Tb6Jqa9XuGnos7bwmIpK5lNSTKFIfoaadZNiRx9Y+DSncdzy/qBgiEQafewl3LGbf3d4c3HH6PAYWD6S8b/nuddxjaec1EZHMpaSeRIlsuxrrw8iH3g8Ba/l5JDDE3kH/uhwIhyEa5YqXYONcmPoqHL/O+77xzt5c8fYBgLd+fcja/tPQzmsiIplLo9+TKJFtV/eRYD98o2sO/loGvbfVQM2evdUH1sLdj7UutBOqvWb1kgJvh7XY0e8hC2nnNRGRDKaknkS7t13tRGIvzT8gofLmwCXwOp/kNUJ5uTcYr7aNLoKiIijb06w+4ZAJbLh6A+FVYaq3VVNWWkblmEoldBGRDKaknkSVoyu57PHLEj/RwdsbViV2SoIdJwf3GeRtujJrVtsFQqF9NmUpzi/WzmsiIt2I+tSTqKSghILEZpp5DDZHI9767wGFguwE18LBoPKjocTffKWkxKuZg/e9RJuyiIhkA9XUk+yUuv4802dz4k3whjeQLaC8KNQH/UhmUNqnv/fzhAneJizhsNeHXlbm1dCV0EVEuj0l9SRrOKCw8ydXVUHvYEUf33gak4cuD/zhYVPNpj0PiotTOn1ORETSQ83vSbapbnOnz33tkGD7oxeE8pl0/FeY+VJO4FHzr296vdNxiYhI96CknmTNHRdp16TauwKVK8wvgh072FwQfFrbx3Uf70dkIiLSHSipJ1moqJN90wabdgar5Q/s3Y/IdVfz4Ojgl+9f1L9zcYmISLehpJ5kQ4eO6tyJLcu2dlTzdvD1t4r44akJ7NvuYNrR6kMXEcl2SupJds6Y8xJeHS5RX162mV8cT0Iryl0+7vJUhiQiIhlAST3Jpo6dSk6n14oNZsb4rUQTeIkccrQSnIhID6CknmQlBSWc25DCXcwcPDiiPqF58L1yg42qFxGR7k1JPQUmDTiB/MYUXbwTjQA7m2q56umrWPDKAiL1wVetExGR7kVJPQUqL76RgkSWcU1Ugok96uDWF29l5tMzGXLLEFauW5mauEREJK2U1FOgpO9gFp94ByX1UFif7mjY/SGgtrGWSEOEioUV1DTUpDcmERFJOiX1FJlw9hVsuGYjv/jkuOSPht/P60VdlPCq4OvMi4hI96CknkLFpQOZ1ueU5F94PwfX1zbWUr2tOjmxiIhIxlBST7Xy8qRfMi+R+WxtKMoroqw0hSP0RUQkLZTUUywy8eTghQPm6gIXdCm5toUsROWYyv26hoiIZB4l9RS79g8JrOQWpK/cwa/eOjx4v7qDHH+XmaK8IkryS1h80WItRiMikoW0n3qK3VX7p+AfnQLU1POicMuw9QnFMOczV7PlwDzKSsuoHFOphC4ikqWU1FPMJdL9HWBDl0PqclndZ2fg6/1s0xhm/XhuAkGIiEh3peb3TBJgh7Zj60vJyc3vsBwOfjbqSq775evJik5ERDKcknp3YjB02GhOGX5qh+VOP+x0rrvwF10SloiIZAYl9UyQwKC3USefx2nDT+uwaEGoYP9iEhGRbkdJPVMETOyVx07lw8iHHZY7a+RZ+xmQiIh0NxoolwkCDqbLtxDF+cVs27ktbrkcy2Hq2KlJCExERLoT1dQzRYCaupn3Geyg3gfFLXf+kedr2pqISA+kpJ4pAtTWm5y3SfuY/mPoldOrzTIFoQLOGHFGMiMTEZFuQkm9G2l2XnW+cnQleTl5bZbJz83XErAiIj2UknqGMH9ueRAlBd5SryX5JRTlFQFaAlZERDRQLjM4OCCay79ym+KWKXR7PoNNOGQCG67eQHhVmOpt1VoCVkRElNQzxRnNw3kwp7r9vnWD83oft9eh4vxiph0zLfXBiYhIt6Dm9wzxvcFfjvu8ReGUky/qomhERKQ7UlLPELcOeId5K/Lb7Vcvzu1N5bGaey4iIu1TUs8EBm9tf5dZNyzj0Ud6kdvs1cwBQg5KQr1ZPPUZ9ZeLiEhc6lPPBA5G9hsJEyZwzvNb+OT+ewi/v4jqPjDsgEPYcO1mJXQREemQknqGuGniTd4PxcUUXzadaUwHYMWKFUroIiISSMY0v5vZzWb2ppm9ZmaPmNmB6Y5pf0W2bghcdmDxwBRGIiIiPUHGJHXgWWCMc+4zwNvAdWmOZ7/NufuydIcgIiI9SMYkdefcM865ltVXXgCGpjOeZLhtx9LAO7CJiIjsL3Mu4NqkXcjMngDCzrn72nn+cuBygAEDBhz7wAMPdGV4gb2y8R9BV37l2EHHtnm8pqaG4uLs71PvKfcJPedee8p9Qs+5155yn5DZ93raaaf9wzk3rq3nujSpm9lSoK3O4+udc4/5Za4HxgHnugDBjRs3zr388svJDTRJ7Efm1dQ7qq07cD9u+1ZXrFjBqaeemuzQMk5PuU/oOffaU+4Tes699pT7hMy+VzNrN6l36eh359zEeM+b2VTgbOBzQRJ6xguY0ANX50VEROLImCltZjYFuBY4xTlXl+54uoz63EVEJEkyZqAccDtQAjxrZq+a2V3pDkhERKQ7yZiaunOuLN0xiIiIdGeZVFMXERGR/aCknkpBBsA56G0Z02AiIiLdmJJ6CvWtIVBif/K8R1Mei4iIZD8l9RSavD430Oj208eclfpgREQk6ympp9B7/XM0B11ERLqMknoKNRT11jx0ERHpMkrqKTQ6Z1DHNXUlfRERSRIl9RS68ZD/6LBMr5xeXRCJiIj0BErqKTT4om9w6Hbi1tbHDhjbZfGIiEh2U1JPpZISnpn8+7hFFp63sIuCERGRbKeknmKHT/kyVx073XsQU2O/6virGFE6ouuDEhGRrKSk3gVu+ffbqb6ymhOHncjg4sGcOPREqq+s5pbJt6Q7NBERySJan7SLjCgdwfPTnk93GCIiksVUUxcREckSSuoiIiJZQkldREQkSyipi4iIZAkldRERkSyhpC4iIpIllNRFRESyhJK6iIhIllBSFxERyRJK6iIiIllCSV1ERCRLKKmLiIhkCSV1ERGRLKGkLiIikiWU1EVERLKEkrqIiEiWUFIXERHJEuacS3cM+8XMtgDvpzuOFOoHfJzuILpAT7lP6Dn32lPuE3rOvfaU+4TMvtdDnXMHt/VEt0/q2c7MXnbOjUt3HKnWU+4Tes699pT7hJ5zrz3lPqH73qua30VERLKEkrqIiEiWUFLPfL9OdwBdpKfcJ/Sce+0p9wk95157yn1CN71X9amLiIhkCdXURUREsoSSeoYysylm9paZVZvZ7HTHkypmNszMlpvZG2a22sxmpDumVDKzHDP7XzN7Mt2xpJKZHWhmD5nZm/7v9oR0x5QKZnaV/3e7yszuN7Ne6Y4pWczst2a22cxWtTpWambPmlmV//2gdMaYLO3c683+3+9rZvaImR2YzhiDUlLPQGaWA9wBnAmMAr5sZqPSG1XKNAFXO+eOBI4HpmfxvQLMAN5IdxBdYD7wtHPuCGAsWXjPZjYE+A4wzjk3BsgBvpTeqJLqbmBKzLHZwDLnXDmwzH+cDe5m33t9FhjjnPsM8DZwXVcH1RlK6plpPFDtnHvXOdcAPACck+aYUsI5t9E594r/cwTvP/8h6Y0qNcxsKHAWsCDdsaSSmfUBTgb+P4BzrsE596/0RpUyuUBvM8sFCoENaY4naZxzfwa2xRw+B7jH//ke4AtdGlSKtHWvzrlnnHNN/sMXgKFdHlgnKKlnpiHA+laPPyBLE11rZjYcOBp4Mb2RpMytwHeBaLoDSbHDgC3A7/yuhgVmVpTuoJLNOfchMBdYB2wEtjvnnklvVCk3wDm3EbwP5ED/NMfTVf4DeCrdQQShpJ6ZrI1jWT1NwcyKgYeBmc65HemOJ9nM7Gxgs3PuH+mOpQvkAscAv3TOHQ3Ukj3NtLv5/cnnAJ8CBgNFZnZxeqOSZDOz6/G6CRemO5YglNQz0wfAsFaPh5JFzXqxzCwPL6EvdM79Md3xpMhJwOfNbC1ed8rpZnZfekNKmQ+AD5xzLS0uD+El+WwzEXjPObfFOdcI/BE4Mc0xpdomMxsE4H/fnOZ4UsrMpgJnAxe5bjL/W0k9M70ElJvZp8wsH2/wzeNpjiklzMzw+l7fcM7dku54UsU5d51zbqhzbjje7/M551xW1uqccx8B681spH/oc8CaNIaUKuuA482s0P87/hxZOCAwxuPAVP/nqcBjaYwlpcxsCnAt8HnnXF264wlKST0D+YMzvg0swftP4g/OudXpjSplTgK+ildzfdX/qkh3ULLfrgQWmtlrwFHAz9IcT9L5LREPAa8Ar+P9f9otVyFri5ndD/wNGGlmH5jZNOAm4AwzqwLO8B93e+3c6+1ACfCs///SXWkNMiCtKCciIpIlVFMXERHJEkrqIiIiWUJJXUREJEsoqYuIiGQJJXUREZEsoaQuPYKZ/djMXKuvDWb2sJmNCHDu1/xzipMc06n+dcck87r+tYf71z47QNkBZnarmb1jZvVm9omZPWVmk5MdVzYys/Fm9uOAZceZ2d3+DoxRM7s7tdFJT6OkLj3JduAE/+savPnTywKsS77IPyfZC1C84l/3nSRfNzB/gZj/xdtoZi4wCbgEWAs8bmZj0xVbNzIe+FHAsicBE/AWmPooZRFJj5Wb7gBEulCTc+4F/+cXzGwd8BegAngwtrC/BW6Oc24L3gYlSeWvcf9ChwVTayHe7lQnxqy5/4SZ/RLI1t3V0uU259x8ADN7Od3BSPZRTV16spbNVYYD+M2iL5vZF8xsNbALOC62+b1V0/aFZvYrM9vur0L1EzPb69+UmX3GzJ4ws3+ZWY2Z/d3MzvCf26f53X88y8zmm9k2/7zb/OWCW8oMMrPfmtm7ZrbTzN42sxtalwnCzE4GjgWua2sTHefca865da3KX2hmr/tN9OvN7Kf+lqMtz7e8T8eY2Qozq/NX4jrGzIrM7Hf+e/WumX05JpYVZvaQmV1uZmv9+1pk3p7lrcv1M7N7zGyrf/0VZjYupsxaM5trZlf5v5dPzOwBMzswplyp//vbZGa7zOyvZnZcTBlnZjPM7GdmtsXMNpvZHWZW0HLPwG2tyjozW9Hee+6cy/Yd+iTNlNSlJxvuf/8o5tgc4Ea8Gvx7cc6fA9QA5wP3AT/0fwbAzI4AngcGAd8Evgg8wt6b9bTlarxNfC4CbgAuB37a6vl+eLXrWcAU4GbgUvzkkoBTgGZgaUcFzWwSEMbrMjjHf61r8JbSjHUPcD9wHt6Ogw/hre+/Ae/9eRG417z95Vs7AW952VnANOAzwKMxZR4FJvuvXYn3f9hyMyuLKXch3lrsl+Ot3302rZaq9ZPyUrylTv8Tb1/wLcBSMxsYc62r8XZhuxjvvf4GMMN/bhEwr1X8JwBXtPGeiHQN55y+9JX1X8CPgY/xupxygcOB5cAOYJBf5m68LW6Pijn3a/7xYv/xcP/xvTHlXgUeaPX4frwdy3q3E9Op/nXGtDrmgDeBUKtj1+P155e2c51c4Ct4LQv5MTGeHec9uQvYGPD9ewFYHnPsu3gfCobGvE9TW5Wp8I/9ttWxA4BG4Futjq3wjx3a6thJ/rlT/MdT/MentCpThJeMf9Xq2Fq8cQq5rY7dCnzU6vE0oAEoj3kf3wFujvl9/Dnmvh8FXmj1+Nvef6UJ/02+DNyd7n8b+squL9XUpSfpi5c4GoG3gMOASufcxlZlPnTOvRrwes/EPF6DV8NucToQds7tTDDOx9zezbR/BHoDY8Db2c7MZprZGjPbiXc/C4EC4JAEX6vDzR/8sQXHsO+4gzBeTfmEmOPLWv1c7X9/bvcLOrcdLxHv1bQOvOKce79VuefxtvYc7x8aD2xxzv2pVZla4Em8wWetLXfexkgt1gD9W3VRTMTrfnnPzHJbdSP8CdirOZ+Of88iGUMD5aQn2Y73n7nDa3Lf4JyLTWqbErhe7CCyBqBXq8d9gY0kLnaP6pbHg/zvM/FGqt+El4Q+Af4NuCPm9TvyIXCwmfVyzu2KU64fkMe+703L49KY463fl4Y2jrUcj421rb25N7Pnvge1EUNLHPFiaHk9A/L9n/sBx+N9IIoVOxshSOwiGUFJXXqSJudcRyOOk7lt4Vb2JKRE9G/nccsHhAuAB51z17cUMLNRnXidFcB/4fU9L4pT7mO85Bcb1wD/+7ZOvHZbYq/fcqzlvje2U2ZAJ2LYhtf8/a02nqtP8FoiGUPN7yKpswy40MwSrdWdEzOK/lxgJ7DKf9ybfRPPRYkG55z7C14T9M/MrCT2eTP7tJkNc841++UuiClyIRDF24c6GY4xs93dB2Z2El4S/7t/6EW8JvSTW5UpxJtjvzLB11oGlAHrnHMvx3y9nuC1GvxYVHuXtFNNXSR1foK3yMifzWweXs39aGCrc+63cc4rAR40s98Ao/FG1d/unGupjT4LfMfMXsRrKr4IL0F1xkV4AwZfNrP/xusv7oM3wvwy4DhgPd7iKkvM7HfAA8Cngf8H/MY590EnXzvWZuBJ81Zn6wX8HK+f/WkA59wSM3seCJvZbLz38xq8Dzk3J/ha9+LNSFhhZnOBd/G6S8bjDaj77wSu9ab/fYaZPQfscM691VZBMzsYb9YBwEHAoWZ2PoBz7qEE70FkH0rqIininHvLzCbg9X0v8A+vAb7Xwanz8Abx3Y/XmrYg5pz/Ag7Gm+4G3kC67wBPdDLGY4Dr8EazD8Ebaf934CvOuX/65Z4xsy8B38f7ILDZjzPoSmpB/A1vmtmtePe3Am9KWmtf9F/3VrzE/3fgdOdcNQlwzu0ys9Pw3suf4DXhb/av93iCcf8F70PFDLypkH/Gm9nQltHsPeDwsFZlLcHXFdmH7TtOSETSxcwccKVzrq3531nLX7DlY+fc+R2VFZH2qU9dREQkSyipi4iIZAk1v4uIiGQJ1dRFRESyhJK6iIhIllBSFxERyRJK6iIiIllCSV1ERCRLKKmLiIhkif8DxCjHnwm2fxIAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#visualizing pca\n", + "pca_visualize_df = pd.concat([principalDf, y_train], axis = 1)\n", + "fig = plt.figure(figsize = (8,8))\n", + "ax = fig.add_subplot(1,1,1) \n", + "ax.set_xlabel('Principal Component 1', fontsize = 15)\n", + "ax.set_ylabel('Principal Component 2', fontsize = 15)\n", + "ax.set_title('2 component PCA', fontsize = 20)\n", + "targets = [1,0]\n", + "colors = ['r', 'g']\n", + "for target, color in zip(targets,colors):\n", + " indicesToKeep = pca_visualize_df['Y'] == target\n", + " ax.scatter(pca_visualize_df.loc[indicesToKeep, 'principal component 1']\n", + " , pca_visualize_df.loc[indicesToKeep, 'principal component 2']\n", + " , c = color\n", + " , s = 50)\n", + "ax.legend(targets)\n", + "ax.grid()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, there is no clear linear separation for the target attributes for 2 pca components, justifying the above percentages. Nonetherless, we will continue to use PCA by finding the optimmum number of PC components which retains 90% of information" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Perform PCA to retain 90% of information\n", + "perform PCA to reduce components so we can run SVM model" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": {}, + "outputs": [], + "source": [ + "#setting pca threshold to 90%\n", + "pca = PCA(0.9)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PCA(copy=True, iterated_power='auto', n_components=0.9, random_state=None,\n", + " svd_solver='auto', tol=0.0, whiten=False)" + ] + }, + "execution_count": 73, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pca.fit(X_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "No. of components before pca: 44\n", + "No. of components after pca: 13\n" + ] + } + ], + "source": [ + "#get number of components after pca\n", + "print('No. of components before pca: {}'.format(len(X_train.columns)))\n", + "print('No. of components after pca: {}'.format(pca.n_components_))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, the number of components is reduced from 26 components to 13 components" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": {}, + "outputs": [], + "source": [ + "#perform pca on training and test attributes\n", + "X_train_pca = pca.transform(X_train)\n", + "X_test_pca = pca.transform(X_test)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Apply SVM model\n", + "Next, we will used the reduced attributes train set to train our SVM model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First, we train our SVM model without parameter tuning\n", + "nor pca reduction" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n", + " max_iter=-1, probability=True, random_state=None, shrinking=True, tol=0.001,\n", + " verbose=False)" + ] + }, + "execution_count": 76, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn import svm\n", + "#train svm model without standardization and no parameter tuning\n", + "clf_original = svm.SVC(kernel = 'rbf', probability = True, gamma = 'scale')\n", + "clf_original.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.1660749149902864\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAEWCAYAAAAkUJMMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gUVffA8e8hgYSQhBZaCKH33kWKKKCAoLyCCCIW9KeIYn0VK9hRxA72V7GCCqIIKkVRQFSK9N4htFDSQ0LK/f0xQ1xCygaymezmfJ6Hh+zUM7Ozc2bu3LlXjDEopZRSTinldABKKaVKNk1ESimlHKWJSCmllKM0ESmllHKUJiKllFKO0kSklFLKUV6TiERkhIgscDoOp4lIpIgkiohfEa6zjogYEfEvqnV6kohsEpGe5zHfeR2DIlJWRH4QkTgR+aag83urghyrhXlci8g0EXnOzWl96th2ZW9Xg2IQR77fx3klIhHZKyKn7APniL2i4PML0z3GmC+MMZd7ch3Fkb2ve5/5bIzZb4wJNsZkOBmXUwrjx2WMaW6M+S2f9ZxzgrqAY3AIUA2obIy59jzmzym+x0Rkj/0bjBKRr+zh74nIpzlM30pEUkWkkog8ZW/bPdmmuc8e/lRhxFiQY9UbjmsR6SkiURcw/5ljKtH+t1dEHsk2jeu5NUZE5olILZfx00TktMsyEkXkugvZruLgQu6IBhpjgoE2QFvg0cIJqWg5eSXki1dh+Smh+7s2sN0Yk17QGXOKWURuAkYCve3fYAfgF3v0NOAaESmXbbYbgbnGmJP25+3ATTlMs72gMbobt8pSwf7ehgBPikifbOPPnFtrAEeBt7KNn2Qn7TP/vvJksEVR+nLBRXPGmCPAfKyEBICIBIjIZBHZLyJHReRdESnrMv5qEVkrIvEisktE+trDy4vI/0TksIgcFJHnzuwEEblZRJbZf78rIpNd4xCR70XkAfvvcBGZJSLH7KvGe1yme0pEZorI5yISD9ycfZvsOD61598nIk+ISCmXOP4QkbfsopatItIr27x5bcMfIvKaiJwEnhKR+iLyq4icEJHjIvKFiFSwp/8MiAR+sK98Hs5+pS4iv4nIs/ZyE0RkgYiEucRzo70NJ0TkScl2h5Vtu8uKyCv29HEissz1ewNG2N/pcRF53GW+TiLyp4jE2ts9RUTKuIw3InKXiOwAdtjD3hCRA/YxsFpEurtM7yfWFf8ue5tWi0gtEVliT7LO9UpQRAbYx1OsiCwXkVYuy9orIuNEZD2QJCL+rvvAjn2VHcdREXnVnvXMumLtdXVxPQbteZuLyEIROWnP+1gO+/RpYDxwnb2cW0WklH1M7RORaPtYK29Pf+b7vVVE9gO/5vBVdQTmG2N2gfUbNMa8b//9J3AQGOy6P4HrgU9clrESCBKR5me2BShrD89RQePO4VitKyJL7O90kYhMFZHPs83v7nH9jVilMXH2MpvnFne2bfAT69x0XER2A1dmG3+LiGyx17lbRO6wh5cDfgLC5d87kfD8jv28GGNWAZtwOXdmG58CzASaubO8vIhIN/v3dqn9uYnLsbtNRIa6TDtNRN4RkR9FJAm41B42Vaw7tAQR+VtE6rvMk+vy3GKMKfA/YC/W1RhABLABeMNl/OvAHKASEAL8AEy0x3UC4oA+WImwJtDEHvcd8B5QDqgKrADusMfdDCyz/+4BHADE/lwROAWE28tcjfXjLwPUA3YDV9jTPgWkAYPsacvmsH2fAt/bsdfBukq81SWOdOB+oDRwnb09ldzchnRgLOCP9cNvYO+LAKAK1gnw9Zz2tf25DmAAf/vzb8AuoJG9vN+AF+1xzYBEoJu9Lybb2947l+91qj1/TcAPuNiO68w6P7DX0RpIBZra87UHLrK3qQ6wBbjPZbkGWIh1PJS1h90AVLbneRA4AgTa4x7COqYaA2Kvr7LLshq4LLsdEA10tmO+yd5nAS77by1Qy2XdWfsU+BMYaf8dDFyU037O4RgMAQ7bsQfanzvnsl+fAj53+TwK2Il1bAYD3wKfZVvvp1jHUE7H5w3ASXs/dQD8so1/HFjk8vkK4BhQ2jUe4DHgJXvYJKxSjc+Bp3LZjgLFnX0f2vt6Mtax2A2IP7Nfcpj2N3I5rl1iCcE6Pl8H1rqMmwY8l8s2jAa22sdDJWBxtvVeCdTHOu4uAZKBdva4nkBUtuXleexnmzb7Nl5kL/8/uZxbg7AuHj51Z9tyWJ/BOr9cgXW+7GQPL2d/vsWOux1wHGjuso44oCvWOTLQHnYS6/ztD3wBzCjA8vKM+UISUSKQYG/sL1i3m9hfYBJQ32X6LsAe++/3gNdyWGY1rJNbWZdhw4HFOZwEBNgP9LA//x/wq/13Z2B/tmU/Cnzs8iNckse2+dlxNHMZdgfwm0sch7CToD1sBVZRiTvbsD+3ddvTDALW5HRg5vGDfcJl/BjgZ/vv8cB0l3FBwGlySET2AXcKaJ3HDygi2zYPy2Ub7gNmZ/tBXJbPdsecWTewDbg6rx+Xy+d3gGezTbMNuMRl/43K4fg982NfAjwNhOWyzbklouGu31M+2/YUZyeiX4AxLp8bY10g+Lust14+yxwBLML6rZ0AHnEZF2kvL8L+/AVnXyg+hZVwIrF+R6Xt/2uRdyIqUNyu+9BeVzoQ5DL+c/JORDke1znEVcGet7z9eRq5J6JfgdEuny/P/j1nm/474F77755kS0T5Hfu5HFOxWL81g5WYXc8le7HOrbH2/joEtHQZPw1IscfHAsfziMVgnfv2ZVvGdcDSbNO+B0xwWcen2cZPAz50+dwf2FqA5eWZiC6kaG6QMSbE/nKaAGdum6tgnfBW27erscDP9nCwDvZdOSyvNtYP4rDLfO9h3VWcxVhbNwPrZABWscMXLssJP7MMezmPYSWJMw7ksV1hWFds+1yG7cO6SzjjoB2D6/hwN7fhrHWLSFURmSFWMV481o8zjII54vJ3MtbVKnZMWeszxiRjnbRyEoZ15ZPTd5PnekSkkYjMtYtK4oEXOHcbsm/3g3YRSJy9n8q7zJPbMZKT2sCD2b7vWljbnuO6s7kV66p7q4isFJEBbq63IDFmF865x5c/7h+jGKviRG+sk/Bo4BkRucIetx8rwd4gViWiQZxdLIfLdDuxvq8dxpg813mBcYcDJ+1jML9pz8jtePMTkRfFKrqNxzp5g3u/m7N+E5y9PYhIPxH5yy5iisU64ea6XDeP/ezC7G35L9b5s3S28YOMMRWw7vbuBn4Xkeou4ycbYyrY//Jb133A18aYDS7DagOds/1mRgCu68jpu8ntPOPO8vJUGM+IfsfKeGee2RzHyvbNXXZWeWM9fANrA+ufuyQOYN1NhLnMF2qMya3sdzowRERqY90FzXJZzh6XZVQwxoQYY/q7hp3HJh3Husqr7TIsEqvc/YyaIiLZxh9ycxuyr3uiPayVMSYUq9hF8pi+IA5jFZ0C1jMgrOKwnBzHutLK6bvJzztYxR0N7W14jLO3AVy2Q6znQeOAoUBF+0cX5zJPbsdITg4Az2f7voOMMdNzWnd2xpgdxpjhWBcLLwEz7ecB+e33gsSY3SHOPb7SsR5MZ4XmzoKMMWnGmG+A9UALl1GfYFU+GIz1e/gnl0V8ilW8eE5Nu0KO+zBQSUSCXIbVymXa/FwPXA30xrqAqWMPz37M5RaH63ojz/whIgFY55HJQDX7uPzRZbk5bZs7x/45jDEZxphXsH5zY/KY5lsgA6so83xcCwwSkftchh0Afs/2mwk2xtzpuvoCrMOd5eWpsN4jeh3oIyJtjDGZWM8SXhORqgAiUvPM1RrwP+AWEekl1sPPmiLSxBhzGFgAvCIiofa4+iJySU4rNMaswSr3/hDrwW2sPWoFEC/WA+qy9tVTCxHp6M6GGKv66NfA8yISYie6B7DuVM6oCtwjIqVF5FqgKfBjQbfBFoJ9Ky4iNbHK/V0dxSqTPx8zgYEicrH9APVpcvmR2N/bR8Cr9kNYP7Ee0Ae4sZ4QrPL+RBFpAuR3AIZgncCOAf4iMh4IdRn/IfCsiDQUSysROZNAs++PD4DRItLZnraciFwpIiFuxI2I3CAiVeztP3MMZdixZZL7vp8LVBerynOAfax0dmedWBdR94v18D4Y6yr6K+NmrTqxKk1caa+zlIj0A5oDf7tMNgvrhPs0OdwNufgKq3jqa0/GbYzZB6zCqqBTRkS6AAPdWGdOQrAu+E5glb68UIB5v8b67UaISEXAtfp0Gay7kGNAur1fXavrHwUqi11BwyWWghz72b0IPCwigdlH2Mfz1VjPwLcUcLlnHAJ6YW3zmYQ3F2gkIiPtc1hpEekoIk3Pcx0XvLxCSUTGmGNYV1RP2oPGYd3y/2Xfri7CKk/GGLMC66HWa1hXwb/z71XWjVgHw2asZwYzsaow5mY61lXRly6xZGAd4G2APVhX+h9iXTm5ayxW2ftuYJm9/I9cxv8NNLSX/TwwxBhzpsiroNvwNNbDvThgHtYDYFcTgSfsW97/FmAbMMZssrdlBtaVYALWg/3UXGb5L1YlgZVYDyZfwr1j5L9YV6kJWIkhv+qk87FqIG3HKhpJ4eyigFexThgLsH7k/8N6YA3W841P7P0x1Fg1j/4PmIK1v3eSQ03IPPQFNolIIvAG1nOvFLsI6XngD3tdF7nOZIxJwKpkMhCryGIHcKmb6/wI+Ayr+GwP1vaPLUDM8VhX3vuxkuck4E5jTFaNPmNMEv8moy9yWog93SljzCJjzKkiiHsE1vPiE8BzWMdJbsdiXj7FOm4OYv3O/irAvB9gHX/rgH9w+b3Z3+k9WMdeDNYxPcdl/Fasc85u+5gIp+DHfnbz7HX9n8uwH+zjMR7rGLzJ/i2fF7sIthcwTkRus7fzcmAYVqI6gvVbd+eiM6flX/DyztQ6U24SkZuB24wx53ur7Bj7KjYWqxhhj9PxqJJNrJdwtxpjJjgdi3KW1zTxo86PiAwUkSD7ucdkrDuevc5GpUoiu7imvl2c2BfrOc93TselnKeJyPddjXW7fAirOHGY0dtg5YzqWNWyE4E3sYoT1zgakSoWtGhOKaWUo/SOSCmllKO8rmHCsLAwU6dOHafDUEopr7J69erjxpgq+U9Z9LwuEdWpU4dVq1Y5HYZSSnkVEdmX/1TO0KI5pZRSjtJEpJRSylGaiJRSSjlKE5FSSilHaSJSSinlKE1ESimlHOWxRCQiH4nVr/3GXMaLiLwpIjtFZL2ItPNULEoppYovT94RTcNqYj83/bDaPmsI3I7VwZRSSqlClplZvJty89gLrcaYJSJSJ49JrsbqF91g9VtUQURq2J3LKaWUukCHY08x+M1lHEo+7XQoeXKyZYWanN0ZWpQ97JxEJCK3Y901ERkZmX20UkopF3HJaVz91lL2xlj9HUqyW53/OsbJygo5dVmd4/2jMeZ9Y0wHY0yHKlWKZVNJSilVLOw+lkjrZxawN+YUJj2TS0OC2PHq+fbKXjScvCOKwurG+IwIrD5zlFJKFVBKWgaDXl/K1hNJALSuFsKUIa2pVau8w5Hlz8lENAe4W0RmAJ2BOH0+pJRSBXfseBI9XlrMKT8hAHj35o5c2qSq02G5zWOJSESmAz2BMBGJAiYApQGMMe8CPwL9gZ1AMnCLp2JRSilfZIzhw2lreObXHfjVDKaKERY/2Yvg4ACnQysQT9aaG57PeAPc5an1K6WUrxsybj6rS2XgVzMYgAXjexNcrozDURWc1/VHpJRSJdmpU2nsOZLI60t3sbpUBgAP9G7EHT3rEeDv53B050cTkVJKeYFjCam8MH0ts5fvh+pBWcPfvaEdfVvUcDCyC6eJSCmliiFjDKfSMpj441YWbzlKVFyKNaJqWdpUCeb6S+pxbfsIRHJ6E8a7aCJSSqli5r3fdzHxp61nDUvdn8BVjaryxiPdCQws7VBknqGJSCmlioHohBRmro7ivd93E3cqDYBOdSvRuVYF/vpsExOfvZwGDSo5HKVnaCJSSikHxaek8eqC7UxbvjdrWNUMkD8OM/35fvj5lYL+TZ0LsAhoIlJKqSK25XA8k+dv45et0WcNH1SrEl+/sJxVhxMZM6YjqakZBAX5frdxmoiUUsrDjDG8v2Q3P286wpr9sWeNu6ZtTbrUqsDHz/3BGy+tom3b6nz/3TA6dqzpULRFTxORUkoVsuj4FLYdTSAlLZNv/4nip41Hssa1jihP0xqhDO1Yi3aRFQFITU3n5ePJvP76Fdx1Vyf8/X3/LsiVJiKllLoAp05n8N3ag+w5nsSCTUc4GHuKtIxzOxIY1bUu9/ZuSPmyVo23JUv2ccX/zWPWrKEEB5fhr79uo1Qp76+KfT40ESmllJui41OYveYgcafSOJWWwYJNRzkYe+qsacKCy3B959o0Dw8lvHxZAkuXok5YOUr7WXc5x48n89BDC5k2bS116lRg795YWrSoWmKTEGgiUkqpPKVlZLJw81Emz9/G7uNJWcNDAv0JCfCne8MwBrYOZ2CrcMqWyb2JHWMMH3+8loceWkh8fCqPPtqNJ57oQVCQb70TdD40ESmllIuk1HTeX7Kb+ZuOEJN8mqPxqVnjWtQM5e5LG3Bpk6rn1a7b55+vp1mzKrz77pU0b+493TR4miYipVSJd+BkMm/8soM5aw9xOiMza3jDqsG0qVWBljXLM7B1OLUrlyvQcpOT03jhhaWMHt2BiIhQZs0aSvnygSW6GC4nmoiUUiVKWkYmc9cfYvHWYxyNT2HDwTiST2dkje/dtBo9G1dhaIdalLmA2ms//riDu+76kb17Y6lZM4Q77+xIxYplC2MTfI4mIqVUiXAiMZWHZ64/5yXSjnUqEhzgz/Wda9OjUdgFd6UQFRXPfff9zKxZW2jaNIzff7+ZHj1qX9AyfZ0mIqWUzzDGkJCazsaoOE6lZXAw9hSJqeks23Gc5btOZE13S9c63NurIRWCCr8TueefX8K8eTt44YXLePDBiymTRwUGZRGro1Tv0aFDB7Nq1Sqnw1BKOcgYw+bD8ayPiuOrlQfYfDiezExDhjHkdkoL8C/Fi4NbMrBVOP5+hfvC6IoVBylb1p+WLatx4kQycXGp1KtXsVDXcaFEZLUxpoPTceRE74iUUl7jWEIqN3+8gk2H4s8aXrtyEJGVgmhbqwIBpf0oX7Y0rSLKE+DvR8Wg0gQH+hPg74dfIVcSiItL4bHHfuGdd1YxYEAj5swZTuXKQVSuHJT/zCqLJiKlVLGWlJrOJ3/u5ZPle7OqUtcoH0iriPKM6lqX1rUqEFi6aIu/jDF89dUm7r9/PtHRSYwd24lnn72sSGPwJZqIlFLFzuG4U7yyYDtbDsefdffTLrICg9tHMKKzsw//P/98PTfe+B0dOoQzd+5w2rcPdzQeb6eJSClVLMQlp7HlSDwfLt3Noi3/1mzrXLcSlzapys0X1ynyOx9Xqanp7N4dQ9OmVRg6tDnp6ZnceGNrq78gdUE0ESmlHBGdkMK6A3H8uOEwi7YcJSEl/azxL17TkmGdIh2K7myLF+/hzjvnkZycxo4dYwkI8OeWW9o6HZbP0ESklCoSGZmGv/ec4JkfNrP1SMJZ40IC/GlYNZhhnSLpUq8yzcJDHYrybNHRSfz3vwv47LP11KtXkfffH0hAgJ42C5vuUaVUoTt1OoN/9sewPiqO5buOs3TH8bPGVw8NpFPdSvRpVo2mNUJoUDXEoUhzt3PnSTp1+oDExNM8/nh3Hn+8O2XLagOlnqCJSCl1waITUvjy7/3MXB2FMZzTNQJAtwZh9GgURvvalWhfu3i9Y+MqPj6V0NAA6tevyK23tmXUqLY0bVrF6bB8miYipVSBRSekcNsnq0hMST+rawSAsOAAbuxSm2qhgfRuWo3alYMcrWTgrqSk0zzzzO988ME/rF9/JxERobz88uVOh1UiaCJSSuXJGENqeibvL9nN6n0x/L792Fnjr20fQXqmoX/LGvRuWhUR72tZ+ocftnH33T+xf38ct97aVvsIKmKaiJRSZ5m7/hC/bolm7YFYDsQkn9Ptdac6lQgoXYoh7SMY2Crcq7s0SE/PZOjQb5g9eyvNm1dh6dJb6NateNTUK0k0ESlVgkUnpPDNqihW74vhYMwpth39tzZbuTJ+NK0RSlhwAB3qVCQksDTXto/wimK2/BhjEBH8/UtRo0YwL77Yi/vv76INlDpEE5FSJYgxhvVRccxYeYB56w8R7/LuTrcGYdSoEEiFsqUZ168JNcr7Zt85f/0VxV13/cgHHwykXbsaTJ16pdMhlXiaiJQqIaITUrh6yh8cjkvJGta9YRjXd4qkb4vqXvlspyBiYk7x2GO/8N57qwkPDyEm5tyafcoZHk1EItIXeAPwAz40xryYbXwk8AlQwZ7mEWPMj56MSSlfZozhYOwpZv9zkM2H40lNz+R0eiZ/7zmR9aynb/PqjLm0Pi1rlvf55HPGV19t5J57fub48WTuu+8inn66JyEhAU6HpWweS0Qi4gdMBfoAUcBKEZljjNnsMtkTwNfGmHdEpBnwI1DHUzEp5UsyMg3bjiSwaMtRft1qVS7IrmmNUILK+NGhdiX8/YRhHSO5slUNB6J11tatx6lTpwI//zyCtm1L3vYXd568I+oE7DTG7AYQkRnA1YBrIjLAmbY8ygOHPBiPUl7PGMM/+2N5fPaGc5rJaVI9hLph5WgeHkrj6qH0alLVq2u0XYiUlHReemkZ7drVYODAxjz2WHeeeKKHNlBaTHkyEdUEDrh8jgI6Z5vmKWCBiIwFygG9c1qQiNwO3A4QGalVK1XJYYxh1b4Ypq/Yz8LN5zYMemu3uvRoVIXOdSv5RG22wrBo0W7GjJnHjh0nefDBLgwc2JjSum+KNU8mopwuxbJ34jscmGaMeUVEugCfiUgLY0zmWTMZ8z7wPlhdhXskWqWKkW1HEvho2R6+XRN11ns8PRtXoVXN8vRtUaPYNAxaXBw9msgDDyzgyy830KBBJRYsuIE+feo7HZZygycTURRQy+VzBOcWvd0K9AUwxvwpIoFAGBCNUiXMos1H+XZNFHuOJ7PlsNUZXIWg0gxoVYMbLqpNk+qaePKycOFuZs7czPjxPXj00e4EBmqlYG/hyW9qJdBQROoCB4FhwPXZptkP9AKmiUhTIBA4hlIlRGJqOl+vPMD7S3ZzJN6qVh1YuhQ3XBTJ4HYRtI0svo2DFgfr1h1hx46TDBnSjBEjWtK1ay3q1tV95m08loiMMekicjcwH6tq9kfGmE0i8gywyhgzB3gQ+EBE7scqtrvZGKNFb8onZWYajiakEBVzim//iWLN/tizKhz0a1Gdp65qTrXQQAej9A6JiaeZMGExb7zxN3XqVGDQoCb4+5fSJOSlPHrvar8T9GO2YeNd/t4MdPVkDEo5xarhFkN0fCpLdhxn+or950wzuF0EXepXpluDMKqX1wTkju++28rYsT8RFRXP7be3Y+LE3vj7a204b6aFqEoVkrSMTLYfTWD+xiNsOZLA2gOxHEtIzRof4F+KSxtXpX+rGkRULEs7LXYrsA0bjvKf/3xFy5ZV+eqrIVx8ca38Z1LFniYipc7T0fgUvvh7P3/tPsHO6EROJp0+a3z9KuVoU6sC91zWkKqhAVrkdp7S0jJYunQ/l11Wl5YtqzFv3vX06VNPq2T7EE1EShXApkNx/LIlmgWbj7DxYHzW8MrlytC9YRgX1atMz8ZVaFYjtMQ0n+NJy5cfYPTouWzadIxt2+6mQYNK9O/f0OmwVCHTRKSUG+JT0nhz0Q4+XLYna9gljapww0W1S3QLBp5y8uQpHnlkER988A+1aoXy7bdDadCgktNhKQ/RRKRULpJPpzN5/nY++uPf5FOrUlneHNaWRtVCKBegPx9PSElJp02bdzl0KIEHH+zCU0/1JDi4jNNhKQ/SX5JS2SzZfoxPlu/ll63/vlfdvWEYQzvUon/LGvjp3Y9HREXFExERSmCgP88+eylt2lSndevqToelioAmIlXinU7PZPmu47z7+y52RidyPNGqdNClXmUaVw/hkX5NtB03Dzp1Ko2JE5fx0kt/MHPmtQwc2JibbmrjdFiqCLmViESkDBBpjNnp4XiUKhKxyadZuPkoryzYntWiwRlDO0TwcN8mhAVrfzWetmDBLsaMmceuXTHccEMrOnWq6XRIygH5JiIRuRJ4FSgD1BWRNsAEY8x/PB2cUoVtwaYjfLh0Dyv2nswaFlGxLNd1qMXA1uHUrhyktd2KyNixPzJlykoaNqzEokUj6dWrntMhKYe4c0f0DFb3DYsBjDFrRaSBR6NSqpAcT0zluzUHmfTzNk5n/Nuoe3CAP+P6NaFLvco0qBrsYIQlS4b9Hfj5leKiiyIICwti3Lhu2kBpCefOt59mjInNdpWo7cGpYssYwyOzNjB7zcGzkk+FoNLc3qMe17SN0OZ0HPDPP4cZPXouI0e2YuzYzowY0crpkFQx4U4i2iIiQ4FSdkva9wJ/eTYspQom7lQae48nMXf9IT5Y+m9164Gtw+nVpCqXNa1KaGBpByMsuRISUhk/fjFvvrmCKlWCqFEjxOmQVDHjTiK6GxgPZALfYrWm/agng1LKHfEpaSzeGs3sNQf5bdvZvYe0r12RL27rrLXdHLZgwS5GjfqeQ4cSGD26Ay+80IsKFfRuVJ3NnUR0hTFmHDDuzAARuQYrKSlVZDYfimdHdAJr9scybfnes8ZVCw3glq516VC7Im1qVcDfT1tjLg7KlPGjatVyzJo1lM6dI5wORxVTkl/3PyLyjzGmXbZhq40x7T0aWS46dOhgVq1a5cSqlQNW7j3J9L/38+2ag+eMK+NXivv7NGJg6xqEly+rzewUA2lpGbz66p/Ex6fy/PO9AKsfJv1unGeftzs4HUdOcr0jEpErsLrxrikir7qMCsUqplOq0KVlZPLbtmP8ti2aOesOkZCSDkApgdt71KdbgzAaVQumSkiAVrMuZpYt25/VQOm11zbLSkCahFR+8iqaiwY2AinAJpfhCcAjngxKlTwbD8bx0Mz1bDn8b4vW9cLK0bRGKE9e2Yxm4aHatE4xdeJEMuPGLeJ//1tDZGR5fvhhOAMGNHI6LOVFck1Expg1wBoR+cIYk5LbdAamQ68AACAASURBVEqdr40H45g0fxu7ohM5GHsKgNBAf27tVo/hnWpRVfvv8QonTpxixoyNPPzwxYwffwnlymkDpapg3KmsUFNEngeaAVlnBmOMXvKo87LjaAIPz1rPmv2xWcMub1aNMZc2oE2tCg5Gpty1Zcsxvv56ExMm9KRRo8rs338/lSqVdTos5aXcSUTTgOeAyUA/4Bb0GZEqoPiUNFbvjWHq4p2s2hcDQNMaoTw3qAXta2uX2d4iOTmN559fwssvLyc4uAy33tqOiIhQTULqgriTiIKMMfNFZLIxZhfwhIgs9XRgyvut2HOSj5btYXt0AruPJWUNDyrjxyejOtGxjnZ05k1+/nknY8bMY8+eWG66qTUvv9yHKlXKOR2W8gHuJKJUsaon7RKR0cBBoKpnw1LeJjE1ncVbo/lz9wnW7o9ls0ulgwD/UgzvFEn72hVpHVGeBlWDtcabl0lMPM3IkbOpXLksixffRM+edZwOSfkQdxLR/UAwcA/wPFAeGOXJoJT3yMg0vPTzVt5fsvus4RWDSjOgVTgjLoqkcbUQTTxeKCMjk+nTNzJ8eAuCg8uwaNFImjQJI0B7plWFLN8jyhjzt/1nAjASQET0FekSbPW+k7y2cAd7jidl1XYDeHlIK3o3rUZFrTXl9VavPsQdd8xl9erDlC3rz+DBzbS3VOUxeSYiEekI1ASWGWOOi0hzrKZ+LgM0GZUw248mcN+MtVnFbhWDStO/ZXVaRVTgxi61CSqjV8reLi4uhSefXMzUqSupWrUcM2YM5pprmjodlvJxebWsMBEYDKzDqqAwG6vl7ZeA0UUTnnJaRqbhuzUHefqHTcTbrRw0rBrMm8Pb0rRGqMPRqcI2ePDX/PrrHu66qyPPPXcZ5bW7DFUE8rqEvRpobYw5JSKVgEP2521FE5pyUkJKGrd9soq/9/zbk2mriPI8dVVz2kVqdWtfsnt3DFWqBBESEsDzz19GqVJCx47aZbcqOnklohRjzCkAY8xJEdmqScj3pWdksmTHMUZN+7dh2Zu61ObOng20Mzkfc/p0BpMnL+fZZ5dwzz2deOmlPtpCtnJEXomonoic6epBgDounzHGXOPRyFSR23ciiQFvLctqaPSq1uG8Obytw1EpT1iyZB+jR89ly5bjDBnSjHvu6ex0SKoEyysRDc72eYonA1HO2XQojsdnb2TtAavJncbVQvjolo7UrKBvy/ui1177kwceWECdOhWYN+96+vdv6HRIqoTLq9HTX4oyEFX0UtMzuGf6GuZvOgpA1ZAA3hzelovqVXY4MlXYMjMNSUmnCQkJ4MorG3HsWDJPPNGDoCDtPl05T+vbllAnElNp/9wiwEpA741sT1uthOCTNm2KZvToeVk9pTZqVJkXXujldFhKZfFof8oi0ldEtonIThHJsQ8jERkqIptFZJOIfOnJeJR1Zfzy/K1ZSah1RHmWjbtMk5APSk5O49FHF9GmzXts2XKMAQMakl+PzEo5we07IhEJMMakFmB6P2Aq0AeIAlaKyBxjzGaXaRoCjwJdjTExIqJt2HnQ/5bt4dm5Wbufide0ZHinSAcjUp6yZs1hrrnma/bujeWWW9owaVIfwsKCnA5LqRzlm4hEpBPwP6w25iJFpDVwmzFmbD6zdgJ2GmN228uZgfVu0maXaf4PmGqMiQEwxkQXfBNUfg6cTGbs9DVZlRFGXlSbx69sSmBpP4cjU4XNGIOIEBlZnsjI8nzyySB69KjtdFhK5cmdO6I3gQHAdwDGmHUicqkb89UEDrh8jgKy1xFtBCAifwB+wFPGmJ/dWLZyQ1pGJg98vY4f1h3KGvb5rZ3p1jDMwaiUJ6SnZzJlygrmzNnGwoUjqVw5iN9/v9npsJRyizuJqJQxZl+21pMz3Jgvp+aWsxdQ+wMNgZ5YbdctFZEWxphY14lE5HbgdoDISC1KyosxhlX7YvhuzUG++Ht/1vA3h7flqtbhDkamPGXFioOMHj2XNWuO0K9fA+LjU6lYUaveK+/hTiI6YBfPGfu5z1hguxvzRQG1XD5HYDUTlH2av4wxacAeEdmGlZhWuk5kjHkfeB+gQ4cO+rQ1F5mZhnbPLSQ2OQ2wKiIMbB3OjV3qUMbfo/VSlAMSE08zbtxC3nlnFTVqhPDNN9cyeHBT7XJDeR13EtGdWMVzkcBRYJE9LD8rgYYiUherM71hwPXZpvkOGA5ME5EwrKK63agCO3AymSvfXEp8Sjo1K5Tlk1EdaVA1xOmwlAeVLl2K337bx9ixnXj22csIDQ1wOiSlzos7iSjdGDOsoAs2xqSLyN3AfKznPx8ZYzaJyDPAKmPMHHvc5SKyGau47yFjzImCrqskOxh7ilfmb+PbNQcBGNCqBm8Nb6tXxT5q586TPPPM70yd2p+QkABWr76dwEB9HVB5N8nvvQIR2QVsA74CvjXGJBRFYLnp0KGDWbVqVf4TlgAbD8Yx4K1lWZ+nXN+WAa30OZAvSk1NZ9KkP3j++aWUKePHvHnX07271oZT7hOR1caYDk7HkRN3emitLyIXYxWtPS0ia4EZxpgZHo9O5Sgj0zB2+j/8uOEIANe2j+DFwa3wK6V3Qb5o8eI93HnnPLZtO8F11zXn1VevIDxci12V73Drnt4YsxxYLiJPAa8DXwCaiIrY3uNJjJ+ziSXbj2UNmzu2Gy1qlncwKuVJxhief34paWmZ/PzzCK64ooHTISlV6Nx5oTUY60XUYUBT4HvgYg/HpVwkpabTZeIvWT2kgtVH0O2X1NcWsn1QZqbhf//7h759G1CrVnk+++w/VKgQSNmy2kCp8k3u3BFtBH4AJhljlno4HmU7GHuKndGJRMUk8/jsjQBUDw3klaGt6dpAX0j1VevXH2X06Ln8+WcU48f34OmnL6VGDS2GU77NnURUzxiT6fFIFGA9/3lk1nq+WR111vCWNcvz1R0XEVRGa0j5osTE0zz99G+89tpfVKxYlmnTrubGG1s7HZZSRSLXs5qIvGKMeRCYJSLnVK3THloL3z/7Yxj92WqiE6y2Zd8Y1oZG1UKoGhJA5WB9R8SXPfXUb7zyyp/cdltbXnyxN5UrawOlquTI6/L6K/t/7ZnVw9IzMnnwm3V8v9ZqeKJ306q8el0bQgP1mYAvO3AgjqSkNJo0CeORR7oxaFATunXTJqxUyZNXD60r7D+bGmPOSkb2i6rag2shWLw1mlumWS0a+ZUSfr63Ow2r6TMBX5aensmbb/7N+PGLad8+nN9/v5mwsCBNQqrEcueBwyjOvSu6NYdhqgA2Hozjri//Yd+JZAC6Nwzj45s74u+nbcL5sr/+imL06LmsW3eUK69syJQp/Z0OSSnH5fWM6DqsKtt1ReRbl1EhQGzOc6n8pGdk8vKCbbz3u9WkXveGYdzZsz4X19eacL5u3rztDBw4nfDwEL79diiDBjXRppiUIu87ohXACaxWs6e6DE8A1ngyKF9ljOGOz1bzy1ar/79593Sjebi+jOrLjDEcOpRAzZqh9O5dj2eeuZR77+1MSIhWPlHqjHzbmituvLGtOWMMD369LqthUoBNT19BuQCtiu3Ltm8/wZgx89i+/QSbN99FcHAZp0NSJZhXtjUnIr8bYy4RkRjO7tBOAGOMqeTx6HyAMYbRn69m/qajVC5Xhi71KzNpSCt9H8iHpaSk8+KLy5g4cRlly/ozcWIvypbV71up3OT16zjTHbg+vDhPmZmG+75ay/xNRwFY+XhvSmnDpD7tyJFEevT4mB07TjJ8eAteffUKqlcPdjospYq1vKpvn2lNoRZwyBhzWkS6Aa2Az4H4IojPa63Yc5Kh7/0JQGigP8seuUyTkA9LS8ugdGk/qlUrR48etZk6tT99+tR3OiylvII7dYW/w+omvD7wKVbDp196NCovFZ+SxvQV+2n8xE9ZSah1RHmWP9pLX071UZmZhnffXUX9+m8SFRWPiPDhh1dpElKqANwpuM40xqSJyDXA68aYN0VEa83ZPv9rH2/+sgMDHLOb5jljzt1daRVRwZnAlMetW3eEO+6Yy99/H+Syy+qSlpbhdEhKeSW3ugoXkWuBkcAge5he3gP93ljKlsNWCWW3BmEMaFWD2pWC+E/bCMoH6S7yVcYYHnpoIa+//heVKpXls8/+w4gRLfWdIKXOk7stK4zB6gZit4jUBaZ7NqziLSbpNA98vTYrCS24vweNtFmeEkNEiIk5xa23Wg2UVqyofUIpdSHceo9IRPyBM11D7jTGpOc1vSc5/R7R/hPJDH53OccSUqlVqSyfjepMnbByjsWjisa+fbHce+/PjB9/Ce3a1SAz02jlE+VVivN7RPlWVhCR7sBO4H/AR8B2Eenq6cCKo+iEFHq8vJhjCalc0bwaSx++TJOQj0tLy2DSpD9o1uxtFi7czbZtxwE0CSlViNwpmnsN6G+M2QwgIk2Bz4BimVk9JSPT8N9v1gNwb6+G3N+nkcMRKU9bvvwAd9wxl40bo7n66sa8+WY/IiO1SSalCps7iajMmSQEYIzZIiIlqq2SpNR0uk9azMmk00RWCuK+3g2dDkkVgUWLdhMXl8J3313H1Vc3cTocpXxWvs+IRGQakIp1FwQwAggyxtzk2dByVtTPiA6cTKb7pMUAhJcP5Nf/9iSwtF+RrV8VHWMMn322nipVgujXryGpqemkpWVqG3HKJ3j1MyJgNLALeBgYB+wG7vBkUMXFt/9EZSWhPs2qsfzRXpqEfNTWrce57LJPuemm7/j447UABAT4axJSqgjkWTQnIi2B+sBsY8ykogmpeEhJy+CBr9cB8Mq1rRncPsLhiJQnnDqVxgsvLOWll/6gXLkyvPfeAG67rZ3TYSlVouR6RyQij2E17zMCWCgio4osqmLg3hlW4xHjBzTTJOTDfvhhO889t5TrrmvB1q13cfvt7bVGnFJFLK87ohFAK2NMkohUAX7Eqr7t8xZuPsr8TUfxLyXcdHEdp8NRhezIkUTWrj1C374NuPbaZtSpcxudOtV0OiylSqy8nhGlGmOSAIwxx/KZ1mccjjvF/31qVYZY9MAl+OnVsc/IyMjk7bdX0rjxFEaOnM2pU2mIiCYhpRyW1x1RPRH51v5bgPounzHGXOPRyByQmJpOl4m/AvD8f1roy6o+5J9/DjN69FxWrjxE7971ePvt/pQtq+0BKlUc5JWIBmf7PMWTgTgtPSOTFhPmA1YNuRGdazsckSose/bE0KnTB4SFBfHll9cwbFgLbaBUqWIkr47xfinKQJw2drpVOSEsOID3R7Z3OBp1oYwxbNgQTatW1ahbtyIff3w1Awc2pkKFQKdDU0plUyKe++QnNvk0P208QkiAPyse66VXy15uz54YBgyYTtu277F+vdVN+8iRrTUJKVVMeTQRiUhfEdkmIjtF5JE8phsiIkZEivyt3/SMTNo8sxCA165ro1V3vdjp0xm8+OIymjd/m99/38vkyX1o1qyK02EppfLhTltzAIhIgDEmNf8ps6b3A6YCfYAoYKWIzHFtt86eLgS4B/jb3WUXpnVRsQB0rluJ3s2qORGCKgQZGZlcfPH/WL36MNdc05TXX7+CWrW0gVKlvIE73UB0EpENwA77c2sRecuNZXfC6rtotzHmNDADuDqH6Z4FJgEp7oddeOatPwLAY/2bOrF6dYHi461rIz+/Uowa1ZYffhjOrFlDNQkp5UXcKZp7ExgAnAAwxqwDLnVjvprAAZfPUfawLCLSFqhljJmb14JE5HYRWSUiq44dO+bGqt2z5XA8H/2xB4Bm4aGFtlzlecYYpk1bS716b/D991sBGDOmIwMGaPccSnkbdxJRKWPMvmzDMtyYL6eHLVlNfYtIKay+jh7Mb0HGmPeNMR2MMR2qVCm8Mv/J87cB8PUdXSjtp/U2vMXmzcfo2fMTbrnle5o0CaN+/UpOh6SUugDuPCM6ICKdAGM/9xkLbHdjviiglsvnCOCQy+cQoAXwm11LrTowR0SuMsZ4vJ+HPceT+GVrNB1qV6RTXT2ReYtJk/7g8cd/JTQ0gA8/HMgtt7TVCiZKeTl3EtGdWMVzkcBRYJE9LD8rgYYiUhc4CAwDrj8z0hgTB4Sd+SwivwH/LYoktDM6gd6vLgFgUFtt3sUbGGMQEapXD2bEiJa8/HIfqlTRli+U8gX5JiJjTDRWEikQY0y6iNwNzAf8gI+MMZtE5BlglTFmToGjLSRP/2BV3JswsBk3XKQtKBRnhw4lcO+9P9O9eyT33NOZG29szY03tnY6LKVUIco3EYnIB7g82znDGHN7fvMaY37EarXbddj4XKbtmd/yCkNCShpLdxwnLLgMt3StWxSrVOfhTAOljz/+K2lpmVx8sXbFoZSvcqdobpHL34HAfzi7NpxX+XpVFACjL6nvcCQqN2vXHuG22+awevVhLr+8Pm+/3V8rJCjlw9wpmvvK9bOIfAYs9FhEHnQ6PZNn51rFckM71spnauWUuLgUDh1K4KuvhnDttc20ySWlfJzbLSu4qAt45YOVyQus6tr39W5IaKB2AVBcGGP45pvN7Nhxgscf78Ell9Rh9+57CQw8n8NTKeVt3GlZIUZETtr/YrHuhh7zfGiFKz4ljfeX7KZ5eCj39mrodDjKtmvXSfr3/5LrrpvJ999vIy3NekVNk5BSJUeev3axykRaY1W/Bsg0xpxTccEbvPf7LgCGdaylRT3FQGpqOpMnL+e555ZSunQp3nijL2PGdMTfX18sVqqkyTMRGWOMiMw2xnh9Bz1z1h0iwL8U13WMdDoUBRw4EM+zzy5h4MDGvP76FdSsqU0sKVVSuXP5uUJE2nk8Eg/aezyJAydPMaZnA8roFbdjjh1LYsqUFQA0aFCJzZvv4ptvrtUkpFQJl+sdkYj4G2PSgW7A/4nILiAJqw05Y4zxmuR09/R/AGitLTI7IjPT8PHHa3j44UUkJKTSp089GjcOo169ik6HppQqBvIqmlsBtAMGFVEsHpGZaTgYc4qypf3o2biq0+GUOBs3RnPnnfNYtmw/3btH8u67A2jcOCz/GZVSJUZeiUgAjDG7iigWj3jr153EJKfx9FXNnQ6lxDl9OoPLL/+M06cz+Oijq7j55jZaUUQpdY68ElEVEXkgt5HGmFc9EE+hSsvI5LVF26kYVJobu3jlq09e6ddf93DJJbUpU8aPr7++liZNwggLC3I6LKVUMZXXk3s/IBiru4ac/hV7248mAHBtB62yXRSiouIZPPhrevX6lE8/XQdAt26RmoSUUnnK647osDHmmSKLxANe/MnquXNAqxoOR+Lb0tMzmTJlBU8+uZiMjEwmTuzFiBGtnA5LKeUl8n1G5K3SMjJZuuM4IQH+tIqo4HQ4Pm3kyNnMmLGRfv0aMHVqf+rW1dpwSin35ZWIehVZFB7w3RqrMYjRPbWVbU+IjU3B378UwcFluOuujgwe3JTBg5tqEahSqsByfUZkjDlZlIEUtn/2xwJwnbayXaiMMcyYsZGmTafy5JO/AtZzoCFDtJVspdT58dlmBvadSAIgLDjA4Uh8x86dJ7niis8ZPnwWERGh3HCDPgdSSl04n23iePmuE7SsqS0pFJYvv9zAqFHfExDgz5Qp/Rg9ugN+fj57HaOUKkI+mYiW7zwOQNMaXlHLvFhLS8ugdGk/OnQIZ8iQZkya1IfwcN2vSqnC45OJ6KeNRwB48PLGDkfivaKjk3jwwQUkJZ3m22+vo1Gjynz++TVOh6WU8kE+Wbbyz/4YAKqFBjociffJzDS8//5qGjeewldfbaR58ypkZGQ6HZZSyof55B3R0fgUIiqWdToMr7N7dww33PAtf/4ZRc+edXjnnStp0kQbKFVKeZbPJaKVe09yPPE0wztpte2CKl8+gNjYFD75ZBAjR7bS6thKqSLhc0Vzs1ZHAXBly3CHI/EOc+Zs45prviIjI5PKlYPYuHEMN97YWpOQUqrI+FwimrHyAAAX1avkcCTF2/79cQwaNIOrr57B9u0nOHw4EYBSpTQBKaWKlk8VzZ1MOg1Ah9oV8dd3XHKUnp7J66//xYQJv2GM4aWXenP//RdRurSf06EppUoon0pEv22LBrRZn7xkZGTy4Yf/cNlldXnrrX7UqaMNwiqlnOVTtw17j1vN+lzSuIrDkRQvMTGnGDduIQkJqQQE+PPHH6OYM2eYJiGlVLHgU4lo9/EkwoLLUDVE3x8Cq4HSL75YT5MmU3nllT9ZvHgvAJUrB2llBKVUseFTRXNLth+jcXVtfgZg+/YTjBkzj19+2UOnTjWZP/8G2rSp7nRYSil1Dp9JRAdOJhOfkk7zcG3oFOC++35m1apDvP12f26/vb02UKqUKrZ8JhEtsxs67dui5F71L1y4iyZNwqhVqzzvvHMlAQH+VK8e7HRYSimVJ49eJotIXxHZJiI7ReSRHMY/ICKbRWS9iPwiIrXPd13Ld50AoFl46AVE7J2OHEnk+utncfnln/PSS38AULt2BU1CSimv4LFEJCJ+wFSgH9AMGC4izbJNtgboYIxpBcwEJp3v+o4npAIQGlj6fBfhdTIzDe++u4omTaYwa9YWJky4hMmTL3c6LKWUKhBP3hF1AnYaY3YbY04DM4CrXScwxiw2xiTbH/8CIs53ZWsOxNCrSdXzDtYbTZy4lDvvnEf79uGsXz+ap57qSWCgz5S2KqVKCE+etWoCB1w+RwGd85j+VuCnnEaIyO3A7QCRkZHnjF+9L4aUtEwuqlf5vIP1FgkJqRw/nkzduhUZPboDdetWZPjwFlodWynltTx5R5TTmdHkOKHIDUAH4OWcxhtj3jfGdDDGdKhS5dyXVXcfs9pJ6+zD7csZY5g9ewvNmr3NddfNxBhD5cpBXH99S01CSimv5slEFAW4trUTARzKPpGI9AYeB64yxqSez4pS062O26r7aEd4+/bFctVVM7jmmq+pVKksb77ZT5OPUspneLJobiXQUETqAgeBYcD1rhOISFvgPaCvMSb6fFe0aMtRAALL+F7DnX/+eYDevT8DYPLkPtx770X4++s7QUop3+GxRGSMSReRu4H5gB/wkTFmk4g8A6wyxszBKooLBr6xr/D3G2OuKui6ytotR/tSjbn4+FRCQwNo164Go0a14aGHuhIZqS/rKqV8j0erWBljfgR+zDZsvMvfvQtjPccTU2lZ0zdO0idOJPPII4tYsGA3mzaNITi4DG+91d/psJRSymO8vozHGMPKvTEEeHlxlTGGTz9dR5MmU/n447Vcd11z9DGQUqok8PqXTs60qBBZOcjhSM5fXFwKgwZ9xW+/7aVLlwjefXcArVpVczospZQqEl6fiD79cy8AT13V3NE4zocxBhEhNDSAsLAg3n9/ALfe2k6761ZKlSjeXZ4FxJ9KJyTQ3+sqKsyfv5N27d4nKioeEeGbb67l//6vvSYhpVSJ4/WJaP/JZGp7UbHc4cMJDBs2k759vyA5OY3o6CSnQ1JKKUd5ddGcMYaj8Sk0reEdLW5PnbqCxx77ldTUdJ5+uifjxnUlIMCrvwKllLpgXn0W3Hw4nvRMQ6sI76i6vXr1YTp3rsnUqf1p2ND328VTSil3eHUi+mWL1RhD57rFs425+PhUxo9fzMiRrWjfPpy3376SgAA/bZ5HKaVceHUi2hFtNXbaoU7xSkTGGGbN2sK99/7M4cMJREaWp337cO2iQSmlcuDVZ8Y9xxOpEFQav2JU02zPnhjuvvsnfvxxB23aVOfbb4fSufN5d7OklFI+z2sTUXpGJjuOJnJNu+J1kv/iiw0sWbKP1167grvv7qQNlCqlVD68NhElpKSTmp5J/SrlnA6FpUv3kZqaQe/e9XjooYu5+eY2RER4R00+pZRymtderqekZwBQzsHqz8ePJzNq1Pf06DGNZ575HYCAAH9NQkopVQBee0d0NN7qQ8+Jp0PGGKZNW8tDDy0kLi6VceO68uSTPRyIRBV3aWlpREVFkZKS4nQoqoQIDAwkIiKC0qW9p7UZr01E248mAM40dvrjjzsYNWoOXbvW4t13B9CiRdUij0F5h6ioKEJCQqhTp45W21ceZ4zhxIkTREVFUbduXafDcZvXFs1tORwPQP0qwUWyvuTkNP74Yz8A/fs35Pvvh7FkyS2ahFSeUlJSqFy5siYhVSREhMqVK3vdHbjXJqL1UXEAVC5XxuPr+umnHbRo8Tb9+n1BbGwKIsJVVzXWBkqVWzQJqaLkjceb1yYi/1JC2dJ++Pt5bhMOHozn2mu/oX//LwkI8OeHH4ZToUKgx9anlFIlkdcmovRMQ7vaFTy2/OjoJJo1e5u5c7fz3HOXsm7daC65pI7H1qeUp/j5+dGmTRtatGjBwIEDiY2NzRq3adMmLrvsMho1akTDhg159tlnMcZkjf/pp5/o0KEDTZs2pUmTJvz3v/91YhPytGbNGm677bazhl199dV06dLlrGE333wzM2fOPGtYcPC/Rfvbt2+nf//+NGjQgKZNmzJ06FCOHj16QbGdPHmSPn360LBhQ/r06UNMTMw50yxevJg2bdpk/QsMDOS7774DYMSIETRu3JgWLVowatQo0tLSAJg7dy4TJky4oNiKFWOMV/1r3769McaY2uPmmhs+/MsUtqiouKy/33jjL7Nz54lCX4cqOTZv3ux0CKZcuXJZf994443mueeeM8YYk5ycbOrVq2fmz59vjDEmKSnJ9O3b10yZMsUYY8yGDRtMvXr1zJYtW4wxxqSlpZmpU6cWamxpaWkXvIwhQ4aYtWvXZn2OiYkxERERpkmTJmb37t1Zw2+66SbzzTffnDXvmX1z6tQp06BBAzNnzpyscb/++qvZsGHDBcX20EMPmYkTJxpjjJk4caJ5+OGH85z+xIkTpmLFiiYpKckYY8y8efNMZmamyczMNMOGDTNvv/22McaYzMxM06ZNm6zpssvpuANWmWJwDs/pn1fWmku13yFKSk0vtGXGxaXwxBO/8t57q/nrr9to164G99zTgp2FggAAEK1JREFUudCWr9TTP2xi86H4Ql1ms/BQJgx0v3fiLl26sH79egC+/PJLunbtyuWXXw5AUFAQU6ZMoWfPntx1111MmjSJxx9/nCZNmgDg7+/PmDFjzllmYmIiY8eOZdWqVYgIEyZMYPDgwQQHB5OYaLUHOXPmTObOncu0adO4+eabqVSpEmvWrKFNmzbMnj2btWvXUqGCVcLRoEED/vjjD0qVKsXo0aPZv9+qJPT666/TtWvXs9adkJDA+vXrad26ddawWbNmMXDgQKpVq8aMGTN49NFH890vX375JV26dGHgwIFZwy699FK392tuvv/+e3777TcAbrrpJnr27MlLL72U6/QzZ86kX79+BAVZtYH79++fNa5Tp05ERUUB1nOgnj17MnfuXIYOHXrBcTrNKxPRicTTAPRqWu2Cl2WM4ZtvNnPffT9z5Egid9/difr1K17wcpUqbjIyMvjll1+49dZbAatYrn379mdNU79+fRITE4mPj2fjxo08+OCD+S732WefpXz58mzYsAEgx+Kn7LZv386iRYvw8/MjMzOT2bNnc8stt/D3339Tp04dqlWrxvXXX8/9999Pt27d2L9/P1dccQVbtmw5azmrVq2iRYsWZw2bPn06EyZMoFq1agwZMsStRLRx48Zz9kVOEhIS6N69e47jvvzyS5o1a3bWsKNHj1KjRg0AatSoQXR0dJ7LnzFjBg/8f3t3HxxVfS5w/PvIWwIEqrxdKSp0pJgXQ4TQggUUUbTKhcKoMbxUHK0DiNSiONzBmStoLW0FNIIFLtebVNsQdCowoFDgxoLIi+EaCQJNU7ryYm5MMRdTCiaQ5/5xTjabsCEbZffsJs9nZmd2z8ueZ3+z2Se/3znn+c2Zc9Hy6upqXn/9dV5++WX/svT0dHbu3GmJyCu7Sv4O8I3nIVJVJk5cy7p1Rxg06Go2bMgkPb335QjRmIs0p+dyOZ09e5a0tDR8Ph+DBw/mjjvuAJzvf2NXWDXnyqtt27axZs0a/+srr2z6H7n77ruPNm3aAJCRkcHChQt56KGHWLNmDRkZGf73PXTokH+fL7/8ksrKShISEvzLSktL6dGjh/91WVkZJSUlDB8+HBGhbdu2HDx4kJSUlKCfqblXmCUkJFBYWNisfUJVWlpKUVERd95550XrZs6cyciRI+slwZ49e/LZZ5+FJZZIi8mLFWpvZh3wLwlNbBlcdbUztCciDB9+DVlZd7Fv3yOWhEyLFB8fT2FhIZ9++ilVVVUsX74cgOTkZAoKCupte/ToUTp37kxCQgLJycns37+/yfdvLKEFLmt4X0unTnU1IocNG0ZJSQnl5eWsW7eOiRMnAlBTU8Pu3bspLCyksLCQkydP1ktCtZ8t8L3z8vKoqKigX79+9O3bF5/P50+S3bp1q9db++KLL+jevbu/LUL5rJWVlfUuLAh8BCbNWr169aK0tBRwEk3Pno3fd7h27VomTJhwUUWEBQsWUF5ezpIlS+otP3fuHPHx8U3GHAtiMhHV3kPUo3OHZu/73ns+UlNXsH79EQCefPJmHn/8+7QJ42XgxkSDrl27kpWVxYsvvkh1dTWTJ0/m/fffZ9u2bYDTc5o9ezZPP/00AHPnzuWFF16guLgYcBJDwx9DgDFjxrBs2TL/69of+169enH48GH/0FtjRIQJEyYwZ84cEhMT6datW9D3DdYTSUxMpKSkxP86NzeXzZs34/P58Pl87N+/35+Ibr31VvLy8qiqcob2s7Oz/eeBJk2axAcffMCmTZv877V582b/cGOt2h5RsEfDYTmAcePGkZOTA0BOTg7jx49vtB1yc3PJzMyst2z16tVs2bKF3Nxcrrii/m9UcXHxRcOSsSomf32LTp7m6q5xzepWl5ef4cEH1zFqVA5ffXWehITmJzFjYt1NN93EwIEDWbNmDfHx8axfv57nn3+eAQMGcOONNzJkyBBmzZoFQGpqKi+99BKZmZkkJiaSkpLi/+8+0DPPPENFRQUpKSkMHDiQ/Px8ABYtWsTYsWO57bbb/OdJGpORkcEbb7zhH5YDyMrKoqCggNTUVJKSklixYsVF+91www2cPn2ayspKfD4fx44dY+jQof71/fr1o0uXLuzdu5exY8cyYsQIBg8eTFpaGrt27fJfOBAfH8/GjRt55ZVX6N+/P0lJSWRnZ1+yBxOKefPmsXXrVvr378/WrVuZN28e4JzbCrzk3Ofzcfz4cW655ZZ6+0+fPp2ysjKGDRtGWloaCxcu9K/Lz8/nnnvu+UbxRQvRgHsGYsHg9HQ9dfsC7kzuxcqp6SHtk5tbxGOPvcM//lHF3Lk3M3/+SDp2jJ2CgCZ2HT58mMTERK/DaNGWLl1KQkLCRfcStWRlZWVMmjSJ7du3B10f7HsnIvtVNbQfzQiLuR5RTY2TOAf0Cv380PnzNaSk9KSwcDo///loS0LGtCAzZsygQ4fWNcJx7NgxFi9e7HUYl03MXTV33k1Evb/V+Em6M2eqeO65HVx7bVdmzhzClCmpTJmSGpM1mIwxlxYXF8fUqVO9DiOihgwZ4nUIl1XM9YguuIkorl2boOs3biwmOflVfvnLXRQXnwKck6GWhIxXYm3428S2WPy+xVyP6PTZatoBfa6s3yM6ceJLZs9+l7ffPkJSUg927JjGiBHXeROkMa64uDhOnTplU0GYiFB3PqK4uNgqzhxziaj6Qg3tgNQ+9QueHj1awZYtf+UXvxjNnDnDaN8+eI/JmEjq06cPJ06coLy83OtQTCtRO0NrLIm5RHS26gKj+l1F+7ZXsG/fSXbvPs5PfzqUkSOv49ixJ+jmwYytxjSmXbt2MTVTpjFeCOs5IhG5S0T+LCIlIjIvyPoOIpLnrt8rIn2bes+qCzVc360TM2duYujQ1SxZsoczZ5wb1CwJGWNM7AlbIhKRNsBy4IdAEpApIg1vPX4YqFDV64GlQONlaQP819K9rFy5n9mzv09R0Qw6RWCWVmOMMeERzqG57wElqnoUQETWAOOBwIJM44Fn3edvActERLSJyz56V8HGD3/CoEGXvlvbGGNM9AtnIvo2cDzg9Qmg4QQ//m1U9byInAa6AX8P3EhEHgUedV9+tf9/Hz0YQsX21qA7DdqqFbO2qGNtUcfaos4ArwNoTDgTUbBrVRv2dELZBlVdBawCEJGCaC1TEWnWFnWsLepYW9SxtqgjIgVNb+WNcF6scAK4JuB1H6Dh5Bn+bUSkLdAV+CKMMRljjIky4UxEHwL9RaSfiLQHHgA2NNhmA/Cg+/xe4L+bOj9kjDGmZQnb0Jx7zmcWsAVoA7ymqp+IyEKgQFU3AP8JvC4iJTg9oQdCeOtV4Yo5Bllb1LG2qGNtUcfaok7UtkXMTQNhjDGmZYm5oqfGGGNaFktExhhjPBW1iSgc5YFiVQhtMUdEDonIARHZLiIttux4U20RsN29IqIi0mIv3Q2lLUTkfve78YmI/D7SMUZKCH8j14pIvoh85P6d3O1FnOEmIq+JyOcicrCR9SIiWW47HRCRQZGOMShVjboHzsUNfwW+A7QHPgaSGmwzE1jhPn8AyPM6bg/bYhTQ0X0+ozW3hbtdArAD2AOkex23h9+L/sBHwJXu655ex+1hW6wCZrjPkwCf13GHqS1GAoOAg42svxt4F+cezqHAXq9jVtWo7RH5ywOpahVQWx4o0Hggx33+FjBaWuaEL022harmq+o/3Zd7cO7ZaolC+V4APAf8CjgXyeAiLJS2+AmwXFUrAFT18wjHGCmhtIUCXdznXbn4nsYWQVV3cOl7MccDv1XHHuBbIuJ5rbRoTUTBygN9u7FtVPU8UFseqKUJpS0CPYzzH09L1GRbiMhNwDWqujGSgXkglO/Fd4HvisguEdkjIndFLLrICqUtngWmiMgJ4B3g8ciEFnWa+3sSEdE6H9FlKw/UAoT8OUVkCpAO3BLWiLxzybYQkStwqrhPi1RAHgrle9EWZ3juVpxe8k4RSVHV/wtzbJEWSltkAtmqulhEhuHcv5iiqjXhDy+qROXvZrT2iKw8UJ1Q2gIRuR2YD4xT1a8iFFukNdUWCUAK8J6I+HDGwDe00AsWQv0bWa+q1ar6N+DPOImppQmlLR4G1gKo6m4gDqcgamsT0u9JpEVrIrLyQHWabAt3OGolThJqqecBoIm2UNXTqtpdVfuqal+c82XjVDVqiz1+A6H8jazDuZAFEemOM1R3NKJRRkYobXEMGA0gIok4iag1zt++Afixe/XcUOC0qpZ6HVRUDs1p+MoDxZwQ2+LXQGfgTfd6jWOqOs6zoMMkxLZoFUJsiy3AGBE5BFwA5qrqKe+iDo8Q2+JJ4D9E5Gc4Q1HTWuI/riKSizMU2909H/bvQDsAVV2Bc37sbqAE+CfwkDeR1mclfowxxngqWofmjDHGtBKWiIwxxnjKEpExxhhPWSIyxhjjKUtExhhjPGWJyEQdEbkgIoUBj76X2LZvY5WGm3nM99zqzR+7JXEGfI33mC4iP3afTxOR3gHrVotI0mWO80MRSQthnydEpOM3PbYx4WKJyESjs6qaFvDwRei4k1V1IE4x3V83d2dVXaGqv3VfTgN6B6x7RFUPXZYo6+J8ldDifAKwRGSiliUiExPcns9OEfkf93FzkG2SRWSf24s6ICL93eVTApavFJE2TRxuB3C9u+9odw6bIneulw7u8kVSNwfUi+6yZ0XkKRG5F6fm3+/cY8a7PZl0EZkhIr8KiHmaiLzyNePcTUDBShH5jYgUiDP30AJ32WychJgvIvnusjEistttxzdFpHMTxzEmrCwRmWgUHzAs97a77HPgDlUdBGQAWUH2mw68rKppOInghFvOJQP4gbv8AjC5ieP/K1AkInFANpChqjfiVCKZISJXAROAZFVNBZ4P3FlV3wIKcHouaap6NmD1W8DEgNcZQN7XjPMunDI+tearajqQCtwiIqmqmoVTS2yUqo5yS/08A9zutmUBMKeJ4xgTVlFZ4se0emfdH+NA7YBl7jmRCzh10xraDcwXkT7AH1T1LyIyGhgMfOiWP4rHSWrB/E5EzgI+nGkCBgB/U9Vid30O8BiwDGeuo9UisgkIecoJVS0XkaNuna+/uMfY5b5vc+LshFPOJnCGzftF5FGcv+urcSaAO9Bg36Hu8l3ucdrjtJsxnrFEZGLFz4AyYCBOT/6iSe9U9fcishe4B9giIo/glL3PUdV/C+EYkwMLpIpI0Pmt3Npm38MpovkAMAu4rRmfJQ+4HzgCvK2qKk5WCDlOnFlIFwHLgYki0g94ChiiqhUiko1T2LMhAbaqamYz4jUmrGxozsSKrkCpO3/MVJzeQD0i8h3gqDsctQFniGo7cK+I9HS3uUpErgvxmEeAviJyvft6KvAn95xKV1V9B+dCgGBXrlXiTEsRzB+AH+HMkZPnLmtWnKpajTPENtQd1usCnAFOi0gv4IeNxLIH+EHtZxKRjiISrHdpTMRYIjKx4lXgQRHZgzMsdybINhnAQREpBG7AmRL5EM4P9h9F5ACwFWfYqkmqeg6nOvGbIlIE1AArcH7UN7rv9yec3lpD2cCK2osVGrxvBXAIuE5V97nLmh2ne+5pMfCUqn4MfAR8AryGM9xXaxXwrojkq2o5zhV9ue5x9uC0lTGeserbxhhjPGU9ImOMMZ6yRGSMMcZTloiMMcZ4yhKRMcYYT1kiMsYY4ylLRMYYYzxlicgYY4yn/h+f8Vryc7oj6AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7220704605012441" + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#plot roc for svm\n", + "get_roc(clf_original, y_test, X_test, \"SVM original data RBF kernel\")" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM original data RBF kernel identified 733\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06391338
11287733
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6391 338\n", + "1 1287 733" + ] + }, + "execution_count": 78, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_original.predict(X_test), \"SVM original data RBF kernel\")" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.36 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.66 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_original.predict(X_test)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Evidently, SVM model fit with no tuning or feature reduction with RBF kernal shows low performance. Now, we will fit SVM model with reduced standardized features and access accuracy" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "note that the default values of gamma = 1/13 and c= 1" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=1, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma=0.07692307692307693,\n", + " kernel='rbf', max_iter=-1, probability=True, random_state=None,\n", + " shrinking=True, tol=0.001, verbose=False)" + ] + }, + "execution_count": 80, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#train svm model with feature reduction and no parameter tuning\n", + "clf_reduced = svm.SVC(kernel = 'rbf', probability = True, gamma = 1/13, C = 1)\n", + "clf_reduced.fit(X_train_pca, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.16374485135727915\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAEWCAYAAACUr7U+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdZ3QV1deA8WeTDiShQyAJIQiGXgwdxQKKCIooioqKyquoiIq9K4pdLH9RsFfAhoKIIqA0pUqTLp3QW0II6Tnvh5mEm5AKSSb33v1bi0Xu1D11zzlzZkaMMSillFLKM1RyOgCllFJKlR5N7EoppZQH0cSulFJKeRBN7EoppZQH0cSulFJKeRBN7EoppZQHqZCJXURuEJHfnY7DaSISKSLHRcSnHOcZJSJGRHzLa55lSUTWisj5pzHeae2DIhIkIj+LSIKIfFfS8T2JiMwRkaHlPM/PROSFQvq/ICKHRGRfecblrUTkcRH5yOk4zpSIbBeRnhUgjiEisqCo4YpM7PYCJdsJZp994FQtnTDzZ4z52hhzcVnOoyLKu/MYY3YaY6oaYzKdjMsp9gXGWWcyDWNMC2PMnCLmc8rFzBnsg1cDdYGaxpiBpzF+fvE9LiLb7GMwTkS+sbuPF5Ev8hm+tYikikgNEXnWXrYReYa5z+7+bGnE6A5EJAJ4AGhujKl3htM6X0TiSieyisPeX74qrekZY140xpTJxZ29/ybZx8VuERnjWgiyLyxT7P4JIjJPRFq59H9WRNLt/tn/Hi6LWMtbcUvs/YwxVYG2QDvgsbILqew4WQr1lBJwSXjp+m4IbDLGZJR0xPxiFpGbgRuBnvYxGAvMtnt/BgwQkSp5RrsJmGaMOWL/3gTcnM8wm84kNjfUEDhsjDngdCAesj4rgjb2cdEDuBa4NU//4Xb/msAc4Ms8/b+xC0/Z/14ty2DLa7uXqCreGLMPmIGV4AEQkQAReV1EdorIfhEZJyJBLv2vEJGVInJMRLaISG+7e6iIfCwie+2rrReyr7Zcqxvs6b3uGoeITBGRkfbf9UXkBxE5aJdqRrgM96yIfC8iX4nIMWBI3mWy4/jCHn+HiDwpIpVc4vhLRP5nX/FtEJGL8oxb2DL8JSJvisgR4FkRaSwif4jIYbs68GsRqWYP/yUQCfycfeWYtyRpX4E+b083UUR+F5FaLvHcZC/DYRF5SgqpPhKryvgNe/gEEVngut2AG+xtekhEnnAZr6OILBSReHu53xURf5f+RkTuFpH/gP/sbm+LyC57H/hHRM51Gd5HrBLpFnuZ/hGRCBGZZw+yyl4f19rD97X3p3gR+VtEWrtMa7uIPCIiq4EkEfF1XQd27MvsOPaLyBh71Ox5xdvz6iJ5qrxEpIWIzBSRI/a4j+ezTp8Dngautadzm4hUsvepHSJywN7XQu3hs7fvbSKyE/gjn03VAZhhjNkC1jFojPnA/nshsBu4ynV9AtcDn7tMYylQWURaZC8LEGR3z1d++6/d/VYRWS8iR0Vkhog0dBmnl32MJIjIu4C49MtVEsxn364hIp+KyB572j+5DFvYNm8nIsvtfecbILCA5ekJzATq29vmM7t7Z3ua8SKySlxu24jILfayJorIVhG5w+5eBfjVZVrHxToP5boNIHlK9QXsn4WdvwraX/Mu2/li1eQ8YO9je0XkFpf+BZ7j8kynN/A4J/ffVS5x93QZLmdbumzHmyX/80VJhg0Skc/t7b9erHNgsWpFjDGbgb9wyU15+mcAk4DmxZleYUQkxt5Wg+zfJcpBdrdv7W2SKNbtwliXcR6Vk+fDdSJyZYmDNMYU+g/YjlVaAAgH/gXedun/FjAVqAEEAz8DL9n9OgIJQC+si4gGQIzd7ydgPFAFqAMsAe6w+w0BFth/nwfsAsT+XR1IBurb0/wH62TqD0QDW4FL7GGfBdKB/vawQfks3xfAFDv2KKxSzG0ucWQA9wN+WFeECUCNYi5DBnAP4It1Ij3LXhcBQG2shPJWfuva/h0FGMDX/j0H2AI0tac3B3jZ7tccOA50t9fF6/ay9yxgu461x28A+ABd7biy5/mhPY82QCrQzB7vHKCzvUxRwHrgPpfpGqwTaI3s9Q0Mxrpi9sWqCt0HBNr9HsLap87GSgRtsKqxs6d1lsu02wMHgE52zDfb6yzAZf2tBCJc5p2zToGFwI3231WBzvmt53z2wWBgrx17oP27UwHr9VngK5fftwKbsfbNqsBk4Ms88/0Cax/Kb/8cDByx11Ms4JOn/xPALJfflwAHAT/XeLBO2K/Y3V7FqnX7Cni2gOUYwqn7b397WZrZ3Z4E/raHrwUcw7oV4Yd1zGQAQwtYL7nWOfAL8A3W8e0H9Chqm2Pt5zs4eXxejbXPv1DAMp0PxLn8bgAcBvpgnR962b9r2/0vAxpj7Zc9gBNA+/ymZXf7zHXe+cxvOy77J0Wfv/LdXwtYrgxglL0e+tixVi/qHFfU/lvAeSlnGIo+X5Rk2JeBufY+EA6szruO88SVc34AYrCO0ftd+s/h5P7nD4wG5hW2rIXMazvQE2t/3An0tbuXOAfZ3VLs7eQDvAQscpnXQE7mt2uBJCAs73mp0HiLuUDHgUR7Rc4Gqtn9xJ5pY5fhuwDb7L/HA2/mM8269gYNcul2HfBnPidVsVfkefbv/wP+sP/uBOzMM+3HgE9dVuq8QpbNx46juUu3O4A5LnHswb6osLstwaoaLc4y7Cxo3vYw/YEVhRxAUZya2J906X8X8Jv999PARJd+lYE08kns9g6TjFWNlbdf9jzD8yzzoAKW4T7gxzwH24VFLPfR7HkDG4Erijpw7d/vA8/nGWYjJ5PAduDW/A5I++95wHNArQKWuaDEfp3rdipi2Z4ldwKbDdzl8vtsrAPd12W+0UVM8wZgFtaxdhh41KVfpD29cPv31+S+8H4WK4FHYh1Hfvb/ERSd2PMeW7/ikhDs/egEVhX3TeQ+OQkQRzESOxAGZGEnojzzLHCbY1305z0+/6b4if0R7Issl24zgJsLGP8n4N78pmV3+4yiE/utLr+LOn/lu78WsFzJ5N5/D2BdgBd6jitq/817DOUdhiLOFyUcNich2r+H5l3HeeIyWBeTSfbfE7Ev8u3+c7D2z3isc2ECcFGe5Uiz+2f/q1/AvLbb2yIOuKAE2/BZ8uQgu5vrxXhzILmQ5VyJfY6kmIm9uFXx/Y0xwVg7UAzW1TlYpc7KwD92VVY88JvdHayTx5Z8ptcQ6wSz12W88Vil3lyMtTSTsE6uYFUzfu0ynfrZ07Cn8zhW0s22q5DlqsXJq/5sO7Cu5LPttmNw7V+/mMuQa94iUkdEJolVbX8M68Rai5Jxbc17AutKHjumnPkZY05gJYH81MIqeea3bQqdj4g0FZFpYjWkPAa8yKnLkHe5H7Cr1hLs9RTqMk5B+0h+GgIP5NneEVjLnu+887gNq7Zjg4gsFZG+xZxvSWLMqz6n7l++FH8fxVgN+XoC1YBhwCgRucTutxMrAQwWq1Frf3JXw+My3Gas7fWfMabQeRYQV0PgbZd1fwQrgTfg1P3PFLVcLiKAI8aYo/n0K2yb1yf/47O4GgID80y7O9aFBiJyqYgsEuv2SzxWCaukx2teruukqPNXSfbXwyZ3u47sY7Y457jSUNB5qSTD5tqHKN7+094e/1qsJJu3vckIY0w1rPNdX+B7cbmVA3xrjKnm8m9PIfMahlVD9adLt9PNQXnXQaCcvC11k5y89RQPtKSE+11J77HPxboqzb7nfQjrSrGFy4oJNVZjBbAWqHE+k9qFdRVZy2W8EGNMiwJmPRG4Wqz7eZ2AH1ymsy3Phgk2xvRxDbuQRTqEVdpp6NItEuu+ZbYGIiJ5+u8p5jLknfdLdrfWxpgQrGpWKWT4ktiLVX0FWPersKq/83MIqyoov21TlPeBDUATexkeJ/cygMtyiHU//RHgGqwSWTWsK+fscQraR/KzCxidZ3tXNsZMzG/eeRlj/jPGXId18fUK1kFepbBxTiPGvPZw6v6VAex3Da04EzLGpBtjvsOqomzp0utzrBLzVVjHw/ICJvEF1u2EU1rSFzTLPL93Yd1qcl3/QcaYv7H2v4jsAe1jJsJl3CSsQkA211bpu4AaYrc3yWeeBW3zveR/fBbXLqwSu+u0qxhjXhaRAKzzzOtAXXu/nc7J/Ta/bVbYMmZzHa/Q81ch+2tJFOccV1B82YqzXKUh1zmM3PtPgYzlW6xbF08XMEyWMWY+1sXt6T5xNQyIFJE3XbqdaQ7Kxc5xHwLDsW5JVgPWcOo5tlCn8xz7W0AvEWlrjMmyg3hTROrYgTXILk0AHwO3iMhFYjUiaiAiMcaYvcDvwBsiEmL3aywiPfKboTFmBdZ9w4+wGhLF272WAMfEapASJFZDrJYi0qE4C2Ksx8i+BUaLSLC9UkdilaSz1QFGiIifiAzEur84vaTLYAvGuq0RLyINsO6butqPdY/mdHwP9BORrmI1ZnuOAnYGe7t9AoyxG374iNVgLKAY8wnGqv46LiIxwJ3FGD4Da/v5isjTQIhL/4+A50WkiVhai0j2BUne9fEhMExEOtnDVhGRy0QkuBhxIyKDRaS2vfzZ+1CmHVsWBa/7aUA9sR4RC7D3lU7FmSfWRen9ItLILlG/iNUSt1it5sVqxHaZPc9KInIp0AJY7DLYD1gnwefIp7Tu4husk9q3xYw9r3HAY3KyEV6ofUyAdY+8hYgMsEseI8idAFYC54n1boZQXJ6ssY+lX4H3RKS6faydZ/cubJsvxNq3RojVEG0AVrue4voK65i5xD4GAsVqiBaOVcoNwNo3Muz17poQ9gM17WVxXcY+YjUErId1m6owhZ6/Ctlfi62Y5zhX+4Eoyd24biUwyN4usVhtGcrCt1j7V3X7/Di8hOO/DNxur/tTiEgXrGrvtacZXyLQG2s/ftnudkY5KB/ZBY2Ddsy3kPsivlhKnNiNMQexrvifsjs9gnUVtEisqtlZWPcRMcYsAW4B3sQqpc3l5JXjTVgHzzqse67fY1eBFWAiVuOFCS6xZAL9sFpCbsO6Ov0Iq6q3uO7BuiLdCiywp/+JS//FQBN72qOBq40x2VXcJV2G57CqjhKwToST8/R/CXhSrCqYB0uwDBhj1trLMgnryjcR6z5bagGjPIjVaG0pVpXqKxRvf3gQ63ZIItZJ95sihp+BddLehFUFmELuqqkxWAf071gXDB9jNTAB617U5/b6uMYYswyrjcW7WOt7M/k86VCI3sBaETkOvI11by/Fvm0xGvjLnldn15GMMYlYDav6YVWh/QdcUMx5foL1iM08rH00BWs7FdcxrFqRnVgn91eBO40xOS32jTFJnEzuX+c3EXu4ZGPMLGNMcgnm7zr+j1j7yST7WF8DXGr3O4TV6OdlrFtATbBaKWePOxNrX1mN1dhoWp7J34hVstyAtd/eZ49X4DY3xqQBA+zfR7GqY/MeU4Utzy7gCqz1exBrv3wIqGRv8xFY++ZRrH1+qsu4G7DOSVvtfaY+1nZehXU/9neKODaKcf7Kd38t7vK5KOoc5yr7pUqHRSS75ucprBqro1jnsAn5jVgKRmHdw96GlUe+p+Dz1ymMMf9i5RjXAtO7Yj+5gLV9njTG/Hq6AdqFyl7ApSLyfCnlINfprwPewLpo3Q+0wuU4Kq7sluYqHyIyBKvxT3enYykpu3QYj1Vlvs3peJRSqiRE5E6si5nCakFVPirkK2XV6RGRfiJSWaz7cK9jlci3OxuVUkoVTUTCRKSbfcvpbKz2ID86HZc70sTuWa7Aaqy1B6sqdJDRKhmllHvwx3qyKBHrZU1TgPccjchNaVW8Ukop5UG0xK6UUkp5EP0QQSmrVauWiYqKcjoMpZRyK//8888hY0ztoodURdHEXsqioqJYtmyZ02EopZRbEZGSvDVQFUKr4pVSSikPooldKaWU8iCa2JVSSikPooldKaWU8iCa2JVSSikPooldKaWU8iBem9hF5BMROSAiawroLyLyjohsFpHVItK+vGNUSimlSsprEzvwGdZnEQtyKdb71psAtwPvl0NMSinldQ4cO52v0aqCeO0Laowx80QkqpBBrgC+sD+iskhEqolImDFmb7kEqJRSHu7EiXSuHDWbjWQ6HYpH8drEXgwNgF0uv+PsbqckdhG5HatUT2RkZLkEp5RS7uzLH9by1NLtTofhkby5Kr4okk+3fD+FZ4z5wBgTa4yJrV1bX3WslFKFeXTCipykXivIj3+e7OlsQB5GS+wFiwMiXH6HY33nXCmlVAllZGQxZ842fj50jCmrrVPp/Rc14d5eTR2OzPNoYi/YVGC4iEwCOgEJen9dKaVKbsmS3dxx9y8c6loXnyBfAnwrMWtkDyJqVHY6NI/ktYldRCYC5wO1RCQOeAbwAzDGjAOmA32AzcAJ4BZnIlVKKfe0KS6eW9/8ix3p6fhc1AAfu/uqZy4m0M+n0HHV6fPaxG6Mua6I/ga4u5zCUUopj2CMYcHmQzw++V92HU0GPwjw9SMmLIQB54QzuHND/H21eVdZ8trErpRSqnQYY/h4wTb+2XGUX9fsy+ke5FOJB7o2YuhlMQ5G5300sSullDptXy7awdNT1mDsZ4ZMZhZ1KwfwzT3diKpZGZH8HjBSZUkTu1JKqRLZuC+Rt2ZtylU6l8Q0dnzwL9dc1ZwxY86nfq0qDkbo3TSxK6WUKtKJtAx+WL6bsX9sZp/LK2DDs4SF760gqm4w03++nt69z3IwSgWa2JVSSuUjK8uwdPsRJizZyfZDSayKS8jp17lRDYb1aEyPs2szefJ6zvcP5IknziUoyM/BiFU2TexKKaVyfL14B5//vZ1N+4/n6t6nVT2ah4XQoVoVRt7zG4uOZnL+o3W46qrmXHVVc4eiVfnRxK6UUorktEzOffVPDh1PBeCKtvWpXy2IPi3DaFE/hOTkdJ57bi73jVlI9epBREaGOhyxKogmdqWU8lJr9yQwbu5Wth06zprdxwDw8xFmjexBw5onG7/9+ec2hgyZws6dCQwd2o6XX+5JzZr61riKShO7Ukp5kaTUDF74ZR0Tl+zK1b1nszp0jq7Jbd0bnfKIWuXKfoSGBjB//i10765fsKzoNLErpZSXMMbQ8tkZOc+cX9Y6jNu6N6JdRLVcyTwjI4t33lnM7t3HeOONS+jUKZyVK4dRqZI+k+4ONLErpZQH23YoiQ17jzHvv0N8t2wXxkCX6JpMvL1zvsMvWhTHsGHTWLVqP/36NSUjIwtf30qa1N2IJnallPIwGZlZrIqL55Ef/mXzgdyt2/u1qc/rA1ufMk58fAqPPTaL8eP/oX79YCZPvob+/WP0zXFuSBO7Ukp5CGMMXy3eyaif15KeadW3X9YqjCvbNeDsesGEhQbi65P/B1ji41P4+ut/uf/+zjz77PkEBweUZ+iqFGliV0opN7f/WAqf/b2d75bt4tDxNACubNeAQR0i6BRds8DxNm06zJdfrmLUqAuIiqrG9u33UaNGUHmFrcqIJnallHJDx1Mz+Hj+NrYcPM7UVXtyut/WvRF3nd+YmlULLnGnpGTw8ssLeOmlBQQF+XLLLe2Ijq6uSd1DaGJXSik3EX8ijQ/nb2VvQgqTl+/O6X5Ow+pcGFOH28+Lxq+AqvZss2Zt5a67fuG//45w/fWteOONi6lXr2pZh67KkSZ2pZSq4IwxPPHTGiYs3pnT7ey6wfRtHcaw8xsXmcyzJSenc9NNP1Klij+//z6YXr0al1XIykGa2JVSqgJKTstkysrdzFy3n9kbDuR0v7VbIx7vE1NgI7i8srIMEyb8y7XXtiAoyI8ZMwbTpElNAgP19O+pdMsqpVQFkpiSzpiZm/j0r+053YIDfOnepBZvXtuWQD+fYk9r5cp9DBs2jcWLrWr7wYNb06pV3dIOWVUwmtiVUsphSakZfLloB7+t2cfKXfEA+FYShl94FiMubFLil8McP57GM8/8ydtvL6ZGjSC+/PJKbrihVVmEriogTexKKeWQw8dTeej71fzhUtXeJbomV7ZrwID2DYpd3Z7Xddf9wLRpm7jjjnN46aWLqF5dW7t7EzHZLw1WpSI2NtYsW7bM6TCUUhWUMYbF245w8ydLSM3IAiCmXjA3dmnIwHMi8Pc9vWS+Y0c8oaGBVKsWyPLle0lNzaBLl4jSDL1Micg/xphYp+PwBFpiV0qpcjB300EmL49jysqTz5xXDfDlpQGt6NMqDJ/TfBd7enomb765iOeem8ttt7XjnXcupX37sNIKW7khTexKKVXKNh9IZNrqvSzaepjElAzW7jmW069Jnao0qlWFZy5vQYNqZ1ZF/tdfOxk27BfWrDnA5ZefzYMPdj3T0JUH0MSulFJnKCk1g1/X7GPWuv2sjotnT0IKAJX9fagkQseoGjQLC+amrlE0rl06L4N5//2l3HXXdCIiQvjpp2u54oqYUpmucn+a2JVS6jR9/08c3y7bxZJtR3K6BfpV4oKzazOsR+NC39N+OowxJCamERISwKWXNuHhh7vy1FM9qFrVv1Tno9ybJnallCqBQ8dTmbJyD18v3sHWg0kA1AkOoG/r+gw7P5o6wYFlMt8NGw5x552/EBjoy/Tp1xMVVY1XXulVJvNS7k0Tu1JKFUNqRiav/baRjxZsy+kWFhrIlLu7USekbJI5WK+BffHF+bzyyl9UqeLPK6/0LLN5Kc+giV0ppQqRlWV4dcZGxs3dktNtcOdInunXotjvaD9da9YcoH//SWzZcpTBg1vz+uu9qFtXP9iiCqeJXSml8khJz2TtnmO8NWsT6/Yc43CS9Y3zns3q8uFN5yByeo+mFZcxBhEhIiKE8PAQxo/vy0UXRZfpPJXn0MSulPJ6xhiS0jKZt+kg90xcQWZW7hd3Xd6mPqOvbElwoF+ZxpGZmcW4ccuYOHENf/55M6GhgcyZM6RM56k8jyZ2pZTXmrfpIC9OX8+GfYmn9Hvu8hac07A6LRuElkssy5fvZdiwaSxduoeePaOJj0+hdu0q5TJv5Vm8OrGLSG/gbcAH+MgY83Ke/pHA50A1e5hHjTHTyz1QpVSp2ZuQzDuz/+PnVXs5npoBWK3aL25Rl6Z1g7mkRT3qlmFjuLySk9N57LHZ/O9/S6hduzITJgxg0KCWZV7drzyX1yZ2EfEBxgK9gDhgqYhMNcascxnsSeBbY8z7ItIcmA5ElXuwSqkzMnPdfn5etYcdh5NYFZeQ0/2a2HDu6NG41F4aczp8fSsxb94Ohg07h9GjL6JatfK7qFCeyWsTO9AR2GyM2QogIpOAKwDXxG6AEPvvUGAPSqkKL+7oCVbtSmBvQjI/rdzNmt3WK11j6gXTJbom13WK5OLmdUv0bfPStG3bUZ55Zg5vv92b6tWDWLjwNgICvPl0rEqTN+9JDYBdLr/jgE55hnkW+F1E7gGqAPk+QCoitwO3A0RGRpZ6oEqpomVkZjF+3lZem7HxlH7NwkL44taO1A4OcCCyk9LSMhkzZiGjRs2lUiXh5pvbcNFF0ZrUVany5r0pvxtYeb9hex3wmTHmDRHpAnwpIi2NMVm5RjLmA+ADsD7bWibRKqVOkZKeyUvT1/PTyj0kJKfndI+oEcTDl8TQqkEo4dWDTvu75qVp/vwdDBv2C+vWHWTAgGa89dYlRESUT8M85V28ObHHAa4fKw7n1Kr224DeAMaYhSISCNQCDpRLhEqpfP29+RBfLNzBb2v35XRrH1mN2KgaDOvRmBpVKt6701999W+SktL4+efr6Nu3qdPhKA/mzYl9KdBERBoBu4FBwPV5htkJXAR8JiLNgEDgYLlGqZQXM8Zw9EQ6mw8c5+dVe0hMSWfFrnh2HD6RM8zAc8J5+arWp/0987JijOHzz1dx3nkNiY6uzkcf9aNqVX+qVMCLDuVZvDaxG2MyRGQ4MAPrUbZPjDFrRWQUsMwYMxV4APhQRO7HqqYfYozRqnalykBmlmHlrqOs3JXAwi2H2bQ/kZ1HTpwyXLOwEJqHhfBw77M5/+w6DkRatHXrDnLnnb8wb94OHnqoK6++qq+CVeXHaxM7gP1M+vQ83Z52+Xsd0K2841LKk2VmGY6eSCM5LZM/Nhxgy8Hj/L52P/uOpZwybNfGNYmoXpk2EdVoHR5K87AQKlWwkrmrEyfSeeGFebz22t+EhATw0Uf9uOWWdk6HpbyMVyd2pVTZMcaw9VASU1bsZu5/h6gkkJqexbq9x04ZtmqAL4M6RNAqPJQW9UOJqRfs2KNoZ+Kll+bz0ksLuPnmNrz2Wi99c5xyhCZ2pVSpMMYwZeUe9h1LYeO+RH5csTtX/3ohgbRsEEKjWlUI9POhU3QNgvx8uKhZHSr7u++paM+eRI4cSaZlyzo8+GBXevaMpkePKKfDUl7MfY8mpZRjjialkZCczq9r9jFr/X427kvMeT1rtph6wbSNqMZlrcPo1Kgm/r7OP3JWmjIzsxg7dilPPvkHMTG1WLx4KKGhgZrUleM0sSul8pWZZXh79n+s3BXPyp1H8fetRHqmyfW8eLaompVpExFK18a1uLlrFJX9fCr0vfAztWzZHu64YxrLl+/l4osb8957ffTd7qrC0MSulMpxMDGVSUt2MnbOZlLST76HKTjAl8a1qxJTL5gAPx+SUjNoH1mdapX96HZWLbe8H366Zs/eSq9eX1K3blW++eZqBg5srkldVSia2JXyYhmZWfy4Yjff/RPHmt0JnEjLBCDIz4d2kdW4NjaCgbERFe4Z8fJmjGH37kTCw0M477yGPP/8BQwf3pHQUP1gi6p4NLEr5YUOHU/lo/nbGDd3S063QL9K9Gpel76tw+jXur5HV6WXxNatR7n77umsWLGXDRuGU61aIE88cZ7TYSlVII9I7CLiD0QaYzY7HYtSFVFWlmHTgURe+XUDh5PSWG1/ujTAtxIXt6jHC1e0JLSyn8NRVixpaZm8/vrfPP/8PPz8KvHCCxcSHKxvjVMVn9sndhG5DBgD+AONRKQt8Iwx5kpnI1PKeZv2J/Lyrxv4Y0Puzxt0jq7BVe3DubxtfQJ8vef+eHEdPnyCc8/9lPXrD3H11c15661LaNAgpOgRlaoA3D6xA6OwPrf6J4AxZqWInOVsSEqVP2MMM9buY5t9riAAACAASURBVO6mg/y95TBJqZkcOp6a039w50j6tAojtmENj3v0rLSkp2fi5+dDjRpB9OjRkNdfv5g+fZo4HZZSJeIJiT3dGBOfp1Wqvs9deQVjDJOW7uK7ZbtYvjM+p3utqgG0CQ+lXmgg/drUp3N0TQejrPiysgyffrqCZ5+dy9y5Q4iOrs777/d1OiylTosnJPb1InINUMn+Utu9wCKHY1KqTBhj+HPjAVbHJbB+7zFmrN0PgL9PJeqGBNCjaW1u6daIZmFabVxca9Yc4M47f2HBgp2ce24kWVlaLlDuzRMS+3DgaSALmIz1tbbHHI1IqVI0/7+DfPbXdkRg1vrc98rDqwdxZbsG3Nezqdc/klZSxhiefPIPXn31b0JDA/jkk8sZMqStPpOu3J4nJPZLjDGPAI9kdxCRAVhJXim3MmPtPv7ZcZSVu+JZv+cYiS6vaQ3y86FjVA2qV/Hjkd4xRNSojJ+P3is/XSJCQkIqN93Umlde6UWtWpWdDkmpUiHu/nlxEVlujGmfp9s/xphznIgnNjbWLFu2zIlZKzeSmpHJxn2JJKVmsnZPAjsOn+DXNftyGruFBvlRSaBlg1C6nVWL3i3qEVVLvxR2puLijnHffb8xcmQXunaNICvL6PP6FYR93o51Og5P4LYldhG5BOgNNBCRMS69QrCq5ZWqcDbuS6Tf/xaQlnnqLtq4dhVa1A/hxQGtaFAtyIHoPFdGRhb/+99inn56DpmZWfTr15SuXSM0qSuP5LaJHTgArAFSgLUu3ROBRx2JSKl8bD+UxKSlu5i2eg9xR5Nzuj92aQytwkOp7O/L2XWDCfLX58nLwpIlu7njjmmsXLmPSy89i7Fj+9CoUXWnw1KqzLhtYjfGrABWiMjXxpgUp+NRCqx3r8cdTeb3dfvYfTSZSUt3kZpxsnTeon4Ij14aQ/ezamkjrXIyb94ODhxI4rvvBnLVVc10vSuP5wn32BsDo4HmQM4XGYwxTZ2IR++xe5+sLMOs9ft5a9Z/rNt7LKe7byWhbkggETWCGNajMec1qa1Vv+XAGMOkSWsICvKjf/8Y0tMzSU7OICQkwOnQVCH0HnvpcdsSu4vPgBeA14FLgVvQe+yqHBxJSuOBb1fy1+bDOffMa1X154ZODTmnYXXOa1rb4Qi9z+bNR7jrrl+YOXMr/fo1pX//GPz8fPDzos/KKuUJib2yMWaGiLxujNkCPCki850OSnkeYwxLtx9l++Ek/lh/gN/W7svpd3/PpvRrE0Z07aoORui9UlMzeOWVv3jxxfkEBPjy7ruXMmyYFv6Ud/KExJ4q1k2zLSIyDNgN1HE4JuVhTqRl0PzpGbm6+ftWom+rMF4f2Ear2B32++9beOaZOQwa1JIxYy4mLCzY6ZCUcownJPb7garACKx77aHArY5GpDzG2j0JTFi8kwlLdgIQFhrIeze0p0ndYKoGeMLh474OHEhiyZLd9O3blL59m7JkyVA6dGjgdFhKOc7tz0zGmMX2n4nAjQAiEu5cRMrdHT6eyntztvDxgm25ug/t3ogn+zZ3KCqVLSvL8NFHy3nkkVkYY9i1636CgwM0qStlc+vELiIdgAbAAmPMIRFpgfVq2QsBTe6q2Ob/d5B3/9jMil3xpLk8nnZdxwiu7RBJm/BQfUyqAli9ej/Dhk1j4cI4evRoyPvvX0ZwsLZ2V8qV2yZ2EXkJuApYhdVg7kesL7u9AgxzMjblHlLSM5mweCejpq3L6dYxqgYhQb4M7tyQc5vU1g+rVCB79iTSocOHhIQE8Pnn/bnxxtZ6saVUPtw2sQNXAG2MMckiUgPYY//e6HBcqoJKSs1gy8HjzFi7j68W7SQhOT2nX3CgL5Nu70yL+qEORqjys3r1flq3rkv9+sF8+ukVXHJJY2rW1A+2KFUQd07sKcaYZABjzBER2aBJXeW1+cBxFm09zITFO3O9PAYgulYVbuzSkGs7RFDZ350PBc+0c2cCI0b8ypQpG3Maxl1/fSunw1KqwnPns1m0iGR/mlWAKJffGGMGOBOWqihem7GBsX9uAaCSQNuIalzWKoy2kdXoEFXD4ehUQdLTM3nnncU888wcsrIMr7zSk7Zt6zkdllJuw50T+1V5fr/rSBSqwtl68DhPT1nLgs2HAHhjYBsuax1GoL59rMLLyjL06PEZCxfG0bdvU/73v0uJiqrmdFhKuRW3TezGmNlOx6AqhrSMLNbsSWDSkp18uywup3tMvWAm/l9nqlfxdzA6VRzHjqUSHOxPpUrCLbe05aGHutK/f4w2jlPqNLhtYlcKrAZx/d5dwNaDSQD4+QgXxdTltnMbaXW7GzDG8PXX//LAA7/zzju9ufbalvzf/53jdFhKuTWvTewi0ht4G/ABPjLGvJzPMNcAzwIGWGWMub5cg1T5Ss/M4kBiKl8v2sF7c6x76BfF1OH+Xk1pHhair3d1Exs3HuKuu6bzxx/b6NixAWefXcvpkJTyCB6T2EUkwBiTWsxhfYCxQC8gDlgqIlONMetchmkCPAZ0M8YcFRF9/7zDNh9I5MN52/hm2a5c3WMbVufjIR0cikqdjnfeWcxDD80kKMiX99+/jP/7v/b4+FRyOiylPILbJ3YR6Qh8jPWO+EgRaQMMNcbcU8hoHYHNxpit9jQmYT0Xv85lmP8DxhpjjgIYYw6URfyqcMlpmTz43Sp+W7uPzCwDWJ9Gvap9OG0iqnFx87r4akJwG8YYRISwsKpcfXVz3njjYurV0y/iKVWa3D6xA+8AfYGfAIwxq0TkgiLGaQC4FvvigE55hmkKICJ/YVXXP2uM+a1UIlZF+nPjAR76bjWHjp+shLk2NoLrO0XSJkJbSbubffuO88ADv9OqVR0efbQ7Awe2YODAFk6HpZRH8oTEXskYsyNP69nMIsbJ7yasyfPbF2gCnI/13vn5ItLSGBN/ysREbgduB4iMjCxm2Cqv7YeSmLh0J3M3HmTDvkQAu2QeyrUdIgjw1cfV3E1WlmH8+GU89thskpMzaNVK72gpVdY8IbHvsqvjjX3v/B5gUxHjxAERLr/DsV5Jm3eYRcaYdGCbiGzESvRL807MGPMB8AFAbGxs3gsEVYSJS3by2OR/c3ULDfLj3evbcW6T2g5Fpc7UmjUHGDp0KosX7+bCCxvx3nt9tIGcUuXAExL7nVjV8ZHAfmCW3a0wS4EmItII2A0MAvK2eP8JuA74TERqYVXNby3FuL3StkNJrNtzjKTUDFbFxbNg8yF2HD4BQJuIatx3URPOa6ofX/EESUlp7NiRwJdfXskNN7TSZ9KVKieekNgzjDGDSjKCMSZDRIYDM7Dun39ijFkrIqOAZcaYqXa/i0VkHVbV/kPGmMOlHbwnS0nPZMXOeNbvPcayHUdYuTOePQkppwzXtG5VPrwploY1qzgQpSotxhh++mkDK1fu47nnLqBTp3C2bbuXwEBPOM0o5T7EGPeuORaRLcBG4BtgsjEm0cl4YmNjzbJly5wMwXGpGZncM2EFv6/bn6u7v08lmtaryogLmxBTL4SQIF+qVda3wnmCHTviGT78V6ZN20TbtvVYuPA2TeiqRETkH2NMrNNxeAK3P/KMMY1FpCtWdfpzIrISmGSMmeRwaF5nX0IK3y3bxRszTzZxGHHhWXRpXIv2Datp4zcPlJ6eyZtvLuK55+YiAq+/3ot77+2Mr68+gqiUU9y+xO7K/i77W8ANxhhHsog3ltj3H0vhmvELc+6V+/tU4vpOkTxxWTP89Blzj7ZzZwLNmo3l4osb8/bbvYmM1O/Zq9OjJfbS4/YldhGpivVymUFAM2AK0NXRoLzI5gOJ9BwzD4DawQE8f0ULLmlRTxtKebDDh0/w5ZeruffeTkRGhrJmzZ00alTd6bCUUja3T+zAGuBn4FVjzHyng/F0WVmGD+dvZXVcAgs2HyIhOR2wqtxHXny2w9GpsmSM4YsvVvHggzM5ejSZCy9sROvWdTWpK1XBeEJijzbGZDkdhKdLSc/kmSlrc72nPSw0kJYNQniqb3Ni6oU4GJ0qa+vXH+TOO39h7twddOkSzrhxfWnduq7TYSml8uG2iV1E3jDGPAD8ICKnNBQwxgxwICyPcyAxhQ/mbuWjBdsA6NioBpe2rMf1nSK1MZyXyMjIok+fCcTHpzB+fF+GDm2vX9BTqgJz28SO9XgbwLuORuGhth48zsPfr2bZjqM53Z7q25zbujdyMCpVnubM2U63bhH4+fkwYcIAGjeuQZ06+q4BpSo6t03sxpgl9p/NjDG5krv98pnZ5R+VZ9h84Dg9x8zN+T32+vZ0jq5BzaoBDkalysvevYncf/8MvvlmLWPH9uGuuzrQpUtE0SMqpSoEt03sLm7l1FL7bfl0U8Xg+t72Ry+N4fZzo7Xa1UtkZmbx/vvLeOKJP0hNzWDUqPO57bZ2ToellCoht03sInIt1iNujURkskuvYOCUL7Cpgp1Iy+DNmZv4cL51Hz040Je7LziLYT0aOxyZKk9Dh/7MZ5+tpGfPaN57rw9NmtR0OiSl1Glw28QOLAEOY32ZbaxL90RghSMRuRljDAu3Hub6DxcDUNnfhw5RNfjgpnO0YZyXOHYsFWMMoaGB3HlnLBdfHM2gQS31PQRKuTG3TezGmG3ANqyvuakSWLLtCC9OX8/KXScrNtpFVmPynV31hO4ljDH88MN67r33N/r1a8q4cX3p2LEBHTs2cDo0pdQZctvELiJzjTE9ROQo4Pq4mwDGGFPDodAqrF9W7+XuCctzdbu1WyOu7RDB2fWCHYpKlbdt244yfPivTJ/+H23b1uOWW9o6HZJSqhS5bWIHLrD/r+VoFG7AGMMlb81j0/7jAMTUC+Z/17WjSV1N5t5m8uT1DB48GR+fSrz55iUMH95RP9iilIdx28Tu8ra5CGCPMSZNRLoDrYGvgGOOBVeBGGNo9Nj0nN/zH76AiBqVHYxIOSE9PRM/Px/atw+jf/8YXn21F+Hh+rZApTyRJ1yq/wQYEWkMfIH1IZgJzoZUMexNSM6V1Dc831uTupc5dOgEt902hcsvn4QxhqioakyYcJUmdaU8mCck9ixjTDowAHjLGHMP4PUtgIwxXPbOAgB6NqvLv89eTKCftnT3FsYYPv10BTEx7/LFF6tp06YumZme84lmpVTB3LYq3kWGiAwEbgT62938HIzHUWkZWTwzdQ3TVu0lMTWD5mEhfHSzfuLYm+zcmcDgwZOZP38n3bpFMG5cX1q2rON0WEqpcuIJif1W4C6sz7ZuFZFGwESHY3JEVpbhnBdmkpiSAcCAdg0YfWUrh6NS5S00NIAjR5L56KN+3HJLO31zoFJexu0TuzFmjYiMAM4SkRhgszFmtNNxOWHEpBU5SX37y5c5HI0qT9On/8f77y9j8uRrCA0NZPXqOzWhK+Wl3P4eu4icC2wGPgY+ATaJSDdnoyp/v/67l2mr9wKw8YXeDkejysvu3ccYOPA7LrtsAps3H2H37kQATepKeTG3L7EDbwJ9jDHrAESkGfAl4DU3lhNT0rnza+vFM7+M6K6vg/UCmZlZjB27lCef/IP09CxGj76QBx/sir+/bnulvJ0nJHb/7KQOYIxZLyL+TgZUnjIys2jz3O8AXN8pkhb1Qx2OSJWHrCzDxx+voFu3SMaO7UN0dHWnQ1JKVRCekNiXi8h4rFI6wA14yUdg/tufSL93F5BlILpWFUb3b+l0SKoMJSSk8OKL83n00e5Urx7En3/eTPXqgfp+f6VULm5/jx0YBmwBHgYeAbYCdzgaUTlYtv0Ivd6cR0p6FrefF83sB3roCd5DGWP45ps1xMSM5bXX/mbmzK0A1KgRpNtcKXUKty6xi0groDHwozHmVafjKS+ZWYarxy0E4IMbz+HiFvUcjkiVlS1bjnD33dOZMWML7duH8fPP1xEbW9/psJRSFZjblthF5HGs18neAMwUkVsdDqncPDVlDQBtI6ppUvdwjzwyi7//3sXbb/dmyZKhmtSVUkVy5xL7DUBrY0ySiNQGpmM97ubRjiSlMWHxTgAm/l9nh6NRZWHu3O1ERIQSHV2dt97qjQg0aKDvdldKFY/bltiBVGNMEoAx5iDuvSzFciItg46jZwEw6ooWBOmjTR7l4MEkhgz5ifPP/5zRo+cBEB4eokldKVUi7lxijxaRyfbfAjR2+Y0xZoAzYZWdTxZsIyPL0Kt5XW7qEuV0OKqUZGUZPvlkBQ8/PJPjx9N4/PHuPPHEeU6HpZRyU+6c2K/K8/tdR6IoJ3sTknn9900AvH51G4ejUaXprbcW8cADv3PuuZGMG9eX5s1rOx2SUsqNuW1iN8bMdjqG8tTlpT8AuCY2nNDKXvvxOo+RlJTG3r3HOeusGtx2Wztq167M4MGt9fE1pdQZ8/j70p5gzEyrpF41wJdXtbTu9qZN20SLFu8xYMA3ZGUZQkMDufHGNprUlVKlwqsTu4j0FpGNIrJZRB4tZLirRcSISLm/f37XkRO8M/s/AOY/fEF5z16Vori4YwwY8A39+k2kShV/xo7tox9rUUqVOretis9LRAKMMaklGN4HGAv0AuKApSIy1fW98/ZwwcAIYHFpxltc906y3o774U2xVK/iNa/A9zjLl++lR4/PyMzM4qWXLmLkyC76wRalVJlw+xK7iHQUkX+B/+zfbUTkf8UYtSPWt9u3GmPSgEnAFfkM9zzwKpBSWjEX176EFJbvjKd2cAC9mtct79mrUnDsmHWt2apVHW69tS1r197Fo49216SulCozbp/YgXeAvsBhAGPMKqA4ddYNgF0uv+PsbjlEpB0QYYyZVtiEROR2EVkmIssOHjxYktgLdf83KwF4/ooWpTZNVT7i41O4665fiIl5l/j4FPz8fHj77Utp1Ei/wqaUKluekNgrGWN25OmWWYzx8ru5aXJ6ilTC+tb7A0VNyBjzgTEm1hgTW7t26TyqtHZPAgu3HiY40JdL9LWxbsMYw8SJ/xIT8y7jx//Dtde2wMdH76MrpcqPJ9xj3yUiHQFj3ze/B9hUjPHigAiX3+HAHpffwUBLYI7dWrkeMFVELjfGLCuVyAsxaPwiAN4e1FZbS7uJ48fTGDDgG2bO3EpsbH2mT7+B9u3DnA5LKeVlPCGx34lVHR8J7Adm2d2KshRoIiKNgN3AIOD67J7GmASgVvZvEZkDPFgeSX3jvkQSUzNoVKsKF8bovfWKzhiDiFClih+1alXm3XcvZdiwWHx8PKFCTCnlbtw+sRtjDmAl5ZKOlyEiw4EZgA/wiTFmrYiMApYZY6aWcqjFdveE5QC8dnVrp0JQxfTHH9sYOXIGkydfS3R0dSZMyPtCRKWUKl9un9hF5ENc7o1nM8bcXtS4xpjpWF+Fc+32dAHDnn+aIZbI6F/WsfnAcQDOaagNrSqq/fuP8+CDM/nqq9VER1fn0KETREfr9lJKOc/tEztW1Xu2QOBKcrd2dxvGGD6cvw2AX0Z013vrFdRHHy3noYdmkpSUxlNPncdjj3UnKEhf86uUqhjcPrEbY75x/S0iXwIzHQrnjExbvReAQR0iaFE/1OFoVEFWrNhL27b1eP/9y4iJqVX0CEopVY7cPrHnoxHQ0OkgTsfv6/YTHODLi1e2cjoU5eL48TSee24OV17ZjK5dIxgz5hL8/X20RkUpVSG5fWIXkaOcvMdeCTgCFPje94oqM8vw86o9DGjfQN8fXoFMmbKBe+75lV27jlG9ehBdu0YQEOD2h41SyoO59RlKrCJTG6zH1QCyjDGnNKRzB+/9uRmAs+pUdTgSBbBjRzwjRvzG1KkbadmyDhMnXkW3bpFOh6WUUkVy68RujDEi8qMx5hynYzlTk5Za7f2Gdo92OBIF8P3365g1ayuvvtqT++7rjJ+fvttdKeUe3Dqx25aISHtjzHKnAzld+xJS2B2fzFl1quLvqy81ccqiRXHEx6fQu/dZjBjRiYEDWxAZqY0YlVLuxW2ziIhkX5R0x0ruG0VkuYisEBG3SvLfLbNK6y/0b+lwJN7p6NFkhg2bRteuH/P0039ijMHPz0eTulLKLblziX0J0B7o73QgZ+qjBduo7O9D5+iaTofiVYwxfP31v4wcOYMjR5K5//7OPPfcBdraXSnl1tw5sQuAMWaL04GcicPHU0lITqdZWIjToXidP//czo03/kinTg34/fcbadtWv6KnlHJ/7pzYa4vIyIJ6GmPGlGcwp+ufHUcBuP28Rg5H4h1SUjJYunQ3557bkAsuiGLKlEFcdlkT/WCLUspjuPPZzAeoivV51fz+VXjGGG7/8h8Azm9ax+FoPN/MmVto1ep9LrnkKw4eTEJEuPzyszWpK6U8ijuX2PcaY0Y5HcSZePzHf3P+rlZZ3zVeVvbtO87IkTOYOHENTZrUYOrU66hdu4rTYSmlVJlw58Tu1i2cDhxLYeISqzX8ulGXaIOtMnL0aDItWrzH8eNpPPNMDx59tDuBge682yulVOHc+Qx3kdMBnIkZa/cBMOaaNlT2d+fNUDHt3n2MBg1CqF49iOefv4CePaNp2lSfOlBKeT63vblojDnidAxn4oP5W/GpJPRv28DpUDxKYmIqI0fOICrqbf7+26oRueuuDprUlVJeQ4uKDjDGEH8inaoBvvrBl1JijOHHHzcwYsSv7NmTyB13nEOzZvpJVaWU99HE7oCFWw+TmJLByF5NnQ7FY1x//WQmTVpD69Z1+f77a+jcOdzpkJRSyhGa2B1w48dLAOjRtLbDkbi39PRMfH0rISJ06xZBbGwY997bGV99375SyovpGbCcGWPIzLK+LNsmoprD0bivBQt20q7deL79di0Aw4d35IEHumpSV0p5PT0LlrM9CSkADDxHq4pPx+HDJxg6dCrnnvspx46lEhoa6HRISilVoWhVfDn7etEOAIZ0i3I2EDf0ww/rGDbsF44eTeahh7ry9NM9qFrV3+mwlFKqQtHEXs72HbNK7DH19KMvJWUMNGlSg3Hj+tK6dV2nw1FKqQpJE3s523XkBCGBvvjoY25FSk5OZ/To+dSoEcTIkV246qpmDBjQTB8RVEqpQug99nK2dPtRzm2ireGLMmPGZlq2fJ/Ro+ezadNhAEREk7pSShVBS+zlaHVcPAA1quh94YLs3ZvIfffN4Ntv19K0aU1mz76JCy/UT9oqpVRxaWIvR5OWWq84valLQ4cjqbh27kzg5583MmrU+Tz8cDcCAnQXVUqpktCzZjmKP5EGQJO6bvG5+HKzfPle/vhjGw8+2JVOncLZtet+atas7HRYSinllvQeeznJyjIs2nqEuiEBTodSYRw7lsp99/1Ghw4fMmbMQhLsZ/w1qSul1OnTxF5OZm84wJGkNG7opNXwxhi+/34dzZqN5Z13FjNs2DmsW3e3vmxGKaVKgVbFl5Ml26yW3YM6RDgcifP270/i5pt/omnTmvz447V07KifrlVKqdKiib2cbDt0AoCaVb2zKj4tLZPvvlvL9de3ol69qsydO4S2bevpu92VUqqUee1ZVUR6i8hGEdksIo/m03+kiKwTkdUiMltEzqgOfdP+RBrVquKVL6aZP38H7dqNZ/DgH5k/fycAsbH1NakrpVQZ8Mozq4j4AGOBS4HmwHUi0jzPYCuAWGNMa+B74NXTnZ8xhp1HTnC2l7WGP3ToBLfeOoXzzvuMpKQ0fv75Os47T9sYKKVUWfLWqviOwGZjzFYAEZkEXAGsyx7AGPOny/CLgMGnO7P0TOszrTFh3pPYjTFcdNEXrFt3kEcf7caTT55HFX0xj1JKlTlvTewNgF0uv+OAToUMfxvwa0E9ReR24HaAyMjIU/qnZ2YBUNnfp+SRupkNGw4RHV0df38f3nzzEurUqULLlnWcDksppbyGV1bFA/nd6Db5DigyGIgFXitoYsaYD4wxscaY2Nq1T30PfFqGldj9fDx3dZ84kc7jj8+mVav3eeedxQBceGEjTepKKVXOvLXEHge4PncWDuzJO5CI9ASeAHoYY1JPd2ar7HfEe2qzuenT/+Puu6ezfXs8Q4a0ZciQtk6HpJRSXstzi5CFWwo0EZFGIuIPDAKmug4gIu2A8cDlxpgDZzKzqSuta4bOjWueyWQqpCef/IPLLptAYKAvc+bczKefXkGtWvrmOKWUcopXltiNMRkiMhyYAfgAnxhj1orIKGCZMWYqVtV7VeA7EQHYaYy5/HTmdzw1A4CYeiGlEb7jMjKySE3NoEoVf/r1a0pQkC8PPdQNfy9oQ6CUUhWdVyZ2AGPMdGB6nm5Pu/zds7TmNWfTQUICPWNVL126m2HDfqFDh/qMG9eXTp3C6dQp3OmwlFJK2by1Kr5cVfb3IdLNP2ySkJDC8OHT6dTpI/buTeSii/Qb6UopVRF5RjGygktMyaBLtPveX583bweDBn3P/v1JDB/ekRdeuJAQ/UqdUkpVSJrYy1j8iTQysww+ldyvcsQYg4jQsGEoTZrUZOrU64iNre90WEoppQqhib2MLd52BID61dznk6SpqRm89trf/PPPXiZPvoaGDasxd+4Qp8NSSilVDO5XjHQzq3ZZz7D3blHP4UiKZ86c7bRtO56nnvoTP79KJCdnOB2SUkqpEtDEXsamrtrDWXWqUiekYpfYjxxJ5uabf+KCCz4nNTWD6dOv59tvB1K5sp/ToSmllCoBTexlbE98MqFBFT85+vgIc+du5/HHu7NmzV1cemkTp0NSSil1GvQeexnKyMwiy0BkjYr5qNuaNQcYM2Yh48f3JTQ0kA0bhhPoIc/bK6WUt9ISexnan2i9Xj6mXsX6XGtSUhqPPDKTdu3GM3XqRjZsOASgSV0ppTyAnsnL0L6EFADqhVac++vTpm1i+PDp7NiRwK23tuXVV3tR081fnqOUUuokTexlaO5G69sxZ1eQEntmZhZPPvkHVar4M2/eEM49t6HTISmllCplmtjL0JaDSQA0qlXFsRgyMrIYN24ZN9zQiurVg5g69Trq1auqH2xR1H1MTgAAFDZJREFUSikPpYm9DK3fe4yOjWoQ4OtMEl28OI477pjGqlX7EYG77+5IZGSoI7EopZQqH5rYy9DWQ0m0Di//RBofn8Ljj89m3LhlhIUF8/33AxkwoFm5x6EqvvT0dOLi4khJSXE6FOUlAgMDCQ8Px8+v4j8G7K40sZeRJPsb7JUDyn8V33vvb3z11WruvbcTo0ZdQHCwfrBF5S8uLo7g4GCioqIQEafDUR7OGMPhw4eJi4ujUSP9QmRZ0cReRlbHJQBwdt3yaTj333+H8fPzISqqGqNGnc+993aiffuwcpm3cl8pKSma1FW5ERFq1qzJwYMHnQ7Fo+lz7GVk8bbDAGVeFZ+amsGoUXNp1ep9HnzwdwAaNqymSV0VmyZ1VZ50fyt7WmIvIxv2JgLQJrxamc1j9uyt3HXXdDZtOsygQS0ZM+biMpuXUkop96Al9jLiU0nw8xEqVSqbq9MvvlhFz55fkpmZxYwZg5k48SrCwirG8/JKlYSPjw9t27alZcuW9OvXj/j4+Jx+a9eu5cILL6Rp06Y0adKE559/HmNMTv9ff/2V2NhYmjVrRkxMDA8++KATi1CoFStWMHTo0FzdrrjiCrp06ZKr25AhQ/j+++9zdatatWrO35s2baJPnz6cddZZNGvWjGuuuYb9+/efUWxHjhyhV69eNGnShF69ev1/e/ceHVV1L3D8+4OgIYA8JYXGGkOwBAKEhw+Ellfl4W20YCQKlcelFZDqqijea+1apGLro76qwYvcXhvQJiBQEMVqi2C1CEooMUGCGCGl0AghBkkgSEh+949zMpmECZlAMgOT32ets9bMmX32+c2PYfacfXb2pri4+IwymzZtIiEhwbOFh4ezdu1aAPbt28d1111Hz549SU5O5tSpUwCkpqbyhz/84bxiM+dBVW1rxG3QoEGqqnrlf72piS98oI2poqJSDx48pqqqR4+W6a9//b6eOHGqUc9hmpddu3YFOwRt06aN5/HUqVP10UcfVVXVEydOaExMjL7zzjuqqnr8+HEdN26cpqamqqpqTk6OxsTEaG5urqqqlpeX66JFixo1tvLy8vOuIykpSbOysjzPi4uLNSoqSnv16qV79+717J82bZquXLmyxrFVuSkrK9PY2Fhdt26d57WNGzdqTk7OecU2f/58feyxx1RV9bHHHtMHH3zwrOWLioq0Y8eOevz4cVVVve222zQjI0NVVWfNmqUvvviiqjr/VgkJCXXW4+tzB2TqBfAdHgqbdcU3AeczCkdPlDdanZ988iWzZ6+npOQbduyYRfv24fziF99rtPqN+dUbn7Lr38catc7e3S9jQWIfv8sPGTKE7OxsANLT0xk6dChjxji3mCIiIkhNTWXEiBHMnTuXJ598kocffphevXoBEBYWxt13331GnaWlpdxzzz1kZmYiIixYsIBbb72Vtm3bUlpaCsCqVat48803SUtLY/r06XTq1IkdO3aQkJDAmjVryMrKokMH57ZabGwsmzdvpkWLFsyePZv9+/cD8NxzzzF06NAa5y4pKSE7O5v+/ft79q1evZrExEQiIyNZvnw5Dz30UL15SU9PZ8iQISQmJnr2jRw50u+81uX111/nvffeA2DatGmMGDGCJ554os7yq1atYvz48URERKCqbNy4kfT0dM/xKSkpzJkzh4iICKKjo/n444+59tprzztO0zDWsDeB/KITACRfc8V511VaeoqUlPd47rmtdOrUmqefHkNYmN1BMaGnoqKCd999l5kzZwJON/ygQYNqlOnRowelpaUcO3aMnTt3cv/999db78KFC2nfvj05OTkAPruba9uzZw8bNmygZcuWVFZWsmbNGmbMmMFHH31EdHQ0kZGRTJ48mfvuu49hw4axf/9+xo4dS25ubo16MjMziY+Pr7EvIyODBQsWEBkZSVJSkl8N+86dO8/IhS8lJSV873u+f/Cnp6fTu3fvGvsOHTpEt27OQNtu3bpx+PDhs9a/fPly5s2bB0BRUREdOnQgLMxpRqKiojh48KCn7ODBg/nggw+sYQ8Ca9ibwJYvnBHxV5zncq2ff17E6NHL+Ne/jvHTnw7k8cd/QKdOrRsjRGPO0JAr68ZUVlZGQkIC+fn5DBo0iBtvvBFwer7qGkHdkJHVGzZsYPny5Z7nHTt2rPeY2267jZYtnRkjk5OTeeSRR5gxYwbLly8nOTnZU++uXbs8xxw7doySkhLatase61JQUMDll1/ueX7o0CHy8vIYNmwYIkJYWBg7d+4kPj7e53tq6Ajydu3akZWV1aBj/FVQUEBOTg5jx44FqnsmvXnH27VrV3bv3t0ksZizs0u/JrC30Oneuyn+W+d0fHl5BQDR0R0YNuw7bN78nyxZkmiNuglJrVu3Jisri3/+85+cOnWKRYsWAdCnTx8yMzNrlN27dy9t27alXbt29OnTh+3bt9dbf10/ELz31Z55r02b6vUdhgwZQl5eHoWFhaxdu5aJEycCUFlZyZYtW8jKyiIrK4uDBw/WaNSr3pt33StWrKC4uJirrrqK6Oho8vPzPT86OnfuXKM34auvvqJLly6eXPjzXktKSmoMdPPevH+EVImMjKSgoABwGu6uXbvWWfdrr73GhAkTPDPGdenShaNHj3L6tDMZ14EDB+jevbun/MmTJ2nd2r6zgsEa9ibwRWEpl7RsQVjLhqW3vLyCp576kF69FlFcXEarVi1JT7+VG244/y59Yy507du35/nnn+epp56ivLycKVOm8Pe//50NGzYAzpX9vffey4MPPgjA/Pnz+c1vfsOePXsAp6F95plnzqh3zJgxpKamep5XNZ6RkZHk5uZ6utrrIiJMmDCBefPmERcXR+fOnX3W6+tKOS4ujry8PM/zjIwM3n77bfLz88nPz2f79u2ehn3EiBGsWLHCM7I8LS3Ncx998uTJfPjhh6xfv95T19tvv+25vVCl6ord11a7Gx7g5ptvZunSpQAsXbqUW265pc48ZGRkcMcdd9TIy8iRIz0j+Wsfv2fPnjNuQ5gACfbovVDbBg0apD0eWq/jn3tfG2Lz5v3at++LCimamJiuBQUlDTremHNxoY2KV1X94Q9/qMuWLVNV1ezsbB0+fLheffXV2qNHD01JSdHKykpP2TfeeEMHDhyovXr10ri4OH3ggQfOqL+kpESnTp2qffr00X79+unq1atVVXXlypUaExOjw4cP17lz5+q0adNU1ffo9G3btimgaWlpnn2FhYU6adIk7du3r8bFxemsWbN8vr/4+Hg9duyY7tu3T7t3714jflXVAQMG6NatW1VVNSUlRePj47V///46ceJEPXz4sKdcbm6ujh07VmNjYzUuLk6Tk5P1yy+/PGtu63PkyBEdNWqUxsbG6qhRo7SoqMjzfmfOnOkpVxV7RUVFjeO/+OILveaaa7RHjx6alJSkJ0+erPG+CgsLfZ7XRsU37SZOPk1j6dN/gB4f/yizvh/DQzfVv/DKqVMV3HPPWyxZ8g+ioi7jhRfG86Mf9QpApMZAbm4ucXG2QFBTevbZZ2nXrt0Zf8seynbs2MEzzzzDK6+84vN1X587EdmuqoMDEV+os674RlZ2yrk/Ps7P++utWrXg8OETzJt3Pbm5c61RNybEzJkzh0svbV4LMR05coSFCxcGO4xmy0bFN7KKSkWAb3eoe9DIZ58dYd68v/DCC+OJienI6tWTmmyGOmNMcIWHh3PnnXcGO4yAqvrLBhMcdsXeyEpOnqZjRCsu97FU6smTp1mwYBP9+i1m8+b97N59BMAadRNUdjvOBJJ93pqeXbE3sm9OVxBxSdgZf16zYcNe5sxZT17eV0yZ0pennx5DZGTbOmoxJjDCw8MpKiqic+fOtuqWaXKqznrs4eHhwQ4lpFnD3sgqKpWrurQ5Y//rr+9GBDZsuJPRo2OCEJkxZ4qKiuLAgQO2PrYJmPDwcKKiooIdRkizUfGN7NJuPfUXS9byy5vieOml7QwY8C2GDLmC0tJThIW1IDzcfksZY0xtNiq+8TTre+wiMk5EPhORPBH5bx+vXyoiK9zXPxKRaH/q7VgJN9zwMnPnvsWrrzoLWrRte4k16sYYY5pcs21pRKQlsAi4ETgAbBORdarqPe/iTKBYVWNF5HbgCSC5vrrvS/oTXTq15tVXJzB5ct+mCN8YY4zxqTlfsV8L5KnqXlU9BSwHas+neAuw1H28Chgt9Ywwqiyv5K6fDGD37rlMmdLPBiQZY4wJqGZ7xQ58G/iX1/MDwHV1lVHV0yLyNdAZOOJdSETuAu5yn36zeHHizsWLmyTmi00XauWqGbNcVLNcVLNcVPtusAMIFc25Yfd1KV17JKE/ZVDVJcASABHJtAEgDstFNctFNctFNctFNRHJrL+U8Udz7oo/AHgvmxYF/LuuMiISBrQHvgpIdMYYY8w5aM4N+zagp4hcJSKXALcD62qVWQdMcx8nARvV/j7QGGPMBazZdsW798x/BrwDtAReVtVPReQRnOUD1wH/B7wiInk4V+q3+1H1kiYL+uJjuahmuahmuahmuahmuWgkNkGNMcYYE0Kac1e8McYYE3KsYTfGGGNCiDXs56CppqK9GPmRi3kisktEskXkXRG5MhhxBkJ9ufAqlyQiKiIh+2dO/uRCRCa5n41PRSQ90DEGih//R74jIptEZIf7/+SmYMQZCCLysogcFpGddbwuIvK8m6tsERkY6BhDgqra1oANZ6DdF0AMcAnwCdC7Vpm7gcXu49uBFcGOO4i5GAlEuI/nNOdcuOXaAe8DW4HBwY47iJ+LnsAOoKP7vGuw4w5iLpYAc9zHvYH8YMfdhPn4PjAQ2FnH6zcBf8aZQ+R64KNgx3wxbnbF3nBNMhXtRareXKjqJlU94T7dijNfQCjy53MBsBB4EjgZyOACzJ9c/BRYpKrFAKp6OMAxBoo/uVDgMvdxe86cTyNkqOr7nH0ukFuAZerYCnQQkW6BiS50WMPecL6mov12XWVU9TRQNRVtqPEnF95m4vwaD0X15kJEBgBXqOqbgQwsCPz5XFwNXC0im0Vkq4iMC1h0geVPLlKAH4vIAeAt4J7AhHZBauh3ivGh2f4d+3lotKloQ4Df71NEfgwMBoY3aUTBc9ZciEgL4FlgeqACCiJ/PhdhON3xI3B6cT4QkXhVPdrEsQWaP7m4A0hT1adFZAjO3BnxqlrZ9OFdcJrLd2eTsiv2hrOpaKv5kwtE5AfAw8DNqvpNgGILtPpy0Q6IB94TkXyc+4frQnQAnb//R15X1XJV3Qd8htPQhxp/cjETeA1AVbcA4TiLwzRHfn2nmLOzhr3hbCraavXmwu1+fgmnUQ/V+6hQTy5U9WtV7aKq0aoajTPe4GZVDcWFL/z5P7IWZ2AlItIFp2t+b0CjDAx/crEfGA0gInE4DXthQKO8cKwDprqj468HvlbVgmAHdbGxrvgG0qabivai42cufgu0BVa64wf3q+rNQQu6ifiZi2bBz1y8A4wRkV1ABTBfVYuCF3XT8DMX9wP/KyL34XQ7Tw/RCwFEJAPn9ksXd0zBAqAVgKouxhljcBOQB5wAZgQn0oubTSlrjDHGhBDrijfGGGNCiDXsxhhjTAixht0YY4wJIdawG2OMMSHEGnZjjDEmhFjDbkwDiUiFiGR5bdFnKRtd10pWDTzne+4KYZ+407B+9xzqmC0iU93H00Wku9drvxeR3o0c5zYRSfDjmJ+LSMT5ntsY47CG3ZiGK1PVBK8tP0DnnaKq/XEWGPptQw9W1cWqusx9Oh3o7vXaT1R1V6NEWR3ni/gX588Ba9iNaSTWsBvTCNwr8w9E5B/udoOPMn1E5GP3Kj9bRHq6+3/stf8lEWlZz+neB2LdY0e763jnuGtdX+ruf9xd6zxbRJ5y96WIyAMikoQzb/8f3XO2dq+0B4vIHBF50ivm6SLywjnGuQWvBTxE5H9EJFOc9dd/5e67F+cHxiYR2eTuGyMiW9w8rhSRtvWcxxjjxRp2YxqutVc3/Bp332HgRlUdCCQDz/s4bjbwO1VNwGlYD7hTiCYDQ939FcCUes6fCOSISDiQBiSral+cmSTniEgnYALQR1X7AY96H6yqq4BMnCvrBFUt83p5FTDR63kysOIc4xyHM3VslYdVdTDQDxguIv1U9XmcucBHqupId3rZXwI/cHOZCcyr5zzGGC82pawxDVfmNm7eWgGp7j3lCpy5z2vbAjwsIlHAn1T1cxEZDQwCtrlT7rbG+ZHgyx9FpAzIx1na87vAPlXd476+FJgLpOKs9/57EVkP+L1MrKoWished57uz91zbHbrbUicbXCmUB3otX+SiNyF873TDegNZNc69np3/2b3PJfg5M0Y4ydr2I1pHPcBh4D+OD1hJ2sXUNV0EfkI+A/gHRH5Cc4ylUtV9SE/zjHFe9EYEensq5A7P/m1OAuL3A78DBjVgPeyApgE7AbWqKqK08r6HSfwCfA4sAiYKCJXAQ8A16hqsYik4Sx2UpsAf1XVOxoQrzHGi3XFG9M42gMF7hrad+JcrdYgIjHAXrf7eR1Ol/S7QJKIdHXLdBKRK/08524gWkRi3ed3An9z70m3V9W3cAam+RqZXoKzlKwvfwJ+hLNO+Ap3X4PiVNVynC71691u/MuA48DXIhIJjK8jlq3A0Kr3JCIRIuKr98MYUwdr2I1pHC8C00RkK043/HEfZZKBnSKSBfQClrkj0X8J/EVEsoG/4nRT10tVT+KsfrVSRHKASmAxTiP5plvf33B6E2pLAxZXDZ6rVW8xsAu4UlU/dvc1OE733v3TwAOq+gmwA/gUeBmne7/KEuDPIrJJVQtxRuxnuOfZipMrY4yfbHU3Y4wxJoTYFbsxxhgTQqxhN8YYY0KINezGGGNMCLGG3RhjjAkh1rAbY4wxIcQadmOMMSaEWMNujDHGhJD/B1A8TuGEh9jrAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7040433457077317" + ] + }, + "execution_count": 81, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#plot roc for svm\n", + "get_roc(clf_reduced, y_test, X_test_pca, \n", + " \"SVM reduced features no tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM reduced features no tuning RBF kernal identified 738\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06387342
11282738
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6387 342\n", + "1 1282 738" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_reduced.predict(X_test_pca), \"SVM reduced features no tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.37 0.48 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.66 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_reduced.predict(X_test_pca)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, by reducing features through pca, Although the AUROC did not change much (0.001 increase), the number of correctly idendtified defaulters increased from 297 to 307, suggesting a better recall." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will now try to find best parameters for SVM model" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'C': 1, 'gamma': 0.1}" + ] + }, + "execution_count": 84, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn.model_selection import GridSearchCV\n", + "def svc_param_selection(X, y, nfolds):\n", + " Cs = [0.01, 0.1, 1,10]\n", + " gammas = [0.01, 0.1, 10]\n", + " param_grid = {'C': Cs, 'gamma' : gammas}\n", + " grid_search = GridSearchCV(svm.SVC(kernel='rbf'), param_grid, cv=nfolds)\n", + " grid_search.fit(X, y)\n", + " grid_search.best_params_\n", + " return grid_search.best_params_\n", + "svc_param_selection(X_train_pca, y_train,2)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With 2 folds, it can be found that C = 10, and gamma = 0.01 will have the best svm model with RBF kernel" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma=0.1, kernel='rbf',\n", + " max_iter=-1, probability=True, random_state=None, shrinking=True, tol=0.001,\n", + " verbose=False)" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#train svm model with feature reduction and cost = 10, gamma = 0.01\n", + "clf_reduced_tuned = svm.SVC(kernel = 'rbf', probability = True, C= 10, gamma = 0.1)\n", + "clf_reduced_tuned.fit(X_train_pca, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.16537143473786733\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAf4AAAEWCAYAAACHePXKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KCJCQECCElkJC7zUCiiAoXRHsqOjFhiAoqNxPRb32eu1XsCt2LFgp0kRQUKnSe08IhJZCetnfHzMJJyEVkkxyst7n4SHT18yZmTWz954ZMcaglFJKqarBw+kAlFJKKVV+NPErpZRSVYgmfqWUUqoK0cSvlFJKVSGa+JVSSqkqRBO/UkopVYVUyMQvIjeKyAKn43CaiISKyCkR8SzHZYaJiBGRauW1zLIkIptFpN9ZTHdW+6CIeIvIzyISJyLflHR6dyIiv4nI7eW8zBki8nQhw58WkWMicrg846pMRGSfiAwop2W9LSKPlseyypJ9zmxRAeJ4XEQ+K2q8IhO/vRMk2wnosH1g+ZZOmPkzxnxujBlUlsuoiPIecMaYA8YYX2NMppNxOaU0DiZjTHtjzG9FLOeMi51z2AevBhoCAcaYa85i+vzimyoie+1jMFJEvrL7vyMin+QzficRSRWRevaJwIjIPXnGmWz3f7w0YqwMRCQEuB9oZ4xpdI7z6icikaUTWeVR1IVVSRljxhljniqt+WVzOaZP2f/2iciDecZxzW0nRWSOvY9kD58hImku8zglIteVdqxOKO4d/3BjjC/QBegKPFR2IZUdJ+9i3eUOuiSq6PZuCuwwxmSUdML8YhaRfwE3AQPsYzACWGwPngFcKSK18kx2MzDbGHPC7t4B/CufcXacS2yVUFPguDEmxulA3GR7VgZ17OPmauBRERmYZ3h2bmsMHAH+l2f4i/bNV/a/r8oy2PIq3S1RUb8x5jAwH+sCAAARqSEiL4nIARE5YhfdeLsMHyEi/4hIvIjsFpEhdn9/EflARKJFJMougvO0h40RkT/sv98WkZdc4xCRH0XkPvvvJiIyS0SO2ndF97iM97iIfCsin4lIPDAm7zrZcXxiT79fRB4REQ+XOJaLyP/sotttInJJnmkLW4flIvKqiJwAHheR5iLyq4gcF6u48XMRqWOP/ykQCvxsX1n+X947UbGKTp+y55sgIgtEpL5LPDfb63BcRB6VQorsxCqSftkeP05E/nD93YAb7d/0mIg87DJdDxH5U0Ri7fV+U0Squww3IjJBRHYCO+1+r4vIQXsfWCMifVzG9xTrjna3vU5rRCRERJbZo6x3vdIWkcvs/SlWRFaISCeXee0TkQdEZAOQKCLVXLeBHftqO44jIvKKPWn2smLtZZ3vug/a07YXkYUicsKedmo+2/QJ4D/AdfZ8bhMRD3uf2i8iMfa+5m+Pn/373iYiB4Bf8/mpzgPmG2N2g3UMGmPetf/+E4gCrnLdnsANwMcu81gF+IhI++x1Abzt/vnKb/+1+98qIlvFukOaLyJNXaYZaB8jcSLyJiAuw3IVQeazb9cTkY9E5JA97x9cxi3sN+8qImvtfecroGYB6zMAWAg0sX+bGXb/XvY8Y0VkvbhUC4nILfa6JojIHhG50+5fC5jnMq9TYp2Hct0NS55SgQL2z8LOXwXtr3nXra6IzLbncdL+O9hleFHnjZvk9Hnj4fyWYY83FrgR+D97nX+2++cqmXPdDtnbQETut/f/aBG55SzHDRCrGi1eRFaJdb7NOUYLY4xZDWzGJXflGZ4CfAu0K878CiMiF4p1vutvd7eR0+eO7SJyrcu4M0TkLRGZKyKJQH+73zSxSiASRORvEWnuMk2B59NiM8YU+g/Yh3W3ARAMbARedxn+GvATUA/wA34GnrOH9QDigIFYFxlBQBt72A/AO0AtoAGwErjTHjYG+MP+uy9wEBC7uy6QDDSx57kG62RbHWgG7AEG2+M+DqQDI+1xvfNZv0+AH+3Yw7Dugm5ziSMDuBfwAq6z16deMdchA7gbqIZ1om1hb4saQCBWwnktv21td4cBBqhmd/8G7AZa2fP7DXjeHtYOOAVcaG+Ll+x1H1DA7zrNnj4I8AQusOPKXuZ79jI6A6lAW3u67kAve53CgK3AZJf5GqwTbL3s7Q2MBgLsae4HDgM17WH/xtqnWmMlis5YxeTZ82rhMu9uQAzQ0475X/Y2q+Gy/f4BQlyWnbNNgT+Bm+y/fYFe+W3nfPZBPyDajr2m3d2zgO36OPCZS/etwC6sfdMX+A74NM9yP8Hah/LbP0cDJ+ztFAF45hn+MLDIpXswcBTwco0HmAq8YPd7EavU7jPg8QLWYwxn7r8j7XVpa/d7BFhhj18fiMe6s/LCOmYygNsL2C65tjkwB/gK6/j2Ai4q6jfH2s/3c/r4vBprn3+6gHXqB0S6dAcBx4FhWOeHgXZ3oD38UqA51n55EZAEdMtvXna/Ga7Lzmd5+3DZPyn6/JXv/prPegVgXfz5YO2b3wA/uAz/jaLPG33tbfqK/bsVdN7ItY4FHKc549jbIAN40v6Nhtnbse5ZjDvT/udjx30Q+xjNJ84wcu9fvex5XVFAbvPBulj+pLB1Lehf9jbAOv4OAj3s/rXs7luwjpluwDGgvcsy4oDe9v5Q0+53Ait/VgM+B2bmOScUdD59HJfjrMB4i7FC++wdI8FeucVYxSdgHRCJQHOX8c8H9tp/vwO8ms88G2IlE2+XftcDS/I56QpwAOhrd98B/Gr/3RM4kGfeDwEfuWyEZYWsm6cdRzuXfncCv7nEcQj7osPutxKr6LU463CgoGXb44wE1uW3Ixaw8/4GPOIy/C7gF/vv/wBfugzzAdLI5wC2d7BkoHMhB0xwnnUeVcA6TAa+z3MAXFzEep/MXjawHRhR2MHk0v0W8FSecbZzOknsA27NZ//NPriXAU8A9QtY54IS//Wuv1MR6/Y4uRPcYuAul+7WWMmpmstymxUxzxuBRVjH2nHgQZdhofb8gu3uz8l9Yf44VoIPxTqOvOz/Qyg68ec9tuZhXxS77EdJWEXoNwN/uQwTIJJiJH6sYtYs7BN8nmUW+JtjJau8x+cKip/4H8C+CHPpNx/4VwHT/wBMym9edr8ZFJ34b3XpLur8le/+Wox9sAtw0qX7Nwo/b7gmlVoUcN7Ibx0LOE5zxrG3QTK5j60YTl90F2tcrHN1OtDaZdjTFJ34Y+15GqybIdd9ZR9WbovFuuA4BHTMsx4p9vBY4Fgh29zYv93+PPO4Dvg9z7jvAI+5LOOTPMNnAO+7dA8DthWybNfz6eMUI/EXt6h/pDHGz/5h2mBd3YN11+oDrLGLymKBX+z+YJ1cduczv6ZYJ6Bol+newbprzsVYazMT6+QLVjHm5y7zaZI9D3s+U7GScraDhaxXfU7fNWTbj3UnkC3KjsF1eJNirkOuZYtIAxGZKVa1QDzWibc+JePaGjkJ604AO6ac5RljkrCSRH7qY11Z5vfbFLocEWllFyUettfhWc5ch7zrfb9dZBpnbyd/l2kK2kfy0xS4P8/vHYK17vkuO4/bsO56ttlFhZcVc7kliTGvJpy5f1Wj+PsoxmpoOACoA4wDnhSRwfawA1gJYrRYjW5HkruYH5fxdmH9XjuNMYUus4C4mgKvu2z7E1gJPogz9z9T1Hq5CAFOGGNO5jOssN+8Cfkfn8XVFLgmz7wvxLoQQUSGishfdhFtLNYJuKTHa16u26So81ex9lcR8RGroed++5hcBtSR3PXFxT1vZF9clqbjJnebF9flF3fcQKzjxnX7FWf/qm9PPwUrf3nlGT7SGFMHq7RjIrBURFwbfr5kjKlj/yvqt58MfG2M2ejSrynQM89vfCPguoz81qOg36uo82mxlLSOfynW1Uh2nfsxrKup9i4bx99YjSXAWqHmZ86Jg1h3y/VdpqttjGlfwKK/BK4Wqz6xJzDLZT57XeZRxxjjZ4wZ5hp2Iat0DOsqsqlLv1CsetNsQSIieYYfKuY65F32c3a/TsaY2lhFNlLI+CURjVUVA1h1+FjFQfk5hnUlm99vU5S3gG1AS3sdppJ7HcBlPez6pweAa7Hu6OpgFW1lT1PQPpKfg8AzeX5vH2PMl/ktOy9jzE5jzPVYF2cvAN+KVV9b1HYvSYx5HeLM/SsDqyFRTmjFmZExJt0Y8w2wAejgMuhjrDvuq7COh7UFzOITrKLBM54EKGiReboPYlVluW5/b2PMCqz9z7VFtLh2Y5VW+Lh05z3x1RO7vUs+yyzoN48m/+OzuA5i3fG7zruWMeZ5EamBdZ55CWho77dzOb3f5vebFbaO2VynK/T8Vcj+mtf9WCVJPe1jsq/dP+9xmZ+8v5sPBZ838safLYmi1/tcHcU6boJd+oUUMG4uxphMY8zLWOe8uwoZ5zsgE+vi72xcA4wUkcku/Q4CS/P8xr7GmPGuiy/uAopxPi2Ws3mO/zVgoIh0McZkYdUFvyoiDezAgrLvRoAPgFtE5BKxGjkFiUgbY0w0sAB4WURq28Oai8hF+S3QGLMO64d/H6uhU6w9aCUQL1aDGW+xGop1EJHzirMixnpM7mvgGRHxsy8s7sO6E8/WALhHRLxE5Bqs+s25JV0Hmx920ZKIBGHV27o6glXPdza+BYaLyAViNbZ7ggJ2Bvt3+xB4RazGRZ5iNWirUYzl+GHV5Z4SkTbA+GKMn4H1+1UTkf8AtV2Gvw88JSItxdJJRLJPPHm3x3vAOBHpaY9bS0QuFRG/YsSNiIwWkUB7/bP3oUw7tiwK3vazgUZiPQJXw95XehZnmVgXrfeKSLh9R/4s8JUpZqt/sRrZXWov00NEhgLtgb9dRpuFdRJ8gnzu9l18BQzC2ufPxtvAQ3K6kaC/fUyAVUffXkSuFKvB3j3kTgD/AH3FejeFPy5PBtnH0jxgulgN1bxEJDt5Ffab/4m1b90jVkO5K7HqRYvrM6xjZrB9DNQUq4FZMFZJYA3shGNvd9fHO48AAfa6uK7jMLEaKjbCugMsTKHnr0L217z8sG7AYkWkHvBYCbbBt8BlYjVIq45Vv15YXsjvHPUPcIMd/xCsaphSZZ+rv8NqJO1jn3tuLuFsnsdqmHhGA1B73xqB1cZk61mGeQi4BGt/zL7AmA20EqsBpZf97zwRaXuWyyjqfFosJU78xpijWHcM2S9deACrCPEvsYqZFmFdfWKMWYnVqOFVrKuSpZy++7kZ6+DaglVH8S12EVsBvgQGAF+4xJIJDMeq09qLdSf7PlbRR3HdjXWlvgf4w57/hy7D/wZa2vN+BrjaGJNdFFbSdXgCq3FHHNaJ8rs8w58DHhGrSGhKCdYBY8xme11mYl3FJ2DVj6UWMMkUrEZ1q7CKbF+gePvDFKzqlgSsk3JRj7fMxzqp78Aqhk0hd9HWK1iJaAHWBcUHWA2QwKqv+tjeHtcaq2XuHcCbWNt7F/k8qVGIIcBmETkFvI7VbiHFrhZ5BlhuL6uX60TGmASshl/DsYrgdgL9i7nMD4FPsYpf92Kt/90liDkeq1TlANbJ/0VgvDEmpzWzXTybnfw/z28m9njJxphFxpjkEizfdfrvsfaTmfaxvgkYag87hnXH8zxWUXFLYLnLtAux9pUNWA3aZueZ/U1YpW/bsPbbyfZ0Bf7mxpg04Eq7+yRWfWreY6qw9TkIjMDavkex9st/Ax72b34P1r55Emuf/8ll2m1Y56Q99j7TBOt3Xo9Vd7yAIo6NYpy/8t1f85nVa1jHzDHgL6zq1uJug83ABKzzXrS9roW9n+ADoJ29ztlPXkyy1yO7GPuHgiY+RxOxts1hrG39JQWf3/IzB2v97nDp97O9feOxzgH/srfJWbGr1C4BHhCR2+39aBAwCuvC4DDWMVScm6z8FHU+LZbslvIqHyIyBqtx0tkW/TjGvruMxSqS3+t0PEopVZpE5AWgkTHmX07HUtlUyFf2qrMjIsPtYrBaWHWTG7HuPpRSqlIT63n4TnaxfA+sxo/fOx1XZaSJ372MwCpOOoRV1DrKaJGOUso9+GFV5SRiVcG8jPUOFlVCWtSvlFJKVSF6x6+UUkpVIfqhiDJWv359ExYW5nQYSilVaaxZs+aYMSaw6DHV2dDEX8bCwsJYvXq102EopVSlISIleQOjKiEt6ldKKaWqEE38SimlVBWiiV8ppZSqQjTxK6WUUlWIJn6llFKqCtHEr5RSSlUhmvhtIvKhiMSIyKYChouIvCEiu0Rkg4h0K+8YlVJKqXOlif+0GVifwSzIUKz337cExgJvlUNMSimlVKnSxG8zxizD+i59QUYAnxjLX0AdEWlcPtEppZT7i4tLYeh9c50Ow+1p4i++IOCgS3ek3e8MIjJWRFaLyOqjR4+WS3BKKVVZGWP45pvNtO0wnS2emU6H4/Y08Ref5NMv308bGmPeNcZEGGMiAgP1ddNKKVWYNWuiuXbULKrf2Brx1LRU1nQLF18kEOLSHYz13XullFIllJaWyZIlewHo3r0xLR6IAGBElyZOhlUlaOIvvp+Am+3W/b2AOGNMtNNBKaVUZbNs2X66dHmbQYM+Y//+WLo/vYj0LKsA9eVrOjscnfvTr/PZRORLoB9QX0QigccALwBjzNvAXGAYsAtIAm5xJlKllKqcjh1L4v/+byEfffQPYWF1+Oa7axjw7grSM62kv/HxQVTTov4yp4nfZoy5vojhBphQTuEopZRbSUpKp1Ontzh6NImHHrqQQaM7MOYT65PlnYL9+frO86np5elwlFWDJn6llFJlJjIynuDg2vj4ePHMMxfTo0cQAcG16fXcYgCu7xHCc1d2cjjKqkXLVJRSSpW6pKR0HnpoEeHhrzN//i4AbrmlK8HN6vLQdxsAeO/mCE36DtA7fqWUUqVqzpwdTJw4j337YhkzpgvduzchMTWDL1ce4Ok5WwGoV6s6/Vvr485O0MSvlFKq1Nxxx0+8//462ratz9yFo9mYkc5tX61j/cHYnHHuHdCKOy9qpg35HKKJXyml1DnJyMjCw0Pw8BB69w6laVgd5nmmM37RNgAC/WrQIag2vZvX59YLw2lYu6bDEVdtmviVUkqdtVWrohg3bg7X39KZ+JBazD90jCMJqQB4CLx4dWeu7h7scJTKlSZ+pZRSJRYXl8KURxYz8/d91O3agDcjj0HkMQD6tQ6ke2hdbu/TDO/q+oheRaOJXymlVLFlZhle/3I9r6/cj9SqRsCQMABC6/kweUBLrugahEh+nzZRFYUmfqWUUsWy/mAsI6YtB0C8rfTx+qgu9GvVAH8fLydDUyWgiV8ppVSBMjKzeHPxTt5dspskY71ad0j7Rtw7oCWtG9d2ODp1NjTxK6WUyteumFMMeGVpTnfNDMMXd19At6b1HIxKnStN/EoppXKJjkvmoW828Nsuq7EeR5J469ouDB3a0tnAVKnQxK+UUgqA1IxMJn6xjoVbjuT0G+BRnf9NH4S3t9bhuwtN/EoppcjIzKL1I7/kdP/v+q70DqlLvXreDkalyoImfqWUqqKMMcxaG8X8jdEs2hoDApmHEvnjucE0DfJ3OjxVRjTxK6VUFWKMYc+xRB6atZGV+07k9E/eH0/3+r58+uplBAT4OBihKmua+JVSys2lpGey+VAcP6+PZvG2Ixw8kZwzLG3rCepEJfHZa0O58MJQB6NU5UUTv1JKuaGTiWlMW7KL2RuiORyfkmtYh9rePHRNJ3q3rM8//xymfftAvLz01bpVhSZ+pZRyI9+vi+Szvw6wZv/JnH7X9wghLKAWNU9l8MojS5jzzxHGdQqGlvXp0qWRg9EqJ2jiV0qpSurA8SRenL+NY6dSWbP/JOmZJmdYt9A6dAquw0PD2pCUkMbUqYt55501NGnix6xZ13LppfpMflWliV8ppSqJlPRM9hxNZFNUHG8v3c2eY4k5w3qE1yPQrwYhdX24NiKYZoG+OcMGXD6TFSsOMnlyL554oh9+fjWcCF9VEJr4lVKqEvh+XST3frX+jP4vXZP/9+537jxOUFBtfHy8eOGFAXh7V6Nr18blEaqq4DTxK6VUBRWblMZv24/y+uKd7LXv7sdcEEa/1oE0D/QlpN6Zj92lpGTwwgt/8Oyzf/DAA7158sn+XHBBSHmHriowTfxKKVVBGGOISUjl0z/38+P6qFyP3QHMm9SHtoV8EW/Roj3cddccdu48wahRHRg/PqKsQ1aVkCZ+pZRyUHpmFt+sjuSVhds5diot17CB7RpyXlhdru4eQr1a1Qudz/PP/8FDDy2mefO6zJ8/mkGDmpdl2KoS08SvlFIO2BWTwOM/beGP7C/gARe2qE+rhn50b1qXXs3qEeBbeCO8rCxDUlI6vr7VGT68FUlJ6Tz00IX6QR1VKE38SilVxmISUvjy74MkpKSzct8JMrMMmw/F5wwf0aUJjw1vX+Rdvav16w8zbtwcwsLq8OWXV9G+fQOefLJBWYSv3IwmfqWUKkWZWYat0fFEnkxm0dYjbIqKY9vhhJzhAbWqIyIMaNuAi1o34KZeTUs0/1On0nj88d947bW/qFfPmwkTzivtVVBuThO/UkqVgoMnkrj/6/W5PnwDIAI9w+txQ89QLu/cBBE562WsXBnFVVd9TWRkPGPHduO55wboZ3NViWniV0qpEoqKTWb1vhOsOxDL/uOJ7DhyiqjY0y3wr40IZmSXIILqetM0oNY5L88Yg4jQtKk/zZrV5auvrtZH9NRZ08SvlFL5MMYQFZtMclomu4+e4nhiGnM2RLMxMo6E1Ixc4/ZtFcjAdg3pHOLPFV3PfJnO2UpPz+S11/5iwYI9zJ8/moYNfVm6dEypzV9VTZr4lVIK67G6mIRUvvz7AOsOnmTVvpOkZWSdMZ6HwMT+LWjZ0JfzmwcQ6FvjnIrvC7JixUHGjZvNxo0xXH55axISUvH3r1nqy1FVjyZ+FyIyBHgd8ATeN8Y8n2d4KPAxUMce50FjzNxyD1QpdU4yswx/7j7O3E3RHI5L4ddtMWeM4+/tRb9WgQzv3AQvT6FZoC8Na9fEv4wflYuPT2XKlAW8995aQkJq88MP1zFiRJsyXaaqWjTx20TEE5gGDAQigVUi8pMxZovLaI8AXxtj3hKRdsBcIKzcg1VKlcip1AwWbz3Ckm0x7Iw5letROoAuIXVo08iP5oG++NWsxlXdg/Hy9HAkVi8vD5Yt28+UKefz2GP98PUt/iN+ShWHJv7TegC7jDF7AERkJjACcE38Bsh+X6Y/cKhcI1RKFSk5LZMl22P4ef0h/t57goSU9FyfqwVo17g2rRr6MqF/C1o08C2TovqS2L79GM888ztvv30ZPj5erF8/jho19PSsyobuWacFAQdduiOBnnnGeRxYICJ3A7WAAfnNSETGAmMBQkNDSz1QpdRpx0+l8tZvu9l6OJ6/9lgvx8nWu0UAnYLrULOaJ20a+9ElpA4Na1ecevLk5HSee+4PXnhhOT4+XkyYcISePYM16asypXvXafld8ps83dcDM4wxL4vI+cCnItLBGJOrBZAx5l3gXYCIiIi881BKnSVjDF+uPEhMQgp7jyWyfNexXO+37xBUm5C6PkSE1eOyTo0rVJLPa8GC3dx11xx27z7JjTd25OWXB9Gwoa/TYakqQBP/aZGA64OxwZxZlH8bMATAGPOniNQE6gNntgxSSpWKrCzD1sPxPD17K3/uOZ7Tv5qHIALnhdVldK+mDO/UBA8PZ4vsi8sYw1NPLcPDQ1i06CYuuaSZ0yGpKkQT/2mrgJYiEg5EAaOAG/KMcwC4BJghIm2BmsDRco1SqSpgU1QcCzYfZuHWGLZGn26IF1CrOsM6NubBoW2oVcmKwzMzs3j33TWMHNmGxo39+Oqrq6lXz5uaNSvXeqjKT/c4mzEmQ0QmAvOxHtX70BizWUSeBFYbY34C7gfeE5F7saoBxhhjtChfqVKQkp7Jqn0nuOuztTkvyKnr40UT/5oMaNeQyzs3oXvTuo43xDsba9dGM27cbFatOsTJkylMndqHJk38nA5LVVGa+F3Yz+TPzdPvPy5/bwF6l3dcSrmzlPRMxn66hmU7cheezZ/cl9aNKndyTEhI5T//WcIbb6wkMNCHL764klGjOjgdlqriNPErpcpVYmoGB08msWTbUeZsPMSmqNNF+aN7hfKv88No2bByJ/xsU6cuZtq0VYwbF8Gzz15CnToVt7Ghqjo08SulylRqRiZbDsXz3doo5m6M5nhiWq7hEU3rMqh9Q67uHlKi79FXVPv2xZKWlkmrVgE88khfRo/uRM+epff+fqXOlSZ+pVSpOpqQyh+7jhKfnMFXqw6yJTr3W/Ju6BlKeEAtWjfyo3vTupWukV5B0tIyeeWVP3nyyaVceGEoCxbcRMOGvvqInqpw3OOIU0o56nBcCu8u28O8TdFEx6XkGhZSz5s7+jSjQ5A/XUPqVMrGeUX5/ff9jB8/h82bj3LFFW14/fUhToekVIE08SulSiw5LZNpS3axfPcxNkbGkeHytryuoXWY0K8FrRv50aB2DWpU83Qw0rL33XdbueqqrwkN9eenn0YxfHhrp0NSqlCa+JVSxWKM4bO/D/DsnK0kp2fm9B/UriEicE33EAa0a+hghOXHGMOhQwkEBdVm6NAWPP10fyZP7kUtN2ijoNyfJn6lVIEyMrNYtvMoL8zbzvYjCTn9ezWrx8guQQxu34i6VSzZbd16lPHj53DgQBybNt2Fj48XDz/c1+mwlCo2t0z8IlIdCDXG7HI6FqUqk/TMLOZsiOaPXcf4dk1krmFNA3y4qlswt/cJx6e6W546CpWcnM4zz/zOiy8ux9e3Oi++OFDfuqcqJbfba0XkUuAVoDoQLiJdgMeMMVc4G5lSFdcP66J4eeF2Dp5IzukXVMeb+n41uKJLEwa1b0STOt4ORuisyMh4LrpoBnv2nOTmmzvz3/8OpEGDWk6HpdRZcbvEDzyJ9TndJQDGmH9EpIWzISlV8ew7lsj6yFgmzfwnp1/7JrUZ1rExt/QOq5J39Xmlp2fi5eVJkyZ+9O3blPffH07//uFOh6XUOXHHIzvdGBOb55EhfZ++UsCh2GSembuVXUdO5aqzB1h030W0aKDPnIP1QZ3p01fx3/+uYOXKO2jUyJePPhrhdFhKldlvaV0AACAASURBVAp3TPxbReRawMP+0t4k4C+HY1LKMYdik/ni7wN88uc+4lMycvqP6NKEi1oFclGrQAJ8azgXYAWzZs0h7rxzNmvWRDNoUHPSXZ5gUModuGPinwj8B8gCvsP62t5DjkakVDmLS05nU1QcS3cc5d1le3L6X9KmAdedF8KAtg0rzbfry0tWlmHy5F+YNm0VDRrUYubMq7j22vZu+cIhVbW5Y+IfbIx5AHggu4eIXIl1EaCUW0pJz+SPncd49/c9rNx7Itew2jWr8cSI9ozsEqRJrBAeHkJsbAp33RXB009fjL+/flBHuSdxt8/Ji8haY0y3PP3WGGO6OxFPRESEWb16tROLVm7uSHwKS7cf5evVB1m9/2SuYd2b1uWqbsF0CalDs8Ba1PRy77fnna09e04yadIvPPvsxXTs2JCsLKMlIRWAfc6OcDoOd+U2d/wiMhgYAgSJyCsug2pjFfsrVamdTExj2c6j/LAuiiXbc3+7/rywuozu1ZRuoXUJqefjUISVR1paJi+9tIKnnlqGl5cHO3Ycp2NHrf5QVYPbJH4gBtgEpACbXfonAA86EpFSZykjM4u/9pxgxe5jJKZm8NP6Q5xMSs81zqWdGjOkfSMubtPAbb5wVx6WLt3H+PFz2Lr1GFdf3Y7XXhtMUFBtp8NSqty4zdnCGLMOWCcinxtjUoqcQKkKJCU9k01Rcby5ZBer9p4gMS13S/Im/jWJ98jgqREdGNCuAQ38tP75bC1cuIfk5AzmzLmBYcNaOh2OUuXOHev4mwPPAO2AnLOjMaaVE/FoHb8qyL5jiTw1ewvbjyQQefL0G/PC69fi0o6NqVWjGhe1CqRVQ1+qeXo4GGnllpVl+OijdYSG+jNwYHOSk9MxBnx8vJwOTRVA6/jLltvc8buYATwNvAQMBW5B6/hVBZGemcWUb9Yzb9Nh0jKs3bKJf00ubFGfnuH1uDoimMb+VffVuKVt8+YYxo2bwx9/HOCmmzoxcGBzvL014auqzR0Tv48xZr6IvGSM2Q08IiK/Ox2Uqpr2Hkvk1YU7SErL4K89JziVevoFOld2C+Ka7iGc3zzAwQjdU1JSOk89tZSXXvoTf/8afPjh5YwZ08XpsJSqENwx8aeK9bDybhEZB0QBDRyOSVUx03/bxed/HSAq9nQR/nlhdalezYP+rRsw5oIwLb4vQ7NmbeH555dz661deOGFgdSvr086KJXNHRP/vYAvcA9WXb8/cKujEakq4dipVD5ZsY83fj39NegrugZxeecm9G+j155lLTIyns2bYxg8uAU33tiJtm0DiYho4nRYSlU4bpf4jTF/238mADcBiEiwcxEpd5aRmcWirTE8MGsDccmnH7drVr8WX487n/r6Dvwyl5GRxZtvruTRR5fg61udffsmUaNGNU36ShXArRK/iJwHBAF/GGOOiUh7rFf3Xgxo8lelIjYpjT93H+ezv/ezfNfxnP6N/WvywJA2DOvYmOrVtBi/PKxcGcW4cbNZt+4wQ4a0YNq0YdTQdxooVSi3OUJE5DngKmA9VoO+77G+zPcCMM7J2JR7SM3IZPxna/l1W0yu/rf0DuPOvs1ppO92L1c7dhynV6/3adzYj2++uYarrmqr3yJQqhjcJvEDI4DOxphkEakHHLK7tzscl6qEUtIzWbIthhNJaazce4Lo2BRW7rM+fuNboxq39A7jhp6h+uhdOTPGsHFjDJ06NaRVqwBmzBjJyJFtqF1bq1SUKi53SvwpxphkAGPMCRHZpklflUTkySTeW7aHr1dHkpzPN9iHd25Cj7C6XN8jVFvkO2DXrhNMmDCXX3/dy4YN42jbNpCbb+7sdFhKVTrulPibiUj2p3cFCHPpxhhzpTNhqYoqKjaZ/ccSeWnBdjZExpGRZb3FskUDX1o38qNjkD8Xt2lAHW8v6vvW0A+4OCQ1NYMXX1zOM8/8TvXqnrz66mBatdJ3Hyh1ttwp8V+Vp/tNR6JQFV5qRiYj3lzOtsMJOf18a1RjWMdG3NQrjI7B/g5Gp1ylp2dy3nnvsXFjDNdd155XXhlMkyZ+ToelVKXmNonfGLPY6RhUxXYiMY0Zy/fmes7+7dHdadnQl+aBvg5GpvKKj0+ldu0aeHl5ctttXWnduj5DhrRwOiyl3ILbJH6l8rPjSAJvLN7Jyr0niElIzek/6ZKWTB7QUluBVzBZWYYPPljLAw8s4osvrmLIkBZMmtTL6bCUciua+G0iMgR4HfAE3jfGPJ/PONcCjwMGWG+MuaFcg1TFtvbASZ6ds5XV+0/m9OvXOpAbeoQSEVaPerWqOxidys+GDUcYN242f/4ZyUUXNSUsrI7TISnlltw28YtIDWNMatFjgoh4AtOAgUAksEpEfjLGbHEZpyXwENDbGHNSRPQdrBVMemYW/1u8k69XR3I4PgUAn+qevHdzBL1b1Hc4OlWYZ55ZxmOP/Ubdut58/PFIbrqpk5bGKFVG3C7xi0gP4AOsd/SHikhn4HZjzN2FTNYD2GWM2WPPYybWewG2uIxzBzDNGHMSwBgTc8ZcVLnLzDI8PWcLS7cfZc+xxJz+bRvX5u3R3WgaUMvB6FRRjDGICA0b+nLLLV14/vkBBAToB3WUKktul/iBN4DLgB8AjDHrRaR/EdMEAQdduiOBnnnGaQUgIsuxqgMeN8b8UioRq7Py2/YYxny0CoBqHsLoXqF0CanLVd2C9G6xgjt4MI577vmFwYObM25cBLff3o3bb+/mdFhKVQnumPg9jDH785z4z3wbS275ZQmTp7sa0BLoh/Xe/99FpIMxJvaMmYmMBcYChIaGFjNsVVxr9p/g399uYM9R6w6/S0gdvr/rAk32lUBGRhZvvPE3//nPErKyDAMGhDsdklJVjjsm/oN2cb+x6+7vBnYUMU0kEOLSHYz1yt+84/xljEkH9orIdqwLgVV5Z2aMeRd4FyAiIiLvBYQ6B+//voen52wFrI/ivDW6O11CtBFYZbB69SFuv/0n1q8/wqWXtuTNN4dpAz6lHOCOiX88VnF/KHAEWGT3K8wqoKWIhANRwCggb4v9H4DrgRkiUh+r6H9PKcatChGXnM5zc7cyc5VVI/PWjd0Y2rGxw1Gpkjh5Mpljx5KYNetarriijZbQKOUQd0z8GcaYUSWZwBiTISITgflY9fcfGmM2i8iTwGpjzE/2sEEisgWr6uDfxpjjBc9VnYv0zCz+3nOCGSv2EXkyKddb9r6+83x6hNdzMDpVHMYYvvhiI5GR8TzwwIUMHNicXbvuoWZNdzztKFV5iDHuVRItIruB7cBXwHfGmIQiJilTERERZvXq1U6GUKkkpWXw3/nb+Wj5vlz9R3RpQscgf27tHa7vzK8Eduw4zl13zWHx4r307h3C0qVj8NQPG6liEpE1xpgIp+NwV2536W2MaS4iF2AV1z8hIv8AM40xMx0OTeWRnpnFxqg4/t5zggMnEvlrzwn2ujySd2nHxtw/qBXN9HW6lUZKSgbPP/8Hzz33B97e1Zg+fRhjx3bXpK9UBeJ2iR/AGLMCWCEijwOvAZ8DmvgriOOnUnlg1kYWbT1yxrB2jWvTIag2Dw9rh7+PlwPRqXOxb18szz77O9dc056XXx5Eo0Z60aZUReN2iV9EfLFevjMKaAv8CFzgaFAKgGlLdvHDuih2xpzK6Te6VyhXdQumfRN/qlfTu8LK6PDhU8yatYUJE3rQpk19tm+fSHh4XafDUkoVwO0SP7AJ+Bl40Rjzu9PBKEhJz2Twa8vYfzwJgGEdG3FeWD3GXBCmLbsrsawswzvvrOahhxaTnJzBsGEtCQ+vq0lfqQrOHRN/M2NMltNBKKtV98sLdvDmEuszuOH1a/HDXb21CN8N/PPPYcaNm83ff0dx8cXhTJ8+TBO+UpWE2yR+EXnZGHM/MEtEznhUwRhzpQNhVUmpGZm8unAnby/dndNPP4PrPpKT0xk06FMAPv30Cm68saP+rkpVIm6T+LEe3wN409EoqrCU9EwmzVzH/M2nG+0F1fFm7qQ++HvrXX5lZozh11/30r9/ON7eXsyadS0dOjSgbl1vp0NTSpWQ2yR+Y8xK+8+2xphcyd9+Oc/i8o+q6ohLSmfwa8tyPod7Z99m3DeoFTWqeTocmTpX+/fHcvfd8/j55x18/vmV3HBDR/r0aep0WEqps+Q2id/FrZx5139bPv1UKdgUFcdri3bmPJrXMcifWeMv0Bb6biA9PZNXX/2LJ55YCsBLLw3kmmvaORyVUupcuU3iF5HrsB7hCxeR71wG+QFnfEFPnZu9xxIZ9+kath85/WLERy5ty20Xhmt9r5u49tpv+eGHbVx+eWv+97+hhIb6Ox2SUqoUuE3iB1YCx7G+rDfNpX8CsM6RiNyQMYYPl+/jqdlbAOuTuE+OaE/HIH9N+G7gxIlkatasho+PF5Mm9WTMmM6MGNHG6bCUUqXIbRK/MWYvsBfra3yqlKWkZzL+szUs2X40p9+9A1oxaUBLB6NSpcUYw2efbeD++xdw661def75AfTrF+Z0WEqpMuA2iV9ElhpjLhKRk4Dr43wCGGOMfs7tLCSlZTDxi3X8ui0mp99V3YJ5+NK21KtV3cHIVGnZtu0Y48fP4bff9tGrVzDXX9/B6ZCUUmXIbRI/0N/+v76jUbiJORuimfDF2lz97h/YiokXt9AifTfy8cf/cMcdP1OrVnXeeecybr+9m379UCk35zaJ3+VtfSHAIWNMmohcCHQCPgPiHQuuEsnMMkz5Zj3fr4sCoFOwP9edF8KVXYPxrq6P5rmL9PRMvLw8iYhowvXXd+TFFwfQsKF+UEepqkCMOeMld5Wa/Rne84BQYCEwBwg3xlzmRDwRERFm9erVTiy6xHbFJDDglWU53V/feT49wrWGxJ1ERydw330LMMYwc+bVToejVL5EZI0xJsLpONyV29zxu8gyxqSLyJXAa8aYN0REW/UXIiU9k4lfrGXRVqseP7iuN4vvv0hfvuNGMjOzePvt1Uyd+iupqRlMndoHY4xW2yhVBblj4s8QkWuAm4CRdj99X2wB1uw/yVVvrcjpnnHLefRr3cDBiFRp27HjOKNHf8eqVYcYMKAZ06cPo2XLAKfDUko5xB0T/63AXVif5d0jIuHAlw7HVCF9vfog//ftBgDuG9iKu7Xhnlvy969BQkIaX3xxJaNGddDfWKkqzu3q+AFEpBrQwu7cZYzJcCqWilbHn56ZxdyN0Uya+U9Ov3du6s7g9o0cjEqVJmMM3323lS+/3MTXX1+Dh4eQlWW0tb6qNLSOv2y53R2/iPQBPgWisJ7hbyQiNxljljsbmfPiU9Lp9PiCnO5GtWsy554LCfCt4WBUqjTt3XuSiRPnMXfuTrp0aURMTCKNGvlq0ldK5XC7xA+8CgwzxmwBEJG2WBcCVfrq8eCJJPq8uCSne+2jA/UFPG4kLS2TV175kyefXIqnpwevvjqYiRN7UE0/lqSUysMdE3/17KQPYIzZKiJVNsNlZRleX7yT1xfvBODq7sH89+pOWs/rZjIzs3j//bUMHdqS118fQnBwbadDUkpVUO6Y+NeKyDtYd/kAN1JFP9JzNCGVK6YvJ/JkMgAT+jfn34P1gyvu4tixJF544Q+eeKI/Pj5erFx5B/XqeTsdllKqgnPHxD8OuAf4P6w6/mXA/xyNyAELNh9m7KdrAOjXOpAP/nUenlrP6xaMMXz88XqmTFlAXFwql1zSjCFDWmjSV0oVi1slfhHpCDQHvjfGvOh0PE6JiU/JSfovXdOZq7sHOxyRKi1bthxl/Pg5LFu2nwsuCOHtty+lY8eGToellKpE3Kblj4hMBX7AKtpfKCK3OhySI4wxXP/eXwDc0Sdck76bueeeeWzceIT33hvO77/foklfKVVi7nTHfyPQyRiTKCKBwFzgQ4djKldLtsfwyPebiIpNpm+rQKYOa+t0SKoU/PLLLjp3bkjjxn68995wfH2rExhYy+mwlFKVlNvc8QOpxphEAGPMUdxr3Yq0ZFsMt3y0iqjYZK7qFszbo7tpy/1K7tChBK699huGDv2cl16yXqscHl5Xk75S6py40x1/MxH5zv5bgOYu3RhjrnQmrLKVmWUY+8lqFm+zPrDz3s0RDGynxb+VWWZmFtOnr+Lhh38lPT2Lp5/uz5QpFzgdllLKTbhT4r8qT/ebjkRRzl5duCMn6f84oTedQ+o4HJE6V088sZSnnlrGoEHNmT59GM2b66eRlVKlx20SvzFmsdMxlLf0zCzeXLILgJ3PDMXLs0rVbriVuLgUTp5MISysDhMmnEf79oFce217ra5RSpU6zRSV2G0fWx//Gd0rVJN+JWWM4euvN9O27TRuuul7jDE0bOjLddfpV/SUUmVDs4ULERkiIttFZJeIPFjIeFeLiBERx97/Hx2XzLIdRwF4fHh7p8JQ52DPnpMMG/YF1133LY0b+/Hqq4M12SulypzbFPXnJSI1jDGpJRjfE5gGDAQigVUi8pPre//t8fyw3gz4d2nGWxJbo+MZ+vrvAEwd1oZqerdf6Sxduo8hQz7Hy8uD118fwoQJ5+Gpv6NSqhy43ZlGRHqIyEZgp93dWUSK88reHsAuY8weY0waMBMYkc94TwEvAimlFXNJ3TpjFQAPDW3D2L7NnQpDnYX4eOtatEePIG6/vStbt07gnnt6atJXSpUbdzzbvAFcBhwHMMasB/oXY7og4KBLd6TdL4eIdAVCjDGzC5uRiIwVkdUisvro0aMlib1Ip1IziI5LoZqHcOdFmvQri6NHExkz5ge6dn2HpKR0vL29+N//hhEUpF/RU0qVL3dM/B7GmP15+mUWY7r8KldNzkARD+BV4P6iZmSMedcYE2GMiQgMDCzGootn+a5jdHhsPoC+la+SyMoyfPDBWtq0mcYXX2xk1Kj2aDW+UspJ7ljHf1BEegDGrre/G9hRjOkigRCX7mDgkEu3H9AB+M1ugNUI+ElELjfGrC6VyAthjOHG961mBRP6N+fWC8PLepHqHB0/nsTIkV/xxx8H6NMnlLffvox27UrvQlAppc6GOyb+8VjF/aHAEWCR3a8oq4CWIhIORAGjgBuyBxpj4oD62d0i8hswpTySPsCK3ccBaOBXg38PblMei1RnyRiDiFC3rjcBAd58+OHljBnTRVvsK6UqBLdL/MaYGKykXdLpMkRkIjAf8AQ+NMZsFpEngdXGmJ9KOdRi2xAZm3O3P/ueC50KQxXDnDk7ePjhX/nll9E0auTLDz+UeFdUSqky5XaJX0Tew6VuPpsxZmxR0xpj5mJ91c+1338KGLffWYZYYpe/uRyARy5tSwO/muW1WFUCkZHxTJr0C999t5W2betz9GgijRr5Oh2WUkqdwe0SP1bRfraawBXkbq1fqeyKOQXABc0DuL1PM4ejUXkZY3j99b959NElZGZm8dxzl3DffedTvbqn06EppVS+3C7xG2O+cu0WkU+BhQ6Fc84mfrEWgLF9NelXRCLCqlWH6NMnlGnThhEeXtfpkJRSqlBul/jzEQ40dTqIs/Hz+kNsO5wAwEWttDV4RREbm8Ijj/zK2LHd6dSpIR98cDk1anhq4z2lVKXgdolfRE5yuo7fAzgBFPje/Yrsx3+iAFh0X19NKhWAMYaZMzdx773zOXo0iTZt6tOpU0Nq1nS7w0gp5cbc6owlVnbsjPU4HkCWMeaMhn6VQVJaBou2xhBevxYtGvg5HU6Vt3PncSZMmMvChXuIiGjCnDk30L17E6fDUkqpEnOrxG+MMSLyvTGmu9OxnKt3l+0B4PoeIUWMqcrDJ5+s5++/o3jzzaGMGxeh79ZXSlVabpX4bStFpJsxZq3TgZytrCzDjBX76N60rn6Ex0FLluxFROjXL4ypU/swfvx5NGmipS9KqcrNbRK/iFQzxmQAFwJ3iMhuIBHrHfzGGNPN0QBL4Id/oohNSmdI+0ZOh1IlxcQkMmXKAj79dAMDBjSjX78wvL298Pb2cjo0pZQ6Z26T+IGVQDdgpNOBnKv7vl4PwOVdtA65PGVlGd5/fy0PPLCIxMQ0HnmkD1On9nE6LKWUKlXulPgFwBiz2+lAzsWTP28BoGmADw1r61v6ytMPP2zjzjtnc9FFTXn77cto06Z+0RMppVQl406JP1BE7itooDHmlfIM5mwkp2Xy4fK9ACy67yKHo6kaEhPTWL/+CBdcEMLIkW348cdRDB/eSh+fVEq5LXdK/J6AL/adf2U0Z2M0AFMGtcJLW42XuZ9+2s7dd88jPj6VAwcm4+dXg8svb+10WEopVabcKfFHG2OedDqIs5WVZXh14Q4ALu2kdftl6eDBOO6+ex4//ridDh0a8MUXV+LnV8PpsJRSqly4U+KvtHf6AG/8upOo2GR6NatHeP1aTofjtqKi4mnXbjqZmVm88MIA7r23F15e+kEdpVTV4U6J/xKnAzgXm6LiAJhxSw+HI3FPUVHxBAXVJiioNk891Z+RI9sQFlbH6bCUUqrcuU1FsjHmhNMxnIu/9pzgguYB1NS7z1J18mQy48bNplmzN9i48QgAkyf30qSvlKqy3OmOv9LKyMziVGoGPtX15ygtxhg+/3wj99+/gOPHk5g0qacme6WUQhN/hRCXnA5Am0b6OtjSkJVluOyyL5g3bxc9egQxf/5ounTRtyAqpRRo4q8Q1kfGAmijvnOUnp6Jl5cnHh5C375NGT68FWPHdtcP6iillAs9I1YA05ZYLxsc1L6hw5FUXgsX7qZdu+n88ssuAB588ELGjz9Pk75SSuWhZ0WHHYpNZs3+k9T3rYFfTf0ITEkdPnyKG26YxaBBnwHg46PbUCmlCqNF/Q77fl0UAE9c3t7hSCqfTz5Zzz33zCM5OYPHHruIBx+8kJo1dZdWSqnC6FnSYTuOJAAwrKM2PiuptLRMundvwvTpw2jdWj+oo5RSxaFF/Q7bczQRQD8KUwwJCancf/983ntvDQC33daVRYtu0qSvlFIloInfYfuPJxJUx9vpMCo0Ywzff7+Vdu2m88orf7Fzp/WuJhHRCyallCohLep3kDGG+JQMOofoi2UKsn9/LBMnzmP27B107NiAr7++mvPPD3E6LKWUqrQ08Tto2c5jAHQI8nc4koprx47jLFmyl5deGsg99/TUD+oopdQ50sTvoHkbowG4oUeow5FULCtWHGTt2mgmTuzBwIHN2b9/MgEBPk6HpZRSbkHr+B00c9VBAmpVJ6SeJjWAEyeSGTv2Z3r3/pBXX/2LlJQMAE36SilVijTxOyQ2KQ2A5oG+DkfiPGMMn3yyntat3+TDD9cxZcr5rF8/Tp/JV0qpMqBnVodsORQPwMB2+prevXtjue22n4iIaMI771xGp066TZRSqqzoHb9DDpxIAqB7WF2HI3FGcnI6M2duAqBZs7osX34ry5ffqklfKaXKmCZ+m4gMEZHtIrJLRB7MZ/h9IrJFRDaIyGIRaXouyzuVatVfN62C9fsLFuymY8e3uP76WaxffxiAHj2C8PDQZ/KVUqqsaeIHRMQTmAYMBdoB14tIuzyjrQMijDGdgG+BF89lmYu2HgGgVo2qU9sSHZ3AqFHfMnjwZ3h6erB48c107qyvKlZKqfJUdbJO4XoAu4wxewBEZCYwAtiSPYIxZonL+H8Bo892YemZWWyNtt7RX7OKPJeekZHFBRd8SHR0Ak8+2Y//+7/e1KhCFz1KKVVR6JnXEgQcdOmOBHoWMv5twLyCBorIWGAsQGjomc/oz95wiLjk9CrxRb7Nm2No2zaQatU8mD59GC1a1KNlywCnw1JKqSpLi/ot+VUum3xHFBkNRAD/LWhmxph3jTERxpiIwMDAM4Znf5hnZJegswq2MoiPT2Xy5F/o1OltPvxwHQBDh7bUpK+UUg7TO35LJOD6Avhg4FDekURkAPAwcJExJvVsF7bzyCkA/H28znYWFZYxhlmztjJp0i9ERycwfnwEV1+dt7mEUkopp2jit6wCWopIOBAFjAJucB1BRLoC7wBDjDEx57KwTGOoXs09C1vuvnse06atokuXRnz33bX07BnsdEhKKaVcaOIHjDEZIjIRmA94Ah8aYzaLyJPAamPMT1hF+77AN/anYA8YYy4/m+XtOJLA+c3cp8g7LS2TrCxDzZrVuOKKNjRvXpe77+5JNTe9uFFKqcpME7/NGDMXmJun339c/h5QGss5mZjG/uNJ9G/doDRm57jff9/PuHFzGD68Fc8/P4BLLmnGJZc0czospZRSBdBbsnK2M8aq3+/etHK/se/YsSRuvfVH+vadQWJiGn366BcGlVKqMtA7/nJ2JD4FgPD6tRyO5Oz98ssuRo/+jri4VB54oDePPtqXWrWqOx2WUkqpYtDEX84WbrHe2NfYv6bDkZScMQYRITTUn06dGvLGG0Pp0ME9qiyUUqqq0MRfzk4mpVGjmgcBvjWcDqXYkpLSefrpZURGxvPJJ1fQrl0gv/76L6fDUkopdRa0jr+c/XMwtlLV78+bt5MOHabz3HN/4OEhZGRkOR2SUkqpc6B3/OXoaEIqCSkZNPb3djqUIh05coqJE+fx7bdbaNOmPkuW/It+/cKcDksppdQ50sRfjvYft17V27tFxX+GX0RYseIgTz/dn3//uzfVq1eNjwkppZS708RfjlbvPwlAuya1HY4kf6tXH+K999bw1luX0aBBLXbvvoeaNXUXUUopd6J1/OVoQ2Qs1T09aNOoYiX+uLgU7r57Lj16vMfPP+9g717rAkWTvlJKuR89s5ejuOR02gdVnKRvjOGbb7YwefIvHD58igkTzuPppy/GvxI+aqiUUqp4NPGXo+2HT9GmkZ/TYeRIS8tk6tTFNG7sx08/XU9ERBOnQ1JKKVXGNPGXo9T0TBLTMpyNITWDt95azdix3fHx8WLRopsJDq6tH9RRSqkqQhN/OYlJSCEhNYOuIc49w//bb/sYP34O27YdIzDQhxtv7ERYWB3H4lFKKVX+NPGXBwBoOwAAFAFJREFUk49X7AOggwN1/EePJvLvfy/k44/XEx5ehzlzbmDYsJblHoeq+NLT04mMjCQlJcXpUFQVULNmTYKDg/Hy8nI6lCpFE3852X44AYCRXYLKfdm33/4z8+btZOrUC3n44b74+OhBpvIXGRmJn58fYWFhiIjT4Sg3Zozh+PHjREZGEh4e7nQ4VYpW7JaT6LgUwuvXwsOjfE6mmzbFEB1tXWz8978D+eefcTzzzCWa9FWhUlJSCAgI0KSvypyIEBAQoKVLDtDEX05S0jMpj5yfmJjGgw8uomvXd3j00SUAtGoVQLt2gWW/cOUWNOmr8qL7mjO0qL8cZGRmsftoIjf0DC3T5cyevYOJE+eyf38ct97ahRdeGFCmy1NKKVX56B1/OdgQFQdA80DfMlvGG2/8zfDhX1KrVnWWLRvDBx+MICDAp8yWp1RZ8fT0pEuXLnTo0IHhw4cTGxubM2zz5s1cfPHFtGrVipYtW/LUU09hjMkZPm/ePCIiImjbti1t2rRhypQpTqxCodatW8ftt9+eq9+IESM4//zzc/UbM2YM33777f+3d+/RUVX3Ase/PwmSIAEFSi4tlYBESAghBGjhUm0gJloeQVGJaBW8lJYgaPGSFpesRYsuGxCFInKp7XUFLyWJL4QbWysvLUVeAUKISY25mAaUhhQCBElCHvv+cU4mQzKYCcnMaOb3WWvWmnNmn3N+81uT7Nn77Nn7in3dujX+DyksLGTixIkMGjSI8PBwpk+fTmlpaZtiO3v2LPHx8YSFhREfH095ebnLciUlJSQkJBAeHk5ERATFxcUA7Ny5k5iYGCIjI5k5cya1tdbPl7Oysli6dGmbYlPtRyt+L/jwkzIAJkf1bdfz1tbW889/XgQgKWkoK1bcwZEjP+O22/q363WU8qagoCBycnLIy8ujZ8+evPzyywBUVlaSmJjI4sWLKSws5OjRo3z00UesW7cOgLy8PObPn8/GjRspKCggLy+PgQMHtmtsDRVZWzz33HMsWLDAsX3u3DkOHz7MuXPn+Oyzz9w6R1VVFZMmTSI5OZmioiIKCgpITk6mrKysTbGlpqYSFxfHp59+SlxcHKmpqS7LPfLII6SkpFBQUMCBAwfo06cP9fX1zJw5k4yMDPLy8ujfvz8bNmwAYNKkSWzdupVLly61KT7VPrSr3wv+dbGanjdcT0j39psK98CBz/nZz7IIDAxgz57/ICSkGykp49rt/Er9+n8/Jv+LC+16zohvd2fplKFulx87diy5ubkAbNq0iXHjxpGQkABA165dWbt2LbGxsTz22GOsWLGCp59+miFDhgAQEBDAvHnzmp3z4sWLLFiwgOzsbESEpUuXcu+999KtWzcuXrS+SL/55ptkZWWRlpbGrFmz6NmzJ0eOHCE6OprNmzeTk5PDjTdac2AMGjSIPXv2cN111zF37lxKSkoAWL16NePGXfk3WVFRQW5uLsOHD3fse+utt5gyZQohISFkZGTw1FNPtZiXTZs2MXbsWKZMmeLYN378eLfzejVbtmzhgw8+AGDmzJnExsayfPnyK8rk5+dTW1tLfHw80NgLUVZWRpcuXbj11lsBiI+P5ze/+Q2zZ89GRIiNjSUrK4vp06e3OU7VNtri94KPv7hA93Za8ObcuSrmzXuXMWP+wOnTX7Jo0Vh0fIzqiOrq6tixYweJiYmA1c0/cuTIK8rccsstXLx4kQsXLpCXl9fsdVeeeeYZevTowbFjx8jNzWXChAktHlNYWMj27dtZtWoVU6dOZfPmzQDs37+f0NBQQkJCeOKJJ1i4cCEHDx7krbfeatadD5CdnU1kZOQV+9LT05kxYwYzZswgPT29xVgAt99rRUUF0dHRLh/5+fnNypeWltK3r9Uz2bdvX06fPu0yFzfeeCPTpk1jxIgRpKSkUFdXR+/evampqSE7OxuwvjydOHHCcdyoUaPYvXu3W+9PeZa2+L2grKK6XSbuOXr0n9x550bKyi7x+OPfZ9my8XTv3qUdIlSquda0zNtTZWUl0dHRFBcXM3LkSEfL0hhz1VHgrRkdvn37djIyMhzbN93U8mya999/P506dQIgKSmJZcuW8eijj5KRkUFSUpLjvM6V6YULF6ioqCA4uHF9jlOnTvGtbzX+wqa0tJSioiJ+8IMfICIEBASQl5dHZGSky/fU2lHwwcHB5OTktOqYltTW1rJ7926OHDnCzTffTFJSEmlpacyePZuMjAwWLlxIdXU1CQkJBAQ0VjF9+vThiy++aNdY1LXRFr8XfH6ukm8FX3sFXVNTB1g/y7v99v4cPDiH1avv0kpfdUgN9/j/8Y9/cPnyZcc9/qFDhzpakw2OHz9Ot27dCA4OZujQoRw6dKjF81/tC4Tzvqa/Lb/hhhscz8eOHUtRURFlZWW88847TJs2DYD6+nr27t1LTk4OOTk5fP7551dU+g3vzfncmZmZlJeXM2DAAEJDQykuLnZ8KenVq9cVg+vOnj1L7969Hblw5722tsUfEhLCqVOnAOtLSp8+fZqV6devHyNGjGDgwIEEBARw9913c/jwYUdudu/ezYEDB7j99tsJC2ucIbSqqoqgoKAWY1aepxW/h1Vetirt6+3WQmtUV9eybNmHREWt59KlGoKCOvP66/cTE9O+gwSV+jrq0aMHa9asYeXKldTU1PDQQw/xt7/9je3btwNWz8Djjz/OL37xCwBSUlJ47rnnKCwsBKyK+MUXX2x23oSEBNauXevYbqhcQ0JCKCgooL6+3tGV74qIcM899/Dkk08SHh5Or169XJ7XVUs7PDycoqIix3Z6ejrvvfcexcXFFBcXc+jQIUfFHxsbS2ZmJpcvXwYgLS3NcR//wQcf5KOPPuLdd991nOu9997j2LFjV1yvocXv6hEREdEsvsTERMeAvA0bNjB16tRmZUaPHk15ebljIOHOnTsd52q4NVBdXc3y5cuZO3eu47jCwsJmtzmUb2jF72HnKmsAmBbTuql6d+78jKio9Sxd+gHR0f9GVZVvV/VTyhdGjBjB8OHDycjIICgoiC1btvDss88yePBghg0bxujRo5k/fz4AUVFRrF69mhkzZhAeHk5kZKSj9epsyZIllJeXExkZyfDhw9m1y5roKjU1lcmTJzNhwgTHfe6rSUpKYuPGjY5ufoA1a9aQnZ1NVFQUERERrF+/vtlxQ4YM4fz581RUVFBcXExJSQljxoxxvD5gwAC6d+/O/v37mTx5MrfddhsjR44kOjqaPXv2OAbaBQUFkZWVxUsvvURYWBgRERGkpaW5bKG3xuLFi9m2bRthYWFs27aNxYsXA9bYhIYxC506dWLlypXExcUxbNgwjDHMmTMHgOeff57w8HCioqKYMmXKFeMndu3axaRJk9oUn2of4vwbWNX+eoWGm5FPrOf9hT90q/yXX15m7tx32bgxl4EDb2LduonceecgD0eplKWgoIDw8HBfh9GhrVq1iuDgYJeD/zqq0tJSHnzwQXbs2NHsNVefORE5ZIwZ5a34/I22+D3sYnUtYX2CWy5oCwrqzKlTFSxZcht5ecla6SvVwSQnJ9Oli3+NzykpKeGFF17wdRjKpqP6vaClhXlyc0v55S+38+qrifTtG8z77z/stcV8lFLeFRgYyMMPP+zrMLxq9OjRvg5BOdEWvxcMDnE9Ve/Fi5dJSXmfmJjfkZ39BYWFZ4CWvygo5Ul6+095i37WfENb/F6QNLr54jxbt37C/Pl/4sSJC8yZE0Nq6h307Kk/dVG+FRgYyJkzZ3RpXuVxxhjOnDlDYGD7zWiq3KMVvxfc1LVzs31vvJFPjx6BpKffy7hxnl21Tyl39evXj5MnT7Z5znel3BEYGEi/fv18HYbf0VH9Htalb5ipPvUpNTV1rFmzn/j4W4iKCuHChWqCggLo3Ln1v+9XSqmOTEf1e5be43ciIneJyCciUiQii1283kVEMu3X94tIaEvn7Hzddezde4JRo37PokXbyMzMA6B79y5a6SullPI6rfhtItIJeBn4ERABzBCRplNbzQbKjTGDgFXAclpQV1fPuHGvcvZsJW+/PZ1nn215QRCllFLKU7Tib/Q9oMgYc9wYcxnIAJrOVzkV2GA/fxOIkxZGQNVcqmHhwjHk58/jnnvCdcCUUkopn9LBfY2+A5xw2j4JfP9qZYwxtSJyHugF/Mu5kIj8FPipvVn94ot35bmYMtwf9aZJrvyY5qKR5qKR5sIy2NcBdGRa8Tdy1RRvOvLRnTIYY14BXgEQkWwdpGLRXDTSXDTSXDTSXFhEJLvlUupaaVd/o5PAd522+wFNF492lBGRAKAHcNYr0SmllFLtQCv+RgeBMBEZICLXAw8AW5uU2QrMtJ/fB+w0+ntIpZRS3yDa1W+z79nPB/4CdAJeNcZ8LCLLgGxjzFbgv4H/EZEirJb+A26c+hWPBf3No7lopLlopLlopLmwaB48SCfwUUoppfyIdvUrpZRSfkQrfqWUUsqPaMXfDjwx1e83lRu5eFJE8kUkV0R2iEh/X8TpDS3lwqncfSJiRKTD/ozLnVyIyHT7s/GxiGzydoze4sbfyM0isktEjth/JxN9Eac3iMirInJaRPKu8rqIyBo7V7kiEuPtGDskY4w+2vDAGgj4f8BA4HrgKBDRpMw8YL39/AEg09dx+zAX44Gu9vNkf86FXS4Y+CuwDxjl67h9+LkIA44AN9nbfXwdtw9z8QqQbD+PAIp9HbcH83E7EAPkXeX1icCfseZQGQPs93XMHeGhLf6288hUv99QLebCGLPLGHPJ3tyHNV9CR+TO5wLgGWAFUOXN4LzMnVzMAV42xpQDGGNOezlGb3EnFwbobj/vQfP5RDoMY8xf+eq5UKYCrxnLPuBGEenrneg6Lq34287VVL/fuVoZY0wt0DDVb0fjTi6czcb6Nt8RtZgLERkBfNcYk+XNwHzAnc/FrcCtIrJHRPaJyF1ei8673MnFr4Afi8hJ4E/AAu+E9rXU2v8pyg36O/62a7epfjsAt9+niPwYGAX80KMR+c5X5kJErsNa4XGWtwLyIXc+FwFY3f2xWL1Au0Uk0hhzzsOxeZs7uZgBpBljXhCRsVhzh0QaY+o9H97Xjr/87/QqbfG3nU7128idXCAidwBPA4nGmGovxeZtLeUiGIgEPhCRYqz7l1s76AA/d/9GthhjaowxnwGfYH0R6GjcycVs4HUAY8xeIBBr8R5/5Nb/FNU6WvG3nU7126jFXNjd27/DqvQ76n1caCEXxpjzxpjexphQY0wo1niHRGNMR1ycxJ2/kXewBn4iIr2xuv6PezVK73AnFyVAHICIhGNV/GVejfLrYyvwiD26fwxw3hhzytdBfdNpV38bGc9N9fuN42Yunge6AW/Y4xtLjDGJPgvaQ9zMhV9wMxd/ARJEJB+oA1KMMWd8F7VnuJmL/wR+LyILsbq1Z3XQhgIiko51e6e3PaZhKdAZwBizHmuMw0SgCLgEPOqbSDsWnbJXKaWU8iPa1a+UUkr5Ea34lVJKKT+iFb9SSinlR7TiV0oppfyIVvxKKaWUH9GKX6k2EpE6EclxeoR+RdnQq61E1sprfmCv8HbUnuZ28DWcY66IPGI/nyUi33Z67Q8iEtHOcR4UkWg3jvm5iHRt67WVUq5pxa9U21UaY6KdHsVeuu5DxpjhWAtAPd/ag40x640xr9mbs4BvO732E2NMfrtE2RjnOtyL8+eAVvxKeYhW/Ep5gN2y3y0ih+3Hv7soM1REDti9BLkiEmbv/7HT/t+JSKcWLvdXYJB9bJy9jvsxe63zLvb+VHut+1wRWWnv+5WILBKR+7DWTfijfc0gu6U+SkSSRWSFU8yzROSla4xzL04LrIjIf4lItoh8LCK/tvc9jvUFZJeI7LL3JYjIXjuPb4hItxauo5T6ClrxK9V2QU7d/JvtfaeBeGNMDJAErHFx3Fzgt8aYaKyK96Q9RWsSMM7eXwc81ML1pwDHRCQQSAOSjDHDsGbmTBaRnsA9wFBjTBTwrPPBxpg3gWyslnm0MabS6eU3gWlO20lA5jXGeRfW1LwNnjbGjAKigB+KSJQxZg3WXOzjjTHj7el7lwB32LnMBp5s4TpKqa+gU/Yq1XaVduXnrDOw1r6nXYc193xTe4GnRaQf8LYx5lMRiQNGAgftKY2DsL5EuPJHEakEirGWbh0MfGaMKbRf3wA8BqwFqoA/iMi7gNvLABtjykTkuD1P+qf2NfbY521NnDdgTVEb47R/uoj8FOv/UF8gAshtcuwYe/8e+zrXY+VNKXWNtOJXyjMWAqXAcKyetaqmBYwxm0RkPzAJ+IuI/ARrGdINxpin3LjGQ86L+ohIL1eF7Pnhv4e18MsDwHxgQiveSyYwHfg7sNkYY8Sqhd2OEzgKpAIvA9NEZACwCBhtjCkXkTSsxWiaEmCbMWZGK+JVSn0F7epXyjN6AKfsNdQfxmrtXkFEBgLH7e7trVhd3juA+0Skj12mp4j0d/OafwdCRWSQvf0w8KF9T7yHMeZPWAPnXI2sr8BaKtiVt4G7sdaJz7T3tSpOY0wNVpf9GPs2QXfgS+C8iIQAP7pKLPuAcQ3vSUS6ioir3hOllJu04lfKM9YBM0VkH1Y3/5cuyiQBeSKSAwwBXrNH0i8B3heRXGAbVjd4i4wxVVirl70hIseAemA9ViWaZZ/vQ6zeiKbSgPUNg/uanLccyAf6G2MO2PtaHac9duAFYJEx5ihwBPgYeBXr9kGDV4A/i8guY0wZ1i8O0u3r7MPKlVLqGunqfEoppZQf0Ra/Ukop5Ue04ldKKaX8iFb8SimllB/Ril8ppZTyI1rxK6WUUn5EK36llFLKj2jFr5RSSvmR/weg7rxyhqL8uAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(clf_reduced_tuned, y_test, X_test_pca, \n", + " \"SVM reduced features and tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM reduced features and tuning RBF kernal identified 727\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06383346
11293727
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6383 346\n", + "1 1293 727" + ] + }, + "execution_count": 87, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_reduced_tuned.predict(X_test_pca), \"SVM reduced features and tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.36 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_reduced_tuned.predict(X_test_pca)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From this, we can conclude that fitting SVM model with PCA-reduced features but no parameter tuning is most accurate based on Recall value" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Thus, fitting an SVM with PCA-reduced features with default gamma = 1/13 and C = 1 with kernal = 'rbf' is the best model. However, this is only for rbf kernel." + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [ + { + "ename": "ValueError", + "evalue": "X.shape[1] = 44 should be equal to 13, the number of features at training time", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 1\u001b[0m evaluation.loc[4] = ([\"SVM\" , \n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mclassification_report\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my_test\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mclf_reduced_tuned\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_test\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moutput_dict\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"1\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"recall\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 3\u001b[0m auroc])\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36mpredict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 572\u001b[0m \u001b[0mClass\u001b[0m \u001b[0mlabels\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0msamples\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 573\u001b[0m \"\"\"\n\u001b[1;32m--> 574\u001b[1;33m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 575\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclasses_\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mintp\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 576\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36mpredict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 320\u001b[0m \u001b[0my_pred\u001b[0m \u001b[1;33m:\u001b[0m \u001b[0marray\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mshape\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mn_samples\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 321\u001b[0m \"\"\"\n\u001b[1;32m--> 322\u001b[1;33m \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_validate_for_predict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 323\u001b[0m \u001b[0mpredict\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_sparse_predict\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_sparse\u001b[0m \u001b[1;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dense_predict\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 324\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36m_validate_for_predict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 472\u001b[0m raise ValueError(\"X.shape[1] = %d should be equal to %d, \"\n\u001b[0;32m 473\u001b[0m \u001b[1;34m\"the number of features at training time\"\u001b[0m \u001b[1;33m%\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 474\u001b[1;33m (n_features, self.shape_fit_[1]))\n\u001b[0m\u001b[0;32m 475\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 476\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mValueError\u001b[0m: X.shape[1] = 44 should be equal to 13, the number of features at training time" + ] + } + ], + "source": [ + "evaluation.loc[4] = ([\"SVM\" , \n", + " classification_report(y_test, clf_reduced_tuned.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Neural Networks\n", + "We will now use the train and test sets as defined above and attempt to implement a neural network model on the data\n", + "\n", + "#### Theory\n", + "A neural network is comprised of many layers of perceptrons that take in a vector as input and outputs a value. The outputs from one layer of perceptrons are passed into the next layer of perceptrons as input, until we reach the output layer. Each perceptron combines its input via an activation function. \n", + "\n", + ".\n", + "\n", + "\n", + "![image.png](https://www.researchgate.net/profile/Leslaw_Plonka/publication/260080460/figure/fig1/AS:340931325775876@1458295770470/A-simple-neural-network-diagram.png)\n", + "\n", + "\n", + "The network is at first randomly initialised with random weights on all its layers. Training samples are then passed into the network and predictions are made. The training error (difference between the actual value and the predicted value) is used to recalibrate the neural network by changing the weights. The change in weights is found via gradient descent, and then backpropogated through the neural network to update all layers.\n", + "\n", + "\n", + "This process is repeated iteratively until the model converges (i.e. it cannot be improved further).\n", + "\n", + "#### Training\n", + "Here we create an instance of our model, with 5 layers of 26 neurons each, identical to that of our training data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.neural_network import MLPClassifier" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "mlp = MLPClassifier(hidden_layer_sizes=(26,26,26,26,26), activation = \"logistic\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mlp.fit(X_train,y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "predictions = mlp.predict(X_test)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "confusion(y_test,predictions,\"Neural Network (5x26)\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "auroc = get_roc(mlp, y_test, X_test, \"Neural Network (5x26)\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(classification_report(y_test,predictions))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[5] = ([\"Neural Network\" , \n", + " classification_report(y_test, mlp.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])\n", + "\n", + "evaluation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Deep Learning\n", + "\n", + "#### Theory\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from numpy import loadtxt\n", + "from keras.models import Sequential\n", + "from keras.layers import Dense\n", + "\n", + "# define the keras model\n", + "model = Sequential()\n", + "model.add(Dense(12, input_dim=26, activation='relu'))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1, activation='sigmoid'))\n", + "# compile the keras model\n", + "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n", + "# fit the keras model on the dataset\n", + "model.fit(X_train, y_train, epochs=10, batch_size=10)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# evaluate the keras model\n", + "#recall, accuracy = model.evaluate(df1, target)\n", + "#print('Accuracy: %.2f' % (accuracy*100))\n", + "#print('Recall: %.2f' % (recall*100))\n", + "\n", + "predictions = list(model.predict(X_test).ravel() > 0.5)\n", + "print(classification_report(y_test,predictions))" + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "BT2101 disrudy ", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.1" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/BT2101_Default - EDIT-Copy1.ipynb b/BT2101_Default - EDIT-Copy1.ipynb new file mode 100644 index 0000000..fcf3597 --- /dev/null +++ b/BT2101_Default - EDIT-Copy1.ipynb @@ -0,0 +1,5439 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "-4Rm0wjQMUHi" + }, + "source": [ + "# BUILDING A DEFUALT DETECTION MODEL\n", + "\n", + "---\n", + "\n", + "\n", + "\n", + "## Table of Contents\n", + "1. Problem Description (Brief Write Up)\n", + "2. Exploratory Data Analysis (EDA)\n", + "3. Data Pre-processing\n", + "4. Model Selection\n", + "5. Evaluation\n", + "6. Discussion and Possible Improvements\n", + "\n", + "## 1. Problem Description\n", + "\n", + "The goal of this project is to predict a binary target feature (default or not) valued 0 (= not default) or 1 (= default). This project will cover the entire data science pipeline, from data analysis to model evaluation. We will be trying several models to predict default status, and choosing the most appropriate one at the end. \n", + "\n", + "The data set we will be working on contains payment information of 30,000 credit card holders obtained from a bank in Taiwan, and each data sample is described by 23 feature attributes and the binary target feature (default or not).\n", + "\n", + "The 23 explanatory attributes and their explanations (from the data provider) are as follows:\n", + "\n", + "### X1 - X5: Indivual attributes of customer\n", + "\n", + "X1: Amount of the given credit (NT dollar): it includes both the individual consumer credit and his/her family (supplementary) credit. \n", + "\n", + "X2: Gender (1 = male; 2 = female). \n", + "\n", + "X3: Education (1 = graduate school; 2 = university; 3 = high school; 4 = others). \n", + "\n", + "X4: Marital status (1 = married; 2 = single; 3 = others). \n", + "\n", + "X5: Age (year). \n", + "\n", + "### X6 - X11: Repayment history from April to Septemeber 2005\n", + "The measurement scale for the repayment status is: -1 = pay duly; 1 = payment delay for one month; 2 = payment delay for two months, . . . 8 = payment delay for eight months; 9 = payment delay for nine months and above.\n", + "\n", + "\n", + "X6 = the repayment status in September, 2005\n", + "\n", + "X7 = the repayment status in August, 2005\n", + "\n", + "X8 = the repayment status in July, 2005\n", + "\n", + "X9 = the repayment status in June, 2005\n", + "\n", + "X10 = the repayment status in May, 2005\n", + "\n", + "X11 = the repayment status in April, 2005. \n", + "\n", + "### X12 - X17: Amount of bill statement (NT dollar) from April to September 2005\n", + "\n", + "X12 = amount of bill statement in September, 2005; \n", + "\n", + "X13 = amount of bill statement in August, 2005\n", + "\n", + ". . .\n", + "\n", + "X17 = amount of bill statement in April, 2005. \n", + "\n", + "### X18 - X23: Amount of previous payment (NT dollar)\n", + "X18 = amount paid in September, 2005\n", + "\n", + "X19 = amount paid in August, 2005\n", + "\n", + ". . .\n", + "\n", + "X23 = amount paid in April, 2005. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "aM_aIU6UPHe4" + }, + "source": [ + "## EDA\n", + "\n", + "In this section we will explore the data set, its shape and its features to get an idea of the data.\n", + "\n", + "### Importing packages and the dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "Is0wEkk3LJCt" + }, + "outputs": [], + "source": [ + "import pandas as pd" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "x_Z7u_9vRC5m" + }, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import seaborn as sns" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "KhmX9KWWyrUW" + }, + "outputs": [], + "source": [ + "url = 'https://raw.githubusercontent.com/reonho/bt2101disrudy/master/card.csv'\n", + "df = pd.read_csv(url, header = 1, index_col = 0)\n", + "# Dataset is now stored in a Pandas Dataframe" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 255 + }, + "colab_type": "code", + "id": "FhJ2eAxVQhBm", + "outputId": "7f79bb40-f08f-4709-e7d4-1f747bb8af2f" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...BILL_AMT4BILL_AMT5BILL_AMT6PAY_AMT1PAY_AMT2PAY_AMT3PAY_AMT4PAY_AMT5PAY_AMT6Y
ID
1200002212422-1-1-2...000068900001
212000022226-12000...3272345532610100010001000020001
3900002223400000...1433114948155491518150010001000100050000
4500002213700000...2831428959295472000201912001100106910000
55000012157-10-100...2094019146191312000366811000090006896790
\n", + "

5 rows × 24 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 \\\n", + "ID \n", + "1 20000 2 2 1 24 2 2 -1 -1 \n", + "2 120000 2 2 2 26 -1 2 0 0 \n", + "3 90000 2 2 2 34 0 0 0 0 \n", + "4 50000 2 2 1 37 0 0 0 0 \n", + "5 50000 1 2 1 57 -1 0 -1 0 \n", + "\n", + " PAY_5 ... BILL_AMT4 BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 PAY_AMT3 \\\n", + "ID ... \n", + "1 -2 ... 0 0 0 0 689 0 \n", + "2 0 ... 3272 3455 3261 0 1000 1000 \n", + "3 0 ... 14331 14948 15549 1518 1500 1000 \n", + "4 0 ... 28314 28959 29547 2000 2019 1200 \n", + "5 0 ... 20940 19146 19131 2000 36681 10000 \n", + "\n", + " PAY_AMT4 PAY_AMT5 PAY_AMT6 Y \n", + "ID \n", + "1 0 0 0 1 \n", + "2 1000 0 2000 1 \n", + "3 1000 1000 5000 0 \n", + "4 1100 1069 1000 0 \n", + "5 9000 689 679 0 \n", + "\n", + "[5 rows x 24 columns]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#rename the target variable to \"Y\" for convenience\n", + "df[\"Y\"] = df[\"default payment next month\"] \n", + "df = df.drop(\"default payment next month\", axis = 1)\n", + "df0 = df #backup of df\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "zcuPyfM86AKj", + "outputId": "89bb2e37-a3ba-43e5-99a7-6917f24acc3f", + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data has 24 Columns and 30000 Rows\n" + ] + } + ], + "source": [ + "size = df.shape\n", + "print(\"Data has {} Columns and {} Rows\".format(size[1], size[0]))" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "QVaSnvJP3VbO", + "outputId": "4bf72e64-2d0c-41c3-85b5-3bd6e70920d3" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check for null values\n", + "df.isnull().any().sum() " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "eVYXnIGH9Zq6" + }, + "source": [ + "From the above analyses, we observe that:\n", + "1. The data indeed has 30000 rows and 24 columns\n", + "2. There are no null values\n", + "\n", + "We will now explore the features more in depth." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "W6hhPNl1Slau" + }, + "source": [ + "### Exploring the features" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "1Sp2F3gzXX2F" + }, + "source": [ + "**1) Exploring target attribute:**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "colab_type": "code", + "id": "DCSEICWwXWgX", + "outputId": "9545da56-f31b-48f2-a271-db0e18677beb" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "defaults : 22.12 %\n", + "non defaults : 77.88000000000001 %\n" + ] + }, + { + "data": { + "text/plain": [ + "Text(0, 0.5, 'Frequency')" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAapElEQVR4nO3deZxcZZ3v8c+XgALjAg6LCEhQo4iMsgTEcQNRBBwFRhz16oBeFHVwvDqLouOIG3fQGZfBBQcxF3BUFBFFQTGgiNcrSFBkEZWIKDEBoiyyKYu/+0c9LWWnOl056Uqn6c/79apXn/Oc55zzO51Kffs8p+pUqgpJkrpYZ7oLkCTNXIaIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEpNWU5NAkX27T6yepJFtNd10ASY5OcnybfnSSm6Zw2yckeWOb3ifJ4inc9jOT/HCqtqfRMUTUWZJb+x5/SHJH3/xL1nAtq/3ineTaJE/pm98uyd2TrVdVn6iq53bd77gazk/y0qnY1nhV9dOq2miIGl6d5Owhtveyqnrv6tY16N+uqs6uqies7rY1eutOdwGauarqAWPTSa4GXlFVk774DJJk3aqa9AV7bTNT615dSeZU1T3TXYemn2ciGpkkT05yQZKbkyxN8oEk67ZlY399vibJz4DLWvtzklyZ5KYkHxz/l3mSVyX5SZIbkpyRZMu26Lz28yftTOiAAfVsl+Tctu7yJCcmeWBbdgqwGfD1tv7r2jbn9J1d7dT+Sv9Gko8kuRE4YoK/3A9IcnXbz1FJ0vbzx+GlvprubtPvA3YFjm/7e19r36Ht88YkVww6tr7tPSrJd5LckuSrwMaD9tXmX9lqvCXJVUlekGQn4IPAHq2Ga1vfk5Mck+TrSW4DntTa3jpu/+9ov9+rkrygr338v2P/72yFf7vxw2NJ/iLJt9vz4pIk+/YtO7k9V85qx/KdJNtM9DvSFKsqHz5W+wFcDTxzXNtu9F4U5wCPBBYDr27L1gcKOAPYCNgAeChwK/BXwHrAG4G7gJe2dV4EXAE8ui1/N/DNcdvbaiU1bgc8A7hf29f5wNF9y68FnjKu/93jtvFq4G7gle24NmhtZ4+r46x2XNsCV/Udw9HA8RPto9X00r75BwHLgJe0/e0K3AA8aoJj/D7wb+0Y9wJuH9tf/77ohctNwCPb/JbAY/uO8exx2z257feJ9P74vH9re2tbvk/7vYzt+5lt39tOcFyDfmdb9S3fB1jct/wXwD+2f/dnt+fJtn21XQ/s3JZ/Hjhhuv9PzJaHZyIamar6XlVdWFX3VNXPgOOBp4/rdlRV3VRVdwDPAy6sqq9U1V3AfwA39vV9FfDu6o3t3wW8A3hKks2HrOfHVfWNqrqzqq6l9xf3+HqGcVVVfbwd1x0T9Pm3dlw/Bz4MvLjDfgAOBC6rqk+1/V0IfBl4/viOSR4NbA+8ox3jOcDXJtn+DknWr6pfVdUVk/T9fFVdUFV/qKrfD1h+d9++zwbOBg6a7ACH8NT28/1VdVdVnQUsBF7Y1+dzVfX99rz4NLDjFOxXQzBENDJJtk/y1STXJfkt8DZgk3Hdrumbflj/fFX9AfhV3/JtgI+1IY2bgOX0XriGupie5GFJTknyq1bP8QPqGcY1k3f5kz6/oHdsXWwDPG3smNtxPx/YYkDfhwHLq+p34/a9gqq6kd7ZzeuAa5OcnuRRk9Qy2XEP2nfX4+73MOCXVdV/t9hf0Dt7GnNt3/TtwAPQGmGIaJQ+Tm945ZFV9SDgnUDG9el/YVhGXyAkWYc/faG4BnhZVW3U99igqi4at52J/DtwG7BDq+cV4+oZv42JtjnMvrbum344sLRN3wZs2LfsoZNs+xrg6+OO+QFV9foB+1wGbJJk/XH7HqiqzqiqvWgv0sCxE9QwUW3jDdr3MMc92XaXsuJxPJw//QND08QQ0Sg9ELi5qm5N8jh61xFW5nTgiUn2axfg/4G+C8PAx4C3JnkMQJKNkzwfoA2v3Aw8YpJ6bgV+m+Thbfv9rhu3/vX0LqxP+EK8Em9K8uAkc4HXAp9t7RcDeybZMsnGwJsmqeGLwE5JXphkvST3S7J7G7oa76fAj4F/bf32pHdtYQVt/89JsiHwe3q/l7F3W10HbJ1kvVU85vX69v0M4FnAqX3HfVB6b6jYDnjZ2EpD/Nt9G1gnyeuTrJvkWcDewCmrWJ9GwBDRKL0BeEWSW4GPcO8L6UBVtYzetYNjgF/TOyu5lN6LHFX1GXrXF77QhqMupvdCNeZtwClt2Od5A3bxNuAp9F6wTuPeF7gxRwFHtfVf24Z83gtc1NpWZZz9DOCHwCJ6L3b/3df+FeBH9C42f3Hceh8ADm7vxHpvq+HZwMvpnWkspfeGghVe4NtwzwuBPeldBH9j337HmwO8md4w0G/oXbD/+7bsa/TeKHF9kiWrcMxX0xtevBZYALy8qq5qy95L7yMFy4HjBtQ14b9dGyL7K3rXV34DvB94YbvOpmmWPx1mlNYe7WzkWuC5VfXd6a5H0oo8E9FaJcm+bRhofeBIehdJL5rmsiRNwBDR2uZpwM/pXY/YCziwqu6c3pIkTcThLElSZ56JSJI6m3U3YNxkk01q7ty5012GJM0oF1100a+ratPx7bMuRObOncuiRYumuwxJmlGSDLz7gcNZkqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTOZt0n1meKuUecMd0l3GdcffRzprsE6T7LMxFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISJJ6swQkSR1ZohIkjozRCRJnRkikqTODBFJUmcjC5EkWyf5ZpIrklye5H+19ockWZjkyvZz49aeJMckWZzkkiQ7923rkNb/yiSH9LXvkuTSts4xSTKq45EkrWiUZyJ3A/9YVY8FdgcOT7I9cARwTlXNA85p8wD7AvPa4zDgWOiFDnAk8ERgN+DIseBpfQ7rW2+fER6PJGmckYVIVS2rqu+36VuAK4Atgf2BE1u3E4ED2vT+wEnVcz6wUZItgGcDC6vqhqq6EVgI7NOWPaiqvltVBZzUty1J0hqwRq6JJJkL7ARcAGxeVcugFzTAZq3blsA1fastaW0ra18yoF2StIaMPESSPAA4FXh9Vf12ZV0HtFWH9kE1HJZkUZJFy5cvn6xkSdKQRhoiSdajFyCfqqovtObr2lAU7ef1rX0JsHXf6lsBSydp32pA+wqq6riqml9V8zfddNPVOyhJ0h+N8t1ZAT4BXFFV7+9bdDow9g6rQ4Av9bUf3N6ltTtwcxvuOgvYO8nG7YL63sBZbdktSXZv+zq4b1uSpDVg3RFu+8nA3wKXJrm4tb0FOBr4XJJDgV8CL2jLzgT2AxYDtwMvB6iqG5K8C7iw9XtnVd3Qpl8DnABsAHy1PSRJa8jIQqSq/i+Dr1sA7DWgfwGHT7CtBcCCAe2LgB1Wo0xJ0mrwE+uSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1NnIQiTJgiTXJ7msr+3tSX6V5OL22K9v2ZuTLE7ykyTP7mvfp7UtTnJEX/u2SS5IcmWSzya536iORZI02CjPRE4A9hnQ/oGq2rE9zgRIsj3wIuBxbZ2PJpmTZA7wEWBfYHvgxa0vwHvatuYBNwKHjvBYJEkDjCxEquo84IYhu+8PnFxVv6+qnwOLgd3aY3FVXVVVdwInA/snCfAM4PNt/ROBA6b0ACRJkxoqRJLsMIX7fG2SS9pw18atbUvgmr4+S1rbRO1/DtxUVXePax8oyWFJFiVZtHz58qk6Dkma9YY9E/lYku8l+bskG63G/o4FHgnsCCwD3tfaM6BvdWgfqKqOq6r5VTV/0003XbWKJUkTGipEquopwEuArYFFST6d5FmrurOquq6q7qmqPwAfpzdcBb0zia37um4FLF1J+6+BjZKsO65dkrQGDX1NpKquBN4KvAl4OnBMkh8n+etht5Fki77ZA4Gxd26dDrwoyf2TbAvMA74HXAjMa+/Euh+9i++nV1UB3wQOausfAnxp2DokSVNj3cm7QJLHAy8HngMsBJ5bVd9P8jDgu8AXBqzzGWAPYJMkS4AjgT2S7Ehv6Olq4FUAVXV5ks8BPwLuBg6vqnvadl4LnAXMARZU1eVtF28CTk7ybuAHwCdW+eglSatlqBABPkxv+OktVXXHWGNVLU3y1kErVNWLBzRP+EJfVUcBRw1oPxM4c0D7Vdw7HCZJmgbDhsh+wB19ZwfrAOtX1e1V9cmRVSdJWqsNe03kbGCDvvkNW5skaRYbNkTWr6pbx2ba9IajKUmSNFMMGyK3Jdl5bCbJLsAdK+kvSZoFhr0m8nrglCRjn8XYAnjhaEqSJM0UQ4VIVV2YZDvgMfQ+Lf7jqrprpJVJktZ6w56JAOwKzG3r7JSEqjppJFVJkmaEYT9s+El697y6GLinNRdgiEjSLDbsmch8YPt2uxFJkoDh3511GfDQURYiSZp5hj0T2QT4UZLvAb8fa6yq542kKknSjDBsiLx9lEVIkmamYd/i+60k2wDzqursJBvSu6uuJGkWG/brcV9J7/vM/6s1bQl8cVRFSZJmhmEvrB8OPBn4LfzxC6o2G1VRkqSZYdgQ+X1V3Tk2076W1rf7StIsN2yIfCvJW4AN2nernwJ8eXRlSZJmgmFD5AhgOXApva+0PZPe961LkmaxYd+d9Qd6X4/78dGWI0maSYa9d9bPGXANpKoeMeUVSZJmjFW5d9aY9YEXAA+Z+nIkSTPJUNdEquo3fY9fVdUHgWeMuDZJ0lpu2OGsnftm16F3ZvLAkVQkSZoxhh3Oel/f9N3A1cDfTHk1kqQZZdh3Z+056kIkSTPPsMNZ/7Cy5VX1/qkpR5I0k6zKu7N2BU5v888FzgOuGUVRkqSZYVW+lGrnqroFIMnbgVOq6hWjKkyStPYb9rYnDwfu7Ju/E5g75dVIkmaUYc9EPgl8L8lp9D65fiBw0siqkiTNCMO+O+uoJF8FntqaXl5VPxhdWZKkmWDY4SyADYHfVtV/AkuSbDuimiRJM8SwX497JPAm4M2taT3gv0dVlCRpZhj2TORA4HnAbQBVtRRveyJJs96wIXJnVRXtdvBJ/mx0JUmSZophQ+RzSf4L2CjJK4GzmeQLqpIsSHJ9ksv62h6SZGGSK9vPjVt7khyTZHGSS/pv+JjkkNb/yiSH9LXvkuTSts4xSbIqBy5JWn3D3gr+P4DPA6cCjwHeVlUfmmS1E4B9xrUdAZxTVfOAc9o8wL7AvPY4DDgWeqEDHAk8EdgNOHIseFqfw/rWG78vSdKITfoW3yRzgLOq6pnAwmE3XFXnJZk7rnl/YI82fSJwLr0L9vsDJ7Uhs/OTbJRki9Z3YVXd0GpZCOyT5FzgQVX13dZ+EnAA8NVh65Mkrb5Jz0Sq6h7g9iQPnoL9bV5Vy9p2lwGbtfYt+dP7cC1pbStrXzKgfaAkhyVZlGTR8uXLV/sgJEk9w35i/XfApe1M4Laxxqp63RTVMeh6RnVoH6iqjgOOA5g/f/6E/SRJq2bYEDmjPVbXdUm2qKplbbjq+ta+BNi6r99WwNLWvse49nNb+1YD+kuS1qCVhkiSh1fVL6vqxCna3+nAIcDR7eeX+tpfm+RkehfRb25Bcxbwv/supu8NvLmqbkhyS5LdgQuAg4HJLvRLkqbYZNdEvjg2keTUVdlwks8A3wUek2RJkkPphcezklwJPKvNA5wJXAUspvfW4b8DaBfU3wVc2B7vHLvIDrwGOL6t8zO8qC5Ja9xkw1n91x4esSobrqoXT7BorwF9Czh8gu0sABYMaF8E7LAqNUmSptZkZyI1wbQkSZOeiTwhyW/pnZFs0KZp81VVDxppdZKktdpKQ6Sq5qypQiRJM8+qfJ+IJEl/whCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM7Wne4CJM0sc484Y7pLuE+5+ujnTHcJq8UzEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKmzaQmRJFcnuTTJxUkWtbaHJFmY5Mr2c+PWniTHJFmc5JIkO/dt55DW/8okh0zHsUjSbDadZyJ7VtWOVTW/zR8BnFNV84Bz2jzAvsC89jgMOBZ6oQMcCTwR2A04cix4JElrxto0nLU/cGKbPhE4oK/9pOo5H9goyRbAs4GFVXVDVd0ILAT2WdNFS9JsNl0hUsDXk1yU5LDWtnlVLQNoPzdr7VsC1/Stu6S1TdS+giSHJVmUZNHy5cun8DAkaXabrlvBP7mqlibZDFiY5Mcr6ZsBbbWS9hUbq44DjgOYP3/+wD6SpFU3LWciVbW0/bweOI3eNY3r2jAV7ef1rfsSYOu+1bcClq6kXZK0hqzxEEnyZ0keODYN7A1cBpwOjL3D6hDgS236dODg9i6t3YGb23DXWcDeSTZuF9T3bm2SpDVkOoazNgdOSzK2/09X1deSXAh8LsmhwC+BF7T+ZwL7AYuB24GXA1TVDUneBVzY+r2zqm5Yc4chSVrjIVJVVwFPGND+G2CvAe0FHD7BthYAC6a6RknScNamt/hKkmYYQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHVmiEiSOjNEJEmdGSKSpM4MEUlSZ4aIJKkzQ0SS1JkhIknqzBCRJHU240MkyT5JfpJkcZIjprseSZpNZnSIJJkDfATYF9geeHGS7ae3KkmaPWZ0iAC7AYur6qqquhM4Gdh/mmuSpFlj3ekuYDVtCVzTN78EeOL4TkkOAw5rs7cm+ckaqG022AT49XQXMZm8Z7or0DTx+Tm1thnUONNDJAPaaoWGquOA40ZfzuySZFFVzZ/uOqRBfH6uGTN9OGsJsHXf/FbA0mmqRZJmnZkeIhcC85Jsm+R+wIuA06e5JkmaNWb0cFZV3Z3ktcBZwBxgQVVdPs1lzSYOEWpt5vNzDUjVCpcQJEkaykwfzpIkTSNDRJLUmSGiVZbkhCQHTdJnuyQXJ/lBkkd22Mfbk/xTm35Zkod1rVczX//zYYLlmya5oD3fntph+y9L8uE2fYB3vhieIaJROQD4UlXtVFU/W81tvQwwRLQyewE/bs+3b6/mtg6gdxslDcEQuQ9JMjfJFUk+nuTyJF9PskFbtmOS85NckuS0JBu39nOTvCfJ95L8dNBfcen5cJIfJTkD2Kxv2S5JvpXkoiRnJdkiyX7A64FXJPlm6/fF1ufydgeBsfVv7Zs+KMkJ4/Z9EDAf+FQ7s9lgKn9nWnsl+Zd2c9Wzgce0tkcm+Vp7Ln27nfHuCLwX2G/sOZLk2CSL2vPtHX3bvDrJJm16fpJzx+3zL4HnAf/etrXKZ9GzjSFy3zMP+EhVPQ64CXh+az8JeFNVPR64FDiyb511q2o3ei/8/e1jDqT3n/gvgFcCfwmQZD3gQ8BBVbULsAA4qqrOBD4GfKCq9mzb+J+tz3zgdUn+fJiDqarPA4uAl1TVjlV1xzDraWZLsgu9z33tBPw1sGtbdBzw9+259E/AR6vqYuBtwGf7niP/0j6t/njg6UkeP8x+q+r/0fus2T+3ba3uWfR93oz+nIgG+nn7TwVwETA3yYOBjarqW639ROCUvnW+0N9/wDafBnymqu4Blib5Rmt/DLADsDAJ9D6rs2yCul6X5MA2vTW9sPvNqhyYZpWnAqdV1e0ASU4H1qf3B8wp7fkGcP8J1v+bdsa7LrAFveGpS0Za8SxliNz3/L5v+h5gmOGfsXXuYeLnxKAPFAW4vKqetLKNJ9kDeCbwpKq6vQ0hrD9gu+sj3Wv8c24d4Kaq2nFlKyXZlt5Zyq5VdWMbIh17bt3NvSMwPt+mgMNZs0BV3Qzc2He942+Bb61klfHOA16UZE6SLYCxIaqfAJsmeRL0hreSPG7A+g8GbmwBsh2we9+y65I8Nsk69IbNBrkFeOAq1KuZ7zzgwHZ944HAc4HbgZ8neQH88VrdEwas+yDgNuDmJJvT+76hMVcDu7Tp5zOYz7dVYIjMHofQu1h4CbAj8M5VWPc04Ep611KOpQVQ+w6Xg4D3JPkhcDHtesk4XwPWbft+F3B+37IjgK8A32DiobATgI95YX32qKrvA5+l95w6FRh7x9VLgEPb8+1yBnx/UFX9EPhBW74A+E7f4ncA/5nk2/TOvAc5Gfjnrm9Pn2287YkkqTPPRCRJnRkikqTODBFJUmeGiCSpM0NEktSZISKNSJKHJjk5yc/afcfOTPLoJJdNd23SVPET69IIpHdfjtOAE6vqRa1tR2DzaS1MmmKeiUijsSdwV1V9bKyh3dPsmrH5dtflbyf5fnuM3dhyiyTntQ9XXpbkqe1uASe0+UuTvGHNH5K0Is9EpNHYgd4NLVfmeuBZVfW7JPOAz9C7y/H/AM6qqqOSzAE2pHeXgS2rageAJBuNrnRpeIaINH3WAz7chrnuAR7d2i8EFrRb7X+xqi5OchXwiCQfAs4Avj4tFUvjOJwljcbl3Hujv4m8AbgOeAK9M5D7AVTVefRuv/8r4JNJDq6qG1u/c4HDgeNHU7a0agwRaTS+Adw/ySvHGpLsCmzT1+fBwLKq+gO9OyvPaf22Aa6vqo8DnwB2bt/Gt05VnQr8K7DzmjkMaeUczpJGoKqqfQnXB5McAfyO3m3IX9/X7aPAqe3W5t+kd/tygD3o3UX2LuBW4GBgS+D/tFvmA7x55AchDcG7+EqSOnM4S5LUmSEiSerMEJEkdWaISJI6M0QkSZ0ZIpKkzgwRSVJn/x+T6PNo2DnX9gAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "All = df.shape[0]\n", + "default = df[df['Y'] == 1]\n", + "nondefault = df[df['Y'] == 0]\n", + "\n", + "x = len(default)/All\n", + "y = len(nondefault)/All\n", + "\n", + "print('defaults :',x*100,'%')\n", + "print('non defaults :',y*100,'%')\n", + "\n", + "# plotting target attribute against frequency\n", + "labels = ['non default','default']\n", + "classes = pd.value_counts(df['Y'], sort = True)\n", + "classes.plot(kind = 'bar', rot=0)\n", + "plt.title(\"Target attribute distribution\")\n", + "plt.xticks(range(2), labels)\n", + "plt.xlabel(\"Class\")\n", + "plt.ylabel(\"Frequency\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "tysR0WHw4SGU" + }, + "source": [ + "**2) Exploring categorical attributes**\n", + "\n", + "Categorical attributes are:\n", + "- Sex\n", + "- Education\n", + "- Marriage" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 323 + }, + "colab_type": "code", + "id": "s61SSRII00UB", + "outputId": "69df981f-8c36-43a9-d155-a6553adbba0b", + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2 60.373333\n", + "1 39.626667\n", + "Name: SEX, dtype: float64\n", + "--------------------------------------------------------\n", + "2 46.766667\n", + "1 35.283333\n", + "3 16.390000\n", + "5 0.933333\n", + "4 0.410000\n", + "6 0.170000\n", + "0 0.046667\n", + "Name: EDUCATION, dtype: float64\n", + "--------------------------------------------------------\n", + "2 53.213333\n", + "1 45.530000\n", + "3 1.076667\n", + "0 0.180000\n", + "Name: MARRIAGE, dtype: float64\n" + ] + } + ], + "source": [ + "print(df[\"SEX\"].value_counts().apply(lambda r: r/All*100))\n", + "print(\"--------------------------------------------------------\")\n", + "print(df[\"EDUCATION\"].value_counts().apply(lambda r: r/All*100))\n", + "print(\"--------------------------------------------------------\")\n", + "print(df[\"MARRIAGE\"].value_counts().apply(lambda r: r/All*100))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Uudv5XE828nb" + }, + "source": [ + "**Findings**\n", + "\n", + "- Categorical variable SEX does not seem to have any missing/extra groups, and it is separated into Male = 1 and Female = 2\n", + "- Categorical variable MARRIAGE seems to have unknown group = 0, which could be assumed to be missing data, with other groups being Married = 1, Single = 2, Others = 3\n", + "- Categorical variable EDUCATION seems to have unknown group = 0,5,6, with other groups being graduate school = 1, university = 2, high school = 3, others = 4 " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 357 + }, + "colab_type": "code", + "id": "U3IJzhwwe5KK", + "outputId": "cb61e112-a3ec-4a37-c1a0-0ffc9ebcbf89", + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Total target attributes:\n", + "non defaults : 77.88000000000001 %\n", + "defaults : 22.12 %\n", + "--------------------------------------------------------\n", + "SEX Male Female\n", + "Y \n", + "non defaults 75.832773 79.223719\n", + "defaults 24.167227 20.776281\n", + "--------------------------------------------------------\n", + "EDUCATION 0 1 2 3 4 5 \\\n", + "Y \n", + "non defaults 100.0 80.765234 76.265146 74.842384 94.308943 93.571429 \n", + "defaults 0.0 19.234766 23.734854 25.157616 5.691057 6.428571 \n", + "\n", + "EDUCATION 6 \n", + "Y \n", + "non defaults 84.313725 \n", + "defaults 15.686275 \n", + "--------------------------------------------------------\n", + "MARRIAGE unknown married single others\n", + "Y \n", + "non defaults 90.740741 76.528296 79.071661 73.993808\n", + "defaults 9.259259 23.471704 20.928339 26.006192\n" + ] + } + ], + "source": [ + "#proportion of target attribute (for reference)\n", + "print('Total target attributes:')\n", + "print('non defaults :',y*100,'%')\n", + "print('defaults :',x*100,'%')\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with Sex\n", + "sex_target = pd.crosstab(df[\"Y\"], df[\"SEX\"]).apply(lambda r: r/r.sum()*100).rename(columns = {1: \"Male\", 2: \"Female\"}, index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(sex_target)\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with education\n", + "education_target = pd.crosstab(df[\"Y\"], df[\"EDUCATION\"]).apply(lambda r: r/r.sum()*100).rename(index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(education_target)\n", + "print(\"--------------------------------------------------------\")\n", + "#analysing default payment with marriage\n", + "marriage_target = pd.crosstab(df[\"Y\"], df[\"MARRIAGE\"]).apply(lambda r: r/r.sum()*100).rename(columns = {0: \"unknown\",1: \"married\", 2: \"single\", 3: \"others\"},index = {0: \"non defaults\", 1: \"defaults\"})\n", + "print(marriage_target)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "kOriUQ0wxbhD" + }, + "source": [ + "**Conclusion**\n", + "\n", + "From the analyses above we conclude that\n", + "\n", + "1. The categorical data is noisy - EDUCATION and MARRIAGE contains unexplained/anomalous data.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "77GAylGWnPJO" + }, + "source": [ + "**3) Analysis of Numerical Attributes**\n", + "\n", + "The numerical attributes are:\n", + " \n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 669 + }, + "colab_type": "code", + "id": "HEcCl5Rj-N0T", + "outputId": "a59f7092-366e-47ec-c67b-e18f02d84ac4", + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
012345678910111213141516171819
0LIMIT_BALAGEPAY_0PAY_2PAY_3PAY_4PAY_5PAY_6BILL_AMT1BILL_AMT2BILL_AMT3BILL_AMT4BILL_AMT5BILL_AMT6PAY_AMT1PAY_AMT2PAY_AMT3PAY_AMT4PAY_AMT5PAY_AMT6
\n", + "
" + ], + "text/plain": [ + " 0 1 2 3 4 5 6 7 8 \\\n", + "0 LIMIT_BAL AGE PAY_0 PAY_2 PAY_3 PAY_4 PAY_5 PAY_6 BILL_AMT1 \n", + "\n", + " 9 10 11 12 13 14 15 \\\n", + "0 BILL_AMT2 BILL_AMT3 BILL_AMT4 BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 \n", + "\n", + " 16 17 18 19 \n", + "0 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6 " + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#printing numerical attributes\n", + "pd.DataFrame(df.drop(['SEX', 'EDUCATION', 'MARRIAGE','Y'], axis = 1).columns).transpose()" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
LIMIT_BAL26245.00.2843370.2378750.00.0816330.2244900.4081631.0
AGE26245.00.3685950.2324220.00.1842110.3421050.5263161.0
PAY_026245.00.3721090.7657300.00.0000000.0000000.0000008.0
PAY_226245.00.3373210.8148780.00.0000000.0000000.0000008.0
PAY_326245.00.3246330.8114910.00.0000000.0000000.0000008.0
PAY_426245.00.2782240.7863140.00.0000000.0000000.0000008.0
PAY_526245.00.2387500.7439230.00.0000000.0000000.0000008.0
PAY_626245.00.2432080.7403450.00.0000000.0000000.0000008.0
BILL_AMT126245.00.2001680.1893800.00.0575900.1229230.2652971.0
BILL_AMT226245.00.2549170.1745860.00.1236710.1857490.3089291.0
BILL_AMT326245.00.2981590.1609770.00.1777390.2370200.3517731.0
BILL_AMT426245.00.3176480.1568050.00.2018360.2611940.3706151.0
BILL_AMT526245.00.3300720.1554220.00.2174530.2741640.3743861.0
BILL_AMT626245.00.4279510.1332430.00.3326990.3777600.4635631.0
PAY_AMT126245.00.0727790.1059660.00.0152700.0413830.0880841.0
PAY_AMT226245.00.0551330.0923160.00.0091910.0306370.0615961.0
PAY_AMT326245.00.0618930.1056440.00.0062510.0302270.0694351.0
PAY_AMT426245.00.0617080.1070400.00.0033210.0265660.0690271.0
PAY_AMT526245.00.0628740.1075020.00.0018550.0275580.0710461.0
PAY_AMT626245.00.0553390.1033570.00.0000000.0224120.0601761.0
S_026245.0-0.1335870.879876-2.0-1.0000000.0000000.0000001.0
S_226245.0-0.3004380.883472-2.0-1.0000000.0000000.0000001.0
S_326245.0-0.3273000.895264-2.0-1.0000000.0000000.0000001.0
S_426245.0-0.3644120.886115-2.0-1.0000000.0000000.0000001.0
S_526245.0-0.3959990.877789-2.0-1.0000000.0000000.0000001.0
S_626245.0-0.4281580.900723-2.0-1.0000000.0000000.0000001.0
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "LIMIT_BAL 26245.0 0.284337 0.237875 0.0 0.081633 0.224490 0.408163 1.0\n", + "AGE 26245.0 0.368595 0.232422 0.0 0.184211 0.342105 0.526316 1.0\n", + "PAY_0 26245.0 0.372109 0.765730 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_2 26245.0 0.337321 0.814878 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_3 26245.0 0.324633 0.811491 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_4 26245.0 0.278224 0.786314 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_5 26245.0 0.238750 0.743923 0.0 0.000000 0.000000 0.000000 8.0\n", + "PAY_6 26245.0 0.243208 0.740345 0.0 0.000000 0.000000 0.000000 8.0\n", + "BILL_AMT1 26245.0 0.200168 0.189380 0.0 0.057590 0.122923 0.265297 1.0\n", + "BILL_AMT2 26245.0 0.254917 0.174586 0.0 0.123671 0.185749 0.308929 1.0\n", + "BILL_AMT3 26245.0 0.298159 0.160977 0.0 0.177739 0.237020 0.351773 1.0\n", + "BILL_AMT4 26245.0 0.317648 0.156805 0.0 0.201836 0.261194 0.370615 1.0\n", + "BILL_AMT5 26245.0 0.330072 0.155422 0.0 0.217453 0.274164 0.374386 1.0\n", + "BILL_AMT6 26245.0 0.427951 0.133243 0.0 0.332699 0.377760 0.463563 1.0\n", + "PAY_AMT1 26245.0 0.072779 0.105966 0.0 0.015270 0.041383 0.088084 1.0\n", + "PAY_AMT2 26245.0 0.055133 0.092316 0.0 0.009191 0.030637 0.061596 1.0\n", + "PAY_AMT3 26245.0 0.061893 0.105644 0.0 0.006251 0.030227 0.069435 1.0\n", + "PAY_AMT4 26245.0 0.061708 0.107040 0.0 0.003321 0.026566 0.069027 1.0\n", + "PAY_AMT5 26245.0 0.062874 0.107502 0.0 0.001855 0.027558 0.071046 1.0\n", + "PAY_AMT6 26245.0 0.055339 0.103357 0.0 0.000000 0.022412 0.060176 1.0\n", + "S_0 26245.0 -0.133587 0.879876 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_2 26245.0 -0.300438 0.883472 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_3 26245.0 -0.327300 0.895264 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_4 26245.0 -0.364412 0.886115 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_5 26245.0 -0.395999 0.877789 -2.0 -1.000000 0.000000 0.000000 1.0\n", + "S_6 26245.0 -0.428158 0.900723 -2.0 -1.000000 0.000000 0.000000 1.0" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.drop(['SEX', 'EDUCATION', 'MARRIAGE','Y'], axis=1).describe().transpose()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Analysis of PAY_0 to PAY_6**\n", + "\n", + "We observe that the minimum value of PAY_0 to PAY_6 is -2. The dataset's author has explained these factors (PAY_0 to PAY_6) as the number of months of payment delay, that is, 1= payment delay of one month; 2= payment delay of two months and so on. \n", + "\n", + "However, the presence of -2, -1 in these columns indicates that\n", + "1. There is anomalous data, OR \n", + "2. The numbers do not strictly correspond to the number of months of payment delay. \n", + "\n", + "This means we must conduct some data transformation.\n", + "\n", + "According to **(link)**, the numeric value in these attributes shows the past history of a credit card holder, where -2 means: No consumption of credit card, -1 means that holder paid the full balance, and 0 means the use of revolving credit.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 669 + }, + "colab_type": "code", + "id": "awXnqvLOS-wB", + "outputId": "a77b53b8-011e-4f53-b7b7-20d80bbc1777", + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df7BcZZ3n8ffHIC4VpEiM3IoEDboZZ/mhCHcIrlUz17EIAZwKlj+WHyXBQWMpKWdXtoowyxYK6Ga2lnXBHzhRY0ItENmpYZNRNGZZe3FnRZMoS4KARLzKJUiEQJaLOBj87h/naWgufW/f7j7d/Zx7P6+qru5++ulznpP+5vme55znnqOIwMzMLDevGHQDzMzMmnGCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzMyy5AQ1TZJGJT0raVzSY5K+Lunwhs83SDoo6XUNZaenugsayl4l6T5JH22xvpMk7ZT02/R8Um+2zHqlnzEj6Y8kbZb0G0n7JW2V9ObebZ2Vrc/xskDSP0p6QtJTkn4g6R2927rOOEG15y8i4nDgZOBPgCsAJM0F3gscAC6oV46IbcA3gesalnEF8CiwbrKVSDoU2Az8V2AesBHYnMqtWvoSM8CRwBbgzcAQ8COKGLJq6Ve8jAN/CbyWoo/5G+AfJB1S2paUwAmqAxHxCPBt4IRU9F7gKeAqYOWE6p8E/kzS2ZJOAFYDH4mprzE1AhwC/JeI+KeIuB4Q8OflbYX1U69jJiJ+FBFfi4j9EfF74HPAmyW9puxtsd7rQ7z8LiIeiIg/UPQtz1Mkqvnlbkl3nKA6IOkY4CzgJ6loJXALsAn4Y0kn1+tGxAHgY8CXgfXApyPi5y1WcTxwz4QAuyeVWwX1IWYm+lPg1xHxRLdtt/7rV7xIugf4HcXo+6sRsa+0jSiBE1R7/rukp4D/Dfwv4LOSXg+8E7g5Ih4D7mDCHk5E/ANwF8W/9/XTWM/hFEP5RgeAV3fXfBuAfsXMCyQtAr5IsWdt1dLXeImItwBHAOendWbFCao950TEkRHxhoj4eEQ8C3wQuC8i7k51bgLOl/TKCd+9F7g/DalbGacImkZHAE9303gbiH7FDACSXgt8F/hSRNxSxgZYX/U1XuCFw323AGskvbXrLShRVifEKupC4PWSfp3eHwK8BjiTYtjciXuBSyWp4TDfWyj2iq36ehEzSJpHkZy2RMRnum6l5aIn8dLEK4E3Av+3xGV2xQmqC5LeDrwJeBvwm4aPrqUYgncaPDWKk5afkPRl4COp/H92uDzLRK9iRtIRwFbgHyNiTbfttDz0MF5Oo+j/fwTMAT5BMfvzh920t2xOUN1ZCWyOiF2NhZKuA74vaX5E7G93oRHxnKRzgK8Ca4H7KIb+z5XRaBuonsQM8B6KacnHS7qoofy4iPhVx621QetVvLyK4lzVG4HfA7uAsyNib7cNLpN8R10zM8uRJ0mYmVmWnKAGRNIF6ZImEx/3DrptlifHjLVjJsSLD/GZmVmWWk6SkLQeeDewLyJOSGWfophZVp9V8tcRcXv67HLgYtIstIjYmsqXU1wvag7FXyyvTeXHUvx19Hzgx8AHpzMZYMGCBbF48eJpb2grzzzzDHPnzi1teb0yiHbu3Lnz8Yh4bV9XWrKy4wWqETOOl865j+mfSWMmIqZ8UFwy5WRgd0PZp4B/26TucRRz6F8FHAv8nCIhzUmv3wgcmuocl75zK3Buev1l4GOt2hQRnHLKKVGm733ve6Uur1cG0U5gR0zjN8n5UXa8RFQjZhwv+cRMFeIlIq+YaXkOKiLuBKY7jXEFsCmKC5z+AtgDnJoeeyLioShGR5uAFZLqF0D9u/T9jcA501yXmZnNYN38HdRqSRcCO4BLI+JJ4GiK60HVjaUygIcnlC+l+GvopyLiYJP6LyNpFbAKYGhoiFqt1kXzX2p8fLzU5fVKVdppZtatThPUDcDVQKTnaynuLaImdYPmswVjivpNRcQ60j1OhoeHY2RkpK1GT6VWq1Hm8nqlKu00M+tWRwkqiivqAiDpKxQ3zIJiBHRMQ9VFQP0vk5uVPw4cKemQNIpqrN9Xux45wEVrvjVlndG1Z/epNVYFrWLG8WKNHC/t6+jvoCQtbHj7HmB3er0FODfdcvhYYAnFtZ62A0skHZvuCnsuxQUtA/ge8L70/ZX4LqCVJ2m9pH2SdjeUfUrSI5LuTo+zGj67XNIeSQ9IOqOhfHkq2yNpTUP5sZJ+KOlBSd/wnYarzfFik2mZoCTdAvyA4u6cY5IuBv6jpF3pZlfvBP4NQETcSzEr76fAd4BLIuL5NDpaTXExy/uAW1NdgMuAT0raQ3FO6mulbqENwgZgeZPyz0XESelR/7OE4yh2WI5P3/mSpDmS5lBcvf1Mitmh56W6UNye+nMRsQR4kuLPGqy6NuB4sSZaHuKLiPOaFE+aRKK4zP/LLvWfAuz2JuUPUczysxkiIu6UtHia1V+Y+Qn8Iu2o1ONhT4oPJNVnft5HMfPz/FRnI8WfPdxQTuut3xwvNhlfzdz6qa8zP3s56xNg6DC49MSDk36ew2zLis/6nFEzhasQL5BXzDhBWb/0feZnL2d9Anz+ps1cu2vy/0KjF5S7vk5UeNbnjJspXIV4gbxixgnK+mImzvy03nG8GPhq5tYnnvlp7XC8GHgEZT2QZn6OAAskjQFXAiOSTqI4vDIKfBSKmZ+S6jM/D5Jmfqbl1Gd+zgHWT5j5uUnSNcBP8MzPSnO82GScoKx0nvlp7XC82GR8iM/MzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllqWWCkrRe0j5JuxvK5kvaJunB9DwvlUvS9ZL2SLpH0skN31mZ6j8oaWVD+SmSdqXvXC9JZW+kmZlVz3RGUBuA5RPK1gB3RMQS4I70HuBMYEl6rAJugCKhUdzGeSnFnS2vrCe1VGdVw/cmrsvMzGahlgkqIu4E9k8oXgFsTK83Auc0lN8YhbuAIyUtBM4AtkXE/oh4EtgGLE+fHRERP4iIAG5sWJaZmc1ih3T4vaGIeBQgIh6VdFQqPxp4uKHeWCqbqnysSXlTklZRjLYYGhqiVqt12PyXGzoMLj3x4JR1ylxfp8bHx7Noh5lZr3WaoCbT7PxRdFDeVESsA9YBDA8Px8jISAdNbO7zN23m2l1T/3OMXlDe+jpVq9Uoc7t7QdJ64N3Avog4IZXNB74BLAZGgQ9ExJPpnON1wFnAb4GLIuLH6TsrgSvSYq+JiI2p/BSKQ8+HAbcDf5VG4FZBjhebTKez+B5Lh+dIz/tS+RhwTEO9RcDeFuWLmpRbtW3A5y1t+jbgeLEmOk1QW4D6TLyVwOaG8gvTbL7TgAPpUOBWYJmkeSlolgFb02dPSzot7Rld2LAsqyift7R2OF5sMi0P8Um6BRgBFkgao9hLWQvcKuli4FfA+1P12ymG3nsoht8fAoiI/ZKuBraneldFRD0gP8aLw+9vp4fNPH0/b9nLc5bQ+rxlDucKK3zOctad587ld8opZlomqIg4b5KP3tWkbgCXTLKc9cD6JuU7gBNatcNmrJ6dt+zlOUtofd7S5yx7Ysae584hXiCvmPGVJKxffN7S2uF4MSco6xuft7R2OF6s9GnmZj5vaW1xvNhknKCsdD5vae1wvNhkfIjPzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZWnW3LBw8ZpvTfn5pSf2qSFWGY4Zs8HqKkFJGgWeBp4HDkbEsKT5wDeAxcAo8IGIeFKSgOsobtf8W+CiiPhxWs5K4Iq02GsiYmM37TIz6zfv0JSvjEN874yIkyJiOL1fA9wREUuAO9J7gDOBJemxCrgBICW0K4GlwKnAlZLmldAuMzOrsF6cg1oB1EdAG4FzGspvjMJdwJGSFgJnANsiYn9EPAlsA5b3oF2WAUmjknZJulvSjlQ2X9I2SQ+m53mpXJKul7RH0j2STm5YzspU/8E0ArcZyPEyu3WboAL4rqSdklalsqGIeBQgPR+Vyo8GHm747lgqm6zcZi6Puq0djpdZqttJEu+IiL2SjgK2Sbp/irpqUhZTlL98AUUSXAUwNDRErVabdkMvPfHglJ8PHda6Tjvr65Xx8fEs2lGyFcBIer0RqAGX0TDqBu6SVB91j5BG3QCS6qPuW/rbbBsQx8ss0VWCioi96XmfpNso9k4ek7QwIh5NwbEvVR8Djmn4+iJgbyofmVBem2R964B1AMPDwzEyMtKsWlMXtTyBeZBrd039zzF6wfTX1yu1Wo12tjtD9VF3AH+bftOXjLrTDg941G2Ol1mt4wQlaS7wioh4Or1eBlwFbAFWAmvT8+b0lS3AakmbKIbaB1JwbQU+2zDkXgZc3mm7LHt9G3V3M+KG7kfdOYx0Z8CIe9Ycpcnld8opZroZQQ0BtxWzxzkEuDkiviNpO3CrpIuBXwHvT/Vvp5hivodimvmHACJiv6Srge2p3lX1objNPP0cdXcz4obuR90ecXdvNh2lySFeIK+Y6XiSREQ8FBFvTY/jI+IzqfyJiHhXRCxJz/tTeUTEJRHxpog4MSJ2NCxrfUT88/T4evebZTmSNFfSq+uvKUbLu3lx1A0vH3VfmGZnnUYadQNbgWWS5qWR97JUZjOI48VmzZUkLAsedVs7HC+znBOU9U1EPAS8tUn5E8C7mpQHcMkky1oPrC+7jZYPx4s5QZmZTUOrSxlZ+Zyg2tAqQEfXnt2nlli3cuhsptMGx5Q1mm19kG+3YWZmWXKCMjOzLDlBmZlZlpygzMwsSzNmkkQOJ73NzKw8MyZB5WC2zbAxM+slH+IzM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmlqVsEpSk5ZIekLRH0ppBt8fy5nixdjlmqieLq5lLmgN8ETgdGAO2S9oSET8dbMvKNZ1bgviK563NlngBXyG/LLMpZmaSLBIUcCqwJyIeApC0CVgBOHisGceLtWtWxMxM26HJJUEdDTzc8H4MWDqxkqRVwKr0dlzSA2U14BOwAHi8rOV1Sn/Tssog2vmGPq+vlYHHC7SOmWn8ll1zvEzbwGOmIvECGcVMLglKTcriZQUR64B1PWmAtCMihnux7DJVpZ09NvB4gWr8FlVoY58MPGaq8lvk1M5cJkmMAcc0vF8E7B1QWyx/jhdrl2OmgnJJUNuBJZKOlXQocC6wZcBtsnw5XqxdjpkKyuIQX0QclLQa2ArMAdZHxL19bkbPDgWVrCrt7JlM4gWq8VtUoY09l0nMVOW3yKadinjZYVgzM7OBy+UQn5mZ2Us4QZmZWZacoKjGJVAkjUraJeluSTsG3Z7ZrArxAo6ZnFQhZnKMl1l/DipdAuVnNFwCBTgvt0ugSBoFhiNi4H9MPJtVJV7AMZOLqsRMjvHiEVTDJVAi4jmgfgkUs2YcL9Yux0yHnKCaXwLl6AG1ZSoBfFfSznQ5FhuMqsQLOGZyUZWYyS5esvg7qAGb1iVQMvCOiNgr6Shgm6T7I+LOQTdqFqpKvIBjJhdViZns4sUjqIpcAiUi9qbnfcBtFIcNrP8qES/gmMlIJWImx3hxgqrAJVAkzZX06vprYBmwe7CtmrWyjxdwzGQm+5jJNV5m/SG+TC6B0soQcJskKH6zmyPiO4Nt0uxUkXgBx0w2KhIzWcbLrJ9mbmZmefIhPjMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKCmSdKopGcljUt6TNLXJR3e8PkGSQclva6h7PRUd0FD2ask3Sfpo9Nc70pJIenD5W6R9Vq/YybFyTNpfeOSvtqbLbNeGEC8zJF0jaS9kp6W9BNJR/Zm6zrjBNWev4iIw4GTgT8BroAX7kD5XuAAcEG9ckRsA74JXNewjCuAR4F1rVYmaR5wOZDbzc1s+voaM8BbI+Lw9PBOTfX0M14+DfxL4O3AEcAHgd+VshUlcYLqQEQ8AnwbOCEVvRd4CrgKWDmh+ieBP5N0tqQTgNXAR2J6d4r8D8D1wOOlNNwGpo8xYzNAr+Ml7fz+61Tvl1HYHRFOUFUn6RjgLOAnqWglcAuwCfhjSSfX60bEAeBjwJeB9cCnI+Ln01jHqcBw+p5VXD9iJrlT0q8l/b2kxSU13/qsD/FyInAQeF+Kl59JuqTkzeheRPgxjQcwCoxT7MX8EvgScBjweuAPwEmp3lbguibf/2/ADuAV01jXnFT37el9DfjwoP8N/Mg3ZlL9PwUOBY4EvgDsBg4Z9L+DH/nFC3A+EMDX0jreAvwGOH3Q/w6ND4+g2nNORBwZEW+IiI9HxLMUx23vi4i7U52bgPMlvXLCd+8F7o+IP0xjPR8H7omIH5TXdBuQfsUMEXFnRDwXEU8BfwUcC/yLkrbD+qNf8fJser4qIp6NiHsoRmdnlbERZTlk0A2YAS4EXi/p1+n9IcBrgDOBLR0u810Ux5TrwTIfeJukkyJidVettRz0ImaaCUAlLs8Goxfxck96zvq8phNUFyS9HXgT8DaK4XHdtRTHjDsNnouAf9bw/u+Bv6MYjluF9SpmJB0PvBLYRXHI5hrgEeC+btprg9WreImIn0v6PvDvJH0CeCPwr4DzumtxuZygurMS2BwRuxoLJV0HfF/S/IjY3+5C0yGaxuU9B/y/KE6GWrX1JGaAIeAGYBHwDPB/gHdHxO+7bbANVK/iBYpk9DXgCWAf8O8j4o6uWlsypRNmZmZmWfEkCTMzy5IT1IBIuqDhkjSND181wppyzFg7ZkK8+BCfmZllqbKTJBYsWBCLFy8ubXnPPPMMc+fOLW15vTKIdu7cufPxiHhtX1dasrLjBaoRM46XzrmP6Z/JYqayCWrx4sXs2LGjtOXVajVGRkZKW16vDKKdkn7Z1xX2QNnxAtWIGcdL59zH9M9kMeNzUGZmliUnKDMzy5ITlJmZZamy56DKtuuRA1y05ltT1hlde3afWmNV0CpmHC/WyPHSPo+gzMwsS05QZmaWJScoMxsoSesl7ZO0u6HsU5IekXR3epzV8NnlkvZIekDSGQ3ly1PZHklrGsqPlfRDSQ9K+oakQ/u3ddaNlgnKwWNmPbYBWN6k/HMRcVJ63A4g6TjgXOD49J0vSZojaQ7wRYp7JB0HnJfqAvxNWtYS4Eng4p5ujZVmOiOoDTh4zKxHIuJOYLq3jFgBbIqIf4qIXwB7gFPTY09EPBQRz1HcHXaFJAF/TnE/NYCNwDmlboD1TMsE5eAxswFZLemedBRnXio7Gni4oc5YKpus/DXAUxFxcEK5VUA308xXS7oQ2AFcGhFPUvzwdzXUaQyGicGzlDaDR9IqYBXA0NAQtVqti+a/1NBhcOmJB6esU+b6OjU+Pp5FO6YiaT3wbmBfRJyQyj4FfIQX7wr61w0j78spRs7PA5+IiK2pfDlwHTAH+GpErE3lx1Ls5MwHfgx8MO342MxxA3A1xS3Jr6a4g+xf0vwW9kHzne3Jbnk/6RWyB9nH5PL/Oqc+ptMENZDgiYh1wDqA4eHhKPN6UZ+/aTPX7pr6n2P0gvLW16mKXM9rA/AF4MYJ5Z+LiP/UWDDhsPDrgP8h6Y/Sx18ETqfYcdkuaUtE/JQXDwtvkvRliuR2Q682xvovIh6rv5b0FeCb6e0YcExD1UXA3vS6WfnjwJGSDkk7wo31m613YH1MDv0L5NXHdDSLLyIei4jnI+IPwFcoDuHB5MEzWfkLwTOh3CrMh4WtW5IWNrx9D1CfpLUFOFfSq9JIegnwI2A7sCRNujqUYqdnSxT3E/oe8L70/ZXA5n5sg3WvoxGUpIUR8Wh6OzF4bpb0nyn2huvBI1LwAI9QBM/5ERGS6sGzCQfPTNfXw8K9PFwD1Thkk9PhmslIugUYARZIGgOuBEYknURxRGUU+ChARNwr6Vbgp8BB4JKIeD4tZzWwleKQ8PqIqN+Y7zJgk6RrgJ8AX+vTplmXWiYoB4+VpO+HhXt5uAaqccgmp8M1k4mI85oUT9oPRMRngM80Kb8duL1J+UO8eJTHKqRlgnLwWBkGdU7BzKrLV5KwvvA5BTNrl69mbqXzYWEzK4MTlJXOh4XNrAw+xGdmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMsuQEZWZmWXKCMjOzLDlBmZlZlpygzMwsS05QZmaWJScoMzPLkhOUmZllyQnKzAZK0npJ+yTtbiibL2mbpAfT87xULknXS9oj6R5JJzd8Z2Wq/6CklQ3lp0jalb5zvST1dwutUy0TlIPHzHpsA7B8Qtka4I6IWALckd4DnAksSY9VwA1Q9EkUd25eSnEzyyvr/VKqs6rhexPXZZmazghqAw4ea4N3aqwdEXEnsH9C8QpgY3q9ETinofzGKNwFHClpIXAGsC0i9kfEk8A2YHn67IiI+EFEBHBjw7Iscy1v+R4Rd0paPKF4BTCSXm8EasBlNAQPcJekevCMkIIHQFI9eGqk4Enl9eD5djcbZQO3AfgCRWdQV9+pWStpTXp/GS/dqVlKscOytGGnZhgIYKekLanzqe/U3EVxS/jlOGZmmqGIeBQgIh6VdFQqPxp4uKHeWCqbqnysSXlTklZRxBZDQ0PUarXutqLB0GFw6YkHJ/28zHV1Y3x8PJu2tExQk5h1wQN5BFBOwTMZ79RYDzUbLUcH5U1FxDpgHcDw8HCMjIx00MTmPn/TZq7dNXmXO3pBeevqRq1Wo8zt7kanCWoyMzZ4II8Ayil42jSQnRqrrMckLUyxshDYl8rHgGMa6i0C9qbykQnltVS+qEl9q4BOE5SDx8rSs52aXo64oRqHbKow4p7EFmAlsDY9b24oXy1pE8Uh4QOpH9oKfLbh3PYy4PKI2C/paUmnAT8ELgQ+388Nsc51mqAcPNauvu/U9HLEDdU4ZFOFEbekWyh+6wWSxijOPa4FbpV0MfAr4P2p+u3AWcAe4LfAhwBSX3I1sD3Vu6p+eBj4GMV50cMoDgX7cHBFtExQDh4riXdqrKmIOG+Sj97VpG4Al0yynPXA+iblO4ATummjDcZ0ZvE5eKwt3qkxszKUPUnCzDs1ZlYKX+rIzMyy5ARlZmZZcoIyM7MsOUGZmVmWnKDMzCxLTlBmZpYlJygzM8uSE5SZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZck3LDQzK8HiNd+a8vNLT+xTQ2aQrhKUpFHgaeB54GBEDEuaD3wDWAyMAh+IiCclCbiO4vbevwUuiogfp+WsBK5Ii70mIjZ2065mHDxm1VOlPsbKV8YI6p0R8XjD+zXAHRGxVtKa9P4y4ExgSXosBW4AlqZguxIYBgLYKWlLRDxZQtssM1XqcLxTkw33MbNUL85BrQDqncVG4JyG8hujcBdwpKSFwBnAtojYnwJmG7C8B+2yfLwzIk6KiOH0vt7hLAHuSO/hpR3OKooOh4YOZylwKnClpHl9bL8NlvuYWaLbEVQA35UUwN9GxDpgKCIeBYiIRyUdleoeDTzc8N2xVDZZ+ctIWkXRUTE0NEStVpt2Qy898eCUnw8d1rpOO+vrlfHx8SzaUbIVwEh6vRGoUewRv9DhAHdJqnc4I6QOB0BSvcO5pb/Ntj6YNX1MLv+vc+pjuk1Q74iIvSlAtkm6f4q6alIWU5S/vLAIznUAw8PDMTIyMu2GXtTycM1Brt019T/H6AXTX1+v1Go12tnuDPWtw+mms4GZ0eHk1Nl0KJs+ptUh31bdaas+Jof+BfLqY7pKUBGxNz3vk3QbxeGWxyQtTB3NQmBfqj4GHNPw9UXA3lQ+MqG81k27LGt963C62aGB7ndqcuhwcupsOuE+Znbr+ByUpLmSXl1/DSwDdgNbgJWp2kpgc3q9BbhQhdOAA2mveSuwTNK8dB5hWSqzGaixwwFe0uEAtNHhNCu3GcR9jHUzghoCbismWnEIcHNEfEfSduBWSRcDvwLen+rfTjEbaw/FjKwPAUTEfklXA9tTvavq5xba0Xr4bYOWOplXRMTTDR3OVbzY4azl5R3OakmbKCZEHEh7zVuBzzZMjFgGXN5OWxwvlZBVH2P913GCioiHgLc2KX8CeFeT8gAumWRZ64H1nbbFKsMdjk2b+xjzlSSsb9zhmFk7fC0+MzPLkhOUmZllyQnKzMyy5HNQJWo1M2x07dl9aomZWfV5BGVmZlnyCMrMrCJm21Eaj6DMzCxLHkGZDch0rmYx0/aIzdrhEZSZmWXJCcrMzLLkBGVmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZckJyszMspRNgpK0XNIDkvZIWjPo9ljeHC/WLsdM9WSRoCTNAb4InAkcB5wn6bjBtspy5XixdjlmqimXi8WeCuyJiIcAJG0CVgA/HWirSjadi4O2smH53J63oQIXKJ0V8QLdx0y38TKdNlQgXmCWxMwM+a1eoIgYdBuQ9D5geUR8OL3/ILA0IlZPqLcKWJXevhl4oMRmLAAeL3F5vTKIdr4hIl7b53VOKpN4gWrEzKyPF8gmZqoQL5BRzOQyglKTspdlzohYB6zrSQOkHREx3Itll6kq7eyxgccLVOO3qEIb+2TgMVOV3yKndmZxDgoYA45peL8I2Dugtlj+HC/WLsdMBeWSoLYDSyQdK+lQ4Fxgy4DbZPlyvFi7HDMVlMUhvog4KGk1sBWYA6yPiHv73IyeHQoqWVXa2TOZxAtU47eoQht7LpOYqcpvkU07s5gkYWZmNlEuh/jMzMxewgnKzMyy5ARFNS6BImlU0i5Jd0vaMej2zGZViBdwzOSkCjGTY7zM+sWNACwAAAEnSURBVHNQ6RIoPwNOp5iKuh04LyKy+gtzSaPAcERU4Q/9ZqyqxAs4ZnJRlZjJMV48gmq4BEpEPAfUL4Fi1ozjxdrlmOmQExQcDTzc8H4sleUmgO9K2pkux2KDUZV4AcdMLqoSM9nFSxZ/BzVg07oESgbeERF7JR0FbJN0f0TcOehGzUJViRdwzOSiKjGTXbx4BFWRS6BExN70vA+4jeKwgfVfJeIFHDMZqUTM5BgvTlAVuASKpLmSXl1/DSwDdg+2VbNW9vECjpnMZB8zucbLrD/El8klUFoZAm6TBMVvdnNEfGewTZqdKhIv4JjJRkViJst4mfXTzM3MLE8+xGdmZllygjIzsyw5QZmZWZacoMzMLEtOUGZmliUnKDMzy5ITlJmZZen/A6K07c+5e7/dAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEYCAYAAADmugmLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfbRdVXnv8e9PEKS8lMRImhLqoR3RK8KQQoZJL7Y91EEIYA0doyroNSfIbVovjNqWcWuwtlBBG22xJdXLJbZpkqvyMlpfooIhl+EptQMwwYsEREygUY5JSWkAE17aJn3uH2tusrKz93nZZ7/MvfbvM8Yee++518tc2U/Os9dcc86liMDMzCxHr+h1BczMzJpxkjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTMzy5aTlJmZZctJyszMslWZJCVph6QXJe2T9Iykr0k6JX22VtL16fWQpJB0ZINtXCvpsy3ufzTt9+i68rVpf2+vK/+LVL5M0odSvfdJeknSgdL7R9Ly10naKmm/pGtbqaMdVOV4kXSSpFsk7ZT0nKR/lLSglXraQVWOmbT8NyT9i6QfS/qOpCWt1LPdKpOkkl+NiOOAOcBTwF92Y6eShoBfBAJ4e4NFvg+MlJY/EngH8DhARHwsIo5Ldf8t4N7a+4h4Y1ptO/D7wNc6dRwDqKrxchywGTgbmAmsA74m6biOHdTgqGrMAHwAmBMRJwDLgc9KmtOhQ5q0qiUpACLiJeBvgdO6tMulwH3AWkqBUvIV4BxJM9L7xcBDwD9PdgcRsS4i7gT2Tq+qVq9q8RIRT0TEJyNiV0QciIjVwFHA66ddcwOqFzMAEfFQROyvvQVeCZzSaoXbpZJJStJPAO+i+FK7YSnwufQ4X9Lsus9fAjYAl5SWX9+lutkEqh4vks6kSFLbW92GHaqqMSPpq5JeAu4HRoEtrVa4XaqWpL4k6Vngx8B5wJ92eoeS3gK8Frg9Ih6gOL1+d4NF1wNLJf0k8MvAlzpdN5tQ5eNF0gnA/wH+OCKea63WVlLpmImItwHHAxcCGyPiP1uueJtULUldHBEnAkcDVwJ/L+mnOrzPEeCuiHg6vf88DU7HI+KbwGuADwNfjYgXO1wvm1il40XSMRTNQPdFxJ+0XmUrqXTMpO38R7q0cH59Z4xeOKz3SRVExAHgC5JuBt7Sqf2kPwLvBI6QVGv7PRo4UdKbIuI7dat8Fvgj4NxO1cmmrorxknqAfQn4EfCbrdfaGqlizDRwJPBzbdjOtFTtTAoAFZYAM4BHmyx2tKRXlR61f4tX1JUf3WR9gIuBAxQXT89MjzcA/0DRJlxvFUUTwT0tHNMrJb2K4js7MtXtiKluxw5XtXiR9EqKi/ovAktzaLKpmgrGzH+RdIGkY9Lfmv8G/BLw91PZTidU7UzqK5IOUPRM+QEwEhGPSGq07L669+el50vTo+ZHwNwm+xsB/iYiflgulPQpYJWkD5bLI2IPcPdkDqSBz3DoKf4fAJdR9Pax1lQ1Xv4r8DaKJPVs6XguiIh/aGF7dlBVY0bAtRTJ8ACwDXhXRHy7hW21lXxnXjMzy1Ulm/vMzKwaqtbc13aSfgb4bpOPT6s/DbfB5nixqXLMjM/NfWZmlq2+PZOaNWtWDA0NTXr5559/nmOPPbZzFcpA7RgfeOCBpyPiNb2uT04mipdBiI96jpfx9VvMdLM+3YyZvk1SQ0NDbNky+Rk7RkdHGR4e7lyFMlA7Rkk/6HVdcjNRvAxCfNRzvIyv32Kmm/XpZsy444SZmWXLScrMzLLlJGVmZtnq22tSkzG04uD9Aa86Yz/LSu93rLyoF1WyDJTjoqYWH44La6RZzAx3vyoDx2dSZmaWLScpMzPLlpOUmfWUpDWSdkt6uFR2raQfSXowPS4sfXa1pO2SHpN0fql8cSrbLmlFqfxUSfdL2ibpNklHde/obLomTFKSTpH0DUmPSnpE0gdS+UxJm9IXv0nSjFQuSatSoDwk6azStkbS8tskjZTKz5a0Na2zSk2mFDazSloLLG5Q/ucRcWZ63AEg6TSKW6S/Ma3zvyQdkW5b82ngAoqZvC9NywJ8PG1rHvAMcHlHj8baajJnUvuBqyLiDcBC4Ir05a8A7k5f/N3pPRRBMi89lgM3QZHUgGuABcCbgWtqiS0ts7y0XqOANbMKioh7gD2TXHwJcGtE/FtE/BOwneLvyZuB7RHxRET8O3ArsCT94P0VivtrAayjuEeT9YkJk1RE7KrdUyQi9lLc4OtkimBZlxYrf/FLgPVRuI/iDpJzgPOBTRGxJyKeATYBi9NnJ0TEvVFMJLgeB5GZwZWpNWZN6QftycCTpWXGUlmz8lcDz0bE/rpy6xNT6oIuaQj4eeB+YHZE7IIikUk6KS021SA6Ob2uLzezwXUTcB3FzQWvA24A3kdxc756QeMf3DHO8g1JWk7RqsPs2bMZHR0Fiu7m9WYfw8uf52Dfvn1Z1addJp2kJB0H/B3wOxHx43EuGzULiqmWN6pDwwBqphxYs4859H0Vv8yqBqkNnoh4qvZa0meAr6a3Y8AppUXnAjvT60blT1O05hyZzqbKyzfa72pgNcD8+fOjNhfesibjpN45oHP3ddOkkpSkV1IkqM9FxBdS8VOS5qSzqDnA7lTeLIjG4JCxb3OB0VQ+t8Hyh2kWQM0sqxvMe8PWg4e74z3jr9uPqhqkNnhqf1vS218Daj3/NgCfl/RJ4KcprmF/i+LH7jxJp1Lcjv0S4N0REZK+Afw6xXWqEeDL3TsSm67J9O4T8NfAoxHxydJHGyi+cDj0i98ALE29/BYCz6Vg2wgskjQjtS8vAjamz/ZKWpj2tRQHkdnAkHQLcC/wekljki4HPpF6/D4EnAv8LkBEPALcTnGTwK8DV0TEgXSWdCXF35lHgdvTsgAfBH5P0naKa1R/3cXDs2mazJnUOcB7ga2SHkxlHwJWArengPoh8I702R3AhRS9bl4ALgOIiD2SrgM2p+U+EhG1Hj3vp+iGegxwZ3qY2QCIiEsbFDdNJBHxUeCjDcrvoPj7U1/+BEXvP+tDEyapiPgmja8bAby1wfIBXNFkW2uANQ3KtwCnT1QX6w+S1gBvA3ZHxOmp7FrgN4B/SYt9qDT25WqKsSsHgN+OiI2pfDFwI3AE8FcRsTKVn0rRdDMT+Dbw3tTt2MwqxjNOWCesxYMzzawNnKSs7Tw408zapdK36rDsXClpKbCFYhaTZyjGxN1XWqY8Tq5+XN0CpjA4c6pjXq46Y/9AdeH3kAXrB05S1i1dH5w51TEvN2w9spJDE5rxkAXrB05S1hW9GpxpZv3N16SsK9KA75r6wZmXSDo69dqrDc7cTBqcmW6tcAmwIfUerQ3OBA/ONKs0n0lZ26XBmcPALEljFLPfD0s6k6Jpbgfwm1AMzpRUG5y5nzQ4M22nNjjzCGBN3eDMWyVdD/w/PDjTrLKcpKztPDjTBsVQg+ubNTtWXtTFmlSXm/vMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLLlJGVmZtlykjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTPrKUlrJO2W9HCpbKakTZK2pecZqVySVknaLukhSWeV1hlJy2+TNFIqP1vS1rTOqnTjTOsTTlJm1mtrgcV1ZSuAuyNiHnB3eg9wAcVM+fMobmh5ExRJjWIi4wUU8zpeU0tsaZnlpfXq92UZc5Iys56KiHuAPXXFS4B16fU64OJS+foo3Edxb7E5wPnApojYk+74vAlYnD47ISLuTbd5WV/alvUBz4JuZjmaHRG7ACJil6STUvnJwJOl5cZS2XjlYw3KG5K0nOKsi9mzZzM6OgoUd24+rILHNC6vqa3bLfv27ev6PrvBScrM+kmj60nRQnlDEbEaWA0wf/78GB4eBmBZg1tyXHXGfm7Y2vxP6I73DDf9rBNGR0ep1bdK3NxnZjl6qnY35/S8O5WPAaeUlpsL7JygfG6DcusTTlLWdu6tZW2wAah95yPAl0vlS1PcLASeS82CG4FFkmak2FoEbEyf7ZW0MMXJ0tK2rA84SVknrMW9tWySJN0C3Au8XtKYpMuBlcB5krYB56X3UNyp+QlgO/AZ4H8ARMQe4Dpgc3p8JJUBvB/4q7TO48Cd3Tguaw9fk7K2i4h7JA3VFS8BhtPrdcAo8EFKvbWA+yTVemsNk3prAUiq9dYaJfXWSuW13lr+w9OnIuLSJh+9tcGyAVzRZDtrgDUNyrcAp0+njtY7TlLWLV3vrdVKT60q9o5qpqq9waxanKSs1zrWW6uVnlrd7pHVS1XtDWbV4mtS1i3urWVmU+YkZd3i3lpmNmVu7rO2S721hoFZksYoeumtBG5PPbd+CLwjLX4HcCFFz6sXgMug6K0lqdZbCw7vrbUWOIaiw4Q7TZhVlJOUtV0/99YaanC9CmDHyos6sTszm8CEzX0emGlmZr0ymWtSa/HATDMz64EJk5Sn0Tczs15p9ZpUVtPoN1MetFk/rX4VBzF6cKaZVU27O070ZBr9ZsqDNuun1a/ioE0PzjSzqml1nJQHZpqZWce1mqQ8MNPMzDpuwuY+D8w0M7NemTBJ9fPATDMz62+eu8/MzLLlJGVmZtlykjIzs2w5SZmZWbacpMwsW5J2pAmoH5S0JZW1bYJry5+TlJnl7tyIODMi5qf37Zzg2jLnJGVm/aYtE1x3u9LWGt/00LpK0g5gL3AA2B8R89Mv3duAIWAH8M6IeCbNQnIjxQDxF4BlEfHttJ0R4MNps9dHxDqsigK4S1IAN6f5O9s1wfVhmk1iXZ6cuqZ+0up63Z7suaoTTDtJWS+cGxFPl97Xmm9WSlqR3n+QQ5tvFlA03ywoNd/Mp/gj9oCkDelXslXLORGxMyWiTZK+N86y057Iutkk1ssa3LG5ftLqet2exLqqE0y7uc9y4OYbaygidqbn3cAXKa4ptWuCa+sDA3smNdTglxHAjpUXdbkmA6drzTf93HTTDbk3D0k6FnhFROxNrxcBH+HgBNcrOXyC6ysl3Upx5v1ciqeNwMdKnSUWAVd38VBsGgY2SVnPdK35pp+bbrqhD5qHZgNfLC5NciTw+Yj4uqTNtG+Ca8uck5R1Vbn5RtIhzTfpV+9km2+G68pHO1x167KIeAJ4U4Pyf6VNE1xb/nxNyrpG0rGSjq+9pmh2eZg23Z+si4diZl3iMynrJjffmNmUOElZ17j5xsymys19ZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtzzhhNgnNbu0Cvr2LWSf5TMrMzLLlJGVmZtlykjIzs2z5mlQdX3swM8uHk5SZWQc0+8HrH7tT4+Y+MzPLVjZJStJiSY9J2i5pRa/rY3lzvNhUOWb6UxbNfZKOAD4NnAeMAZslbYiI7/a2Zpaj3OLFzTr5yy1mbPKySFLAm4Ht6fbiSLoVWAJkFUD+Y5SNvo4XcMz0QDYx47iYmlyS1MnAk6X3Y8CC+oUkLQeWp7f7JD022R38NswCnp5OJZvRxzux1ZbUjvG1va5Ih7U9XjoZH41kEjODEi/QJzEzzbjoZgx3LWZySVJqUBaHFUSsBla3tANpS0TMb2XdfjEIx5i0PV4G6N/uZQN2zJWPmdzq0y65dJwYA04pvZ8L7OxRXSx/jhebKsdMn8olSW0G5kk6VdJRwCXAhh7XyfLleLGpcsz0qSya+yJiv6QrgY3AEcCaiHikzbtpqZmwzwzCMXYqXgbi367OwBzzgMRMbvVpC0Uc1ixrZmaWhVya+8zMzA7jJGVmZtkaiCSV43QoktZI2i3p4VLZTEmbJG1LzzNSuSStSvV/SNJZpXVG0vLbJI2Uys+WtDWts0qSWt1HleUYG404XvIy3bjx9zkFEVHpB8VF0seBnwWOAr4DnJZBvX4JOAt4uFT2CWBFer0C+Hh6fSFwJ8VYj4XA/al8JvBEep6RXs9In30L+IW0zp3ABa3so8qPXGPD8ZL3ox1x4+9zCv9Wva5AFwLqF4CNpfdXA1f3ul6pLkN1QfoYMCe9ngM8ll7fDFxavxxwKXBzqfzmVDYH+F6p/OXlprqPXv8bDWpsOF7yfbQrbvx9Tu4xCM19jaZDOblHdZnI7IjYBZCeT0rlzY5hvPKxBuWt7KPK+v2YHS+90alj9/fZwCAkqUlNh5K5Zscw1fJW9lFlVT1mx0tndfvYB/r7HIQk1U/ToTwlaQ5Aet6dypsdw3jlcxuUt7KPKuv3Y3a89Eanjt3fZwODkKT6aTqUDUCth84I8OVS+dLUA2ch8Fw6Vd8ILJI0I/XSWUTRVr4L2CtpYerVs7RuW1PZR5X1U2w04njpjU7Fjb/PRnp9UawbD4qeK9+n6JHzB72uT6rTLcAu4D8ofsVcDrwauBvYlp5npmVFccO2x4GtwPzSdt4HbE+Py0rl84GH0zqf4uDsIlPeR5UfOcaG4yX/x3Tjxt/n5B+eFsnMzLI1CM19ZmbWp5ykzMwsW05SZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLJVmSQlaYekFyXtk/SMpK9JOiV9tlbS9en1kKSQdGSDbVwr6bMt7n807ffouvK1aX9vryv/i1S+TNKHUr33SXpJ0oHS+0fq1vvltN71rdTTClWPl7rj2yfprlbqaQdVPWbSOh+Q9E+Snpf0qKTXtVLXdqpMkkp+NSKOA+YATwF/2Y2dShoCfhEI4O0NFvk+B++GSQred1DcYIyI+FhEHJfq/lvAvbX3EfHG0nqvBG4E7u/QoQyaSscL6fjSY1FnjmbgVDZmJP13ipsvXgQcB7wNeLpTxzRZVUtSAETES8DfAqd1aZdLgfuAtZQCpeQrwDnpFs8Ai4GHgH+e4n6uAu4CvtdaNa2RCseLdUjVYkbSK4BrgN+NiO9G4fGI2DPtmk9TJZOUpJ8A3kXxpXbDUuBz6XG+pNl1n78EbAAuKS2/fio7kPRailtFf2R6VbV6VYyX5HOS/kXSXZLe1HJt7TAVjJm56XG6pCdTk98fp+TVUz2vQJt9SdKzwI+B84A/7fQOJb0FeC1we0Q8QHF6/e4Gi64Hlkr6SeCXgS9NcVergD+MiH3Tqa8dosrx8h5gKO3rG8BGSSe2Wm97WVVjZm56XgScAZwLXErR/NdTVUtSF0fEicDRwJXA30v6qQ7vcwS4KyJqbbefp8HpeER8E3gN8GHgqxHx4mR3IOlXgeMj4rY21NcOqmS8pPX/MSJejIgXIuJPgGcprmnY9FQ1ZmrLfiIino2IHcDNwIUt17pNDut9UgURcQD4gqSbgbd0aj+SjgHeCRwhqdb2ezRwoqQ3RcR36lb5LPBHFL9SpuKtwPzSPn4SOCDpjIhY0mL1LalgvDQSgNqwHaOSMfMY8O8UcZKVqp1JAaDCEmAG8GiTxY6W9KrSo/Zv8Yq68qObrA9wMXCA4uLpmenxBuAfKNqE662iaCK4Z4qH9IfA60r72AB8BrhsituxBqoWL5J+RtI5ko5KdfqfwCzgH6eyHWuuajETES8AtwG/L+l4SXOB3wC+OpXtdELVzqS+IukAxa+BHwAjEfGI1PAHZP21nfPS86XpUfMjDrbX1hsB/iYiflgulPQpYJWkD5bLU0+ZuydzIHXr7QX2lrb/IvB8Dj1v+lwl4wU4HrgJ+DmKC+oPAhdExL+2sC07VFVjBormy9XATorm4c8Aa1rcVtsoIruzOzMzM6CizX1mZlYNVWvuaztJPwN8t8nHp9Wfhttgc7zYVDlmxufmPjMzy1bfnknNmjUrhoaGprTO888/z7HHHtuZCmW03wceeODpiHhN13bYByaKl17FRg77d7w01srfmPH0OsbaoXYM3YyZvk1SQ0NDbNmyZUrrjI6OMjw83JkKZbRfST/o2s76xETx0qvYyGH/jpfGWvkbM55ex1g71I6hmzHjjhNmZpYtJykzM8vWhElK0imSvpFugPWIpA+k8pmSNknalp5npHJJWiVpu6SHJJ1V2tZIWn6bpPK9T86WtDWts0pNRsaZmdlgmcw1qf3AVRHxbUnHAw9I2gQsA+6OiJWSVgArgA8CFwDz0mMBxcj3BZJmUtyvZD7FaO0HJG2IiGfSMssppr2/g+JeKHdO9+CGVnztkPdXnbGfZalsx8qLprt561P1cQEHY8NxYe3S7O+PY2xqJjyTiohdEfHt9HovxTxVJwNLgHVpsXUUc0yRytenm2bdRzER4hzgfGBTROxJiWkTsDh9dkJE3BtFf/j1pW1ZH5K0RtJuSQ+Xyq6V9CNJD6bHhaXPrk5n0Y9JOr9UvjiVbU8/hGrlp0q6P52R3ybpqO4dnZl105R696m4hfHPU9y+fHZE7IIikUk6KS12MvBkabWxVDZe+ViD8kb7X05xxsXs2bMZHR0dt75XnbH/kPezjzlYNtG67bRv376u7i8Da4FPcfhN1/48Iv6sXCDpNIobtb0R+Gng/0p6Xfr40xTznY0Bm9OZ93eBj6dt3Srpf1Pc8+amTh2MmfXOpJOUpOOAvwN+JyJ+PM5lo0YfNLtNwHjlhxdGrKaYAJH58+fHRN05lzU43b5ha3HIO94z/rrtVIWup1MREfekHzSTsQS4NSL+DfgnSduBN6fPtkfEEwCSbgWWSHoU+BUO3vRtHXAtTlJmlTSp3n2SXkmRoD4XEV9IxU+lpjrS8+5UPgacUlp9LsWsuuOVz21QbtVzZepMs6bW0Yapn3m/Gng2IvbXlZtZBU14JpV62v018GhEfLL00QaKaeRXpucvl8qvTL98FwDPpebAjcDHSn+cFgFXR8QeSXslLaRoRlwK/GUbjs3ychNwHcVZ8nXADcD7aH4m3egH1JTOvJs1D9c3A8PBpuBeNcsOYJOw2aRMprnvHOC9wFZJD6ayD1Ekp9slXQ78EHhH+uwOilsObwdeIN2YLyWj64DNabmPlO6H9H6K6xjHUPTqm3bPPstLRDxVey3pMxy8mVqzM2yalD9N0RnnyHQ21fTMu1nzcH0zMBxsCu5mM3DZoDUJm03WhEkqIr5J89tOv7XB8gFc0WRba2hwE62I2AKcPlFdrH9JmlPraAP8GlDr+bcB+LykT1J0nJgHfIsi5uZJOpXipnCXAO+OiJD0DeDXgVs59Cze+pCkUyg62fwU8J/A6oi4MQ1buQ0YAnYA74yIZ1Lrzo0UP4ZfAJbVeiCn8ZcfTpu+PiLWpfKzOfhD+A7gA+HZtfuCZ5ywtpN0C3Av8HpJY+ls+xNpwPZDwLnA7wJExCPA7RS3Kvg6cEVEHEhnSVcCGymGPdyeloViPN7vpU4Wr6Zojrb+VRuL+QZgIXBF6vW5gmIs5jyKu83WhiGUx2IuJ3WaKY3FXEDR+eaa0uWF2ljM2nqLu3Bc1gZ9O8Gs5SsiLm1Q3DSRRMRHgY82KL+D4ldvffkTHOwBaH0unWHXhrPsTT04a2Mxh9Ni64BRih8oL4/FBO6TVBuLOUwaiwmQJh1YLGmUNBYzldfGYvqyQh/wmZSZZWO8sZhAx8diWn58JmVmWchhLOZUJwwYT7PJBPq5F2cveqE6SZlZz403FjMNYZnsWMzhuvJRpjAWc6oTBoyn2WQCvepB2g696IXq5j4z66lJjMWEw8diLk13XFhIGotJ0clmkaQZqcPEImBj+myvpIVpX0txj9C+4TMpM+s1j8W0ppykzKynPBbTxuPmPjMzy5aTlJmZZctJyszMsuUkZWZm2XKSMjOzbDlJmZlZtpykzMwsW05SZmaWLScpMzPLlpOUmZlly9MimZUM1c1cXbNj5UVdromZgc+kzMwsY05SZmaWLScpMzPLlpOUtZ2kNZJ2S3q4VDZT0iZJ29LzjFQuSaskbZf0kKSzSuuMpOW3SRoplZ8taWtaZ5XGuc+4mfU3JynrhLXA4rqyFcDdETEPuDu9B7gAmJcey4GboEhqwDXAAuDNwDW1xJaWWV5ar35fZlYRTlLWdhFxD7CnrngJsC69XgdcXCpfH4X7gBMlzQHOBzZFxJ6IeAbYBCxOn50QEfemm9+tL23LzCrGXdCtW2ZHxC6AiNgl6aRUfjLwZGm5sVQ2XvlYg/LDSFpOccbF7NmzGR0dBeCqM/YfXrljGpfX1NbtlH379nV8H2b9yEnKeq3R9aRoofzwwojVwGqA+fPnx/DwMADLGoyFuuqM/dywtfl/hx3vGW76WTuMjo5Sq5+ZHeTmPuuWp1JTHel5dyofA04pLTcX2DlB+dwG5WZWQU5S1i0bgFoPvRHgy6XypamX30LgudQsuBFYJGlG6jCxCNiYPtsraWHq1be0tC0zqxg391nbSboFGAZmSRqj6KW3Erhd0uXAD4F3pMXvAC4EtgMvAJcBRMQeSdcBm9NyH4mIWmeM91P0IDwGuDM9zKyCnKSs7SLi0iYfvbXBsgFc0WQ7a4A1Dcq3AKdPp45m1h/c3GdmZtlykjIzs2w5SZmZWbacpMzMLFsTJilPFmpmZr0ymTOptXiyUDMz64EJk5QnCzUzs15pdZxU1ycLheYThjZTP2FoeRLRbk7m6clDzcxa0+7BvB2bLBSaTxjaTP1EouVJRDs9YWiZJw81a07SGuBtwO6IOD2VzQRuA4aAHcA7I+KZdM36RopZSl4AlkXEt9M6I8CH02avj4h1qfxsDs5QcgfwgdRyY32g1d59nizUzNplLb7ubU20mqQ8WaiZtYWve9t4Jmzu82ShZtYDfXHdezzNron38/XpXlxfnzBJebJQM8tIVte9x9Psmng3r4e3Wy+ur3vGCTPLka97G+AkZWZ58nVvA3w/KTPrMV/3tvE4SZlZT/m6t43HzX1mZpatgT2TGqrreVOzY+VFXa6JmZk14zMp6ypJO9KtWR6UtCWVte3WL2ZWLU5S1gvnRsSZETE/vW/nFDhmViFOUpaDtkyB0+1Km1nnDew1KeuZAO6SFMDNaYR/u6bAOUSzKW7qp6uBQ2/j0kinp4Lx7VzMGnOSsm47JyJ2pkS0SdL3xll2WlPdNJvipn66Gjj0Ni6NdHoqG9/OxawxN/dZV0XEzvS8G/gixTWldk2BY2YV467mbUoAAATPSURBVCRlXSPpWEnH115TTF3zMG2aAqeLh2JmXeLmPuum2cAXiynUOBL4fER8XdJm2jcFjplViJOUdU1EPAG8qUH5v9KmKXDMctdsIgHwZAKNuLnPzMyy5SRlZmbZcpIyM7NsOUmZmVm2nKTMzCxbTlJmZpYtJykzM8uWk5SZmWXLScrMzLLlJGVmZtnytEhmk+CpbMx6w2dSZmaWLScpMzPLlpOUmZlly0nKzMyy5SRlZmbZcu++Ou7FZWaWD59JmZlZtpykzMwsW27uMzPLRLPLDYN8qSGbJCVpMXAjcATwVxGxssdVsozlFC/+w9IfcooZm7wskpSkI4BPA+cBY8BmSRsi4ru9rdmh/McoD/0SL5YPx0z/yiJJAW8GtkfEEwCSbgWWAH0RQE5eXdcX8eKeolnpi5hpZpD/xuSSpE4Gniy9HwMW1C8kaTmwPL3dJ+mxqezkt2EW8HSrlZwqffzll13dL/DaLu6rF9oeLz2MjZpux0hZ1eMFuvQ3ZjydiLEGcdRptWPoWszkkqTUoCwOK4hYDaxueSfSloiY3+r6/bbfCmt7vPT6O+r1/gdAV/7GjFuBCnzHvTiGXLqgjwGnlN7PBXb2qC6WP8eLTZVjpk/lkqQ2A/MknSrpKOASYEOP62T5crzYVDlm+lQWzX0RsV/SlcBGiu6hayLikQ7sqiOn8Rnvt5I6FC+9/o56vf9K6+LfmPFU4Tvu+jEo4rBmWTMzsyzk0txnZmZ2GCcpMzPL1sAkKUmLJT0mabukFVNYb4ekrZIelLQllc2UtEnStvQ8I5VL0qq0j4cknVXazkhafpukkVL52Wn729O6Gm8f1n6txkZpfceINTXd+JrGftdI2i3p4VJZz+JyvH2MKyIq/6C4UPo48LPAUcB3gNMmue4OYFZd2SeAFen1CuDj6fWFwJ0UYzIWAven8pnAE+l5Rno9I332LeAX0jp3AheMtw8/8okNx4gf3Yivaez7l4CzgIdLZT2Ly2b7mOgxKGdSL0+JEhH/DtSmRGnVEmBder0OuLhUvj4K9wEnSpoDnA9siog9EfEMsAlYnD47ISLujeJbXF+3rUb7sPZqd2zUOEYMOhdfE4qIe4A9dcW9jMtm+xjXoCSpRlOinDzJdQO4S9IDKqZMAZgdEbsA0vNJE+xnvPKxJvVqtg9rr+nERo1jxJppR3y1Uy/jsqV/iyzGSXXBpKZEaeKciNgp6SRgk6TvtbCfqZZb97TjO3CMWDP98v11Iy5b+rcYlDOplqdEiYid6Xk38EWK0/enaqep6Xn3BPsZr3xuk3o124e117Sny3GM2Dhym46pl3HZ0r/FoCSplqZEkXSspONrr4FFwMNp3VovlxHgy+n1BmBp6sWyEHgune5uBBZJmpF6uiwCNqbP9kpamHrGLK3bVqN9WHtNa7ocx4hNILfpmHoZl832Mb5u9DLJ4UHRs+T7FD1t/mCS6/wsRW+c7wCP1NYDXg3cDWxLzzNTuShurPY4sBWYX9rW+4Dt6XFZqXw+xR+1x4FPcXAWkIb78COP2HCM+NGN+Jrmfm8BdgH/QXEWc3kv43K8fYz38LRIZmaWrUFp7jMzsz7kJGVmZtlykjIzs2w5SZmZWbacpMzMLFtOUmZmli0nKTMzy9b/Bx/Qh33ndxUNAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAEYCAYAAADWNhiqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfbAcdZ3v8ffH8CAbxCRGTsUka8KaukuEe4XkQly3ds+KhIBVN9xSVijKnKvczRbCqrVqbVzrXlZRL24V7hV0paJkSdisiE+buEtuNmKOSJU8KiSEyOYEoxwTiBgIHHyAsN/7R/9GO8PMmYfzMPOb83lVdU3Pt3/d/euZ7+nvTHefHkUEZmZmOXtZpztgZmY2Vi5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2pkwxk7RP0i8ljUh6QtI/SDqxNP0mSUckvaYUOze1nV2KHS9pt6Q/b2Kd09P6bqvTn+fLy07xBySFpAWStqT5RyS9kNpXnt8g6ThJX03LCkn9bb9ANqoezZ9lkrZJOiTpZ5K+ImlO+6+S1dKjubNY0n2SnkrDtyQtbv9VGgcRMSUGYB/wljQ+F3gIuCY9nw48C/wc+FDVfDcCG0vPrwa+DaiJdQ6kZR4B5tTozyPAX5Rip6dYAAuq2t8EfLwqdhzwfuAPgQNAf6df514dejR/zgcuAk4CfgdYB/y/Tr/WvTb0aO7MABYAAqYB7wV2dPJ1njLfzMoi4qfAFuC0FHob8DTwMYokKPtL4I8lvVXSacCVwJ9FekcbGABuAHYAl9aYfjOwqqr9hha24/mI+L8RcSfwYrPz2dj0UP5siYivRMQzEfEL4LPAm5qd31rXQ7nzdETsS30Rxf7ndc3OPxGmZDGTNB+4APhBCg0AXwJuAX5f0pmVthFxGLicIjHWAR+NiL1NrON3gX5gYxpW1Wh2F3CSpFMlTQPeAfxjm5tlk6SH8+ePgF1jmN8a6LXckfQ08CvgeuCTrc4/nqZaMfvn9OLfCXwH+GR64/8E+KeIeAK4napPSBHxTYo3/2XAdU2uaxXF1+6HKZL19ZLOqNGu8gnpXOCHwE9b3iqbLD2bP5L+M/C/gQ+1M7811JO5ExEzgFdSfGv8QYPmE+qYTq68Ay6MiG+VA5LeCeyOiAdSaCNwraQPRsQLpaa7gF9HxH80ua5VwBcAImK/pO9QJGr1G34zcAewkBa+5ltH9GT+SHodxaGv90XEd9tZhjXUk7mT1vGcpBuAn0k6NSIOtrussZhq38xqWQWcIulxSY8DnwZmU5wcb4ukPwAWAR8uLfds4BJJR32AiIgfAz+iOPTw9XbXaR2Tdf5Iei3wLeDqiLi53T5bW7LOnSovo7iIaO4Yl9O2qfbN7CiS3gj8HnAG8LPSpGspPslsbnPRA8A2jj5WfQLFydjzgW9Wtb8MmJk+4bT0nkg6nuIELMBxkl5O8SnOP1Q3wXLPH0lzKa6O+1xE3NBmX60NPZA75wJPpuVOBz4OPAXsbrPfYzalixnFG78pInaWg5I+A3xX0qyIONTKAlMx+VNgVUQ8XjXt5rTOoxKqmZO6o3gEeG0a35oeF1JcfmsTK/f8+Z/AKcBVkq4qLe/E+rPYOMk9d2ZQXPQxD/glcC+wIiJ+1ebyxkz+AG9mZrnzOTMzM8uei1mbJF1aur1LefD/6VhDzh9rl3OnNh9mNDOz7GV7Acjs2bNjwYIFADz33HNMnz69sx1qUTf2+f77738yIl7d6X5MtHLuQHe+F83opn47d7pfN/d1XPKn0c0bgZcD9wAPUvzz3kdTfCFwN7AH+DJwXIofn54PpekLSsv6cIo/ApxXiq9IsSFgTTM3lVyyZElUbN++PXLTjX0G7ovxvcFq1+dORHe+F83opn6Pd+5El+ZPzrnTzX0dj/xp5pzZr4E3R8R/Ad4ArJC0DPgU8HcRsYji/wsuS+0vA56KiNcBf5fakX4e4GLg9SmB/l7StHRfsM9R/A/EYop/7uvsTwnYeHHu2Fg4f6xpDYtZKpwj6emxaQjgzcBXU3w9cGEaX5mek6afI0kpfktE/DoifkTxSeisNAxFxKMR8TzFDTdXjnnLrOOcOzYWzh9rRVPnzNInmPspbvH/OWAv8HREHElNhvntbUzmAo8BRMQRSYeBV6X4XaXFlud5rCp+dp1+rAZWA/T19TE4OAjAyMjIb8ZzkWOf29EtuWN56ob8qbffgbz+jnPqazuaKmYR8SLwBkkzgG8Ap9Zqlh5VZ1q9eK1vhzUvsYyItcBagKVLl0Z/fz8A12/cxLV3Ple3//uueWvdaZ0yODhIpf+9rFtyZ7Qd0sFDh7l+46Y6WwCnz31l3Wmd1Os7J+iO/Km334G89j29vs9p6WrGiHha0iCwDJgh6Zj0CWkesD81GwbmA8PpXl+vBA6V4hXleerFrUd0Onca7pB21v9T2Hdpf91pndTrO6eyTuePdb+G58wkvTp9KkLSCcBbKG4muR14e2o2AFQ+2m7mt7/J83bg2+lqlc3AxZKOl7SQ4s7O91Dc02uRpIWSjqM4UdvuTTatizh3bCycP9aKZr6ZzQHWp2PXLwNujYh/kfQwcIukj1P8Ts6Nqf2NwM2Shig+FV0MEBG7JN0KPAwcAa5IhxCQdCXFTXKnAesiYkr/J3sPce7YWDh/rGkNi1lE7KD4mYLq+KMUVwNVx38FXFRnWZ8APlEjfhtwWxP9tYw4d2wsnD/WCt+b0czMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsNi5mk+ZK2S9otaZek96X4LEnbJO1JjzNTXJKukzQkaYekM0vLGkjt90gaKMWXSNqZ5rlOkiZiY80sH973WCua+WZ2BPhARJwKLAOukLQYWAPcHhGLgNvTc4DzgUVpWA18HooEBK4CzgbOAq6qJGFqs7o034qxb5p1mndGNkbe91jTGhaziDgQEd9P488Cu4G5wEpgfWq2Hrgwja8ENkThLmCGpDnAecC2iDgUEU8B24AVadpJEfG9iAhgQ2lZljfvjKxt3vdYK1o6ZyZpAXAGcDfQFxEHoEg64OTUbC7wWGm24RQbLT5cI26Z887Ixov3PdbIMc02lHQi8DXg/RHxzChHc2pNiDbitfqwmuJTOH19fQwODgLQdwJ84PQjdfteaddNRkZGurJfE2W0nZGkCd8Z1csdyDN/YOrkUKf3Pb2SO72eL00VM0nHUiTTxoj4ego/IWlO2hnNAQ6m+DAwvzT7PGB/ivdXxQdTfF6N9i8REWuBtQBLly6N/v5icddv3MS1O+tvyr5L++tO65TBwUEq/e91nd4ZQf3cgTzzB6ZGDnXDvqdXcqfX86WZqxkF3AjsjohPlyZtBion4geATaX4qnQyfxlwOH0K3woslzQzne9YDmxN056VtCyta1VpWZa50XZGaXqzO6N68aY+CFl+vO+xVjRzzuxNwDuBN0t6IA0XANcA50raA5ybngPcBjwKDAFfAN4DEBGHgKuBe9PwsRQDuBz4YppnL7BlHLbNOsw7Ixsj73usaQ0PM0bEndQ+nANwTo32AVxRZ1nrgHU14vcBpzXqi2WnsjPaKemBFPtrip3PrZIuA34CXJSm3QZcQLFj+QXwLih2RpIqOyN46c7oJuAEih2Rd0Y9wvsea0XTF4CYtco7IzObLL6dlZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfYaFjNJ6yQdlPRQKTZL0jZJe9LjzBSXpOskDUnaIenM0jwDqf0eSQOl+BJJO9M810nSeG+kdY7zx9rl3LFWNPPN7CZgRVVsDXB7RCwCbk/PAc4HFqVhNfB5KBIQuAo4GzgLuKqShKnN6tJ81euyvN2E88facxPOHWtSw2IWEXcAh6rCK4H1aXw9cGEpviEKdwEzJM0BzgO2RcShiHgK2AasSNNOiojvRUQAG0rLsh7g/LF2OXesFce0OV9fRBwAiIgDkk5O8bnAY6V2wyk2Wny4RrwmSaspPknR19fH4OBg0ZkT4AOnH6nb2Uq7bjIyMtKV/Zokk54/9XIH8swfmLI55NxpU6/nS7vFrJ5ax5yjjXhNEbEWWAuwdOnS6O/vB+D6jZu4dmf9Tdl3aX/daZ0yODhIpf/2GxOWP/VyB/LMH3AOVXHuNNDr+dLu1YxPpK/ppMeDKT4MzC+1mwfsbxCfVyNuvc35Y+1y7lhN7RazzUDlqqABYFMpvipdWbQMOJwOCWwFlkuamU6+Lge2pmnPSlqWriRaVVqW9S7nj7XLuWM1NTzMKOlLQD8wW9IwxZVB1wC3SroM+AlwUWp+G3ABMAT8AngXQEQcknQ1cG9q97GIqJzYvZziqqUTgC1psB7h/LF2OXesFQ2LWURcUmfSOTXaBnBFneWsA9bViN8HnNaoH5Yn54+1y7ljrfAdQMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLXNcVM0gpJj0gakrSm0/2xfDh3bCycP72hK4qZpGnA54DzgcXAJZIWd7ZXlgPnjo2F86d3dEUxA84ChiLi0Yh4HrgFWNnhPlkenDs2Fs6fHnFMpzuQzAUeKz0fBs6ubiRpNbA6PR2R9Eganw08WW/h+tQ49XJ8jdrnDnltpzvQhrHmDuSZP9BdOZRj7kAT+dNDudNN+VJtzPnTLcVMNWLxkkDEWmDtS2aW7ouIpRPRsYmSY5+71JhyB/J9L3Ltd5dpmD+9kjs59bUd3XKYcRiYX3o+D9jfob5YXpw7NhbOnx7RLcXsXmCRpIWSjgMuBjZ3uE+WB+eOjYXzp0d0xWHGiDgi6UpgKzANWBcRu1pYRM1DAF0uxz53nXHIHcj3vci1311jiu17cupryxTxktMLZmZmWemWw4xmZmZtczEzM7PsZV/MOnErGkn7JO2U9ICk+1JslqRtkvakx5kpLknXpf7tkHRmaTkDqf0eSQOl+JK0/KE0r0Zbh7VnMnNH0jpJByU9VIp1LGdGW4c1Z5Lzx/ucRiIi24HihO1e4BTgOOBBYPEkrHcfMLsq9rfAmjS+BvhUGr8A2ELx/yzLgLtTfBbwaHqcmcZnpmn3AG9M82wBzh9tHR66P3eAPwLOBB7qhpyptw4PXZs/3uc0GHL/ZtZNt6JZCaxP4+uBC0vxDVG4C5ghaQ5wHrAtIg5FxFPANmBFmnZSRHwvigzaULWsWuuw1k1q7kTEHcChqnAnc6beOqw53bDv8T6nJPdiVutWNHMnYb0B/Juk+1Xc6gagLyIOAKTHkxv0cbT4cI34aOuw1nUqd8o6mTPdsP05m+zXz/ucBrri/8zGoKlbGU2AN0XEfkknA9sk/XCUtvX62Grcxlc3v86TkTPdvP05mOzXz/ucBnL/ZtaRW9FExP70eBD4BsUhhycqh2nS48EGfRwtPq9GnFHWYa3rhtsYdTJnumH7czapr5/3OY3lXswm/VY0kqZLekVlHFgOPJTWW7k6aADYlMY3A6vSFUbLgMPp6/pWYLmkmekKoeXA1jTtWUnL0hVFq6qWVWsd1rpuuI1RJ3Om3jqsOZOWP97nNKnTV6CMdaC4cuffKa4s+sgkrO8UiiuXHgR2VdYJvAq4HdiTHmeluCh+/G8vsBNYWlrWu4GhNLyrFF9Kkax7gc/y2zu11FyHh+7PHeBLwAHgBYpPwpd1MmdGW4eH7sof73OaG3w7KzMzy17uhxnNzMxczMzMLH8uZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyyN2WKmaR9kn4paUTSE5L+QdKJpek3SToi6TWl2Lmp7exS7HhJuyX9eRPrnJ7Wd1ud/jxfXnaKPyApJC2QtCXNPyLphdS+8vyGqvmuSvO9pdXXxhrrxfxJbaIUG5H0v9p/layWXsyd1P53JP29pCclHZZ0R7uv0bjo9A+qTeIP6e0D3pLG51L8EN016fl04Fng58CHqua7EdhYen418G3Sj9c1WOdAWuYRYE6N/jwC/EUpdnqKBbCgqv1NwMfrrOf3KH6Eb39lGz04fxrlD7AgtT2m069vLw+9mDsp/o/ALcCrgWnAkk6+zlPmm1lZRPwU2AKclkJvA54GPsZvfyK84i+BP5b0VkmnAVcCfxbp3WxgALgB2AFcWmP6zRQ/UV5uv6HZ7Sj5LPBXwPNtzGst6sH8sUnSK7kj6T8B/w1YHRE/i4gXI+L+ZuefCFOymEmaT/GT5z9IoQGKn7W/Bfh9SWdW2kbEYeByisRYB3w0IvY2sY7fBfqBjWlYVaPZXcBJkk6VNA14B8WnnVa25SLg+Yh4yeEEmxi9lD/JjyUNp8Nfsxs3t3b1UO6cDfwY+Gg6zLhT0ttamH/cTbVi9s+SngbuBL4DfDK98X8C/FNEPAHcTtUnpIj4JsWb/zLguibXtQrYEREPUyTr6yWdUaNd5RPSucAPgZ82uzHpuPsngfc3O4+NSU/lD/Ak8F+B1wJLgFdQ7Pxs/PVa7syj+HZ5GHgNxbfG9ZJObWEZ4+qYTq24Qy6MiG+VA5LeCeyOiAdSaCNwraQPRsQLpaa7gF9HxH80ua5VwBcAImK/pO9QJOoPqtrdDNwBLKT1Q0QfBW6OiB+1OJ+1p6fyJyJGgPvS0yckXQkckHRSRDzTyrKsoZ7KHeCXwAsU59KOAN+RtB1YDuxucVnjYqp9M6tlFXCKpMclPQ58GpgNnN/uAiX9AbAI+HBpuWcDl0g66gNERPwY+BHFoYevt7iqc4D3ltYxH7hV0l+123drWc75U61yLkZjXI41J+fc2dFuHyfKVPtmdhRJb6S4EvAM4GelSddSfJLZ3OaiB4BtHH2s+gSKBDgf+GZV+8uAmRHxXHXCNXAOcGzp+b0UJ423tNxja1nu+SPpbIqLD/YAMykOYw2mczU2gXLPHYpvdD+hKJr/h6Jg9gMfarPfYzalixnFG78pInaWg5I+A3xX0qyIONTKAiW9HPhTYFVEPF417ea0zqMSqpmTurVExM+rlv8i8FQ6fGQTL+v8AU6hOOd6MvAMxU7wkjaXZa3JOnci4gVJK4EvAmsoLgZZFRE/bGd540HNXeVpZmbWvXzOzMzMsudi1iZJl+ro2wBVhl2d7pt1P+ePtcu5U5sPM5qZWfYaXgCSTireARyf2n81Iq6StJDiv9ZnAd8H3hkRz0s6nuJ/FpZQ3BvsHRGxLy3rwxRXz7wIvDcitqb4CuAzFPf3+mJEXNOoX7Nnz44FCxYA8NxzzzF9+vQWNrt3jOe233///U9GxKvHZWHkkTuQZ/50W5/HO3egO/OnF3KnExq9TuOSP41u3kjxPycnpvFjgbuBZcCtwMUpfgNweRp/D3BDGr8Y+HIaXww8SJGYC4G9FAk0LY2fAhyX2ixu1K8lS5ZExfbt22OqGs9tB+6L8b3Batfnzni/hpOl2/o83rkTXZo/vZA7ndDodRqP/Gl4ziytq3Kp97FpCODNwFdTfD1wYRpfmZ6Tpp8jSSl+S0T8Ooo7VgwBZ6VhKCIejYjnKT5xrWzUL+t+zh0bC+ePtaKp/zNLN6K8H3gd8DmKTzNPR3EbE4Bhip82ID0+BhARRyQdBl6V4neVFlue57Gq+Nktb4l1pW7JHUmrgdUAfX19DA4O/mbayMjIUc9zkGOf29EN+dNrudMJk/E6NVXMIuJF4A2SZgDfAGrdTHK0W+HEKPFa3w5rXpVSL6mmckJ1+7Z3S+5ExFpgLcDSpUujv7//N9MGBwcpP89Bjn1uRzfkT6/lTidMxuvU0h1AIuJpSYMUx61nSDomfUKaR/HDkFB8upkPDKfbo7wSOFSKV5TnqRevXn/NpLp+4yauvfO5uv3ed81bm97G3OTyx9Tp3BnNzp8e5n+s+de603s5f3LRrfnj3OkeDc+ZSXp1+lSEpBOAt1DcFXk78PbUbADYlMY389ufMXg78O10gm8zcLGKn/5eSHEzzHso7ie4SNJCScdRnLht975k1kWcOzYWzh9rRTPfzOZQ/E7NNIrid2tE/Iukh4FbJH2c4qcFbkztbwRuljRE8anoYoCI2CXpVuBhip/yviIdQkDFT09spbi6aF1ETOl//ushzh0bC+ePNa1hMYuIHRR3dq6OP0pxNVB1/FfARXWW9QngEzXitwH+peQe49yxsXD+WCt8OyszM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZjZhJE0X9J2Sbsl7ZL0vhSfJWmbpD3pcWaKS9J1koYk7ZB0ZmlZA6n9HkkDpfgSSTvTPNdJ0uRvqU0E54+1omExc0LZGBwBPhARpwLLgCskLQbWALdHxCLg9vQc4HxgURpWA5+HIteAq4CzgbOAqyr5ltqsLs23YhK2yyaH88ea1sw3MyeUtSUiDkTE99P4s8BuYC6wElifmq0HLkzjK4ENUbgLmCFpDnAesC0iDkXEU8A2YEWadlJEfC8iAthQWpZlzvljrWhYzJxQNh4kLQDOAO4G+iLiABT5BZycms0FHivNNpxio8WHa8Stxzh/rJFjWmk8WkJJmvCEkrSa4hscfX19DA4OAtB3Anzg9CN1+11p14tGRka6fvsknQh8DXh/RDwzylHkWhOijXitPtTMHcgzf3J438dLp/On13KnEyYjX5suZp1OKICIWAusBVi6dGn09/cDcP3GTVy7s/6m7Lu0v+603A0ODlJ5HbqRpGMp8mZjRHw9hZ+QNCd9CJoDHEzxYWB+afZ5wP4U76+KD6b4vBrtX6Je7kCe+dPt7/t46Yb86bXc6YTJyNemrmYcLaHS9GYTql68qR2S5SVdyHMjsDsiPl2atBmoXAA0AGwqxVeli4iWAYfTt/+twHJJM9N51uXA1jTtWUnL0rpWlZZlmXP+WCuauZrRCWXtehPwTuDNkh5IwwXANcC5kvYA56bnALcBjwJDwBeA9wBExCHgauDeNHwsxQAuB76Y5tkLbJmMDbNJ4fyxpjVzmLGSUDslPZBif02RQLdKugz4CXBRmnYbcAFFcvwCeBcUCSWpklDw0oS6CTiBIpmcUD0gIu6k9mFkgHNqtA/gijrLWgesqxG/DzhtDN20LuX8sVY0LGZOKDMz63a+A4iZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2GhYzSeskHZT0UCk2S9I2SXvS48wUl6TrJA1J2iHpzNI8A6n9HkkDpfgSSTvTPNdJ0nhvpHWO88fa5dyxVjTzzewmYEVVbA1we0QsAm5PzwHOBxalYTXweSgSELgKOBs4C7iqkoSpzerSfNXrsrzdhPPH2nMTzh1rUsNiFhF3AIeqwiuB9Wl8PXBhKb4hCncBMyTNAc4DtkXEoYh4CtgGrEjTToqI70VEABtKy7Ie4Pyxdjl3rBXHtDlfX0QcAIiIA5JOTvG5wGOldsMpNlp8uEa8JkmrKT5J0dfXx+DgYNGZE+ADpx+p29lKu140MjKS4/ZNev7Uyx3IM38yfd/Hg3MnQ5ORr+0Ws3pqHXOONuI1RcRaYC3A0qVLo7+/H4DrN27i2p31N2Xfpf11p+VucHCQyuvQAyYsf+rlDuSZPz32vo8H504Xm4x8bfdqxifS13TS48EUHwbml9rNA/Y3iM+rEbfe5vyxdjl3rKZ2i9lmoHJV0ACwqRRfla4sWgYcTocEtgLLJc1MJ1+XA1vTtGclLUtXEq0qLct6l/PH2uXcsZoaHmaU9CWgH5gtaZjiyqBrgFslXQb8BLgoNb8NuAAYAn4BvAsgIg5Juhq4N7X7WERUTuxeTnHV0gnAljRYj3D+WLucO9aKhsUsIi6pM+mcGm0DuKLOctYB62rE7wNOa9QPy5Pzx9rl3LFW+A4gZmaWPRczMzPLnouZmZllz8XMzMyy52JmZmbZczEzM7PsuZiZmVn2XMzMzCx7LmZmZpY9FzMzM8uei5mZmWXPxczMzLLnYmZmZtlzMTMzs+y5mJmZWfZczMzMLHsuZmZmlj0XMzMzy56LmZmZZc/FzMzMsudiZmZm2XMxMzOz7HVNMZO0QtIjkoYkrel0fywfzh0bC+dPb+iKYiZpGvA54HxgMXCJpMWd7ZXlwLljY+H86R3HdLoDyVnAUEQ8CiDpFmAl8PB4LHzBmn8ddfq+a946HquxzpjQ3IHG+TMa51bX876nR3RLMZsLPFZ6PgycXd1I0mpgdXo6IumRND4beLLdletT7c7ZFca07VVeO07LmUxjzR0Y39fw6PVOXG5NWJ/blGPuQBP5M5G5k/m+pxWNXqcx50+3FDPViMVLAhFrgbUvmVm6LyKWTkTHut1U3vZkTLkDeb6GOfa5SzXMn17LnU6YjNepK86ZUXwaml96Pg/Y36G+WF6cOzYWzp8e0S3F7F5gkaSFko4DLgY2d7hPlgfnjo2F86dHdMVhxog4IulKYCswDVgXEbtaWETNQwBTxFTe9vHIHcjzNcyxz13H+55JM+GvkyJecnrBzMwsK91ymNHMzIT8XAIAAAPVSURBVKxtLmZmZpa97ItZzreikbRP0k5JD0i6L8VmSdomaU96nJniknRd2s4dks4sLWcgtd8jaaAUX5KWP5Tm1WjrmGo6kTuS5kvaLmm3pF2S3pfifyPppykXHpB0QWmeD6c+PiLpvEb9Txcz3J3e3y+nCxuQdHx6PpSmL5iMbe5VOe97WpHNfioish0oTtjuBU4BjgMeBBZ3ul8t9H8fMLsq9rfAmjS+BvhUGr8A2ELxfzHLgLtTfBbwaHqcmcZnpmn3AG9M82wBzh9tHVNp6FTuAHOAM9P4K4B/p7iN0t8AH6zRfnHq2/HAwtTnaaP1H7gVuDiN3wBcnsbfA9yQxi8Gvtzp9yHXIfd9T4vbmsV+KvdvZr+5FU1EPA9UbkWTs5XA+jS+HriwFN8QhbuAGZLmAOcB2yLiUEQ8BWwDVqRpJ0XE96LIhg1Vy6q1jqmkI7kTEQci4vtp/FlgN8VdKOpZCdwSEb+OiB8BQxR9r9n/9Kn2zcBX0/zVOVR5378KnFP5FGwt68V9Tyu6bj+VezGrdSua0XYM3SaAf5N0v4pb5gD0RcQBKHZ8wMkpXm9bR4sP14iPto6ppOO5kw7znQHcnUJXpkMz60qHVFp9318FPB0RR6riRy0rTT+c2lvrOp4/kyiL/VRX/J/ZGDR1K6Mu9qaI2C/pZGCbpB+O0rbetrYat0JHXx9JJwJfA94fEc9I+jxwderD1cC1wLtH6WetD6KN3nfnxPiZSq9lFvup3L+ZZX0rmojYnx4PAt+gOHTxRPrqTXo8mJrX29bR4vNqxBllHVNJx3JH0rEUhWxjRHwdICKeiIgXI+I/gC9Q5MJo/awXf5Li0M4xVfGjlpWmvxI4NL5bN2Vkve9pRS77qdyLWba3opE0XdIrKuPAcuAhiv5XrvQZADal8c3AqnS10DLgcPrqvRVYLmlmOjS1HNiapj0raVk6L7Kqalm11jGVdCR30ntxI7A7Ij5dis8pNfvvFLlA6tPF6UrEhcAiihPmNfufzjtsB96e5q/Oocr7/nbg26m9tS7bfU8rstpPdfpKmXG40uYCiivC9gIf6XR/Wuj3KRRXQD0I7Kr0neIcxu3AnvQ4K8VF8SOCe4GdwNLSst5NcWHAEPCuUnxpSry9wGf57R1faq5jqg2dyB3gDykOo+wAHkjDBcDN6X3dkf6I55Tm+Ujq4yOkK71G63/KrXtSPnwFOD7FX56eD6Xpp3T6Pch5yHXf0+I2ZrOf8u2szMwse7kfZjQzM3MxMzOz/LmYmZlZ9lzMzMwsey5mZmaWPRczMzPLnouZmZll7/8DgrC436upn5EAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "def draw_histograms(df, variables, n_rows, n_cols, n_bins):\n", + " fig=plt.figure()\n", + " for i, var_name in enumerate(variables):\n", + " ax=fig.add_subplot(n_rows,n_cols,i+1)\n", + " df[var_name].hist(bins=n_bins,ax=ax)\n", + " ax.set_title(var_name)\n", + " fig.tight_layout() # Improves appearance a bit.\n", + " plt.show()\n", + "\n", + "PAY = df[['PAY_0','PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6']]\n", + "BILLAMT = df[['BILL_AMT1','BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6']]\n", + "PAYAMT = df[['PAY_AMT1','PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6']]\n", + "\n", + "draw_histograms(PAY, PAY.columns, 2, 3, 10)\n", + "draw_histograms(BILLAMT, BILLAMT.columns, 2, 3, 10)\n", + "draw_histograms(PAYAMT, PAYAMT.columns, 2, 3, 10)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "C6c_Gz6wUrJ8" + }, + "source": [ + "We observe that the \"repayment status\" attributes are the most highly correlated with the target variable and we would expect them to be more significant in predicting credit default. In fact the later the status (pay_0 is later than pay_6), the more correlated it is.\n", + "\n", + "Now that we have an idea of the features, we will move on to feature selection and data preparation." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "AQBksEyEf4Sf" + }, + "source": [ + "## Data Preprocessing\n", + "\n", + "It was previously mentioned that our data had a bit of noise, so we will clean up the data in this part. Additionally, we will conduct some feature selection.\n", + "1. Removing Noise - Inconsistencies\n", + "2. Dealing with negative values of PAY_0 to PAY_6\n", + "3. Outliers\n", + "4. One Hot Encoding\n", + "5. Train Test Split\n", + "6. Feature selection\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Removing Noise\n", + "First, we found in our data exploration that education has unknown groups 0, 5 and 6. These will be replaced with Education = Others, which has value 4" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([2, 1, 3, 4], dtype=int64)" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['EDUCATION'].replace([0,5,6], 4, regex=True, inplace=True)\n", + "df[\"EDUCATION\"].unique()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly, for Marriage" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([1, 2, 3], dtype=int64)" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['MARRIAGE'].replace([0], 3, regex=True, inplace=True)\n", + "df[\"MARRIAGE\"].unique()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Separating negative and positive values for PAY_0 to PAY_6" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Second, we are going to extract the negative values of PAY_0 to PAY_6 as another categorical feature. This way, PAY_0 to PAY_6 can be thought of purely as the months of delay of payments.\n", + "\n", + "The negative values will form a categorical variable. e.g. negative values of PAY_0 will form the categorical variable S_0." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(0,7):\n", + " try:\n", + " df[\"S_\" + str(i)] = [x if x < 1 else 1 for x in df[\"PAY_\" + str(i)]]\n", + " except:\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dummy variables for negative values\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
S_0S_2S_3S_4S_5S_6
ID
111-1-1-2-2
2-110001
3000000
4000000
5-10-1000
.....................
29995111111
29996000000
29997-1-1-1-100
29998111-100
30000000000
\n", + "

26245 rows × 6 columns

\n", + "
" + ], + "text/plain": [ + " S_0 S_2 S_3 S_4 S_5 S_6\n", + "ID \n", + "1 1 1 -1 -1 -2 -2\n", + "2 -1 1 0 0 0 1\n", + "3 0 0 0 0 0 0\n", + "4 0 0 0 0 0 0\n", + "5 -1 0 -1 0 0 0\n", + "... ... ... ... ... ... ...\n", + "29995 1 1 1 1 1 1\n", + "29996 0 0 0 0 0 0\n", + "29997 -1 -1 -1 -1 0 0\n", + "29998 1 1 1 -1 0 0\n", + "30000 0 0 0 0 0 0\n", + "\n", + "[26245 rows x 6 columns]" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "print('Dummy variables for negative values')\n", + "df[[\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\"]]" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "#attributes representing positive values\n", + "for col in [\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"]:\n", + " df[col].replace([0,-1,-2], 0, regex=True, inplace=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Outliers\n", + "Next, we would like to remove outliers from the continuous variables. Assuming that all the data points are normally distributed, we will consider a point an outlier if it falls outside the 99% interval of a distribution. (Critical value = 2.58) " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...PAY_AMT4PAY_AMT5PAY_AMT6YS_0S_2S_3S_4S_5S_6
count26245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.000000...26245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.00000026245.000000
mean149324.8999811.6089541.8527341.56471735.0065920.3721090.3373210.3246330.2782240.238750...2787.4250712778.8306732822.2850070.230177-0.133587-0.300438-0.327300-0.364412-0.395999-0.428158
std116558.6165300.4879940.7385720.5219368.8320280.7657300.8148780.8114910.7863140.743923...4835.0819064751.2632875271.1981000.4209540.8798760.8834720.8952640.8861150.8777890.900723
min10000.0000001.0000001.0000001.00000021.0000000.0000000.0000000.0000000.0000000.000000...0.0000000.0000000.0000000.000000-2.000000-2.000000-2.000000-2.000000-2.000000-2.000000
25%50000.0000001.0000001.0000001.00000028.0000000.0000000.0000000.0000000.0000000.000000...150.00000082.0000000.0000000.000000-1.000000-1.000000-1.000000-1.000000-1.000000-1.000000
50%120000.0000002.0000002.0000002.00000034.0000000.0000000.0000000.0000000.0000000.000000...1200.0000001218.0000001143.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000
75%210000.0000002.0000002.0000002.00000041.0000000.0000000.0000000.0000000.0000000.000000...3118.0000003140.0000003069.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000
max500000.0000002.0000004.0000003.00000059.0000008.0000008.0000008.0000008.0000008.000000...45171.00000044197.00000051000.0000001.0000001.0000001.0000001.0000001.0000001.0000001.000000
\n", + "

8 rows × 30 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean 149324.899981 1.608954 1.852734 1.564717 35.006592 \n", + "std 116558.616530 0.487994 0.738572 0.521936 8.832028 \n", + "min 10000.000000 1.000000 1.000000 1.000000 21.000000 \n", + "25% 50000.000000 1.000000 1.000000 1.000000 28.000000 \n", + "50% 120000.000000 2.000000 2.000000 2.000000 34.000000 \n", + "75% 210000.000000 2.000000 2.000000 2.000000 41.000000 \n", + "max 500000.000000 2.000000 4.000000 3.000000 59.000000 \n", + "\n", + " PAY_0 PAY_2 PAY_3 PAY_4 PAY_5 \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean 0.372109 0.337321 0.324633 0.278224 0.238750 \n", + "std 0.765730 0.814878 0.811491 0.786314 0.743923 \n", + "min 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "25% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "50% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "75% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "max 8.000000 8.000000 8.000000 8.000000 8.000000 \n", + "\n", + " ... PAY_AMT4 PAY_AMT5 PAY_AMT6 Y \\\n", + "count ... 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean ... 2787.425071 2778.830673 2822.285007 0.230177 \n", + "std ... 4835.081906 4751.263287 5271.198100 0.420954 \n", + "min ... 0.000000 0.000000 0.000000 0.000000 \n", + "25% ... 150.000000 82.000000 0.000000 0.000000 \n", + "50% ... 1200.000000 1218.000000 1143.000000 0.000000 \n", + "75% ... 3118.000000 3140.000000 3069.000000 0.000000 \n", + "max ... 45171.000000 44197.000000 51000.000000 1.000000 \n", + "\n", + " S_0 S_2 S_3 S_4 S_5 \\\n", + "count 26245.000000 26245.000000 26245.000000 26245.000000 26245.000000 \n", + "mean -0.133587 -0.300438 -0.327300 -0.364412 -0.395999 \n", + "std 0.879876 0.883472 0.895264 0.886115 0.877789 \n", + "min -2.000000 -2.000000 -2.000000 -2.000000 -2.000000 \n", + "25% -1.000000 -1.000000 -1.000000 -1.000000 -1.000000 \n", + "50% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "75% 0.000000 0.000000 0.000000 0.000000 0.000000 \n", + "max 1.000000 1.000000 1.000000 1.000000 1.000000 \n", + "\n", + " S_6 \n", + "count 26245.000000 \n", + "mean -0.428158 \n", + "std 0.900723 \n", + "min -2.000000 \n", + "25% -1.000000 \n", + "50% 0.000000 \n", + "75% 0.000000 \n", + "max 1.000000 \n", + "\n", + "[8 rows x 30 columns]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from scipy import stats\n", + "#we are only concerned with the ordinal data\n", + "o = pd.DataFrame(df.drop(['Y','EDUCATION', 'MARRIAGE', \"SEX\",\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\",\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"], axis=1))\n", + "#rows where the absolute z score of all columns are less than 2.58 (critical value)\n", + "rows = (np.abs(stats.zscore(o)) < 2.58).all(axis=1)\n", + "df = df[rows]\n", + "df.describe()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Feature Scaling\n", + "The models used subsequently may have difficulty converging before the maximum number of iterations allowed\n", + "is reached if the data is not normalized. Additionaly, Multi-layer Perceptron is sensitive to feature scaling, so we will use StandardScaler for standardization. We only want to scale the numerical factors." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.preprocessing import MinMaxScaler\n", + "scaler = MinMaxScaler()\n", + "cols = df.drop(['Y','EDUCATION', 'MARRIAGE', \"SEX\",\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\",\"PAY_0\", \"PAY_2\", \"PAY_3\", \"PAY_4\", \"PAY_5\", \"PAY_6\"], axis =1)\n", + "df1 = df.copy()\n", + "df1[cols.columns] = scaler.fit_transform(cols)\n", + "df = df1" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXEDUCATIONMARRIAGEAGEPAY_0PAY_2PAY_3PAY_4PAY_5...PAY_AMT4PAY_AMT5PAY_AMT6YS_0S_2S_3S_4S_5S_6
ID
10.0204082210.07894722000...0.0000000.0000000.000000111-1-1-2-2
20.2244902220.13157902000...0.0221380.0000000.0392161-110001
30.1632652220.34210500000...0.0221380.0226260.0980390000000
40.0816332210.42105300000...0.0243520.0241870.0196080000000
50.0816331210.94736800000...0.1992430.0155890.0133140-10-1000
\n", + "

5 rows × 30 columns

\n", + "
" + ], + "text/plain": [ + " LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 \\\n", + "ID \n", + "1 0.020408 2 2 1 0.078947 2 2 0 0 \n", + "2 0.224490 2 2 2 0.131579 0 2 0 0 \n", + "3 0.163265 2 2 2 0.342105 0 0 0 0 \n", + "4 0.081633 2 2 1 0.421053 0 0 0 0 \n", + "5 0.081633 1 2 1 0.947368 0 0 0 0 \n", + "\n", + " PAY_5 ... PAY_AMT4 PAY_AMT5 PAY_AMT6 Y S_0 S_2 S_3 S_4 S_5 S_6 \n", + "ID ... \n", + "1 0 ... 0.000000 0.000000 0.000000 1 1 1 -1 -1 -2 -2 \n", + "2 0 ... 0.022138 0.000000 0.039216 1 -1 1 0 0 0 1 \n", + "3 0 ... 0.022138 0.022626 0.098039 0 0 0 0 0 0 0 \n", + "4 0 ... 0.024352 0.024187 0.019608 0 0 0 0 0 0 0 \n", + "5 0 ... 0.199243 0.015589 0.013314 0 -1 0 -1 0 0 0 \n", + "\n", + "[5 rows x 30 columns]" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### One-Hot Encoding for Categorical attributes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In some models, categorical variables which are encoded numerically will be erroneously treated as ordinal data. To understand why this is a problem, consider the \"Education\" column for our dataset.\n", + "\n", + "A logistic regression model, for example, will assume that the difference in odds of default between education = 1 and education = 2 is the same as the difference between education = 2 and 3. This is wrong because the difference in odds between a graduate degree and university (1 and 2) is likely to be different from that between univeristy education and high school education (2 and 3).\n", + "\n", + "One hot encoding will allow our models to treat these columns explicitly as categorical features.\n", + "\n", + "The following categorical columns will be one-hot encoded\n", + "\n", + "1. EDUCATION\n", + "2. MARRIAGE\n", + "3. S0 - S6\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.preprocessing import OneHotEncoder" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "onenc = OneHotEncoder(categories='auto')" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
GRADUNIHSMARRIEDSINGLE
00.01.00.01.00.0
10.01.00.00.01.0
20.01.00.00.01.0
30.01.00.01.00.0
40.01.00.01.00.0
\n", + "
" + ], + "text/plain": [ + " GRAD UNI HS MARRIED SINGLE\n", + "0 0.0 1.0 0.0 1.0 0.0\n", + "1 0.0 1.0 0.0 0.0 1.0\n", + "2 0.0 1.0 0.0 0.0 1.0\n", + "3 0.0 1.0 0.0 1.0 0.0\n", + "4 0.0 1.0 0.0 1.0 0.0" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#one hot encoding for EDUCATION and MARRIAGE\n", + "onehot = pd.DataFrame(onenc.fit_transform(df[['EDUCATION', 'MARRIAGE']]).toarray())\n", + "onehot.columns= names = [\"GRAD\",\"UNI\",\"HS\",\"OTHER-EDU\",\"MARRIED\",\"SINGLE\",\"OTHER_MS\"]\n", + "#drop one of each category to prevent dummy variable trap\n", + "onehot = onehot.drop([\"OTHER-EDU\", \"OTHER_MS\"], axis = 1)\n", + "onehot.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PAY_0_No_TransactionsPAY_0_Pay_DulyPAY_0_Revolving_CreditPAY_2_No_TransactionsPAY_2_Pay_DulyPAY_2_Revolving_CreditPAY_3_No_TransactionsPAY_3_Pay_DulyPAY_3_Revolving_CreditPAY_4_No_TransactionsPAY_4_Pay_DulyPAY_4_Revolving_CreditPAY_5_No_TransactionsPAY_5_Pay_DulyPAY_5_Revolving_CreditPAY_6_No_TransactionsPAY_6_Pay_DulyPAY_6_Revolving_Credit
00.00.00.00.00.00.00.01.00.00.01.00.01.00.00.01.00.00.0
10.01.00.00.00.00.00.00.01.00.00.01.00.00.01.00.00.00.0
20.00.01.00.00.01.00.00.01.00.00.01.00.00.01.00.00.01.0
30.00.01.00.00.01.00.00.01.00.00.01.00.00.01.00.00.01.0
40.01.00.00.00.01.00.01.00.00.00.01.00.00.01.00.00.01.0
\n", + "
" + ], + "text/plain": [ + " PAY_0_No_Transactions PAY_0_Pay_Duly PAY_0_Revolving_Credit \\\n", + "0 0.0 0.0 0.0 \n", + "1 0.0 1.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 1.0 0.0 \n", + "\n", + " PAY_2_No_Transactions PAY_2_Pay_Duly PAY_2_Revolving_Credit \\\n", + "0 0.0 0.0 0.0 \n", + "1 0.0 0.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_3_No_Transactions PAY_3_Pay_Duly PAY_3_Revolving_Credit \\\n", + "0 0.0 1.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 1.0 0.0 \n", + "\n", + " PAY_4_No_Transactions PAY_4_Pay_Duly PAY_4_Revolving_Credit \\\n", + "0 0.0 1.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_5_No_Transactions PAY_5_Pay_Duly PAY_5_Revolving_Credit \\\n", + "0 1.0 0.0 0.0 \n", + "1 0.0 0.0 1.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 \n", + "\n", + " PAY_6_No_Transactions PAY_6_Pay_Duly PAY_6_Revolving_Credit \n", + "0 1.0 0.0 0.0 \n", + "1 0.0 0.0 0.0 \n", + "2 0.0 0.0 1.0 \n", + "3 0.0 0.0 1.0 \n", + "4 0.0 0.0 1.0 " + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#one hot encoding for S_0 to S_6\n", + "onehot_PAY = pd.DataFrame(onenc.fit_transform(df[['S_0', 'S_2', 'S_3', 'S_4', 'S_5', 'S_6']]).toarray())\n", + "onehot_PAY.columns= onenc.fit(df[[\"S_0\", \"S_2\", \"S_3\", \"S_4\", \"S_5\", \"S_6\"]]).get_feature_names()\n", + "#drop one of each category to prevent dummy variable trap\n", + "#onehot = onehot.drop([\"OTHER-EDU\", \"OTHER_MS\"], axis = 1)\n", + "names = []\n", + "for X in range(0,7):\n", + " if X == 1:\n", + " continue\n", + " names.append(\"PAY_\"+str(X)+\"_No_Transactions\")\n", + " names.append(\"PAY_\"+str(X)+\"_Pay_Duly\")\n", + " names.append(\"PAY_\"+str(X)+\"_Revolving_Credit\")\n", + " try:\n", + " onehot_PAY = onehot_PAY.drop(\"x\" + str(X) +\"_1\", axis =1)\n", + " except:\n", + " onehot_PAY = onehot_PAY.drop(\"x1_1\", axis =1)\n", + "onehot_PAY.columns = names\n", + "onehot_PAY.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Index(['LIMIT_BAL', 'SEX', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5',\n", + " 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4',\n", + " 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3',\n", + " 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'Y', 'GRAD', 'UNI', 'HS', 'MARRIED',\n", + " 'SINGLE', 'PAY_0_No_Transactions', 'PAY_0_Pay_Duly',\n", + " 'PAY_0_Revolving_Credit', 'PAY_2_No_Transactions', 'PAY_2_Pay_Duly',\n", + " 'PAY_2_Revolving_Credit', 'PAY_3_No_Transactions', 'PAY_3_Pay_Duly',\n", + " 'PAY_3_Revolving_Credit', 'PAY_4_No_Transactions', 'PAY_4_Pay_Duly',\n", + " 'PAY_4_Revolving_Credit', 'PAY_5_No_Transactions', 'PAY_5_Pay_Duly',\n", + " 'PAY_5_Revolving_Credit', 'PAY_6_No_Transactions', 'PAY_6_Pay_Duly',\n", + " 'PAY_6_Revolving_Credit'],\n", + " dtype='object')" + ] + }, + "execution_count": 97, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1 = df.drop(['EDUCATION', 'MARRIAGE','S_0', 'S_2', 'S_3', 'S_4', 'S_5', 'S_6'], axis = 1)\n", + "df1 = pd.concat([df1.reset_index(drop=True), onehot], axis=1)\n", + "df1 = pd.concat([df1.reset_index(drop=True), onehot_PAY], axis=1)\n", + "df1.columns" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
LIMIT_BALSEXAGEPAY_0PAY_2PAY_3PAY_4PAY_5PAY_6BILL_AMT1...PAY_3_Revolving_CreditPAY_4_No_TransactionsPAY_4_Pay_DulyPAY_4_Revolving_CreditPAY_5_No_TransactionsPAY_5_Pay_DulyPAY_5_Revolving_CreditPAY_6_No_TransactionsPAY_6_Pay_DulyPAY_6_Revolving_Credit
\n", + "

0 rows × 45 columns

\n", + "
" + ], + "text/plain": [ + "Empty DataFrame\n", + "Columns: [LIMIT_BAL, SEX, AGE, PAY_0, PAY_2, PAY_3, PAY_4, PAY_5, PAY_6, BILL_AMT1, BILL_AMT2, BILL_AMT3, BILL_AMT4, BILL_AMT5, BILL_AMT6, PAY_AMT1, PAY_AMT2, PAY_AMT3, PAY_AMT4, PAY_AMT5, PAY_AMT6, Y, GRAD, UNI, HS, MARRIED, SINGLE, PAY_0_No_Transactions, PAY_0_Pay_Duly, PAY_0_Revolving_Credit, PAY_2_No_Transactions, PAY_2_Pay_Duly, PAY_2_Revolving_Credit, PAY_3_No_Transactions, PAY_3_Pay_Duly, PAY_3_Revolving_Credit, PAY_4_No_Transactions, PAY_4_Pay_Duly, PAY_4_Revolving_Credit, PAY_5_No_Transactions, PAY_5_Pay_Duly, PAY_5_Revolving_Credit, PAY_6_No_Transactions, PAY_6_Pay_Duly, PAY_6_Revolving_Credit]\n", + "Index: []\n", + "\n", + "[0 rows x 45 columns]" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check for perfect collinearity\n", + "corr = df1.corr()\n", + "for i in range(len(corr)):\n", + " corr.iloc[i,i] = 0\n", + "#corr[corr == 1] = 0\n", + "corr[corr.eq(1).any(1)]" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data has 45 Columns and 26245 Rows\n" + ] + } + ], + "source": [ + "size = df1.shape\n", + "print(\"Data has {} Columns and {} Rows\".format(size[1], size[0]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Train Test Split\n", + "\n", + "Before we conduct feature selection and model selection, we split the data using a train test split according to the project description." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.metrics import *\n", + "from sklearn.model_selection import *" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "VOB68z_hM1jW" + }, + "outputs": [], + "source": [ + "#using holdout sampling for train test split\n", + "ft = df1.drop(\"Y\", axis = 1)\n", + "target = df1[\"Y\"]\n", + "X_train,X_test,y_train,y_test = train_test_split(ft,target,test_size=0.33333)\n", + "#make the results reproducible (according to instructions)\n", + "np.random.seed(123) " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Filter method for feature selection\n", + "The filter method for feature selection entails selecting relevant attributes before moving on to learning phase.\n", + "We will utitlise univariate feature selection to reduce the features to the fewer more significant attributes. " + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Significant values are:\n", + " 0 pval\n", + "LIMIT_BAL 81.747784 3.335170e-04\n", + "PAY_0 4215.816943 0.000000e+00\n", + "PAY_2 3730.186349 0.000000e+00\n", + "PAY_3 3017.725262 0.000000e+00\n", + "PAY_4 3020.555618 0.000000e+00\n", + "PAY_5 2812.887854 0.000000e+00\n", + "PAY_6 2575.321192 0.000000e+00\n", + "PAY_0_No_Transactions 63.305610 2.349730e-02\n", + "PAY_0_Revolving_Credit 454.630595 0.000000e+00\n", + "PAY_2_Pay_Duly 76.591060 1.225866e-03\n", + "PAY_2_Revolving_Credit 241.568619 0.000000e+00\n", + "PAY_3_Pay_Duly 88.958886 4.790331e-05\n", + "PAY_3_Revolving_Credit 140.257620 3.064993e-12\n", + "PAY_4_Pay_Duly 82.933933 2.446182e-04\n", + "PAY_4_Revolving_Credit 98.729838 2.851222e-06\n", + "PAY_5_Pay_Duly 67.720209 9.456393e-03\n", + "PAY_5_Revolving_Credit 70.202609 5.486203e-03\n", + "PAY_6_Revolving_Credit 73.167377 2.782902e-03\n" + ] + } + ], + "source": [ + "from sklearn.feature_selection import SelectKBest\n", + "from sklearn.feature_selection import chi2\n", + "\n", + "selector = SelectKBest( score_func = chi2, k=10)\n", + "selector.fit(X_train, y_train)\n", + "np.set_printoptions(precision=10)\n", + "chi2data = pd.DataFrame(selector.scores_)\n", + "chi2data[\"pval\"] = 1 - stats.chi2.cdf(chi2data, 43)\n", + "chi2data.index = X_train.columns\n", + "\n", + "print(\"Significant values are:\")\n", + "print(chi2data[chi2data[\"pval\"] < 0.05])\n", + "\n", + "cols = chi2data[chi2data[\"pval\"] < 0.05].index\n", + "X_train_filter = X_train[cols]\n", + "X_test_filter = X_test[cols]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "mbhlIlQzZz7c" + }, + "source": [ + "## Model Selection\n", + "\n", + "In this part, we will fit machine learning models learnt in BT2101 to this classification problem, and pick the model that can produce the best results.\n", + "\n", + "We will be attempting to fit the following models:\n", + "\n", + "\n", + "- Decision Tree \n", + "- Logistic Regression\n", + "- Support Vector Machine\n", + "- Neural Network\n", + "\n", + "To make things easier, we define a get_roc function that will plot an ROC curve for all the models we evaluate, and a confusion matrix function.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "def get_roc(model, y_test, X_test, name):\n", + " try:\n", + " fpr = roc_curve(y_test,model.predict_proba(X_test)[:,1])[0]\n", + " tpr = roc_curve(y_test,model.predict_proba(X_test)[:,1])[1]\n", + " thresholds = roc_curve(y_test,model.predict_proba(X_test)[:,1])[2]\n", + " except:\n", + " fpr = roc_curve(y_test,model.predict(X_test))[0]\n", + " tpr = roc_curve(y_test,model.predict(X_test))[1]\n", + " thresholds = roc_curve(y_test,model.predict(X_test))[2]\n", + " plt.plot([0, 1], [0, 1], color='navy', linestyle='--')\n", + " plt.xlim([0.0, 1.0])\n", + " plt.ylim([0.0, 1.05])\n", + " plt.xlabel('False Positive Rate')\n", + " plt.ylabel('True Positive Rate')\n", + " plt.title('Receiver operating characteristic for ' + name)\n", + " plt.plot(fpr,tpr,label='ROC curve (AUC = %0.2f)' % (auc(fpr, tpr)))\n", + " plt.legend(loc=\"lower right\")\n", + " \n", + " #find- best threshold\n", + " optimal_idx = np.argmax(tpr - fpr)\n", + " optimal_threshold = thresholds[optimal_idx]\n", + " print(\"Optimal Threshold: \" + str(optimal_threshold))\n", + " \n", + " plt.show()\n", + " \n", + " return auc(fpr, tpr)" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "def confusion(y_test, predictions, name):\n", + " conf = pd.crosstab(y_test,predictions, rownames=['Actual'], colnames=['Predicted'])\n", + " print(\"Of \" + str(conf[0][1] + conf[1][1]) + \" Defaulters, the \" + name + \" identified \" + str(conf[1][1])) \n", + " return conf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Evaluation \n", + "We will select the model based on the model evaluation. The key metrics we will compute are:\n", + "\n", + "1. Accuracy\n", + "2. Recall\n", + "3. AUROC\n", + "\n", + "Because of the nature of a default detection problem, we would like to prioritise **recall** for defaults. \n", + "This means we will place more importance in correctly identifying a defaulter than avoiding misclassifying a non-defaulter. (Assumming that the bank loses more money when lending to a defaulter than not lending to a non-defaulter)\n", + "\n", + "However, simply predicting every data point as a defaulter will give us 100% recall. We have to also consider accuracy and AUROC to get a better idea of how our model performs.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation = pd.DataFrame(columns=['Model', 'Recall-1', 'AUROC'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "H89tM6NvaN17" + }, + "source": [ + "### Decision Trees\n", + "\n", + "#### Theory:\n", + "The decision tree algorithm aims to recursively split the data points in the training set until the data points are completely separated or well separated. At each iteration, the tree splits the datasets by the feature(s) that give the maximum reduction in heterogeneity, which is calculated by a heterogeneity index.\n", + "\n", + "Below is a binary decision tree that has been split for a few iterations.\n", + "\n", + "![image.png](https://elf11.github.io/images/decisionTree.png)\n", + "\n", + "Since the target for this project is binary (fraud = yes or no) we will be building a binary decision tree, using the the GINI Index as the Heterogeneity index. The GINI is given by:\n", + "\n", + "![image.png](https://miro.medium.com/max/664/1*otdoiyIwxJI-UV0ukkyutw.png)\n", + "\n", + "The GINI index measures how heterogenous a single node is (0 being completely homogenous and 1 being heterogenous). For each possible split, we will calculate the *weighted sum* of the GINI indices of the child nodes, and choose the split that results in the maximum information gain. i.e. reduction in the weighted sum of the GINI Index.\n", + "\n", + "#### Training\n", + "We will now construct a simple decision tree using the GINI index." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.tree import DecisionTreeClassifier" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n", + " max_features=None, max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, presort=False,\n", + " random_state=None, splitter='best')" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tree = DecisionTreeClassifier()\n", + "tree.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 1.00 1.00 1.00 13475\n", + " 1 1.00 1.00 1.00 4021\n", + "\n", + " accuracy 1.00 17496\n", + " macro avg 1.00 1.00 1.00 17496\n", + "weighted avg 1.00 1.00 1.00 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, tree.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training set accuracy is 1, which means the datapoints are completely separated by the decision tree. We evaluate on the test set below." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (GINI) identified 828\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
054151314
11192828
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 5415 1314\n", + "1 1192 828" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, tree.predict(X_test), \"Decision Tree (GINI)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.6666666666666666\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e9NAoEQekINIQkt9A4iVQVFFEFBFBFUUAQUO2L72VCxtxcsgIoiCgIqVQRUTGjSeyehhN5CGgkp5/1jJrjEkCwhm005n+fhITs7O3NmdnbOzD0zd4yIoJRSSl2Oh7sDUEoplb9polBKKZUlTRRKKaWypIlCKaVUljRRKKWUypImCqWUUlnSRJEJY8wAY8wid8fhbsaYQGNMnDHGMw/nGWSMEWOMV17N05WMMduMMV1y8LkcbYPGmJLGmLnGmHPGmBlX+vm85Oz2VVh/j8aYEsaYncaYSnk4z6eMMW9c8QdFJF//A/YD54E44BgwGfB1d1yF8Z+9rru6OYYgQACvfLA+BKhdkJYZGAiszq31B3QB0uzfXxwQBfwEtHb393OVyzXAYZnOZ1jGuDyK4UlgXIZhbYAFQLT9bxswBihnv/8gsNRh/CjgKODjMGwYsMT+28vetoLs1z7AYaDilcRaUM4oeoqIL9AMaA487+Z4csSdR8mF5Qj9ShTR9V0T2C0iKVf6wSxiPmL//koD1wA7gXBjzA05D9O9RGSqiPjay3Uz9jI6DLuEi77Ph4EpDvPoCPwJLAXqikg5oIf9dqMsplMceNSZGYpIArAI64DCee7O7E5k3f04HOUC7wLzHV57A+8DB4HjwBdASYf3ewEbgRhgH9DdHl4W+AorGx8G3gA87ffuB5bZf38BvJ8hptnAU/bf1YBZwEkgEnjMYbxXgZnA9/b8H8xk+coC39mfPwC8BHg4xLEc+D/gHNYP9IYMn81qGZYDHwFn7PdqYW2Ip4FTwFT+PVKZgnVUlX729iwZjnSxNuAx9nRjsTY4P4d4BtnLcBr4X8bvLsNylwQ+sMc/Byyzh6XP8z77Oz0FvJjhiGsl1tHWUWAcUNzhfQEeAfYAkfawT4BD9newDujoML4n8IK9bcTa79cAwuxpxdvr4y57/FuxtqdoYAXQJMO2OhrYDCRhHc1dXAd27GvtOI4DH9rDD9rzSj+ibYfDNmiP0xBYbH+Xx4EXMlmnrwEXgGR7OkOwmpdfstfzCaxtraw9fvq6HmLHEJbJNLsAUZkMHwesdXgd6hDfLqDfFXzX6dvX/UCE/T1EAgMy/h7t19cCa+xprQGudXhvKVlso5fZFi+3jFHAKGALcMEeFgD8wr+/90ccxvfg323pFDANKH+ZeYZgbVueDsNWAR9lE2tmZxSj7fmVsYdd9ozCHnYfsPiK9sM52Xnn5T8u/aEF2F/aJw7vfwzMASpgHfHMBcY6/DDPAd3sL7E6EGq/9yvwJVAKqIR1uv5wxg0T6IS1kzH26/JYO9Nq9jTXAS9jZfUQe0O/yR73VawfbW973JKZLN93WImnNNYPZzcwxCGOFKxT1GLAXfbyVHByGVKAkfbGUhKoba8Lb8Afa2f4cWbrOsOOxDFR7APq2tNbCrxtv9cAa+fUwV4X79vLfrlEMd7+fHWsnfW1dlzp85xoz6Mp1k63vv25llhHtV72uDuAJxymK1g7rArp6xu4F6hof+ZprCbMEvZ76TuCeoCx51fRYVq1HabdAmtn29aO+T57nXk7rL+NWImmZMZ1ipXgBtp/+wLXZLaeM9kGS2MlxaeBEvbrtpdZr68C3zu8Hgzsxdo2fYGfgSkZ5vsd1jaU2fbZhcx3otdjHViUsv8dAh6w13ELrB1XQye/ay97GjFAPfszVR0+77guKgBnsY6IvYD+9uv072wpl9lGs9jHXG4Zo7B+3wH2tDzt7/cFrG28tv393mCP/wxWgqpuf09fpa/rTKbdC9jk8LqMvT47ZBNrZomiC9Y+8FV7WHaJog1w4or2w67YuefmP/uLiMM6OhDgD/49CjZYWbmWw/jt+PdI8ksyydBAZaydj+OZR3/gr0w2TIN1tNXJfv0Q8Kf9d1vgYIZpPw984/Cj/c9RmsO4nnYcDRyGPZy+IdhxHMFOUvaw1faPxJllOHi5edvj9AY2ZFjX2SWKlxzeHwEstP9+GfjR4T0frKPb/yQKrKR5HmiayXvp8wzIsMx3X2YZngB+cXgtwPXZLPfZ9HljHf32usx4GRPF58CYDOPsAjo7rL/BmWy/6YkiDOuo3y/DOJes50y2wf6O31M2y/YqlyaKP4ARDq/rYSVwL4f5hmQxvS5kvhMNtT9bHesAJjzD+18Crzj5XacnimigDxkSVoZ1MRBYneH9lcD92W2jOVjGKGCQw+v2QESGcf4HTLT/3pO+Ldiva2D9Rj0ymfZ9XHqWlL4uHLe3D+11Eg88Zw+7XKJoao9bkewTRX3sMyRn/xWUGkVvESmNtUJCAT97uD/WDmmdMSbaGBMNLLSHg/VF7ctkejWxjtCPOnzuS6yj8kuItWanYf1YAe7BarJJn0619GnY03kBayee7lAWy+WHdWRywGHYAawfX7rDdgyO71dzchkumbcxppIxZpox5rAxJgarScyPK3PM4e8ErKNU7Jguzk+sttDTl5mGH9YRV2bfTZbzMcbUNcbMM8Ycs5fhLf67DBmX+2ljzA77SqBorCa79M9cbhvJTE3g6Qzfdw2sZc903hkMwTrS3WmMWWOMudXJ+V5JjBlV47/blxfOb6OXUx1rBxSNtV7aZlgvA4AqOPddIyLxWAlnGNY2Pd8YE+rE8sB/fzOX20ZzwnHd1AQCMyzns1jLCRAIzHV4bwvWOsrsqqazWGeG6c7Y41ZNHyAiT4lVp5iL9Z1dlohsAn6348lOaazvzWkFJVEAICJ/Y1319L496BTW0UpDESln/ysr/xajDmG1y2d0CCvT+zl8royINLzMrH8E+hpjamKdRcxymE6kwzTKiUhpEenh8FnJODEHp7CO7mo6DAvEqjekq26MMRneP+LkMmSc91h7WBMRKYPVJGOyGP9KHMU6RQesyzSxjm4ycwpIJPPvJjufY9Vq6tjL8AKXLgM4LIddIBwN9MNqLy6H1XyX/pnLbSOZOQS8meH79hGRHzObd0YiskdE+mPtON4BZhpjSmX1mRzEmNER/rt9pWDVOS6GloPp3g6st3fwh4C/M6wXXxEZzhV81yLyu4h0w9pZ7sRqfsxueeC/v5nc5LhuDgF7Mvm997TfjwK6ZXi/hIgc++9k2QzUSr80WERisOpXd1xFrC8Dw/k3cV1OfWDTlUy4QCUK28dAN2NMMxFJw9qYPkq/FtkYU90Yc5M97lfAA8aYG4wxHvZ7oSJyFKvI9YExpoz9Xi1jTOfMZigiG7CKV5OA30UkPRuvBmKMMaPt69c9jTGNjDGtnVkQEUnFutTwTWNMaTsRPYV1pJ+uEvCYMaaYMeZOrC95wZUug600VjNetDGmOlb7vKPjWG3ZOTET6GmMudYYUxyriSXjDhwA+3v7GvjQGFPNXm/tjDHeTsynNFZbdpx9xDncifFTsL4/L2PMy1jtwekmAWOMMXWMpYkxJj3BZVwfE4Fhxpi29riljDG3GGMcjwwvyxhzrzHG317+9G0o1Y4tjcuv+3lAFWPME8YYb3tbaevMPLEOcp40xgQbY3yxzsCmS86uijL2b+gVrCaQFxziq2uMGWhvp8WMMa2NMfWd/a6NMZWNMbfZiTMJaztNzSSMBfa87jHGeBlj7sKqj8270uXJgZXABfsMtYS9LI2NMS3t978A3jLGBMLFM/jbMpuQiOzHatJu6TB4FDDUGPOsMcbfnkYN/psYMyUiu7AOYkdmM2pn4DdnppmuwCUKETmJVXz7nz1oNFaxbpXdFLEEqx0WEVmNVWD7COso8m/+XemDsJp9tmOdBs7E4bQvEz8CXYEfHGJJBXpiXbYbiXX0NAmracNZI7HaICOwrgb5AeuHle4foI497TeBviKS3qRzpcvwGlah8RwwH6uw6Wgs8JJ96vzMFSwDIrLNXpZpWGcXsViF36TLfOQZrFPzNVin3e/g3Pb4DFbzXyzWjnt6NuP/jvWj2I3VRJHIpc0JH2Il60VYCegrrMIlWO3939rro5+IrMWqUY3DWt97sdrPndUd2GaMicO6EutuEUm0m+neBJbb87rG8UMiEot1EUJPrGaVPcB1Ts7za6wr2sKwttFEst+RZFTNjjkO6/tqDHQRkUUO8d0I3I11xH8M6/tMTwbOfNceWMX6I/Y4nbHqC5ewt/1b7XFPYzW13Coip65wma6YnVx7YBWD92P9Jr/k3wOPD7Gavv8wxsRiXRWX1UHjlzhcpmq3mHTF+m732s1Xv2Ht0z5zMszXyKKpzT7T7461D3Va+pU8Kh8yxtyPdUltB3fHcqXso9dorCaiSHfHo1R+Y4wpAWzAKoCfyKN5Pgn4i8gL2Y7soMjdhKVcxxjTE+sqG4NVR9qCdeSllMpARBKxmpLzcp4f5eRzBa7pSeVrvbCaDo5gNZfdLXrKqlSBp01PSimlsqRnFEoppbJU4GoUfn5+EhQU5O4wlFKqQFm3bt0pEfHPfsz/KnCJIigoiLVr17o7DKWUKlCMMRnvaHeaNj0ppZTKkiYKpZRSWdJEoZRSKkuaKJRSSmVJE4VSSqksaaJQSimVJZclCmPM18aYE8aYrZd53xhjPjXG7DXGbDbGtHBVLEoppXLOlWcUk7G6s72cm7H6A6oDDMV6II1SSqlclpic2aM9nOeyG+5EJMwYE5TFKL2A7+xO41YZY8oZY6raD+RRSil1lU7HJfHiN+v44+CZq5qOO+/Mrs6lD5CJsof9J1EYY4ZinXUQGBiYJ8EppVRBtf9UPB//tpPZW44iHgav04lXNT13FrMze0xmpl3ZisgEEWklIq38/XPUVYlSShV66w+eZdiUdVz3wVJ+3XKU8zvPMKBCGXZOuJpHcbv3jCIKqOHwOgDrOQZKKaWclJomLNlxnIlhEaw9cJbSJbwY3rkWLcv4ULdGWWrUuJInM2fOnYliDvCoMWYa0BY4p/UJpZRyTmJyKjPXRfHVskgiT8VTIlU489chBtwSyrPdQ3N1Xi5LFMaYH4EugJ8xJgp4BSgGICJfAAuwHlS+F0gAHnBVLEopVVicjktiyqoDfLfyAGfiL1CtRDES/4wiasNxnnqyHS882z7X5+nKq576Z/O+AI+4av5KKVWYRJ6K56tlEcxYG0VSSho3hFYidfsZvn1tJddeW4PF6x+mcePKLpl3gXsehVJKFSXrDpxhQlgEi7Yfp5iHB7c1qcpdzarTup4/u3adon0dP4YMaYGHR2bXB+UOTRRKKZXPpKYJi7cfZ0LYPtYfjKZsyWI80qU2NRKFF55aRGSzKsya1Y969fyoV8/P5fFoolBKqXzi/IVUZq6P4qvwCPafTqBGhZK8dltDOlQvx/OjFjNqxnbq1avIo4+2ztO4NFEopZSbnYpL4ruVB5iycj9nE5JpGlCW8fe0oHujKiz9K5LmjT/nwoVUxoy5jlGjrsXbO2933ZoolFLKTSJOxjFpWSSz1lkF6q71KzG0Uy1aB5UnJSUNTw9D06ZV6NGjDm+8cT21a1dwS5yaKJRSKg+JCGsPnGVCWARLdhynmKcHfVpUZ0iHEGpX8iUmJoknnljIP/8cZvnywfj5+TBtWl+3xqyJQiml8kBqmrBo2zEmhEew4WA05XyKMfK62gxsF4R/aW9EhBkztvH44ws5diyOESNak5SUio+P+x8bpIlCKaVc6PyFVGasO8Sk8EgOnkkgsIIPr/dqSN+WAfgUt3bBJ0/Gc999v/Lbb3tp3rwKs2ffTevW1d0c+b80USillAucjE1iysr9TFl1gLMJyTSrUY7nbw7lxoZV8Mxwz0OZMt6cOpXAxx/fxCOPtMHLy/1nEY40USilVC7adzKOSeERzFp/mOTUNLrWr8zQTiG0qlkeY/5NEGFhB3jzzXBmzeqHr29xVq160KU3zV0NTRRKKXWVRIQ1+/8tUBf38qBvywCGdAimlr/vJeOeOpXAqFGLmTx5I0FB5di/P5pGjSrl2yQBmiiUUirHUtOEhVutAvWmQ9GU9ynGYzfUYVC7mvj5el8yrojwzTcbGTVqMTExSTz/fAdeeqkTPj7F3BS98zRRKKXUFUq4kMKMtVFMWhbBoTPnCarow5jejejbIoCSxT0v+7nvv99Mgwb+fPHFLTRsWCkPI746miiUUspJJ2IT+W7FAaasOsC588k0DyzHiz3q063BfwvUAAkJybz1VjjDhrUiIKAMs2b1o2zZEvm6mSkzmiiUUiobe0/EMik8kp/XHyY5LY0bG1gF6pY1L3+n9IIFe3jkkQXs3x9N9eqlGT68NeXLl8zDqHOPJgqllMqEiPBP5BkmhkXwx84TeHt5cGcrq0AdkqFA7SgqKoYnnljIrFk7qF/fj7//vp9OnWrmYeS5TxOFUko5SElNY+G2Y0wMi2BT1DkqlCrOE13rMPCamlTMUKDOzJtvhjF//h7eeut6nn76WopnUbMoKIz1oLmCo1WrVrJ27Vp3h6GUKmTik1L4ae0hvloWSdTZ8wT7lWJIh2D6tgygRLGsd/arVx+mZEkvGjeuzOnTCZw7l0RISPk8itw5xph1ItIqJ5/VMwqlVJF2IiaRb1fu5/tVBzl3PplWNcvzv1sb0LV+5UwL1I7OnUvkhRf+4PPP13LrrXWZM6c/FSv6ULGiT94En0c0USiliqQ9x2OZGB7BrxuOkJyWxk0NqvBQpxBa1sz+TEBEmD59G08++TsnTsQzcmQbxoy5Pg+idg9NFEqpIkNEWBVxhonhEfy58wQlinlwV+saDOkQTJBfKaen8/33mxk06FdatarGvHn9admymgujdj9NFEqpQi8lNY0FW60C9ZbD56hYqjhPdavLvdfUpEKp4k5NIykphYiIs9Sv70+/fg1JSUlj0KCmeHrmrw78XEEThVKq0IpLSuGnNVaB+nD0eUL8SvHW7Y25o0X1bAvUjv76K5Lhw+eTkJDMnj0j8fb24oEHmrsw8vxFE4VSqtA5EZPINyv2M3XVAWISU2gdVJ5XeloF6iu5K/rEiXieeWYRU6ZsJiSkPBMm9Mzz51XnB0VviZVShdbu47FMDIvg142HSU0TujeqwoMdQ2gReOWXqu7de4Y2bSYSF3eBF1/syIsvdqRkyfzfgZ8raKJQShVoIsLKfaeZEB7B0l0nKVHMg/5tAhnSIZiaFZ0vUKeLiUmiTBlvatUqz5AhzRk8uDn16/u7IPKCQxOFUqpASk5NY8GWo0wMj2Dr4Rj8fIvztF2gLu9kgdpRfPwFXn/9byZOXM/mzcMJCCjDe+/d6ILICx5NFEqpAiUuKYVpqw/yzfL9VoHavxRv39GY3s2vrEDtaO7cXTz66G8cPHiOIUOaF4hnROQlTRRKqQLh2LlEJq/Yz9R/DhCbmEKb4Aq8dltDrg/N+dPhUlLS6NdvBr/8spOGDf0JD3+ADh0Ccznygk8ThVIqX9t5LIaJYZHM2WQVqG9uXJWHOobQrEa5HE9TRDDG4OXlQdWqvrz99g08+WS7QtGBnytoolBK5Tsiwop9p/kyLIKw3ScpWcyTAW1rMrh9MIFX2Y/SqlVRPPLIAiZO7EmLFlUZP/6WXIq68NJEoZTKN5JT05i/+SgTwiLYfjQGP19vRt1UjwFtAynnc+UFakdnz57nhRf+4Msv11GtWmnOnj2fS1EXfi5NFMaY7sAngCcwSUTezvB+IPAtUM4e5zkRWeDKmJRS+U9sYjLT1xzi62WRHDmXSC3/UrzTpzG9muW8QO1o+vStPPbYQk6dSuCJJ67htde6ULp09s+WUBaXJQpjjCcwHugGRAFrjDFzRGS7w2gvAT+JyOfGmAbAAiDIVTEppfKXY+cS+WZ5JD/8c5DYpBTaBlfgjdsb0aVuzgvUmdm58xRBQeVYuHAAzZtXzbXpFhWuPKNoA+wVkQgAY8w0oBfgmCgEKGP/XRY44sJ4lFL5xI6jMUwMj2DOxiOkidDDLlA3vYoCtaPExBTeeWcZLVpUpWfPerzwQkdeeqlTkejAzxVcmSiqA4ccXkcBbTOM8yqwyBgzEigFdM1sQsaYocBQgMBAvXRNqYJIRFi29xQTwiII33MKn+KeDGxnFahrVMi9B/0sWRLBiBHz2bPnDE8/3Y6ePetRLBear4oyVyaKzM4bMz53tT8wWUQ+MMa0A6YYYxqJSNolHxKZAEwA61GoLolWKeUSyalpzNt8hAlhkew4GoN/6dwrUDs6fjyOp55axA8/bKF27QosWnQv3brVyrXpF2WuTBRRQA2H1wH8t2lpCNAdQERWGmNKAH7ACRfGpZTKAzGJyRfvoD56LpE6lXx5t28TejWrhrdX7h/hL14cwcyZ23n55U48/3xHSpTQizpziyvX5BqgjjEmGDgM3A3ck2Gcg8ANwGRjTH2gBHDShTEppVzsSPR5Jq/Yzw//HCQuKYV2IRV56/bGdK7rn6sFaoBNm46xZ88Z+vZtwIABjWnfvgbBwVfeU6zKmssShYikGGMeBX7HuvT1axHZZox5HVgrInOAp4GJxpgnsZql7hcRbVpSqgDafsQqUM/ddAQBbrEL1I0Dyub6vOLiLvDKK3/xySf/EBRUjt69Q/Hy8tAk4SIuPTez74lYkGHYyw5/bwfauzIGpZTriAjhe04xMfzfAvWgdkEM7hBEQPncK1A7+vXXnYwc+RtRUTEMHdqCsWO74uWlVzO5kjbiKaWu2IWUNOZuOsLE8Ah2HoulUmlvRncP5Z42gZR1Yc+rW7Yc5/bbp9O4cSWmT+/LtdfWyP5D6qppolBKOS0mMZkf/7EK1MdiEqlb2Zf3+jbhNhcVqAGSk1MJDz/I9dcH07hxZebPv4du3UL0ktc8pIlCKZWtw9Hn+WZZJNPWHCIuKYX2tSvydh+rQG1M7haoHa1YcYhhw+axbdtJdu16lNq1K9CjRx2XzU9lThOFUuqyth4+x6TwCOZuPgrArU2sAnWj6rlfoHZ05sx5nntuCRMnrqdGjTL8/HM/ateu4NJ5qsvTRKGUuoSI8Pfuk0wMj2D53tOUKu7JA9cG8UCHYKqXK+ny+ScmptCs2RccORLL00+349VXu+Drm3s35qkrp4lCKQVYBeo5m44wMSyCXcdjqVzGm+duDqV/m0DKlnT9o0GjomIICChDiRJejBlzHc2aVaFp0youn6/KniYKpYq4c+eT+eGfg0xeEcnxmCRCq5Tmgzub0rNpNYrnwWWn588nM3bsMt55ZzkzZ95Jz571uO++Zi6fr3KeU4nCGFMcCBSRvS6ORymVR6LOJvD1sv1MX3OQ+AupdKjtx3t9m9Kxjp9LC9SOFi3ax4gR89m37yz33tuENm2q58l81ZXJNlEYY24BPgSKA8HGmGbAKyJyu6uDU0rlvq2HzzEhLIL5W45igJ5Nq/Fgx2AaVnNtgTqjkSMXMG7cGurUqcCSJQO54YaQPJ2/cp4zZxSvY3UP/heAiGw0xtR2aVRKqVwlIizdfZKJYRGs2HcaX28vBrcP4oH2wVTLgwJ1utRUq2NoT08PrrkmAD8/H0aP7qAd+OVzznw7ySISneFUVPtjUqoASEpJZfbGI0wKj2D38TiqlCnBCz1CubtNIGVKuL5A7Wj9+qMMGzaPgQObMHJkWwYMaJKn81c550yi2GGM6Qd42D3BPg6scm1YSqmrcS4hmamrDzB5+X5OxFoF6g/7NeXWJnlToHYUG5vEyy//xaefrsbf34eqVUvn6fzV1XMmUTwKvAykAT9j9Qb7vCuDUkrlzKEzCXy9PJLpaw6RcCGVjnX8+KBfUzrUzrsCtaNFi/YxePBsjhyJZdiwVrz11g2UK1ciz+NQV8eZRHGTiIwGRqcPMMbcgZU0lFL5wOaoaCaERbBgy1E8jOG2ptV4sGMIDaqVyf7DLlS8uCeVKpVi1qx+tG0b4NZYVM6Z7B7/YIxZLyItMgxbJyItXRrZZbRq1UrWrl3rjlkrla+kpQlLd59gQlgEqyLOUNrbi3vaBnJ/+yCqls27ArWj5ORUPvxwJTExSbz55g0X48ztBxapK2fvt1vl5LOXPaMwxtyE9ZjS6saYDx3eKoPVDKWUcoOklFRmbzjChPAI9p6Io1rZErx0S33ual2D0nlcoHa0bNnBix343Xlng4sJQpNEwZdV09MJYCuQCGxzGB4LPOfKoJRS/xWdcIGp/xxk8or9nIxNokHVMnx8VzNuaVKVYp7ue3DP6dMJjB69hK++2kBgYFnmzu3PrbfWdVs8KvddNlGIyAZggzFmqogk5mFMSikHh84k8NWySH5aaxWoO9f1Z+hdIVxbq6JbCtQZnT59nmnTtvLss9fy8sudKVVKO/ArbJwpZlc3xrwJNAAuXq4gInrIoJQLbToUzYTwCH7bchRPD8NtTavzUKdgQqu4t0ANsGPHSX76aRuvvNKFunUrcvDgk1So4J66iHI9ZxLFZOAN4H3gZuABtEahlEukpQl/7TrBl2ERrI60CtQPdQrhgWuDqVLW/ZeVJiQk8+abYbz33gp8fYszZEgLAgLKaJIo5JxJFD4i8rsx5n0R2Qe8ZIwJd3VgShUlicmp/LrhMBPDI9h3Mj7fFKgdLVy4lxEj5hMZGc199zXlvfe64e9fyt1hqTzgTKJIMlZD6D5jzDDgMFDJtWEpVTScjb/A96sO8O3K/ZyKu0DDamX45O5m9Gjs3gJ1RnFxFxg48BcqVizJX3/dR5cuQe4OSeUhZxLFk4Av8BjwJlAWGOzKoJQq7A6eTuCrZRH8tDaK88mpdKnnz9COIbTLJwVqsDrw+/HHrfTv3whf3+IsWTKQ0FA/vL21A7+iJttvXET+sf+MBQYCGGP0FkulcmDjoWgmhO1j4dZjeHoYejerzoMdQ6hXJX/1f7Ru3REefnge69YdpWRJL/r0aaBPmyvCskwUxpjWQHVgmYicMsY0xOrK43pAk4VSTkhLE/7YeYKJYRGs3n+G0iW8eLhzLe6/NojKZdxfoHZ07lwi//vfX4wfv4ZKlUoxbVof7rijvrvDUm6W1Z3ZY4E+wHLVKeQAACAASURBVCasAvYvWD3HvgMMy5vwlCq4EpNT+cUuUEecjKd6uZK8fGsD+rWugW8+bb7p0+cn/vwzkkceac0bb1xP2XxwpZVyv6y21l5AUxE5b4ypAByxX+/Km9CUKpjO2AXq71ZaBepG1cvwaf/m9GhUBa98VKBOFxFxFn9/H0qX9ubNN6/Hw8PQurU+klT9K6tEkSgi5wFE5IwxZqcmCaUu78DpeCaFRzJj3SESk9O4PrQSD3UM4ZqQCvmmQO3owoVU3n9/BWPGhPHYY214551u2sOrylRWiSLEGJPelbgBghxeIyJ3uDQypQqI9QfPMjEsgoXbjlHMw4Peza0uvutWzl8FakdhYQcYNmweO3acom/fBjz2WFt3h6TysawSRZ8Mr8e5MhClCpK0NGHxjuNMDItg7YGzlCnhxXC7QF0pnxWoM/roo5U89dQigoLKMX/+PfToUcfdIal8LqtOAf/Iy0CUKggSk1OZtT6KSeGRRJ6KJ6B8SV7p2YB+rWpQKp8WqMFKbPHxFyhd2ptbbqnLyZMJvPRSJ3x88sdd3yp/y/bBRfmNPrhIucOZ+At8t3I/3608wJn4CzQJKMvQTiF0b5g/C9SOtm07wbBh8y8+aU4VTS55cFFuMMZ0Bz4BPIFJIvJ2JuP0A14FBNgkIve4MialrkTkqXi+WhbBzHVRJCancUNoJR7qFELb4PxZoHaUkJDMmDF/8/77Kylb1pvBg5shIvk+bpX/OJ0ojDHeIpJ0BeN7AuOBbkAUsMYYM0dEtjuMUwd4HmgvImeNMdqHlMoX1h04y4SwfSzafpxiHh7c0aI6D3YMpnal/FugdrRhw1HuuOMn9u+P5oEHmvHuu93w8/Nxd1iqgMo2URhj2gBfYfXxFGiMaQo8KCIjs/loG2CviETY05mGdW/GdodxHgLGi8hZABE5ceWLoFTuSE0TFm8/zsTwCNYdOEvZksV4pEttBl1bk0ql83eBOl36GUNgYFkCA8vy7be96dSpprvDUgWcM2cUnwK3Ar8CiMgmY8x1TnyuOnDI4XUUkPEavLoAxpjlWM1Tr4rIQiemrVSuOX8hlZnro/gqPIL9pxOoUaEkr93WkDtbBeBTPP8WqB2lpKQxbtxq5szZxeLFA6lY0Ye//77f3WGpQsKZX4GHiBzI0K6Z6sTnMmsIzVg59wLqAF2w+o4KN8Y0EpHoSyZkzFBgKEBgYKATs1Yqe6fjkvhu5QGmrLIK1E1rlOOz7qHc1LAKnh4Fpx1/9erDDBs2jw0bjnHzzbWJiUmifHl9kJDKPc4kikN285PYdYeRwG4nPhcF1HB4HYDVDUjGcVaJSDIQaYzZhZU41jiOJCITgAlgXfXkxLyVuqyIk3FMWhbJrHVRJKWk0bV+ZYZ2CqF1UPkCVeiNi7vA6NGL+fzztVStWpoZM+6kT5/6BWoZVMHgTKIYjtX8FAgcB5bYw7KzBqhjjAnGetjR3UDGK5p+BfoDk40xflhNURHOha6U80TELlBHsHjHcYp5etCnRXWGdAihdiVfd4eXI8WKebB06QFGjmzDmDHXU6aMt7tDUoWUM4kiRUTuvtIJi0iKMeZR4Hes+sPXIrLNGPM6sFZE5tjv3WiM2Y7VnDVKRE5f6byUupzUNGHRtmNMCI9gw8FoyvkUY+R1tRnYLgj/0gVvx7p37xlef/1vxo/vQenS3qxbN5QSJQpGHUUVXNnecGeM2QfsAqYDP4tIbF4Edjl6w51yxvkLqcxcd4hJyyI5cDqBwAo+PNgxmL4tC06B2lFSUgrvvrucN98Mp3hxT+bPv4eOHfVqJuU8l95wJyK1jDHXYjUdvWaM2QhME5FpOZmhUq50Ki6J71bsZ8qqA5xNSKZZjXI81z2UGwtYgdrRX39FMnz4fHbtOs1ddzXkww9volq1gnE/hyocnDq0EpEVwApjzKvAx8BUQBOFyjf2nYxjUngks9ZHkZz6b4G6Vc2CVaDOSER4881wkpPTWLhwADfdVNvdIakiyJkb7nyxbpS7G6gPzAaudXFcSmVLRFiz3ypQL9lxnOJeHvRtGcCQDsHU8i+YBWqwOvD76qv1dO9emxo1yjJlyu2UK1eCkiW1Az/lHs6cUWwF5gLviki4i+NRKlupacLv244xISyCjYeiKe9TjMduqMOgdjXx8y14BWpHmzcfZ9iweaxcGcXLL3fitdeuo2pVbWZS7uVMoggRkTSXR6JUNhIupDBjbRRfLYvk4JkEgir6MKZ3I/q2CKBkcU93h3dV4uIu8NprS/noo1WUL1+SyZN7MWhQU3eHpRSQRaIwxnwgIk8Ds4wx/7k0Sp9wp/LKydgkvltpFaijE5JpHliOF3qE0q1BwS1QZ/Tqq0v54IOVPPhgc95+uysVK2oHfir/yOqMYrr9vz7ZTrnF3hOxTAqP5OcNh0lOTePGBlaBumXNCu4OLVccOnSO+PhkQkP9eO65DvTuHUqHDtpFjcp/snrC3Wr7z/oickmysG+k0yfgqVwnIqyOPMPE8AiW7DiBt5cHd9oF6pACXKB2lJKSxqef/sPLL/9Fy5bV+Pvv+/Hz89EkofItZ2oUg/nvWcWQTIYplWMpqWn8vu04E8L2sSnqHBVKFeeJrnUYeE1NKhbwArWjVauiGDZsHps2HeeWW+owblwPd4ekVLayqlHchXVJbLAx5meHt0oD0Zl/SqkrE5+Uwoy1h/hqeSSHzpwn2K8Ub/RuRN+WAZQoVrAL1BnNn7+bnj1/pFq10vz8cz969w4t0Pd4qKIjqzOK1cBprF5fxzsMjwU2uDIoVfidiE3k2xX7+X7VQc6dT6ZVzfK8dEsDutavXGgK1GA1pR05Ekv16mXo2jWE11+/jscfb0vpAtjPlCq6su3rKb/Rvp4Ktj3HrQL1LxsOk5yWxk0NqvBQpxBa1izv7tBy3e7dpxkxYj67d59m+/ZH8PUt7u6QVBHmkr6ejDF/i0hnY8xZLn3gkAFERArHpSfK5USEVRFWgfrPnScoUcyDu1rXYEiHYIL8Srk7vFyXmJjC228vY+zYZZQs6cXYsTdQsmTB64hQqXRZbb3pjzv1y4tAVOGTkprGb1uPMTE8gs1R56hYqjhPdavLvdfUpEKpwnl0fexYHJ06fcOePWfo378RH354E1WqFI6rtVTRldXlsel3Y9cAjojIBWNMB6AJ8D0QkwfxqQIoPimF6WsO8dWySA5HnyfErxRv3d6YO1pUL3QF6nTJyakUK+ZJ5cql6NSpJuPH96Bbt1ruDkupXOHM8yg2Aq2xnnC3GJgPBIvIra4P77+0RpF/nYhJZPKK/Xy/6gAxiSm0DirP0E61uCG0Eh6FqEDtKC1NmDBhHW+9Fc6KFUMICCjj7pCUypRLn0cBpIlIsjHmDuBjEfnUGKNXPamLdh+PZWJYBLM3HiElLY3ujarwYMcQWgQWvgK1o02bjvHww/P455/DXH99MMnJqe4OSSmXcOpRqMaYO4GBQG97mPZ3XMSJCCsjTjMxLIK/dp2kRDEP7m5jFahrVix8BWpHIsKoUYv5+ONVVKhQkilTbmfAgMZ6T4QqtJy9M3sEVjfjEcaYYOBH14al8quU1DTmbznKxPAIth6Owc+3OE/bBeryhbRAnZExhrNnzzNkiNWBX/nyJd0dklIu5dR9FMYYLyD90Vp7RSTFpVFlQWsU7hFnF6i/Ti9Q+5diaMcQejcvvAVqRwcORPP44wt5+eXOtGhRlbQ0KbR1F1U4ubRGYYzpCEwBDmPdQ1HFGDNQRJbnZIaqYDkek8g3y/cz9Z8DxCam0Ca4Aq/d1pDrC3GB2lFycioffbSK1177G4C77mpIixZVi8SyK5XOmaanj4AeIrIdwBhTHytx5CgzqYJh17FYJoZHMHvjYVLThJsbV+WhjiE0q1HO3aHlmRUrDvHww/PYuvUEvXrV49NPbyYwsKy7w1IqzzmTKIqnJwkAEdlhjCkajdFFjIiwYt9pJoRF8Pfuk5Qs5smAtjUZ3D6YwCL4IJ0lSyI4dy6RX3+9i169Qt0djlJu48x9FJOBJKyzCIABgI+I3Ofa0DKnNYrcl5yaxoItR5kQFsG2IzH4+XrzQPsgBrQNpJxP0TkmEBGmTNmMv78PN99ch6SkFJKT07SPJlUouPo+imHAY8CzWDWKMOD/cjIzlb/EJaUwbfVBvl4WyZFzidTyL8U7fRrTq1nRKFA72rnzFMOHz2fp0v3ceWcDbr65Dt7eXnhrJ69KZZ0ojDGNgVrALyLybt6EpFzt2LlEvlkRyQ//HCQ2MYW2wRV44/ZGdKlbNArUjs6fT+att8J5553llCpVnC+/vJUHH2zh7rCUyley6j32Bawn2a0HWhtjXheRr/MsMpXrdhyNYWJ4BHM2HiFNhB52gbppESpQZzR37m7eeCOce+9twvvvd6NyZe3AT6mMsjqjGAA0EZF4Y4w/sADQRFHAiAjL955mQngEYbtP4lPck4HtrAJ1jQpFr0ANVg+vGzceo3v32tx5ZwOCgh6kTZvq7g5LqXwrq0SRJCLxACJy0hjjkUcxqVyQnJrGvM1HmBAWyY6jMfiX9mbUTfWKXIHaUWpqGl9+uY7nn/+D4sU9OXjwCUqWLKZJQqlsZJUoQhyelW2AWo7PzhaRO1wamcqR2MRkpq0+xNfLIzl6LpE6lXx5t28TejWrhrdX0SpQO1q//ijDhs1jzZojdO0awmef9aBkSe2yTClnZJUo+mR4Pc6VgaircyT6PJNX7OfHfw4Sm5RCu5CKvHV7YzrX9S9yBeqMIiPP0qbNRPz8fPjhhzu4++5G2oGfUlcgqwcX/ZGXgaic2X7EKlDP3XQEAW6xC9SNA4r2HcQiwpYtJ2jSpDLBweX55pte9OxZj3LlSrg7NKUKHH2QbwEkIoTvOcXE8AjC95zCp7gng9oFMbhDEAHli2aB2lFk5FkeffQ3Fi7cy4YND9OkSWUGDmzq7rCUKrBcmiiMMd2BTwBPYJKIvH2Z8foCM4DWIqK3XWdh38k4Hvp2LRGn4qlU2pvR3UO5p00gZX20vf3ChVQ+/HAlr7/+Nx4ehvff70aDBv7uDkupAs/pRGGM8RaRpCsY3xMYD3QDooA1xpg5jv1G2eOVxrrz+x9np11UXUhJ45kZm4g4Fc8LPUK579qgIl2gdpSamsa1137FunVHueOO+nz88U3UqFG0m9+Uyi3ZXvJqjGljjNkC7LFfNzXGONOFRxusZ1dEiMgFYBrQK5PxxgDvAonOh100jf1tBxsORjPunuYM7VRLkwQQE2Mdu3h6ejB4cHPmzu3PrFn9NEkolYucuTfiU+BW4DSAiGwCrnPic9WBQw6vo+xhFxljmgM1RGReVhMyxgw1xqw1xqw9efKkE7MufOZuOsI3y/fzQPsgbm1Szd3huJ2IMHnyRkJCPmH27J0AjBjRmltvrevmyJQqfJxJFB4iciDDMGeeIp/Z9YcXu6q1b+D7CHg6uwmJyAQRaSUirfz9i16b894TsYyetZmWNcvz/M313R2O223ffpIuXb7lgQdmExrqR61aFdwdklKFmjM1ikPGmDaA2HWHkcBuJz4XBdRweB0AHHF4XRpoBCy1r2mvAswxxtymBe1/xSelMOz79ZQs5sn4e1pQ3Kto3yD/7rvLefHFPylTxptJk3rywAPNi/x9Ikq5mjOJYjhW81MgcBxYYg/LzhqgjjEmGOsxqncD96S/KSLnAL/018aYpcAzmiT+JSI89/MWIk7GMWVIW6qULbr3AIgIxhiqVPFlwIDGvPdeN/z9S7k7LKWKhGwThYicwNrJXxERSTHGPAr8jnV57Nciss0Y8zqwVkTmXHG0Rcy3K/Yzd9MRRt1Uj/a1/bL/QCF05Egsjz++kI4dA3nssbYMGtSUQYP0ngil8lK2icIYMxGH2kI6ERma3WdFZAFWr7OOw16+zLhdspteUbLuwFneXLCDG0IrMbxzLXeHk+dSU9P47LM1vPjinyQnp3HttQHuDkmpIsuZpqclDn+XAG7n0quZVC47HZfEI1PXU6VsCT7s16zItcFv3HiMBx+cw7p1R7nxxlp89lkPLVgr5UbOND1Nd3xtjJkCLHZZREVcaprw2LQNnEm4wM/Dry2Sd1yfO5fIkSOxTJ/elzvvbKAd+CnlZjnpwiMYqJnbgSjLx0t2s3zvad7p05hG1YvGTWMiwowZ29mz5zQvvtiJzp2DiIh4nBIltCsypfIDZ+7MPmuMOWP/i8Y6m3jB9aEVPX/uPM7//bmXfq0CuKt1oLvDyRP79p2hR48fuOuumcyevYvkZOsWHU0SSuUfWf4ajXXO3xTr8laANBH5T2FbXb1DZxJ4cvomGlQtw+u9Grk7HJdLSkrh/fdX8MYb4RQr5sEnn3RnxIjWeBXx+0SUyo+yTBQiIsaYX0SkZV4FVBQlJqcyfOo60kT44t6WlChW+PtwOnQohjFjwujZsx4ff3wT1auXcXdISqnLcObwbbUxpoXLIynCXpu7ja2HY/iwXzMCKxbe50mcPBnPuHGrAahduwLbtz/CjBl3apJQKp+77BmFMcZLRFKADsBDxph9QDxWH04iIpo8csGMtYf4cfUhRnSpRbcGld0djkukpQnffLOBZ59dQmxsEt26hVCvnh8hIeXdHZpSyglZNT2tBloAvfMoliJn+5EYXvp1K+1CKvJUt8LZ6+nWrScYPnw+y5YdpGPHQL744lbq1Suad5krVVBllSgMgIjsy6NYipRz55MZPnUd5XyK8Wn/5nh5Fr4i7oULqdx44xQuXEjl669v4/77m+k9EUoVQFklCn9jzFOXe1NEPnRBPEWCiPDMjE0cPnueaUOvwb+0t7tDylV//hlJ5841KV7ck59+upPQUD/8/Apv7UWpwi6rw1hPwBerO/DM/qkc+jIsgsXbj/N8j/q0Cio8XVNERcXQp89P3HDDd3z33SYAOnQI1CShVAGX1RnFURF5Pc8iKSJW7jvNuwt3ckvjqgxuH+TucHJFSkoa48at5n//+4vU1DTGjr2BAQOauDsspVQuybZGoXLP8ZhERv64gSC/UrzTt0mhaa8fOPAXpk3bys0312b8+B4EB+vVTEoVJlklihvyLIoiIDk1jUd/WE98Ugo/PNQWX++C3UVFdHQiXl4e+PoW55FHWtOnT3369KlfaJKfUupfl61RiMiZvAyksHvnt52s2X+Wt/s0pm7lglviERGmTdtK/frj+d///gSsOkTfvtrLq1KFVeG7JjMfWrDlKJOWRTKoXU16Navu7nBybO/eM9x00/f07z+LgIAy3Huv1iGUKgoKdvtHAbDvZBzPztxMsxrlePGW+u4OJ8d++GELgwfPxtvbi3HjbmbYsFZ4FsJ7P5RS/6WJwoUSLqQw/Pt1FPM0fDagBd5eBa+zv+TkVIoV86RVq2r07duAd9/tRrVqBbfpTCl15TRRuIiI8OIvW9lzIo7vBrehWrmS7g7pipw4Ec/TTy8iPv4CP/98F3XrVuT77+9wd1hKKTfQtgMX+f6fg/yy4TBPdq1Lxzr+7g7HaWlpwoQJ66hXbxzTp2+lYUN/UlPT3B2WUsqN9IzCBTYeimbM3O10qefPo9fVdnc4TouIOMu99/7MypVRdOkSxOef30JoqHbgp1RRp4kil52Jv8AjU9fjX9qbj+9qhodHwblktGxZb6KjE/n2294MHFh4bghUSl0dbXrKRalpwhPTN3IyNonP721BOZ/i7g4pW3Pm7OKOO6aTmppGxYo+bN06gkGDmmqSUEpdpIkiF/3fn3sI232SV29rSJOAcu4OJ0sHD56jd+9p9Oo1jd27T3P0aBxAgToDUkrlDW16yiVLd53gkz/2cEeL6vRvU8Pd4VxWSkoaH3+8ildeWYqI8M47XXnyyWsoVgSe062UyhlNFLkg6mwCT0zfSL3KpXmzd+N83WyTmprGpEnruf76YP7v/24mKCh/n/kopdxPm56uUlJKKo9MXU9qqvD5vS0pWTz/HZmfPXue0aMXExubhLe3F8uXD2bOnLs1SSilnKKJ4iqNmbedTVHneO/OpgT7lXJ3OJcQEaZO3Uxo6Hg++GAlf/21H4CKFX3y9VmPUip/0aanq/DLhii+X3WQhzuF0L1RFXeHc4ndu08zYsR8/vgjkjZtqvP77/fSrFn+ilEpVTBoosihncdieP7nLbQJrsCom+q5O5z/eOKJhaxde4TPPuvB0KEttQM/pVSOaaLIgdjEZIZ/v57SJYox7p7meOWTnfDixfsIDfWjRo2yfP75LXh7e1Gliq+7w1JKFXAu3cMZY7obY3YZY/YaY57L5P2njDHbjTGbjTF/GGNqujKe3CAijJqxmYNnEhjXvzmVSpdwd0gcOxbHPffM4sYbv+edd5YDULNmOU0SSqlc4bJEYYzxBMYDNwMNgP7GmAYZRtsAtBKRJsBM4F1XxZNbvloWycJtx3iueyhtQyq6NZa0NOGLL9YSGjqOWbN28MornXn//RvdGpNSqvBx5RlFG2CviESIyAVgGtDLcQQR+UtEEuyXq4AAF8Zz1VZHnmHsbzvp3rAKD3YMdnc4jB0bzvDh82nZshqbNw/j1Ve7UKKEtiYqpXKXK/cq1YFDDq+jgLZZjD8E+C2zN4wxQ4GhAIGBgbkV3xU5EZvIIz+sJ7CCD+/e6b4O82Jjkzh1KoHg4PIMG9aK4ODy9O/fSC93VUq5jCvPKDLbc0mmIxpzL9AKeC+z90Vkgoi0EpFW/v55/2yHlNQ0Rv6wgdjEZD6/twVlShTL8xhEhF9+2UGDBp9x110zEREqVvThnnvy953gSqmCz5WJIgpw7PQoADiScSRjTFfgReA2EUlyYTw59t6iXfwTeYa3bm9MaJUyeT7/Aweiue22adxxx09UqFCSTz+9WZODUirPuLLpaQ1QxxgTDBwG7gbucRzBGNMc+BLoLiInXBhLjv2+7Rhf/h3BgLaB3NEi70soK1ceomvXKQC8/343Hn/8Gry88sfluEqposFliUJEUowxjwK/A57A1yKyzRjzOrBWROZgNTX5AjPsI+SDInKbq2K6UvtPxfPMT5toElCWl3tmvGDLtWJikihTxpsWLaoyeHAzRo1qT2Bg2TyNQSmlAIxIpmWDfKtVq1aydu1al8/n/IVUbv9sOcdiEpn7aAdqVPBx+TwBTp9O4LnnlrBoUQTbto3A1zf/P/xIKZX/GWPWiUirnHxWr6XMhIjw0q9b2XU8lm/ub50nSUJEmDJlM08/vYizZ8/z1FPt0DKEUio/0ESRiWlrDjFrfRSP3VCHLvUquXx+584l0rv3dJYu3U+7dgF88cWtNGlS2eXzVUopZ2iiyGBL1Dlemb2NjnX8ePyGOi6dl4hgjKFMGW/8/HyYMOFWhgxpoY8jVUrlK3r5jIPohAsMn7oOP9/ifHJ3czxduMP+/fe9tGgxgaioGIwxzJhxJw891FKThFIq39FEYUtLE56cvpHjMYl8dm9LKpRyTRH56NFY7r57Jt27TyUhIZkTJ+JdMh+llMot2vRkG//XXv7adZIxvRrSrIZrHhE6fvxqXnjhT5KSUnjttS6MHt0eb2/9CpRS+ZvupYDwPSf5cMluejerxr3XuK6n83XrjtK2bXXGj+9BnTru7XlWKaWcVeQTxZHo8zw+bSN1Kvny1h25229STEwSL7/8FwMHNqFly2p89tkteHt7avcbSqkCpUgnigspaYyYup6k5FQ+v7clPsVzZ3WICLNm7eDxxxdy9GgsgYFladmymnYBrpQqkIr0nuutBTvYeCiazwa0oJZ/7jwNLjLyLI8++hsLFuyhWbMq/PxzP9q2zdeP2VBKqSwV2UQxe+NhJq/Yz5AOwfRoXDXXpjt16hbCwg7w0Uc38eijbbQDP6VUgVck+3raczyW28Ytp2G1Mvw49BqKeV7dzjw8/ABJSal07RpCUlIKJ08mEBCQ992RK6XU5VxNX09F7nA3LimFYd+vo5S3J+MHtLiqJHHqVAKDB8+mU6fJvP763wB4e3tpklBKFSpFqulJRBg9azORp+KZ+uA1VC5TIsfTmTx5I6NGLebcuSRGj27P//7XKZejVYVBcnIyUVFRJCYmujsUVUSUKFGCgIAAihXLvSdxFqlE8c3y/czffJTR3UNpVyvn9zEsWLCHwYPn0L59Db744lYaNXJ9x4GqYIqKiqJ06dIEBQXpZdHK5USE06dPExUVRXBwcK5Nt8g0Pa07cIa3Fuyga/3KDOsccsWfT0hIZvnygwD06FGH2bPvJizsAU0SKkuJiYlUrFhRk4TKE8YYKlasmOtnsEUiUZyKS2LE1PVUK1eSD/o1veIf7W+/7aFRo8+4+eapREcnYozhttvqaQd+yimaJFRecsX2VugTRWqa8NiPG4hOSObze1tQtqTz7XaHD8dw550z6NHjB7y9vZg7tz/lyuWsrqGUUgVVoU8UHy7exYp9pxnTuxENqzn/zOkTJ+Jp0OAz5s3bzRtvXMemTcPo3DnIdYEq5SKenp40a9aMRo0a0bNnT6Kjoy++t23bNq6//nrq1q1LnTp1GDNmDI6XzP/222+0atWK+vXrExoayjPPPOOORcjShg0bePDBBy8Z1qtXL9q1a3fJsPvvv5+ZM2deMszX998bbXfv3k2PHj2oXbs29evXp1+/fhw/fvyqYjtz5gzdunWjTp06dOvWjbNnz2Y63sGDB7nxxhupX78+DRo0YP/+/QCMGzeO2rVrY4zh1KlTF8efN28er7zyylXFdkVEpED9a9mypThr8bZjUnP0PBk9c5PTn4mKOnfx708+WSV79552+rNKZbR9+3Z3hyClSpW6+PegQYPkjTfeEBGRhIQECQkJkd9//11EROLj46V79+4ybtw4ERHZsmWLhISEyI4dO0REJDk5WcaPH5+rsSUnJ1/1NPr27SsbN268+Prs2bMSEBAgoaGhEhERcXH4fffdJzNmzLjks+nr5vz581K7dm2ZM2fOxff+Q002TQAAEWVJREFU/PNP2bJly1XFNmrUKBk7dqyIiIwdO1aeffbZTMfr3LmzLFq0SEREYmNjJT4+XkRE1q9fL5GRkVKzZk05efLkxfHT0tKkWbNmF8fLKLPtDlgrOdzvFtqrng6eTuCpnzbSqHoZXr2tYbbjnzuXyEsv/cmXX65j1aoHadGiKo891jYPIlVFxWtzt7H9SEyuTrNBtTK80jP77Ttdu3bt2Lx5MwA//PAD7du358YbbwTAx8eHcePG0aVLFx555BHeffddXnzxRUJDQwHw8vJixIgR/5lmXFwcI0eOZO3atRhjeOWVV+jTpw++vr7ExcUBMHPmTObNm8fkyZO5//77qVChAhs2bKBZs2b88ssvbNy4kXLlrO79a9euzfLly/Hw8GDYsGEcPGhdRPLxxx/Tvn37S+YdGxvL5s2badq06cVhs2bNomfPnlSuXJlp06bx/PPPZ7tefvjhB9q1a0fPnj0vDrvuuuucXq+XM3v2bJYuXQrAfffdR5cuXXjnnXcuGWf79u2kpKTQrVs34NKznObNm2c6XWMMXbp0Yd68efTr1++q48xOoUwUicmpDJ+6DoDPB7SkRDHPy44rIsyYsZ0nnljIsWNxPPpoG2rVKp9XoSqVZ1JTU/njjz8YMmQIYDU7tWzZ8pJxatWqRVxcHDExMWzdupWnn3462+mOGTOGsmXLsmXLFoDLNq842r17N0uWLMHT05O0tP9v796Do6qzBI5/z4K8M6AgqSBiwIg2CQECOKIgCoz4WtAMEuVN4aiIUoJg7Ra7Jaur4oyPlQUXnRkrODWEiA6IL1AQBlSCgMQQCWYixIgiiRghIhAgZ/+4N50m6aQ7mXQn6ZxPVVf1ff/6VPc9fX/33nPLWL16NdOnT2f79u3ExsYSHR3NhAkTmDNnDkOHDqWgoIDRo0eTk5Nzznp27txJQkLCOePS0tJ49NFHiY6OZty4cUEliuzs7Cqx8KekpIRhw4b5nbZixQr69OlzzrjDhw8TE+OUCIqJiaGwsLDKcrm5uXTq1Ink5GQOHDjAqFGjWLRoES1aVL/fAhg0aBBbt261RFFXj775BV98d4w/Tx3ExRe0q3Y+VSU5+TXWrNlHUlIMa9fexaBB3cLYUtOc1Oaff306ceIE/fv3Jz8/n4EDB3r/uar7zHZ/anPlzIYNG1i5cqV3+PzzA//RuuOOO7w7wpSUFB577DGmT5/OypUrSUlJ8a5379693mWOHTtGSUkJUVFR3nGHDh3iwgsv9A4fPnyYvLw8hg4diojQsmVLsrOzSUhI8PuZanuFUFRUFJmZmbVaJpAzZ86wdetWdu/eTY8ePUhJSSE1NdWb0KvTtWtXvvvuu3ptS3Ui7mT2azu+IX3nNzxwfRwjPdF+5zl9+izgfEmGDr2YxYtv5NNP77YkYSJS27ZtyczM5Ouvv6a0tJSlS5cCEB8fT+W6afv376dDhw5ERUURHx/Prl27Aq6/uoTjO67ydf3t27f3vh8yZAh5eXkUFRWxZs0akpOTASgrK2Pbtm1kZmaSmZnJt99+e06SKP9svutOT0+nuLiYnj17EhsbS35+vjeJde7c+ZyjnR9//JEuXbp4YxHMZy0pKaF///5+X75JrVx0dDSHDh0CnKTWtWvV+666d+/OgAED6NWrFy1btuS2227js88+C9iWkydP0rZt24Dz1YeIShTZ3x7lP9/M5pq4zsz5TW+/82zenE9i4jLefHMfAA8/fDUPPvhrWvyThQGNaew6duzI4sWLeeaZZzh9+jQTJ07ko48+YsOGDYBz5DF79mweeeQRAObPn8+TTz5Jbm4u4Oy4n3vuuSrrveGGG1iyZIl3uHxnHB0dTU5OjrdrqToiwu23387cuXPxeDx07tzZ73r9/ZP3eDzk5eV5h9PS0li3bh35+fnk5+eza9cub6K47rrrSE9Pp7S0FIDU1FTveYgJEybwySef8M4773jXtW7dOm93WrnyIwp/r8rdTgBjxoxh+fLlACxfvpyxY8dWmWfw4MEUFxdTVFQEwIcffuh3XZXl5uZW6XYLmbqeBW+oV3VXPf30S6kOe/pD/fUTG7So5GSV6YWFP+uUKasVFmrPnv+jGzfu97MWY+pXY7vqSVX11ltv1VdffVVVVbOysnT48OHau3dvvfTSS3XhwoVaVlbmnfett97SpKQkveKKK9Tj8ei8efOqrL+kpESnTJmi8fHxmpiYqG+88Yaqqq5atUp79eqlw4cP11mzZunUqVNV1f/VRzt27FBAU1NTveOKiop0/Pjx2rdvX/V4PHrvvff6/XwJCQl67NgxPXDggHbr1u2c9quqDhgwQDMyMlRVdeHChZqQkKD9+vXT5ORkLSws9M6Xk5Ojo0eP1ri4OPV4PJqSkqLff/99jbEN5IcfftARI0ZoXFycjhgxQo8cOeL9vDNmzPDO9/7772vfvn01ISFBp06dqqdOnVJV1RdeeEEvuugibdGihcbExJyzzC233KJZWVl+t1vfVz1FRJnxsjLlnr/sZPOXRaTfO4SBl5zbR5qWtodZs97l559LmT//ahYsuJZ27eqvYJYx1cnJycHj8TR0MyLa888/T1RUVJV7KSLZ4cOHmTBhAhs3bvQ73d/3rtmXGV+25Ss25BSy4BZPlSQBcOZMGQkJXcnMvI8nnhhpScKYCDJz5kxat27d0M0Iq4KCAp599tmwba/JX/X0yVc/8Mz6L7k1MYZpV8cCcPx4KY8/voUePTpy//2DmTQpkUmTEq3mjjERqE2bNkyePLmhmxFWgwcPDuv2mvQRxfdHTzI7bTe9LuzA0791EsHbb+cSH/8iTz/9Mbm5RwDnZJklCdNQmlr3rmnaQvF9a7JHFKfPljFrxWf8UnqWlfckUVz0C5Nnv8fq1fvo0+dCtmyZxrBhlzR0M00z16ZNG44cOWKlxk1YqPs8ijZt6rd4aZNNFE+9u49dXxez+K4BxHWNYsuWr1m//iueemokc+cOoVWrmu9qNCYcunfvzsGDB72XPhoTauVPuKtPTTJRvJN1iFc+PsBNl3bhwOYC6NeNa6+9hIKCh+jcufo7sY0Jt/POO69enzRmTEMI6TkKEblRRL4UkTwR+Tc/01uLSLo7fbuIxAZa56kzZcx//XM6nYGXZq7nuecyOH7cuYHGkoQxxtS/kCUKEWkBLAVuAvoAd4lI5dsNZwDFqhoHPA88TQD7C3/m+LFT7P1jFrMfuJI9e2bSvn2r+m6+McYYVyi7nq4E8lR1P4CIrATGAr4FUcYCC933rwNLRES0htP2Z8qUTnt/YvWmaSQlxYSm5cYYY7xCmSguAr7xGT4IVH7Ag3ceVT0jIkeBzsAPvjOJyD3APe7gqawPpmUP/CAkbW5qulApVs2YxaKCxaKCxaLC5XVdMJSJwt+1gJWPFIKZB1V9GXgZQER21vU29EhjsahgsahgsahgsaggIjsDz+VfKE9mHwQu9hnuDlQunu6dR0RaAh2BH0PYJmOMMbUUykSxA7hMRHqKSCvgTmBtpXnWAlPd9+OAD2s6P2GMMSb8Qtb15J5zeABYD7QAXlHVL0TkMZxyt2uBPwN/EZE8nCOJO4NY9cuhanMTZLGoYLGoYLGoYLGoUOdYNLky48YYY8KrSRcFNMYYE3qWKIwxxtSo0SaKUJT/aKqCiMVcEdkrIlkislFEIrZsbqBY+Mw3TkRURCL20shgYiEi493vxhcisiLcbQyXIH4jPURkk4jsdn8nNzdEO0NNRF4RkUIRya5muojIYjdOWSKSFNSK6/oM1VC+cE5+fwX0AloBnwN9Ks1zP7DMfX8nkN7Q7W7AWFwPtHPfz2zOsXDniwK2ABnAoIZudwN+Ly4DdgPnu8NdG7rdDRiLl4GZ7vs+QH5DtztEsbgWSAKyq5l+M/Aezj1sVwHbg1lvYz2i8Jb/UNVSoLz8h6+xwHL3/evASInMgv8BY6Gqm1T1F3cwA+eelUgUzPcC4HHg98DJcDYuzIKJxe+ApapaDKCqhWFuY7gEEwsFfuW+70jVe7oigqpuoeZ70cYCr6ojA+gkIgFrITXWROGv/MdF1c2jqmeA8vIfkSaYWPiagfOPIRIFjIWIDAAuVtW3w9mwBhDM96I30FtEPhaRDBG5MWytC69gYrEQmCQiB4F3gQfD07RGp7b7E6DxPo+i3sp/RICgP6eITAIGAcND2qKGU2MsRORfcKoQTwtXgxpQMN+LljjdT9fhHGVuFZEEVf0pxG0Lt2BicReQqqrPisgQnPu3ElS1LPTNa1TqtN9srEcUVv6jQjCxQERGAQuAMap6KkxtC7dAsYgCEoDNIpKP0we7NkJPaAf7G3lTVU+r6gHgS5zEEWmCicUM4DUAVd0GtMEpGNjcBLU/qayxJgor/1EhYCzc7paXcJJEpPZDQ4BYqOpRVe2iqrGqGotzvmaMqta5GFojFsxvZA3OhQ6ISBecrqj9YW1leAQTiwJgJICIeHASRXN8Pu1aYIp79dNVwFFVPRRooUbZ9aShK//R5AQZiz8AHYBV7vn8AlUd02CNDpEgY9EsBBmL9cANIrIXOAvMV9UjDdfq0AgyFg8DfxSROThdLdMi8Y+liKThdDV2cc/HPAqcB6Cqy3DOz9wM5AG/ANODWm8ExsoYY0w9aqxdT8YYYxoJSxTGGGNqZInCGGNMjSxRGGOMqZElCmOMMTWyRGEaHRE5KyKZPq/YGuaNra5SZi23udmtPvq5W/Li8jqs4z4RmeK+nyYi3Xym/UlE+tRzO3eISP8glnlIRNr9s9s2zZclCtMYnVDV/j6v/DBtd6Kq9sMpNvmH2i6sqstU9VV3cBrQzWfa3aq6t15aWdHOFwmunQ8BlihMnVmiME2Ce+SwVUQ+c19X+5knXkQ+dY9CskTkMnf8JJ/xL4lIiwCb2wLEucuOdJ9hsMet9d/aHb9IKp4B8ow7bqGIzBORcTg1t/7qbrOteyQwSERmisjvfdo8TUT+t47t3IZPQTcR+T8R2SnOsyf+yx03GydhbRKRTe64G0RkmxvHVSLSIcB2TDNnicI0Rm19up1Wu+MKgd+oahKQAiz2s9x9wAuq2h9nR33QLdeQAlzjjj8LTAyw/X8F9ohIGyAVSFHVvjiVDGaKyAXA7UC8qiYC/+27sKq+DuzE+effX1VP+Ex+HUj2GU4B0uvYzhtxynSUW6Cqg4BEYLiIJKrqYpxaPter6vVuKY//AEa5sdwJzA2wHdPMNcoSHqbZO+HuLH2dByxx++TP4tQtqmwbsEBEugN/U9V/iMhIYCCwwy1v0hYn6fjzVxE5AeTjlKG+HDigqrnu9OXALGAJzrMu/iQi7wBBlzRX1SIR2e/W2fmHu42P3fXWpp3tccpV+D6hbLyI3IPzu47BeUBPVqVlr3LHf+xupxVO3IypliUK01TMAQ4D/XCOhKs8lEhVV4jIduAWYL2I3I1TVnm5qv57ENuY6FtAUET8Pt/ErS10JU6RuTuBB4ARtfgs6cB4YB+wWlVVnL120O3EeYrbImApkCwiPYF5wGBVLRaRVJzCd5UJ8IGq3lWL9ppmzrqeTFPRETjkPj9gMs6/6XOISC9gv9vdshanC2YjME5EurrzXCDBP1N8HxArInHu8GTg726ffkdVfRfnRLG/K49KcMqe+/M34DacZySku+Nq1U5VPY3ThXSV2231K+A4cFREooGbqmlLBnBN+WcSkXYi4u/ozBgvSxSmqXgRmCoiGTjdTsf9zJMCZItIJnAFziMf9+LsUN8XkSzgA5xumYBU9SROdc1VIrIHKAOW4ex033bX93eco53KUoFl5SezK623GNgLXKKqn7rjat1O99zHs8A8Vf0c5/nYXwCv4HRnlXsZeE9ENqlqEc4VWWnudjJwYmVMtax6rDHGmBrZEYUxxpgaWaIwxhhTI0sUxhhjamSJwhhjTI0sURhjjKmRJQpjjDE1skRhjDGmRv8P5tHaIz14QuoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(tree, y_test, X_test, \"Decision Tree (GINI)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.82 0.80 0.81 6729\n", + " 1 0.39 0.41 0.40 2020\n", + "\n", + " accuracy 0.71 8749\n", + " macro avg 0.60 0.61 0.60 8749\n", + "weighted avg 0.72 0.71 0.72 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, tree.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[0] = ([\"Decision Tree\" , \n", + " classification_report(y_test, tree.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Random Forest Classifier\n", + "\n", + "#### Theory\n", + "Random Forest is an ensemble method for the decision tree algorithm. It works by randomly choosing different features and data points to train multiple trees (that is, to form a forest) - and the resulting prediction is decided by the votes from all the trees. \n", + "\n", + "Decision Trees are prone to overfitting on the training data, which reduces the performance on the test set. Random Forest mitigates this by training multiple trees. Random Forest is a form of bagging ensemble where the trees are trained concurrently. \n", + "\n", + "#### Training\n", + "To keep things consistent, our Random Forest classifier will also use the GINI Coefficient.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.ensemble import RandomForestClassifier\n", + "randf = RandomForestClassifier(n_estimators=300)" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n", + " max_depth=None, max_features='auto', max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, n_estimators=300,\n", + " n_jobs=None, oob_score=False, random_state=None,\n", + " verbose=0, warm_start=False)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "randf.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 1.00 1.00 1.00 13475\n", + " 1 1.00 1.00 1.00 4021\n", + "\n", + " accuracy 1.00 17496\n", + " macro avg 1.00 1.00 1.00 17496\n", + "weighted avg 1.00 1.00 1.00 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, randf.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training set has also been 100% correctly classified by the random forest model. Evaluating with the test set:" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (Random Forest) identified 749\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06310419
11271749
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6310 419\n", + "1 1271 749" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, randf.predict(X_test), \"Decision Tree (Random Forest)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.26\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEWCAYAAADcsGj7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KCCQhjRBCIBASIPROpAsoVUSxYG/YEBv27rWiV71c27WLfnZRsSOIIAI2pErvNaGkkt6T/f0xEzyEQA7l5JycrPd5eMiZumZmz6yZPTN7xBiDUkop5Yl83B2AUkopdSSapJRSSnksTVJKKaU8liYppZRSHkuTlFJKKY+lSUoppZTH8sgkJSKXichP7o7D3UQkRkRyRcS3BucZKyJGROrV1DxdSUTWicjQ4xjvuMqgiASIyPcikiUiXxzr+DXJ2fLlrfujiPiLyEYRiXTDvOvZ+1lsTc/bE4hILxH51Zlhq01SIrJTRArswrxfRN4TkaATD/PIjDEfG2NGunIenshe18MrfhtjdhtjgowxZe6My13snbjtiUzDGNPZGLOgmvkclphPoAyOB5oCjY0xFxzH+JVjGyoi5fb+lysiSSLyuYiccqLTdrZ8uWJ/tBNfxTIVVFrG3JM5r6O4EZhnjEmxY/pIRIrtGDJE5CcRaVdDsbiMiPwmIoWO6/dklJ9jmP9hCdkYswIoEJEzqhvf2Sups4wxQUAPoCfwwHHE6nbuvDrwliuTY1FH13crYLMxpvRYRzxKzHvt/S8Y6AdsBH4VkWHHH6Z72YkvyF6uM7CX0aHbIVy0PW8APqzU7Wl7/tFACvC2C+brDpMc168xZumxjCwiPiJysmvePsbaBkdnjDnqP2AnMNzh93PADw6/GwBTgd1AMvAGEODQfxzwN5ANbANG291DgXeAfcAeYArga/ebAPxm//0GMLVSTN8Cd9p/Nwe+BFKBHcBkh+EeA2YAH9nzv66K5QsFPrDH3wU8DPg4xPE78D8gC+vgMKzSuEdbht+BF4AMu18bYD6QDqTZGynMHv5DoBwoAHKBe4FYwAD17GEWAE/a080BfgIiHOK50l6GdOBflbddpeUOAP5rD58F/GZ3q5jnVfY2TQMechivD/AnkGkv9ytAfYf+BrgZ2ALssLu9BCTa22A5cKrD8L7Ag3bZyLH7twQW2dPKs9fHRfbwY7HKUybwB9CtUlm9D1gNFAH1HNeBHfsyO45k4Hm7+257Xrn2v/44lEF7mM7AXHtbJgMPVrFOHweKgRJ7OtdinQg+bK/nFKyyFmoPX7Gur7VjWFTFNIcCSVV0fwVY5vC7g0N8m4ALj2FbV5SvCcB2ezvsAC6rvD/avwcAS+1pLQUGOPRbwFHK6BHK4pGWMQm4B1gDFNvdWgBf88/+frPD8D78U5bSgOlAoyPMszVW2fJ16PYR8JjD77OBLIff8cAv/LP/flixLR3ivdOONwv4FGjg0P9+YD/WseJae93H2v3C7PmnYpXZBwCx+10HLARexir3W4G+9jQSscrj5UdZv78BE47QbxDWPpEFLAH6VhrvSaz9vcAuL2HA/2Ht+0nAE/xzvGyHtd9m2evnE7v7Hxy6L59vd29ld/M7avk4Wk+HHb9iJ29hb4CXHPq/CHwHhGOd6X0P/NvhoJAFjLALUDTQwe73DfAm0BCItFfQDZV3CmCwvSEqNlgje4U1t6e5HHgEqI9V8LYDoxySVAlwjj1sQBXL9wFW0gu2N8Jm4FqHOEqBOwA/4CJ7ecKdXIZS4Fasg2UA0NZeFw2AJvYGfbGqdV3pIOaYpLbZhSHA/v2M3a+TXQAG2etiqr3sR0pSr9rjR2MligF2XBXzfNueR3esA35He7zeWGfz9exhNwC3V0pSc7HKQ4Dd7XKgsT3OXVg7qr/dr+Ig1B4Qe36NHabV1mHavbAO9H3tmK+y11kDh/X3N1aSC6i8TrF2tivsv4OAflWt5yrKYDDWTnkX4G//7nuE9foY8JHD72uwDiqt7Xl+BXxYab4fYJWhqsrnUKo+gJ+OdVLT0P6XCFxtr+NeWAeJzk5u63r2NLKB9vY4zRzGd1wX4cAB4Ap7vEvs3xXbbAFHKKPHmaSWYx13AuzY/8ZKRPWx9qed2CeOwN1YyTHa3k7vVKzrKqY9DlhVqdvBJGVvq0+B5Q792wHD7HlH2vOaWinexUAUVnnfjH1ijHVytQ9rP20IfM6hSeoTu2wE22VlK3CVQ5Iqtde5L/AM1gnHy/Z2HIN1XAo8liQFRNjjXWJvy8uxEnAjh/F2Ah2xjn/1gJnAa0CgvZzL+ed4+QXWSaKPvf4H2t3rOS5rpRjygU4nI0nlYp0VGeBn/jn7F6xM2MZh+P78cwb9JvBCFdNsinXgc7ziugT4pYqdQrDOMgfbv68H5tt/9wV2V5r2A8D/ORwwDjs7dRjW146jk0O3G4AFDnHsxU6QdrcldmFxZhl2H2ne9jDnACsrrevqktTDDv1vAn60/34E+NShXyDWWf1hScouRAVA9yr6VcyzRaVlvvgIy3A78LXDbwOcXs1yH6iYN9ZZ/7gjDFc5Sb0OPFlpmE3AEIf1d00V5bciSS3CutqJqDTMIeu5ijJ4ieN2qmbZHuPQJPUzcJPD7/ZYJw/1HObb+ijTG0rVB/AO9rjRWCdPv1bq/ybwqJPbuiJJZQLnUylZVloXVwBLKvX/E/sgeLQyehzLmARc6fB7ILC90jD/At62/95SURbs3y2x9lGfKqZ9FQ5Xh3a3j4BCez0YrBPeLkeJezywtFK8Fzv8fh54xf77A2CKQ79O9jxisRJAKdDOof/NWPfLwEpSGxz69bTHbezQLetIsWIlm3x7uTIrth/WSc0flYZdin1VZo/3iEO/aLssOV4dXgHMtf/+BGsfja40zaMlqWQcrsSr+udsHeM5xphgrMLUASsDg3U1EAgsF5FMEckEfrS7g1VItlUxvVb2htnnMN6bWGcnhzDWkkzHOlAAXIpVTVYxneYV07Cn8yBWAqmQeJTlisA6K9rl0G0X1saosMeOwbF/cyeX4ZB5i0ikiEwXkT0iko21U0RwbPY7/J2PdcaHHdPB+Rlj8rHOiqoSgXWmU9W2Oep8RKSdiMy0H6LJBp7m8GWovNx3icgG+4m3TKxq0opxjlRGqtIKuKvS9m6JtexVzruSa7HOhjeKyFIRGevkfI8lxsqac3j5qofzZfRIorF2/Eys9dK30nq5DOtM15ltjTEmDyvZTcIq0z+ISAcnlgcO32eOVEaPh+O6aQXEVFrOe7GWEyAG+N6h3xqsdVTV03sHsK5aKnvGGBMGxGGd5MVX9BCRKPuhlYr99z0OL/tO7Z8cug4jsU6Yj3YcSnb4uwAoM8akV+p2tPV8kzEmzP7XxyGm6rZl5fXfAEh2WMev8k9ZvgvrmLhMRNaIyFVHiadCMFYZPqJjuhFmjFmItWGm2p3SsFZOZ4cVEGr+ufGZiHUfprJErDOcCIfxQowxnY8w60+B8SLSCuvq6UuH6exwmEaYMSbYGDPGMeyjLFIa1lltK4duMVh1xhWiRUQq9d/r5DJUnve/7W7djDEhWJfXcpThj8U+rGoRwHoUGqvKoSppWGeMVW2b6ryOdW8u3l6GBzl0GcBhOUTkVKwqgAuxqhHCsM76KsY5UhmpSiLwVKXtHWiM+bSqeVdmjNlijLkE66DwLDBDRBoebZzjiLGyvRxevko59KBzPNv9XGCFnVwSgYWV1kuQMeZGjmFbG2PmGGNGYFX1baTqhwYqLw8cvs+cTI7rJhHYUsX+fpbdPwkYUam/vzFm/+GTZTXQ5kiP3xtjdmJV8/9PRBrYnZ/F2ue72mV/AoeX/SPZh3WyUyHG4e8UoIyjH4dcwZltWXn952Pd7nA85nUDMMbsM8ZcZ4xphnUl+JaIxHGE8m0fz8G6Aj6i43la40VghIj0MMaUYxXkFyreNRCRaBEZZQ/7DnC1iAyznw6JFpEOxph9WDdU/ysiIXa/NiIypKoZGmNWYt1QnAbMMcZUZN4lQLaI3CfW+ym+ItLF2ccrjfXo7efAUyISbK+0O7GucCpEApNFxE9ELsCqn511rMtgC8aqOs0UkWis+zGOkrHqo4/HDOAsERkgIvWxqrWq3IHs7fYu8LyINLfXW3+HnfFogrHuXeTaZ9o3OjF8Kdb2qycijwAhDv2nAU+KSLxYuolIRXKtvD7eBiaJSF972IYicqaIVHVGfBgRuVxEmtjLX1GGyuzYyjnyup8JRInI7SLSwC4rfZ2ZJ9YJ1h0iEifWqxtPA5+Z43v6T+x96FGsKqAHHeJrJyJX2OXUT0ROEZGOzm5rEWkqImfbSbsIq5xW9Wj6LHtel4r1aPFFWFVXM491eY7Dn0CxfWXuby9LVxHpbfd/A3haRGLgYM3F2VVNyE5Cu7HusVbJGDMbq2xcZ3cKxrq9kSUiLbHugTnrc+AaEelgr+NHHeZTgrX/Pi0iQfaB/Q4OPQ65wkygs4hcZG/LS7Hu882qamBjTCLWAxxTHY55bUVkMICIXGgf1+CfKtMy+zibzuH71xCsKs2SowV5zEnKGJOKVb/6L7vTfVg3+Rbbl8DzsOrdMcYswar3fAHr7Hkh/2TuK7Gq2tZjXXrPwDqDO5JPgeFY9Z4VsZQBZ2E9Gr8D66xxGlZ1krNuxSp427HqYD/B2qkr/IV1yZ8GPAWMd7jMPtZleBzrpnYW8APWjVJH/wYeti+lj2UHwBizzl6W6VhnbTlYZ2hFRxjlbqzqkKVYT4Q9i3Pl4W6sKtccrKTxWTXDzwFmY91E3oV1Vu9YhfA81g78E1byewfrJjlY93fet9fHhcaYZVj3JF/BWt9bsc5mnTUaWCfWezgvYd0/KLSrRp8Cfrfn1c9xJGNMDtYDL2dhVedsAU5zcp7vYj0FtgirjBZibadj0dyOORdre3UFhhpjfnKIbyRwMdbZ8X6s7VmRiJzZ1j5Y1TV77WGGYN1POoRd9sfaw6ZjVbeNNcakHeMyHTM7sY/BeiBrJ9Y++Sb/nPQ8j3W74WcRycF6quxoJ6xvYt1TOZqpwH32id+j/PMw2Hf8U6PjTOzfY1WNLcTaF+ZWGuQmrOrFHfYw72MdZ13GPpafjXUMT8dKjGONMRlHGe1yrPuXFce8L/inurUvsFRE8rCObTcbY3bb/R4FPrH3r/PsbpdhnVgcVcUTc6oKIjIB6+mcQe6O5VjZZ+2ZWNVyO9wdj1KeRkT8gZVYD1ukuDueukREegL/c+bY6pHNIqnjIyJniUigXZ0wFevsead7o1LKM9lX0R01QdU8Y8xKZ0/+NUl5l3FY1TV7saooLzZ6qayUqsW0uk8ppZTH0isppZRSHqtWNnoaERFhYmNj3R2GUkrVKsuXL08zxjSpfkjPUSuTVGxsLMuWLXN3GEopVauISOUWJjyeVvcppZTyWJqklFJKeSxNUkoppTyWJimllFIeS5OUUkopj6VJSimllMdyeZISkXdFJEVE1h6hv4jIyyKyVURWi0gvV8eklFKqdqiJK6n3sD6RcCRnYLUzFw9MxPqonlJKqZOotjaB5/KXeY0xi0Qk9iiDjAM+sBtCXSwiYSLSzP6ooFJKqaMwxpBbVEpKThEp2UWk5BSSmlNk/y5kf3YhiWn57M0scHeox8UTWpyI5tCP4CXZ3Q5JUiIyEetKi5gYxy8vK6WUdyoqLSMlu4hd6fls2JfNhv3ZpOYUUVRSTmFpGVkFJaRkF1FQcvhHlOvX86FJUANKs4vYuSEdKTjmj0F7BE9IUlV94vyw61JjzFvAWwAJCQm187pVKaUqySooYeXuA6zdk8XerEKSswrZl1VIcnYh6XnFhwzbNKQBzcMC8K/nS3jD+rRq3JDI4AbWv5AGRAb727/9CQmox9Spf3Dvv3/loos68/yro4iOvtpNS3n8PCFJJQEtHX63wPoeklJKeYXSsnJyCks5kF/MjrQ8tqbksiUll1WJmWxJyT04XHjD+jQN8adZqD/dW4bRLNSfqBB/WoQH0CEqhPCG9audV0pKHts3pdGzZzNuvrkPPXs2Y/jw1q5cPJfyhCT1HXCLiEwH+gJZej9KKVVbbUnO4Yc1+5i/MYWU7CKyC0vILz68Oi4iqAFdokM4u3tzerdqRLeWYQQ1OP5Dcnm54Z13VnDfffOIigpi7dqbCAz0q9UJCmogSYnIp8BQIEJEkoBHAT8AY8wbwCxgDLAVyAdq3/WoUqpOKis37EjLZd3ebNbtzWb+xhS2puQiAgmtGjG4XQTB/n6E+PsRElCPEH8/WjUOpG1kEGGB1V8VOWv16mQmTZrJn38mMXRoLK+/fiY+PlXdSal9auLpvkuq6W+Am10dh1JKnYiC4jI27reS0fp92azfm83G/dkUlpQDUN/Xh16twriqf2dGdYkiMti/RuL6668kBg58l0aNAnj//XO44opuiHhHggLPqO5TSimPkp5bxPp9dkKyk9L21FzK7Ue2Qvzr0bl5KJf1bUXn5iF0ah5CmyZB+PnWXCM+SUnZtGgRwimnRPPkk6dxww0JhIcH1Nj8a4omKaVUnVVebkg8kM/6vYdeIe3PLjw4THRYAJ2ah3Bm12YHE1J0WIDbrlZ2785i8uTZLFy4i02bbiEysiEPPHCqW2KpCZqklFJeKyu/hF0ZeWQVlJBdUEp2YQnZBSXsyypk/d5sNuzLJqfIen/I10do2ySIAW0a08lORp2ahZzUe0cnoqSkjJde+otHH12AMYbHHhtKo0Y1U6XoTpqklFJeITEjn792ZLBpfzabknPZvD/nkCsiR4H1fenYLIRze0XTqVkInZuHEt80CH8/3xqO2jm5ucUMHPguq1cnM3ZsO/73vzOIjQ1zd1g1QpOUUqrWKC0rt5r7ySkiObuQlJwi1iZl8cf2NBIzrGZ/GtTzIb5pEAPaNqZd02BaRzQkvGF9QgL+ecouwM+3VjxcUFJShp+fL0FB9Rk2LI7HHhvCOed0qBWxnyyapJRSHq2wpIxft6Qxa80+5q1PPlg9VyHEvx792zTmukGt6de6MW0jg/Ct5Y9fG2P45JM13H//z8yefRldukTy/POj3B2WW2iSUkp5lILiMvvJuiyW7TzA/I0p5BaVEhrgx+guUfSMaUTTiiaAQhoQEdSg1iclR5s2pXHTTbOYP38HffpEU4cumqqkSUop5TZWQspiTVIWa/Zks3ZPFltTcymzn/WOCKrPmK5RjOnajIFtI2r0EW93eOqpRTzxxCICAurx2mtjmDixN75evszV0SSllKoReUWlrN+XzZqkLNbuyWLt3iy2pvzz7lFEUAO6RocwqnNTukSH0iU6lGah/nXq/ktubjHjx3fiv/8dSVRUkLvD8QiapJRSJ1VZuWF3Rj6b9uewOTmHTck5bNqfw7bUXCq+uxcZ3ICu0aGM7tKMrtGhdI0OpWlIgzqVkAD278/lrrt+4soruzFqVFueemqY1zRndLJoklJKHRdjDPuyCtmcbCej/blsTs5hS0rOwaaCRCAmPJB2TYMZ2+2fhBQZ4v3v9xxNebnhzTeX8cADP1NQUMrAgdaHIDRBHU6TlFLqqIwxJGcXHUxGW5Jz2ZySw9bk3EOetIsMbkD7qGAu79uKdlHBtG8aTHzTIALr62HG0d9/72fSpJn89dceTj89jtdeG0P79hHuDstjaelRSh1kjGFXej5LdmawcncmW+zElF34TzIKb1if+MggzukZTbumQcQ3tRJSIye+daSsBmF37Mjko4/O5dJLu9a5Ks5jJcbUvo/cJiQkmGXLlrk7DKVqvfJyw6bkHJbsyGDJzgyW7MggNacIsN4/ah8VTLum1r/4pkG0axpMRFADN0dduxhj+OabjRQWlnLJJV0pLzdkZxcRFlbzVZ4istwYk1DjMz4BeiWlVB1SVm5YsyeLxdvTWbojg6U7Mw5eJTUL9WdAm8acEhtO37hw2jQJ0nskJ2jXrkxuuWU2M2duZujQWC6+uAs+PuKWBFVbaZJSysslZxeycHMqizan8tvWNDLzSwBo3aQhY7o245TYcPrEhdOikfta9vY2JSVlvPDCYh5/fCEiMHXqCG67rZ+u3+OgSUopL5SWW8SsNfv47u+9LNt1AIAmwQ04vUMkQ9o1YUCbCJoEa7Wdq/z+eyL33TePc87pwEsvjSYmJtTdIdVamqSU8gIVDzz8tSOdWWv289vWNMrKDe2aBnH3yHac3qEpHZsF65m8C6Wn57No0S7OPbcjQ4fGsmTJdZxySrS7w6r1NEkpVUttT83l1y1pLLHvLaXYDzxEhwUwcXBrxvVoToeoEDdH6f2MMXz44Wruuusn8vKK2b37DiIiAjVBnSSapJSqRbam5DBrzX5mrdnHxv05ADQP9ae//cBDn7hw2uoDDzVm48Y0brzxBxYs2En//i14442xREQEujssr6JJSqla4I9taTz+3Xo2JecgAgmtGvHI2E6M6NSUluF6UHSHjIwCEhLews/PlzffHMt11/XSkwMX0CSllAdLyy3ihbmb+fiv3dT39eH+Mzpwbs9omtbxZoXcadWq/XTvHkV4eAD/93/jGDIklsjIhu4Oy2tpklLKA6XlFvHZ0kReX7CNgpIyJgyIZfKweMK1VQe32bcvhzvumMNnn61j/vwrOe20OC64oLO7w/J6mqSU8hC5RaX8tG4/3/699+DTecM7RnL/GR1pG6mfbXCXsrJyXn99GQ89NJ+iolIef3woAwa0dHdYdYYmKaXcbHVSJm8t2s68DckUlpQTHRbADYNbM65HNO2jgt0dXp1mjGHMmE/46adtDB/emtdeG0N8fGN3h1WnaJJSyk12pOUxdc4mflizj9AAP8b3bsE5PaLpFdNIb8C7WU5OEQ0b1sfHR7jyym5MmNCdiy/uou+ZuYEmKaVq0J7MApbtzGDR5jS++XsPDer5MPn0tlw/uDXB/n7uDq/OM8bw5ZcbuO22H3nssSFcf31vLrusm7vDqtM0SSnlQoUlZSzcnMqPa/ezeHs6+7IKAWhY35dL+8QweVi8Nk/kIXbsOMAtt8xm1qwt9OgRRY8eUe4OSaFJSqmTLr+4lF+3pDFrzT7mrU8mr7iMRoF+DGwbQUKrRiTEhtMhKph6vj7uDlXZpk1bweTJs/H19eGFF0Zxyy19qFdPt48n0CSl1EmQklPIzxtSmLc+md+2plFUWk5YoB9ndW/Omd2a0a91Y/w0KXkcYwwiQsuWIYwe3ZaXXz6DFi20KSlPoklKqeNUXm74eWMKby3axtKdVkvj0WEBXNInhhGdmtInLlwTk4dKS8vn3nvn0rx5MFOmnM6oUW0ZNaqtu8NSVdAkpdQxKi0r54c1+3jtl21sSs6hZXgAd45ox4hOTekQpS2NezJjDO+99zf33DOXrKwi7r9/oLtDUtVweZISkdHAS4AvMM0Y80yl/jHA+0CYPcz9xphZro5LqWNVVFrG1yv28PrCbexKzyc+MogXLurOWd2a6/2lWmDz5nSuu+47fv11NwMHtuSNN8bSpUuku8NS1XBpkhIRX+BVYASQBCwVke+MMesdBnsY+NwY87qIdAJmAbGujEupY1FQXManS3bz1qLt7M8upGt0KG9c3puRnZrq+0y1SHFxGZs3pzNt2llcfXVP3Xa1hKuvpPoAW40x2wFEZDowDnBMUgaouFMZCux1cUxKOSW7sIQP/9zFO7/tICOvmD5x4Tw3vhunxkdolV4tMXv2Fn75ZSfPPTeCLl0i2bXrdho00LsctYmrt1Y0kOjwOwnoW2mYx4CfRORWoCEwvKoJichEYCJATEzMSQ9UqQrpuUW8+/sOPvhjFzlFpQxt34SbT2vLKbHh7g5NOWnPnmxuv30OM2asp0OHCB5+eDAhIQ00QdVCrt5iVZ1umkq/LwHeM8b8V0T6Ax+KSBdjTPkhIxnzFvAWQEJCQuVpKHVCEjPy+XlDMvM2pLB4ezplxnBGlyhuGtqWLtGh7g5POam0tJzXXlvKww/Pp6SknClTTuOeewZSv76vu0NTx8nVSSoJcGwuuAWHV+ddC4wGMMb8KSL+QASQ4uLYVB23Kz2Pmav3MXP1PjbsywagbWQQ154axwW9W2rL47VQRkYBjzzyCwMGtOTVV8fQpo1e/dZ2rk5SS4F4EYkD9gAXA5dWGmY3MAx4T0Q6Av5AqovjUnVUak4RX61IYubqfazZkwVA71aNeGhMR4Z3akpchH68rrbJyirk7bdXcOed/YmMbMjKlTcQGxum9w29hEuTlDGmVERuAeZgPV7+rjFmnYg8ASwzxnwH3AW8LSJ3YFUFTjDGaHWeOqn2ZxXy5qJtfPLXbopKy+neIpSHxnRkTLdmRIcFuDs8dRyMMXz++Tpuv30Oycm5DBjQkgEDWhIX18jdoamTyOV3Ee13nmZV6vaIw9/rAX2jTp1Uxhi2pOTy1/Z0Fm/PYO76ZMqM4dye0dw4tA1tmmhVXm22bVsGN988izlzttGrVzO+//4SEhKauzss5QL6qIvyCuXlhk3JOfy1PZ2/dmSwZEcG6XnFAESF+HNBQgtuGNyGmMaBbo5UnShjDOPGTWf37ixefnk0N910Cr76MrXX0iSlai1jDEt3HuCDP3fy29Y0MvNLAKv9vCHtm9AvrjF9W4cTEx6o9ye8wK+/7qJ37+YEBvrx3nvn0Lx5MM2b65eLvZ0mKVXrpOUWMX9DCh8s3snaPdmEBfoxomNT+rZuTN+4cFqG69WSN0lNzeOee+by/vur+Pe/h3H//YO0aq8O0SSlPF5pWTl/J2ayYFMqCzenHnwqr21kEE+f25Vze0YToO/BeJ3ycsO7767k3nvnkptbzIMPDmLy5MptAShvp0lKeays/BI+WbKb9//Yyf7sQnwEesU04u6R7Rjcrgldo0O1Gs+L3X77j/zvf0s49dQY3nhjLJ06NXF3SMoNNEkpj1JWblidlMk3K/fw+bIkCkrKGNi2MQ+P7cipbZsQGujn7hCVC+XlFVNUVEZ4eADXX9+LXr2acdVV3fVkpA7TJKXczhjD3PXJ/Lh2Pws2p5KRV4yfr3B292iuHRRHp+b6pdS64IcfNnPzzbMYODCGjz8+j65dm9K1a1N3h6XczOkkJSL1gRhjzFYXxqPqmLV7shMjzc4AACAASURBVHj8+3Us3XmARoF+DG0fydD2TRgc34RGDeu7OzxVA5KSsrntth/56qsNdOrUhEmTers7JOVBnEpSInIm8DxQH4gTkR7Ao8aYc10ZnPJe+7IKePnnrUxfuptGgfV55ryuXJDQEl/9xk+d8uOPW7nggi8oKyvn3/8exp139tfGYNUhnL2SegLrExu/ABhj/haRti6LSnmtzck5vLlwO9/+vcdqA2tALLcPb0dogN5rqktKSsrw8/OlR48oxoyJ55lnhmlzRqpKziapEmNMZqWbl9q+nnJKaVk58zak8MmS3SzanEqAny+X92vFtYPi9J2mOiYzs5AHH/yZtWtTWLBgAlFRQXz22Xh3h6U8mLNJaoOIXAj42C2a3wYsdl1YyhsYY/hhzT6e+mED+7IKiQrx584R7biiXyu931THGGOYPn0td9wxh9TUfG69tQ/FxWX4++uzW+ronC0htwCPAOXAV1itmj/gqqBU7bcvq4B/fbOWeRtS6BodyhPjunBa+ybU0zbW6pz9+3O58sqvmTt3OwkJzZk16zJ69Wrm7rBULeFskhpljLkPuK+ig4ich5WwlAKsJ/Xmrk/mz23prEw8gK+P8PCZHZkwIFaTUx0WEtKAtLR8XnnlDCZNStDGYNUxEWc+3SQiK4wxvSp1W26MccuzogkJCWbZsmXumLWqwpqkLF6ct5mfN6YgAl2jQ+nfpjGX9WmlrY7XUfPn72Dq1D/48ssLCQjwo7zc4KNPbrqdfdxOcHccx+KoV1IiMgrr0+7RIvK8Q68QrKo/VYet3ZPFi/O2MG9DMqEBftwzqj2X9Y0hLFDvN9VVycm53H33XD76aDVt2jRi9+4s2reP0ASljlt11X0pwFqgEFjn0D0HuN9VQSnPtn5vNi/O28xP65MJ8a/HXSPaMWFgLMH++hh5XVVebnj77eXcf//P5OUV869/DeaBBwYRoK8WqBN01CRljFkJrBSRj40xhTUUk/JQm/bn8OK8zcxeu59g/3rcPjyeawbFEaLJSQEffLCaHj2ieP31M+nQIcLd4Sgv4eyDE9Ei8hTQCfCv6GiMaeeSqJRH2Z6ay0s/b+G7VXtpWL8ek4fFc+2gOH0Bt47LzS3m6ad/5bbb+tK0aRDff38JjRr5a2Ow6qRyNkm9B0wBpgJnAFej96S8XmJGPi//vIWvVu6hvq8Pk4a0YeKprfUdJ8W3327k1ltnk5iYTdu24VxzTU/CwwPcHZbyQs4mqUBjzBwRmWqM2QY8LCK/ujIw5T77swp55ZctfLY0ERHhqv6x3Di0DU2CG7g7NOVmu3dnMXnybL79dhNdu0Yyffp4Bgxo6e6wlBdzNkkViXUNv01EJgF7gEjXhaXcIS23iNcXbOPDxbsoLzdc3KclN5/WlmaheoasLI8+uoC5c7fz3HPDuf32fvj5aWOwyrWcfU+qL7AeaAQ8BYQCzxpjfndteFXT96ROnryiUr5btZdfNqbw65Y0ikrLOL9XCyYPi9d29RQAf/6ZSGioP506NSE5OZfCwlJatQpzd1jqOHjde1IVjDF/2X/mAFcAiEgLVwWlXC8lu5D3/9zJR4t3k1VQQnRYAOf1iuaaQXG0aRLk7vCUBzhwoID775/HW2+tYPz4TnzxxQU0baplQ9WsapOUiJwCRAO/GWPSRKQzVvNIpwOaqGqR0rJyFm/P4OuVe/h+1V5KyssZ3TmK605tTa+YMH0qSwFWY7Aff7yGO++cQ0ZGAXfe2Y/HHhvq7rBUHVVdixP/Bs4HVmE9LPE1VgvozwKTXB+eOlmW7zrAPTNWsT01j6AG9bjwlBZcN6g1sREN3R2a8jDTpq1g4sSZ9OkTzU8/XUGPHlHuDknVYdVdSY0DuhtjCkQkHNhr/97k+tDUiUrPLeKzZYks33mAXzal0Cw0gP9d0pMRnZrirze8lYPCwlJ27cqkffsILrusG/Xq+XDlld21MVjldtUlqUJjTAGAMSZDRDZqgqod1u7JYuIHy9ibVUjriIZcMzCO24bHa9NF6jBz527jpptmUV5u2LjxZgID/bj66p7uDkspoPok1VpEKj7HIUCsw2+MMee5LDJ1XFJyCnntl218/NcuIoIaMPPWQXSJDnV3WMoD7d+fy513zuHTT9cSHx/Om2+O1UfKlcepLkmdX+n3K64KRJ2YlJzCgx8ZBLjolJbcOaIdEUH6Aq463IYNqfTv/w4FBaU8+ugQ7r9/kH4lV3mk6hqY/bmmAlHHxxjDH9vSufPzv8kqKGHi4NZcmNCSOH0gQlUhO7uIkJAGtG8fwTXX9GTSpATatWvs7rCUOiI9darFdqblcdcXq1i+6wAx4YF8fVMfOjYLcXdYygPl5BTx6KML+PDD1axdeyNNmwbx/POj3B2WUtVyeZISkdHAS4AvMM0Y80wVw1wIPAYYYJUx5lJXx1WblZaV88mS3fxnziZ8fYQnx3VmfO+WBNTX+wnqUMYYvv56I5Mnz2bv3hxuuKE3DRrouamqPY6ptIpIA2NM0TEM7wu8CowAkoClIvKdMWa9wzDxwAPAQGPMARHRNgGPYkdaHjd+tJyN+3Po37oxz43vps0XqSoVF5dx/vmfM3PmZrp3b8qMGRfSr5++f69qF6eSlIj0Ad7BarMvRkS6A9cZY26tZtQ+wFZjzHZ7OtOx3r1a7zDM9cCrxpgDAMaYlGNbhLrjp3X7uffL1QjwxuW9GNU5SluJUIcxxiAi1K/vS1RUQ/7735FMntyXevX0nSdV+zhbal8GxgLpAMaYVcBpTowXDSQ6/E6yuzlqB7QTkd9FZLFdPagcJGcXMunD5Uz8cDnNQgP45uaBjO7STBOUOszvv++md++3WLvWOtd7++2zufPO/pqgVK3lbHWfjzFmV6WDYpkT41V1FK3c7Ho9IB4YitUW4K8i0sUYk3nIhEQmAhMBYmJinAy79puxPInHv1tHcVk5945uz/WntsZPWwFQlaSn53P//fOYNm0lMTGhZGQUuDskpU4KZ5NUol3lZ+z7TLcCm50YLwlw/CJaC6ymlSoPs9gYUwLsEJFNWElrqeNAxpi3gLfA+lSHk3HXWttSc/nPj5v4cd1++saF8+z53bSdPVWljz9eze23z+HAgQLuuWcAjzwyhKAg/Xqy8g7OJqkbsar8YoBkYJ7drTpLgXgRicP6UOLFQOUn974BLgHeE5EIrOq/7U7G5ZVWJWZy5btLyCooYfKweCaf3pZ6evWkjmDdulTi48N5442xdOvW1N3hKHVSOZukSo0xFx/rxI0xpSJyCzAH6xH0d40x60TkCWCZMeY7u99IEVmPVYV4jzEm/Vjn5S1WJ2Vy+bS/CAnw48Nr+9CthX5cTh2qoKCEp576lcGDWzFyZBsee2wo9er54OOj9yiV93E2SS21q+E+A74yxuQ4OwNjzCxgVqVujzj8bYA77X912lcrkrjvy9VEBvvzxaT+NA/Tz7arQ82Zs5WbbprF9u0HKC83jBzZhvr6fpzyYk7VIRlj2gBTgN7AGhH5RkSO+cpKHVlWfgmPf7+ebi3C+O6WgZqg1CH27cvhootmMHr0x/j5+TB//pU8/fQwd4ellMs5faPDGPOHMWYy0AvIBj52WVR1TNKBfK55fynZhSU8Oa4LjbVRWFXJzJmb+fbbjTzxxFBWrZrEaafFuTskpWqEsy/zBmG9hHsx0BH4FhjgwrjqjK0puVz05p8Ul5bz4kU96NRc295TluXL95KYmM0553Tg2mt7MWJEG2Jj9R6lqlucvSe1FvgeeM4Y86sL46lz3vltB3nFpcy89VTaRga5OxzlAbKzi/jXv+bzyitLad++MWef3R4fH9EEpeokZ5NUa2NMuUsjqWP+2JbG07M2sHZPNuf2jNYEpTDGMGPGem677Uf278/lxhsTeOqpYfrUnqrTjpqkROS/xpi7gC9F5LAXaPXLvMcuv7iUe2esZubqfbRoFMDjZ3dmfG9t9FPB8uX7uPDCGfToEcU331xMnz6VWxBTqu6p7krqM/t//SLvSZCVX8Itn67g961p3D48nklD2uCvn+uu04qLy/jjj0SGDo0lIaE5M2dewqhRbbWtPaVs1X2Zd4n9Z0djzCGJyn5JV7/c66SU7EIufmsxuzPyeea8blx4SsvqR1JebdGiXUyaNJMtWzLYtm0yMTGhnHlmO3eHpZRHcfZ07Zoqul17MgPxZttSczn/jT/Yn13IJ9f30wRVx6Wl5XPNNd8yZMh7FBSU8s03FxETE+rusJTySNXdk7oI67HzOBH5yqFXMJBZ9VjK0ZIdGVz/wTL8fIVPru9Hj5b6hFZdlpdXTNeur5OWls999w3kkUeGEBjo5+6wlPJY1d2TWoL1DakWWF/YrZADrHRVUN7AGMNXK/bwwFdraBEewPtX99Ev6NZhSUnZtGgRQsOG9Zky5TT69m1Bly76EWqlqlPdPakdwA6sVs+Vk4pKy7jni9V8t2ovfePCefOK3oQF6qcT6qL8/BKmTFnE1Kl/MHPmpYwc2YZrr+3l7rCUqjWqq+5baIwZIiIHOPRjhYLVNmy4S6OrhfKKSpn00XJ+3ZLGXSPacePQNvqZjTpq1qwt3HzzLHbuzGTChB707Bnl7pCUqnWqq+6r+ER8hKsD8QYpOYVc895S1u/N5rnz9Qm+umzixO95++0VdOwYwYIFVzFkSKy7Q1KqVqquuq+ilYmWwF5jTLGIDAK6AR9hNTSrbA98uYZtKXlMuyqB0zvox+fqmrKyckQEHx+hf/8WxMaGcffdA/RTGkqdAGfrob7B+nR8G+ADrEZmP3FZVLXQ34mZ/LwxhVtOb6sJqg5aunQPffpMY9q0FQBcfXVPHnzwVE1QSp0gZ5NUuTGmBDgPeNEYcyugbbbYvlu1l8mfrqRRoB9XDYh1dziqBmVlFXLLLbPo23ca+/blEBnZ0N0hKeVVnP58vIhcAFwBnGN305c7gEWbU5n86UriI4OYcnFPgho4u0pVbffDD5u57rrvSUnJ45Zb+jBlyumEhOi3wJQ6mZw9ol4D3IT1qY7tIhIHfOq6sGoHYwzPzN5ITHgg3986SNvhq2N8fX2Ijg7m++8vISGhubvDUcorOZWkjDFrRWQy0FZEOgBbjTFPuTY0z7dwcyrr92Xz9LldNUHVAUVFpfznP39QVlbOo48OZfTotowc2UY/paGUCzn7Zd5TgQ+BPVjvSEWJyBXGmN9dGZwny8wv5tZPVtK6SUPO6aln0d5uwYKdTJo0k02b0rn00q4YYw4+yaeUch1nq/teAMYYY9YDiEhHrKSV4KrAPFlRaRlPztxATlEp0y/pR2B9vQ/lrVJT87j77rl88MEq4uLCmDXrUs44I97dYSlVZzh7dK1fkaAAjDEbRKROtvOzN7OAGz5czpo9WUwYEEvn5tp6tTdLTs5jxoz1PPTQqTz00KkEBOjzQkrVJGeT1AoReRPr6gngMupgA7Nr92Qx8YNl5BSW8tYVvRnZWZu58UZr1iTz3XebeOihwXTpEkli4h2Ehwe4Oyyl6iRn35OaBGwD7gXuA7YDN7gqKE+UU1jCVe8uoaTcMP2GfpqgvFBeXjH33TeXXr3e4oUXFpOSkgegCUopN6r2SkpEugJtgK+NMc+5PiTPU1ZuuPmTlWQWlPD5Df21is8Lff/9Jm65ZTa7d2dxzTU9eO65ETRurJ9WUcrdqmsF/UGsL/CuAE4RkSeMMe/WSGQeZObqvSzanMqUc7rQu1Ujd4ejTrLMzEKuvPIbmjcPZtGiCZx6ait3h6SUslV3JXUZ0M0YkyciTYBZQJ1KUsYYXvtlG/GRQVzaJ8bd4aiTpLS0nOnT13LppV0JC/Nn/vwr6dw5UtvaU8rDVHdPqsgYkwdgjEl1Yniv88e2dDYl5zBxcGt9J8ZL/PVXEgkJb3HFFV8ze/YWAHr2bKYJSikPVN2VVGsR+cr+W4A2Dr8xxpznssg8xJuLttO4YX3O6q4v7NZ2mZmFPPjgz7zxxjKaNQtmxowLGDNG33lSypNVl6TOr/T7FVcF4ok+WryLRZtTeWhMR232yAuMHfsJf/6ZxOTJfXnyydMIDtbGYJXydNV99PDnmgrE02QVlPDcjxsZ1DaCawbFuTscdZy2bEknOjqEwEA/nn12OAEBfvTq1czdYSmlnFTn7jE564M/dpJdWMoDYzrgq/eiap2iolIef3wBXbu+zrPP/gbAwIExmqCUqmVcnqREZLSIbBKRrSJy/1GGGy8iRkQ8oj3AhZtT6dEyTN+JqoXmz99Bt25v8NhjCzn33I5MmuQRRUopdRyOKUmJyDFV4ouIL/AqcAbQCbhERDpVMVwwMBn461im7yofLt7Fsl0H6BMX7u5Q1DF65pnfGDbsA8rKypkz53I+/fR8mjULdndYSqnj5OynOvoA7wChQIyIdAeusz8jfzR9sL49td2eznRgHLC+0nBPAs8Bdx9D7C5hjOGpH9bTJy6cW09v6+5wlBPKyw35+SUEBdVn7Nh25OeX8MADg7QxWKW8gLNXUi8DY4F0AGPMKuA0J8aLBhIdfifZ3Q4SkZ5AS2PMzKNNSEQmisgyEVmWmprqZNjHbvrSRApLyrmsbwzB/nqQ83SrVu1n4MB3ueEGq/h06RLJE0+cpglKKS/hbJLyMcbsqtStzInxqnriwBzsKeKD9a2qu6qbkDHmLWNMgjEmoUmTJk7M+tgZY/j4r110ahbC2fpelEfLzS3m7rt/onfvt9i2LYMzztCrXqW8kbOf6ki0q/yMfZ/pVmCzE+MlAS0dfrcA9jr8Dga6AAtEBCAK+E5EzjbGLHMytpPCGMMj365j7Z5sHj6zI3Y8ygMtWbKH8eM/JzExm+uv78UzzwzXlsqV8lLOJqkbsar8YoBkYJ7drTpLgXgRicP69PzFwKUVPY0xWUBExW8RWQDcXdMJCuD3rel8uHgXV/ZvxTUD9b0oT1TxyfaYmFBiY8OYPn08Awa0rH5EpVSt5VSSMsakYCWYY2KMKRWRW4A5gC/wrjFmnYg8ASwzxnx3rNN0lY8W7yIs0I8Hx3TUNvo8TElJGS+99Bc//bSNH3+8nKioIBYtutrdYSmlaoCzT/e9jcO9pArGmInVjWuMmYXVerpjt0eOMOxQZ+I52X7ZlMKP6/YzeVi8Nn/kYf74I5FJk2ayZk0KY8e2IyeniNBQf3eHpZSqIc5W981z+NsfOJdDn9qr1V6cu5nYxoHcfFobd4eibNnZRdxzz0+89dYKWrQI4euvL2LcuPZ6r1CpOsbZ6r7PHH+LyIfAXJdEVMN2peexKimLh8Z0pEE9vYryFPXq+bBgwS7uuqs/jz02lKCg+u4OSSnlBs5eSVUWB3jF50sf/349fr7C2O7appu7bdqUxtNP/8brr59JYKAfq1ZNwt//eIuoUsobOPWelIgcEJEM+18m1lXUg64NzfXScotYuDmVK/rF0ixUH2F2l8LCUh599Be6dXuDb7/dyJo1yQCaoJRS1V9JiXUToDvWI+QA5caYwx6iqI2+XrGHsnLDJX30MWZ3+emnbdx00w9s23aAyy7rytSpI4mKCnJ3WEopD1FtkjLGGBH52hjTuyYCqil/bkvn37M30L1lGPFNtQFSdzDG8MQTC/HxEebNu4Jhw1q7OySllIdxtj5liYj0MsascGk0NeiTJbvx9/Pltct6uTuUOqWsrJxp01YwblwHoqKC+Oyz8TRuHKhVe0qpKh31yCAi9YwxpcAg4HoR2QbkYbXJZ4wxtfIIX1pWzsJNKZzZtRnRYXovqqasXLmPSZN+YMmSPWRkFPDAA6cSHR3i7rCUUh6sutPXJUAv4JwaiKXGrN2bTXZhKUPau6ahWnWonJwiHnnkF15+eQkREYF8/PF5XHJJF3eHpZSqBapLUgJgjNlWA7HUmK9WJAHoRw1ryAMP/Mxrry3lhht68/TTw2jUSK9elVLOqS5JNRGRO4/U0xjz/EmOx+W2pebywZ+7OK9nNJHB2ryOq+zcmUlJSRnx8Y15+OHBXH55N/r1a+HusJRStUx170n5AkFYn9So6l+tUlZueHLmeurX8+GBMR3dHY5XKikp49lnf6NTp1e55ZbZAERFBWmCUkodl+qupPYZY56okUhqwId/7mTBplSeHNeZJsEN3B2O1/n9993ccMNM1q1L5ZxzOvDyy6PdHZJSqpZz6p6UN8gpLOH5uZsZ1DaCy/t5RYtOHuXrrzdw3nmfExMTyrffXszZZ7d3d0hKKS9QXZIaViNR1IB19hN91w6K05a0TxJjDHv35hAdHcKoUW2ZMuU0br+9Hw0bamOwSqmT46j3pIwxGTUViKv9uiUVH4FuLULdHYpX2LAhldNOe5/Bg9+joKCEwEA/HnposCYopdRJ5VQDs7WdMYZZa/YzoE0EjYP0XtSJKCgo4eGH59O9+xusXp3MAw8MokEDbS1CKeUadeLosn5fNjvS8pg4WNuGOxGJiVkMHfo+27cf4IorujF16kgiIxu6OyyllBerE0nqh9X78PURRnWOcncotVJJSRl+fr5ER4dw6qkxTJt2FqedFufusJRSdUCdqO6buz6Z/q0bE673S45JWVk5//vfX7Rt+z+Sk3Px8RHee+8cTVBKqRrj9Ukqq6CELSm59GutTSAdi+XL99K37zQmT/6R9u0bU1xc5u6QlFJ1kNdX963dkwVA95Zhbo6kdigvN9xxx4+88spSIiMbMn36+Vx4YWd9bF8p5RZen6RWJ1lJqlu0Jiln+PgI6ekF3HhjAlOmnE5YmLZvqJRyH6+v7lu8PZ2mIQ0IDfRzdygea/v2A5x99qesW5cCwAcfnMsrr4zRBKWUcjuvTlIp2YUs3JzKJX1i3B2KRyouLuPpp3+lc+fX+OWXnWzcmAZYV1NKKeUJvLq67+/ETAAGtY1wcySeZ9GiXUyaNJMNG9I4//yOvPjiaFq00K/kKqU8i1cnqV82peLv50OXaG0KqbIff9xKQUEpM2dewplntnN3OEopVSWvre7LKyrl0yW7GdOlGf5+vu4Ox+2MMfzf/61k3rztAPzrX4NZt+4mTVBKKY/mtUlqxe4DAAxp38TNkbjfunUpDBnyHtdc8x3vv78KgIAAPwL1YRKllIfz2uq+ZTsP4CNweodId4fiNvn5JTz55EKmTv2TkJAGvPPO2UyY0MPdYSmllNO8N0ntyqBDVAjB/nX3auGLL9bxzDO/M2FCD/7znxFERAS6OySllDomXpmkSsvKWbk7kwt6t3B3KDVuz55s1q9PZcSINlxxRXc6dmxCnz7R7g5LKaWOi8vvSYnIaBHZJCJbReT+KvrfKSLrRWS1iPwsIif8bfdft6aRX1zGKXF1p72+0tJyXnppMR06vMpVV31DcXEZPj6iCUopVau5NEmJiC/wKnAG0Am4REQ6VRpsJZBgjOkGzACeO9H5vvvbDqLDAhjRqemJTqpWWLp0D337TuP22+cwaFAMv/12DfXr6xONSqnaz9VXUn2ArcaY7caYYmA6MM5xAGPML8aYfPvnYuCE6uiy8ktYsiODEZ2a0qCe9x+oN21Ko1+/d9i3L4fPPx/PrFmX0rp1I3eHpZRSJ4Wr70lFA4kOv5OAvkcZ/lpgdlU9RGQiMBEgJubIzRx9tTKJotJyxnvx/ShjDGvXptC1a1Pat4/g3XfP5txzOxIS0sDdoSml1Enl6iupqhqBM1UOKHI5kAD8p6r+xpi3jDEJxpiEJk2qfvfJGMMnf+2me4tQr21lYuvWDEaP/phevd462NbeVVf10ASllPJKrk5SSUBLh98tgL2VBxKR4cBDwNnGmKLjndnc9clsScn1ygZli4pKmTJlEV26vMaffybywgujiI+vOw+GKKXqJldX9y0F4kUkDtgDXAxc6jiAiPQE3gRGG2NSTmRm8zYkA3Bmt2YnMhmPU1JSximnvM2aNSlccEEnXnxxNM2bB7s7LKWUcjmXJiljTKmI3ALMAXyBd40x60TkCWCZMeY7rOq9IOAL++uvu40xZx/P/P5OzGR4x6Ze8wJvdnYRISEN8PPz5dpre9KuXWPOOCPe3WEppVSNcfnLvMaYWcCsSt0ecfh7+MmYT3FpOfsyCzkltvZXgZWXG955ZwX33TeP6dPHM3JkG267rZ+7w1JKqRrnNS1O/LhuPzlFpbX+3ag1a5KZNOkH/vgjkcGDW9GypX7jSSlVd3lNkvp7dyb+fj4Mjq+9rZ4/9dQiHntsIWFh/rz33jiuvLI7dhWoUkrVSV6RpHIKS/hqZRKdm4fWyk+fG2MQESIjG3LVVd159tnhNG6sjcEqpZRXfE9qw74cMvNLmDAg1t2hHJPExCzOO+8z3n57BQDXX9+badPO1gSllFI2r0hSFR847Ne6sZsjcU5paTnPP/8nHTu+yo8/bqW4uMzdISmllEfyiuq+zck5RIX40yTY81tdWLZsL9dd9x2rViVz5pnxvPLKGGJjw9wdllJKeSSvSFI70vKIi2jo7jCckpFRQFpaPl9+eSHnnttBH4xQSqmj8IoktTMtjzO6emYrE8YYPv10LXv2ZHPPPQMZObINW7dOxt/fK1a9Ukq5VK2/J7UrPY8D+SV0iPK8ZoK2bEln5MiPuOyyr/j2202UlZUDaIJSSikn1fok9dtWqyXwgW0j3BzJP4qKSnn88QV07fo6S5bs4dVXx7Bw4QR8fWv96lZKqRpV60/ptyTnUt/Xh9YedE9q27YDTJnyK+PHd+L550fSrJnnXeUppVRtUKuTlDGGBZtS6NemsdsfQEhOzuXLLzdw002n0KlTEzZuvJk2bWp/O4JKKeVOtbr+aW9WITvT8zm9vfuaQiovN7zxxjLat3+FO+6Yw44d1jtbmqCUUurE1eorqZ1peQC0j3JPI6yrVu1n0qQfWLw4idNOi+W1184kLq6RW2JRnq2kpISkpCQKCwvdHYqqA/z9/WnRogV+frX/s0W1Okll5BUDEBFUv8bnXVBQwvDhHyIC4MnDzwAAEzVJREFUH354Lpdd1tXtVY7KcyUlJREcHExsbKyWE+VSxhjS09NJSkoiLi7O3eGcsFqdpHZn5AMQFlhzSWr+/B0MHRpLQIAfM2ZcQNeuTQkPD6ix+avaqbCwUBOUqhEiQuPGjUlNTXV3KCdFrb4n9f2qvbRp0rBGrqR27cpk3LjpDBv2AdOnrwVgyJBYTVDKaZqgVE3xprJWa5NUUWkZG/fnMLxTU5dukJKSMv7zn9/p1Ok15s3bzn/+M4ILLujksvkppZT6R61NUjmFpQBEh7n2SubCC2dw773zGD68NRs23Mzddw/Az8/XpfNUyhV8fX3p0aMHXbp04ayzziIzM/Ngv3Xr1nH66afTrl074uPjefLJJzHGHOw/e/ZsEhIS6NixIx06dODuu+92xyIc1cqVK7nuuusO6TZu3Dj69+9/SLcJEyYwY8aMQ7oFBQUd/Hvz5s2MGTOGtm3b0rFjRy688EKSk5NPKLaMjAxGjBhBfHw8I0aM4MCBA4cN88svv9CjR4+D//z9/fnmm28AOPXUUw92b968Oeeccw4AM2fO5NFHHz2h2DyeMabW/evdu7fZvD/btLpvpvlmZZI52dLT801eXrExxpj587ebr7/ecNLnoeqW9evXuzsE07Bhw4N/X3nllWbKlCnGGGPy8/NN69atzZw5c4wxxuTl5ZnRo0ebV155xRhjzJo1a8z/t3fvwVFVeQLHvz8eQiIBESSFg2NAkCQEiAIuLLAGGRHl4YBIeOgAi6j4qhGEwsIqZtQVZcQHgjI4q8mskiBohMUZHB5hROQVhggRFLPYRjSFGDMQEIaY/PaPe0k6L9JR+snvU9VVfc+9fe8vp7r7l3Pu6XM6deqkBw44n4PS0lJdsmTJeY2ttLT0Z59jzJgxmpubW7FdXFysHTp00Pj4eD106FBF+aRJk3TlypVVXnu2bk6dOqWdO3fWNWvWVOzbtGmT7tu372fFNmvWLJ0/f76qqs6fP19nz559zuOLioq0devWevLkyRr7Ro8erenp6aqqWl5ersnJybUeV9t7DsjREPgOb8gjbAdOfOEOP7+yzfmbaUJVeeONvcyc+TfuuutannpqMIMGhf/oGBNafv+/n7D/m+Pn9ZyJl7dk3ohuPh/fr18/9u7dC8Dy5cvp378/Q4YMASA6OprFixeTkpLC/fffz4IFC5g7dy7x8fEANGnShPvuu6/GOU+cOMGDDz5ITk4OIsK8efO47bbbaNGiBSdOnABg1apVrF27lrS0NCZPnsyll17Knj17SE5OJisri9zcXC65xFm6pnPnzmzdupVGjRpx7733UlBQAMALL7xA//79q1y7pKSEvXv30rNnz4qyt99+mxEjRhAbG0tmZiaPPvpovfWyfPly+vXrx4gRIyrKBg0a5HO91mX16tVs3rwZgEmTJpGSksIzzzxT5/GrVq3i5ptvJjq66gKoJSUlbNq0iddffx1w7j2lpKSwdu1axo4d+7PjDEVhm6QOuUnqfC3R8dln3zF9+ntkZ3vo27cDqam+f+CNCSdlZWVs3LiRqVOnAk5XX69evaocc9VVV3HixAmOHz9OXl4eM2fOrPe8TzzxBK1atWLfvn0AtXZpVXfw4EE2bNhA48aNKS8vJysriylTprBjxw7i4uKIjY1lwoQJPPzwwwwYMICCggJuuukmDhw4UOU8OTk5JCUlVSnLyMhg3rx5xMbGMmbMGJ+SVF5eXo26qE1JSQkDBw6sdd/y5ctJTKx63/rIkSO0b++s1NC+fXu+/fbbc54/MzOTGTNm1CjPyspi8ODBtGxZ+dvQ3r17s2XLFktSoeaLoydp2+IiWkX9/B+rpaXlcs89a4mObsrSpcOYNq0XjRpFzugYE1oa0uI5n06dOkVycjIej4devXpx4403Ak4PQl2DjxoyKGnDhg1kZmZWbLduXf8P22+//XYaN3bu8aampvL4448zZcoUMjMzSU1NrTjv/v37K15z/PhxSkpKiImpnBOzsLCQyy6rnHnmyJEj5OfnM2DAAESEJk2akJeXR1JSUq1/U0MHX8XExJCbm9ug1/iqsLCQffv2cdNNN9XYl5GRUeO+W7t27fjmm2/8EksoCNuBE4e+O0Gnti3qP/AcSkudZdt7976cceOS+PTT+7nnnt6WoExEioqKIjc3ly+//JIzZ86wZMkSALp160ZOTk6VYw8dOkSLFi2IiYmhW7du7N69u97z15XsvMuqz7hx8cWVPSH9+vUjPz+fo0eP8u677zJ69GgAysvL2bZtG7m5ueTm5vL1119XSVBn/zbvc69YsYLi4mI6duxIXFwcHo+nIoG2adOmSivv+++/p23bthV14cvfWlJSUmWQg/fDO6GeFRsbS2FhIeAkoXbt2tV57rfeeotRo0bVmC2iqKiInTt3MmzYsCrlp0+fJioqcn8KE75J6uhJOl3207r6CgtLGD/+bSZNckbOJCW1Iz3918TG/rykZ0w4aNWqFYsWLeLZZ5+ltLSUiRMn8uGHH7JhwwbAaXE99NBDzJ49G4BZs2bx1FNPcfDgQcBJGs8991yN8w4ZMoTFixdXbJ9NBLGxsRw4cKCiO68uIsKoUaOYMWMGCQkJtGnTptbz1taCSUhIID8/v2I7IyODdevW4fF48Hg87N69uyJJpaSksGLFCs6ccWasSUtLq7jvNGHCBD766CPee++9inOtW7euogvzrLMtqdoe1bv6AEaOHEl6ejoA6enp3HrrrXXWQ0ZGBuPHj69RvnLlSoYPH07z5s2rlB88eLBGV2ckCcskVVauFJ080+AkVVZWzssv7yI+fglZWQeIj29bZZitMReKa665hp49e5KZmUlUVBSrV6/mySefpGvXrnTv3p0+ffrwwAMPANCjRw9eeOEFxo8fT0JCAklJSRWtAm+PPfYYxcXFJCUl0bNnT7KzswF4+umnGT58ODfccEPFfZm6pKam8sYbb1R09QEsWrSInJwcevToQWJiIkuXLq3xuvj4eI4dO0ZJSQkej4eCggL69u1bsb9jx460bNmSHTt2MHz4cAYOHEivXr1ITk5m69atFYMYoqKiWLt2LS+99BJdunQhMTGRtLS0c7Z8fDFnzhzWr19Ply5dWL9+PXPmzAGce2ne3Xcej4evvvqK66+/vsY5MjMza01e2dnZNVpXkUTC8Us6vnuynh72X6T/53Vcf7VvM6B//nkREye+w65d3zB4cEdeeWUYXbq08XOkxjgOHDhAQkJCsMOIaM8//zwxMTE17tlEsiNHjjBhwgQ2btxYY19t7zkR2a2qvQMV3/kQli2p0+69pIYsGd+yZTNKSs7w5pujWb/+TktQxkSY6dOn06xZs2CHEVAFBQUsXLgw2GH4VViO7jtdWs4VMc2Ibdm8zmNUlaysT8nMzCMzcwyxsS345JP7bFCEMRGqefPm3HnnncEOI6D69OkT7BD8LmxbUl1j625FeTz/ZMSIDG677S0OHizi6FHnN1WWoEwwhWPXuglPkfReC8skdaasnF+2ia5RXlpaxjPPfEhi4hI2b/bw3HNDyMm520btmaBr3rw5RUVFEfXlYUKTuutJVR8FGK7CsruvrFxrnVj2xx/LWbbsHwwd2pkXXxzKFVe0CkJ0xtTUoUMHDh8+HDFr/JjQdnZl3kgQlkkK4Eq3JVVU9AMLFmxl3rwUoqObsnPnXbSppZVlTDA1bdo0IlZJNSbQ/N7dJyJDReQzEckXkTm17G8mIivc/TtEJM6X87a5+CLS0nLp2nUxCxdu44MPvnTKLUEZY0zE8GuSEpHGwBLgZiARGC8i1X+OPRUoVtXOwPNA3VMDe5nx0DqmTFlN165t2bPnHoYO7Xw+QzfGGBMC/N2Sug7IV9VDqnoGyASqzwdyK5DuPl8FDBYfZnv8/NPvePXVEWzZMoXu3WPPa9DGGGNCg7/vSf0C+Mpr+zDwb3Udo6o/isgxoA3wnfdBInI3cLe7+a8iHs6bNg2mTfNL3OGkLdXq6gJmdVHJ6qKS1UWlrsEOoKH8naRqaxFVH4PryzGo6jJgGYCI5ITb1B7+YnVRyeqiktVFJauLSiKSU/9RocXf3X2HgSu8tjsA1Rc+qThGRJoArYDv/RyXMcaYMODvJLUL6CIiHUXkImAcsKbaMWuASe7zMcAmtV88GmOMwc/dfe49pgeA94HGwGuq+omIPA7kqOoa4L+B/xGRfJwW1DgfTr3Mb0GHH6uLSlYXlawuKlldVAq7ugjLpTqMMcZcGMJy7j5jjDEXBktSxhhjQlZIJyl/TakUjnyoixkisl9E9orIRhG5MhhxBkJ9deF13BgRURGJ2OHHvtSFiIx13xufiMjyQMcYKD58Rn4pItkissf9nNwSjDj9TUReE5FvRSSvjv0iIovcetorItcGOsYGUdWQfOAMtPg/oBNwEfAxkFjtmPuApe7zccCKYMcdxLoYBES7z6dfyHXhHhcDfABsB3oHO+4gvi+6AHuA1u52u2DHHcS6WAZMd58nAp5gx+2nuvgP4Fogr479twB/xfmNal9gR7BjPtcjlFtSfptSKQzVWxeqmq2qP7ib23F+kxaJfHlfADwBLABOBzK4APOlLqYBS1S1GEBVvw1wjIHiS10o0NJ93oqav9mMCKr6Aef+remtwJ/VsR24RETaBya6hgvlJFXblEq/qOsYVf0RODulUqTxpS68TcX5TykS1VsXInINcIWqrg1kYEHgy/viauBqEdkqIttFZGjAogssX+rid8AdInIY+AvwYGBCCzkN/T4JqlBeT+q8TakUAXz+O0XkDqA3cL1fIwqec9aFiDTCmU1/cqACCiJf3hdNcLr8UnBa11tEJElV/+nn2ALNl7oYD6Sp6kIR6Yfz+8wkVS33f3ghJay+N0O5JWVTKlXypS4QkV8Bc4GRqvqvAMUWaPXVRQyQBGwWEQ9On/uaCB084etnZLWqlqrqF8BnOEkr0vhSF1OBtwBUdRvQHGfy2QuNT98noSKUk5RNqVSp3rpwu7j+iJOgIvW+A9RTF6p6TFXbqmqcqsbh3J8bqaphN7GmD3z5jLyLM6gGEWmL0/13KKBRBoYvdVEADAYQkQScJHU0oFGGhjXAb9xRfn2BY6paGOyg6hKy3X3qvymVwo6PdfEHoAWw0h07UqCqI4MWtJ/4WBcXBB/r4n1giIjsB8qAWapaFLyo/cPHupgJvCoiD+N0b02OxH9qRSQDp3u3rXv/bR7QFEBVl+Lcj7sFyAd+AKYEJ1Lf2LRIxhhjQlYod/cZY4y5wFmSMsYYE7IsSRljjAlZlqSMMcaELEtSxhhjQpYlKROyRKRMRHK9HnHnODaurlmfG3jNze5M2h+7Uwl1/QnnuFdEfuM+nywil3vt+5OIJJ7nOHeJSLIPr/mtiET/3GsbE0iWpEwoO6WqyV4PT4CuO1FVe+JMXvyHhr5YVZeq6p/dzcnA5V777lLV/eclyso4X8a3OH8LWJIyYcWSlAkrbotpi4j8w338ey3HdBORnW7ra6+IdHHL7/Aq/6OINK7nch8And3XDnbXIdrnrtfTzC1/WirX8XrWLfudiDwiImNw5lF8071mlNsC6i0i00VkgVfMk0XkpZ8Y5za8JggVkVdEJEec9aN+75Y9hJMss0Uk2y0bIiLb3HpcKSIt6rmOMQFnScqEsiivrr4st+xb4EZVvRZIBRbV8rp7gRdVNRknSRx2p8FJBfq75WXAxHquPwLYJyLNgTQgVVW748zUMl1ELgVGAd1UtQfwpPeLVXUVkIPT4klW1VNeu1cBo722U4EVPzHOoTjTH501V1V7Az2A60Wkh6ouwpmfbZCqDnKnSHoM+JVblznAjHquY0zAhey0SMbgdvdVK2sKLHbvwZThzEVX3TZgroh0AN5R1c9FZDDQC9jlThsVhZPwavOmiJwCPDjLOXQFvlDVg+7+dOB+YDHOelV/EpH3AJ+XBlHVoyJyyJ077XP3Glvd8zYkzotxpgHyXl11rIjcjfP5bo+zwN/eaq/t65Zvda9zEU69GRNSLEmZcPMwcAToidMTUGNRQ1VdLiI7gGHA+yJyF87yBOmq+qgP15joPSGtiNS6Rpk7X9x1OJOWjgMeAG5owN+yAhgLfApkqaqKkzF8jhNnBdqngSXAaBHpCDwC9FHVYhFJw5lItToB1qvq+AbEa0zAWXefCTetgEJ3DaA7cVoRVYhIJ+CQ28W1BqfbayMwRkTaucdcKiJX+njNT4E4Eensbt8J/N29h9NKVf+CMyihthF2JTjLh9TmHeDXOOscrXDLGhSnqpbidNv1dbsKWwIngWMiEgvcXEcs24H+Z/8mEYkWkdpapcYElSUpE25eBiaJyHacrr6TtRyTCuSJSC4Qj7NU9n6cL/O/icheYD1OV1i9VPU0zkzRK0VkH1AOLMX5wl/rnu/vOK286tKApWcHTlQ7bzGwH7hSVXe6ZQ2O073XtRB4RFU/BvYAnwCv4XQhnrUM+KuIZKvqUZyRhxnudbbj1JUxIcVmQTfGGBOyrCVljDEmZFmSMsYYE7IsSRljjAlZlqSMMcaELEtSxhhjQpYlKWOMMSHLkpQxxpiQ9f8SWp8wfNYi6AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(randf, y_test, X_test, \"Decision Tree (Random Forest)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.94 0.88 6729\n", + " 1 0.64 0.37 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.74 0.65 0.68 8749\n", + "weighted avg 0.79 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, randf.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[1] = ([\"Random Forest\" , \n", + " classification_report(y_test, randf.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The random forest ensemble performs much better than the decision tree alone. The accuracy and AUROC are both superior to the decision tree alone." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Gradient Boosted Trees Classifier\n", + "\n", + "#### Theory\n", + "In this part we train a gradient boosted trees classifier using xgBoost. xgBoost is short for \"Extreme Gradient Boosting\". It is a boosting ensemble method for decision trees, which means that the trees are trained consecutively, where each new tree added is trained to correct the error from the previous tree.\n", + "\n", + "xgBoost uses the gradient descent algorithm that we learnt in BT2101 at each iteration to maximise the reduction in the error term. (More details? math?)\n", + " \n", + "#### Training\n", + "For consistency our xgBoost ensemble will use n_estimators = 300 as we have done for the random forest ensemble." + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n", + " learning_rate=0.1, loss='deviance', max_depth=4,\n", + " max_features=None, max_leaf_nodes=None,\n", + " min_impurity_decrease=0.0, min_impurity_split=None,\n", + " min_samples_leaf=1, min_samples_split=2,\n", + " min_weight_fraction_leaf=0.0, n_estimators=300,\n", + " n_iter_no_change=None, presort='auto',\n", + " random_state=None, subsample=1.0, tol=0.0001,\n", + " validation_fraction=0.1, verbose=0,\n", + " warm_start=False)" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn.ensemble import GradientBoostingClassifier\n", + "xgb = GradientBoostingClassifier(n_estimators=300, max_depth = 4)\n", + "xgb.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.86 0.96 0.91 13475\n", + " 1 0.79 0.46 0.58 4021\n", + "\n", + " accuracy 0.85 17496\n", + " macro avg 0.82 0.71 0.74 17496\n", + "weighted avg 0.84 0.85 0.83 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train, xgb.predict(X_train)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that the xgBoost ensemble did not fully separate the data in the training set. (The default maximum depth is 3, so that might be a factor). Evaluating on the test set," + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Decision Tree (Gradient Boosted Trees) identified 750\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06339390
11270750
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6339 390\n", + "1 1270 750" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test, xgb.predict(X_test), \"Decision Tree (Gradient Boosted Trees)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.26114231320864134\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KJySUFDohlEDoHaSISFFBEUVEUBEFP0UQO/aOiKLXdsFrFzsKWBAUBURBQWnSeye0QAjpCSn7+2Mm4RCScICcnJxkvc/DQ6avKWfWzJ49e8QYg1JKKXWhvNwdgFJKqbJBE4pSSqlioQlFKaVUsdCEopRSqlhoQlFKKVUsNKEopZQqFmU+oYjITSLyq7vjcDcRiRCRZBHxLsFlRoqIERGfklqmK4nIRhHpeR7TndcxKCIVRORHEUkQkRnnOn1Jcvb4Kqu/RxEJEJEtIlLN3bEUNxFpJyJLnBm3RBOKiOwRkTT7wDssItNEJMiVyzTGfGGMucyVyyiN7G3dJ7fbGLPPGBNkjMl2Z1zuYie2RhcyD2NMc2PM72dZzhlJ9AKOwcFAdSDUGHP9eUyfP7aeIpJj//6SRSRGRL4RkY4XOm9njy9X/B7tJJW7Tmn51jG5OJdVhLuABcaYWDumH0Tk7Xxx/igibzh01xKRD0XkoB3rThH5SESa2MMb2cdS7rocFpEprrxAExEfe5mRuf2MMauBNBHpd7bp3XGHMsAYEwS0AdoCj7khhgvmzqvusnLFfy7K6fauB2wzxmSd64RFxHzQ/v0FAxcBW4AlItL7/MN0LztJBdnr1Q97HR36ncZF+/NO4DOH7ruAoSLSw17mTUBz4Am7Oxz4G/ADumPtjw7AX0Afh/ngsB5tgB7AaBfEfzZfYK1j0YwxJfYP2AP0ceieDMx16PYHXgX2AUeAd4AKDsMHAmuARGAncIXdvzLwIXAIOAC8AHjbw24F/rT/fgd4NV9MPwAP2H/XAmYBR4HdwD0O4z0LzAQ+t5d/ewHrVxn41J5+L/Ak4OUQx1/Af4EErB9y73zTFrUOfwGvA8ftYQ2B34A44Ji9w6vY438G5ABpQDLwMBAJGMDHHud3YII93yTgVyDMIZ5b7HWIA57Kv+/yrXcF4D/2+AnAn3a/3GWOsPfpMeAJh+k6AcuAE/Z6TwH8HIYbYCywHdht93sT2G/vg1XAxQ7jewOP28dGkj28LrDYnleKvT1usMe/Cut4OgEsBVrlO1YfAdYBGYCP4zawY19px3EEeM3uv89eVrL9rwsOx6A9TnNgvr0vjwCPF7BNnwNOApn2fEZhXQA+aW/nWKxjrbI9fu62HmXHsLiAefYEYgroPwVY6dAd7RDfVmDIOezr3OPrVmCXvR92Azfl/z3a3V2BFfa8VgBdHYb9ThHHaCHHYmHrGAOMB9YDJ+1+dYDvOPV7H+swvhenjqVjwHSgaiHLbIB1bHnn6387sA3rwuAop5/7XsI6PqWIdWkEmHz9XgPezncs/YF1DK8HrnQYVgXrfHUU69h9LHd5QGOs30WCvX5f2v2Xcvpv5Tq7fz27n2+R29/ZZFAc/zj9B1nH3gBvOgx/A5gNhGBl7B+BSQ4/4ASgr72zawPR9rDvgXeBikA1YDlwZ/4DGCu773fYqFWxTrq17HmuAp7GumpogPWDuNwe91msH/c19rgVCli/T7ESVDDWD2wbMMohjizgfsAXuMFenxAn1yELGId1YqtgH2x9sZJwuH1wvFHQts53wnFMKDvtA6uC3f2SPayZfTB1t7fFq/a6F5ZQptrT18Y6qXe148pd5vv2MlpjnZyb2tO1x7pK9rHH3Qzc5zBfg3ViC8nd3sDNQKg9zYPAYSDAHpZ7wmgCiL28UId5NXKYdzusk3JnO+YR9jbzd9h+a7ASUoX82xQrEQ63/w4CLipoOxdwDAZjJc8HgQC7u3Mh2/VZ4HOH7pHADqxjMwj4Fvgs33I/xTqGCjo+e1LwybYX1gVIRfvffuA2exu3wzrhNHdyX/vY80gEmtjT1HSY3nFbhADxwHB7umF2d+4++51CjtEizjGFrWMM1u+7jj0vb3v/Po51jDey929ve/yHsBJZbXs/fZi7rQuY90BgbSHDFtjb78N8/VcCT55lXU5LKHYsG4Bb7G4/rET4MNY5pQ/W77aRPfxL+xgJto+ZHcAIe9gMrAsmL3v9utn9fez9GFlAPKlAsyJjPp/EcL7/7B2WjHW1YYCFnLqqFqwM2NBh/C6cujJ9F3i9gHlWxzpJOd7JDAMWFXAAC9bVWw+7+/+A3+y/OwP78s37MeBjhx/3GVd9DuN623E0c+h3J/C7QxwHcbgiwUoaw51ch32FLdse5xrg33zb+mwJ5UmH4WOAefbfTwNfOQwLxLpaPiOh2AdkGtC6gGG5y6yTb52HFrIO9wHfOXQboNdZ1js+d9lYV9MDCxkvf0L5HzAh3zhbgUsctt/IAo7f3ISyGOsuIizfOKdt5wKOwWGO++ks6/YspyeUhcAYh+4mWInex2G5DYqYX08KPtlG29PWxrrQWZJv+LvAM07u69yEcgK4jnyJLd+2GA4szzd8GXDr2Y7R81jHGOwTsd3dDdiVb5yngPftv7fnHgt2d12s36hXAfMegcNdVwH70ACXFnAs3e7QPcjeZknAT3a/Rva0J+x/BlgCBNnDL8UqzXA8p8zAuov1xboIbewwbCzWcx6wks3/gNr54ioqoRzB4Q6yoH/ueIZyjTEmGGvHRwNhdv9wrBPXKhE5ISIngHl2f7B26M4C5lcPa+MdcpjuXayr/NMYa6tMx/pRA9yIVVSUO59aufOw5/M41sk+1/4i1isM64phr0O/vVg/0lwH7Bgch9dych1OW7aIVBOR6SJyQEQSsW5twzg3hx3+TsW66sWOKW95xphUrKKvgoRhXeEUtG+KXI6INBaROfbDxkTgRc5ch/zr/aCIbLZrPp3AKirMnaawY6Qg9YAH8+3vuljrXuCy8xmFdeW8RURWiMhVTi73XGLMrxZnHl8+OH+MFqY2p05c9YDO+bbLTUANnNvXGGNSsBLTaKxjeq6IRDuxPnDmb6awY/R8OG6bekBEvvV8GGs9ASKAHx2GrcfaRgXV4orHugs4jb3O92GduF/L9+wmDuvODQBjzLfGmCpYd9l+jvMxxlSxh1XEKhb82R5UC+tCM/85pbYdpzeFn48exDrnrBSR9SIyooD1yi8Y6xgplNuqDRtj/gCmYRWngHVbmIZ1a1zF/lfZnHqoth/ruUF++7GuHMIcpqtkjGleyKK/AgaLSD2su5JZDvPZ7TCPKsaYYGNMf8ewi1ilY1hXi/Uc+kVgXUHkqi0ikm/4QSfXIf+yJ9n9WhljKmEVBUkR45+LQ1hFA4BVfRWrmKkgx4B0Ct43Z/M/rGdJUfY6PM7p6wAO6yEiF2Pdpg/BKs+uglVsmDtNYcdIQfYDE/Pt70BjzFcFLTs/Y8x2Y8wwrB/uy8BMEalY1DTnEWN+Bznz+MrCunLMC+085nstsNpOBPuBP/JtlyBjzF2cw742xvxijOmLddLcglXsebb1gTN/M8XJcdvsB7YX8HsfYA+PAfrmGx5gjDl85mxZBzR0rDItIl7AB1jnt3FY54aHHKZZCFyb73xQdPDWhd0nQHcRqYK1/eoWcE45gFWcm00h5yNjzCFjzO3GmJpYdy7viUh9Cjl+7PMlWHduhXL3eyhvAH1FpI0xJgfroHs9ty63iNQWkcvtcT8EbhOR3iLiZQ+LNsYcwnpY9x8RqWQPaygilxS0QGPMv1gPqT4AfjHG5Gbc5UCiiDwiVv1/bxFp4WyVSmNVl/wGmCgiwfYOeADrziFXNeAeEfEVkeuBpli3t+e0DrZgrOLDEyJSG+vKxtERrHLT8zETGCAiXUXED6top8AD395vH2FdgdWyt1sXEfF3YjnBWGXtyfbV3F1OjJ+Ftf98RORpoJLD8A+ACSISJZZWIpKbCPNvj/eB0SLS2R63oohcKSJnXGkWRERuFpFwe/1zj6FsO7YcCt/2c4AaInKfiPjbx0pnZ5aJdTF0v4jUF6u6/YvA1+b8aoGJ/Rt6Buvh8eMO8TUWkeH2ceorIh1FpKmz+1pEqovI1XaCzcA6TguqTvyTvawbxaquegPW87s557o+52EZcNK+4w2w16WliLS3h78DvCgiEZBXInB1QTMyxuzBKkpv79D7bqxj82X73DAKeExEouzhr2KdDz6196eISCWs534FsrfzcKySjtyKJFlYd9q+ItIL6A98Y4zJxPodvygiQXayuB/7fCQiQ+zzBpwqTsu2Y43jzOP3EqzisszC4gM3JxRjzFGsh4hP2b0ewXpw9LdYRSALsMqJMcYsx3pQ+DrWVekfnMq+t2DdJm7Cuv2cicPtZAG+wnqA9aVDLNnAAKyqebuxrsY+wCpScdY4rOdAu7Bqv3yJ9QPM9Q8QZc97IjDYGJNblHSu6/Ac1gPTBGAu1sM3R5OAJ+1b9ofyT1wUY8xGe12mY92tJGFd8WQUMslDWEUCK7BqBr2Mc8fWQ1jFjklYJ/ivzzL+L1i3+9uwbt/TOb0Y4zWspP4rVqL6EOsBLFhl2Z/Y22OIMWYl1jO0KVjbewdW+b6zrgA2ivWew5tYz4XS7avIicBf9rIucpzIGJOEVZliAFZxznassnBnfIRVg28x1jGajrWfzkUtO+ZkrP3VEuhpjPnVIb7LgKFYV8CHsfZnbtJwZl97YRWpHLTHuQTr+cdp7GP/KnvcOKwip6uMMcfOcZ3OmZ2E+2NV9tmD9Zt8l1MXKK9hFbkvFJEkrJN3UReX72Kd7BHrHY4JWBVyMu3lrQfeAt4XETHW+yoXYSWEZVi/gdVYRYpjHWcsp96nOYyVtAba88zAOo4G2vG/BdxojNlmTzoG69nnbqzz5SdY51uwSmdWiEgK1rljrDFmnz3sGeBL+/gdZPe7CSvJFim3tpNyMRG5FeshXHd3x3Ku7KvhE1hFU7vdHY9SpY2IBAD/Yj3Ij3V3PMVJRNoC/3Xm3OXuIi9VSonIABEJtIstXsW6Kt3j3qiUKp3su9OmZS2ZgPWYwNkLYU0oqjADsYosDmIV0w01ejurlCqCFnkppZQqFnqHopRSqlh4XCODYWFhJjIy0t1hKKWUR1m1atUxY0z42cc8fx6XUCIjI1m5cqW7w1BKKY8iIvlbJSh2WuSllFKqWGhCUUopVSw0oSillCoWmlCUUkoVC00oSimlioUmFKWUUsXCZQlFRD4SkVgR2VDIcBGRt0Rkh4isE5F2ropFKaWU67nyDmUaVhPfhemH1UZUFHAH1seWlFJKFaOcHEN6ZkGfoyl+Lnux0Riz2P4uQGEGAp/aDQ7+LSJVRKSm/bEppZRSBTDGsCM2mT+2HQUgK8eQlZ1DZrZhw4EEKvr7kJ6ZzYo9x6no70NMfFqJxebON+Vrc/rHkWLsfmckFBG5A+suhoiIiBIJTimlSlpWdg5bDieRmJbJpkOJeIlwIvUk244ks+lQIr7ews6jKWedT3SNYGoE+bN77wmStsYTElLhrNMUB3cmlII+KVtg08fGmPeA9wA6dOigzSMrpTzSmv0niE85ycnsHDKzc9gbl0pGZjabDiWxLuYEsUmFfRQVwoP98fH24fr2dfDz8aJ/y5q0qF0ZX2/Bx8sLHy/By8s6rRpj6NDhfWK2HuP55y/lnns64/vVUJevnzsTSgxQ16G7Dta3N5RSyuNkZeew73gq62ISmLfhMDnGsD8+jaNJ6YBwLLnwZBEc4EPTmpXoEFmVRuFBXNQwlIp+PtQLDSTA1xs/b6+8ZFGUpUv307JlNYKD/fnggwGEhQVSt+65fMX8wrgzocwG7haR6VjfN07Q5ydKKU9hjGHToURmrIzhWHIGc9adfvoSgdZ1qhBS0Y+mNStRNdAPEejXoiYV/a0k4evtRXiwPxX9L+xUHBeXyqOPLuCDD/7lmWcu4dlne9K2bc0Lmuf5cFlCEZGvgJ5AmIjEYH343hfAGPMO8BPQH9gBpAK3uSoWpZQ6X8YY/th2lDnrDpGdY1i59zj+Pt7siE0+bbxWdSrTuk4VujQMpWNkCOHB/iUS26efruWhh+YTH5/G+PFdGT++q8uXWxhX1vIadpbhBhjrquUrpdS5MsaQmW04kXaS71YfYM66Q2w4mIDjh22jawSTnpnNde2sZxm3X1yfhuFBbon3kUcW8MorS+natS7vvHMlLVtWd0scuTzueyhKKXW+jDHsO55K6slsMrJy2HY4iWxjmL3mIGtjTpB68sz3NRqGV6RH43BGdIkkMqyiG6I+XVpaJikpmYSFBTJqVFuiokIYNaqdU89YXE0TilKqTFq68xixiRkcTkxn1d54jqecZNXe+ELHD/D1YlyvRnkPwStX8KVfyxoEB/iWYNRFmzdvB2PH/kSbNjWYNWsITZqE0aRJmLvDyqMJRSnl8bYeTuKr5fvYcjiRhLQsNh9KPGMcEbg4KozKFXy5qlVN/H288fYSGoRXJDjAl8oVSk/iyO/gwSTuu28eM2ZsokmTUO6+u6O7QyqQJhSllMdKSM1k9toDPPXDxrx+wQE+dGsUSlS1YIZ1iqBSBR8qV/Al0M8zT3cLF+7i2mu/5uTJbCZMuJTx47vif4G1wlyldEallFIOsrJzOJ5ykq1Hkli4OZb1BxLOKL66t3cU9/WJQsT9zxKKQ2ZmNr6+3rRuXYP+/aN44YVeNGoU4u6wiqQJRSlV6mRl5/DblliW7oxjzrpDBb4U2K9FDSoF+NK6bhX6NqteItV0S0JiYgZPPfUb//xzgL/+GklYWCDTpw92d1hO0YSilHKrtJPZbDqUwOJtxziUkMbKvfHsytdeVeu6VejSIJT6YYG0rluFxtWCS0WtpuJkjGHmzE3ce+88Dh9OZsyYjmRkZBMY6DmfrdKEopQqUXHJGcTEp7F0ZxxLdx5jyfZjpw2vEuhL/5Y16BgZwmXNa1CrckCZKcYqzNGjKYwY8T0//7yDtm1r8MMPQ+nYsba7wzpnmlCUUi4TE5/Kz+sPs2pvPCv2HCcu5eQZ44QH+3NjpwgujgqjXUTVMnfn4YxKlfw5diyVN964nLFjO+Hj4zl3JY40oSilLtjOo8ks2xlHZnYOJ7NymLZ0D+mZ2cSnZuaN4+fjxcVRYUSGVqRdvSqEBfnTpUEoPt6eefK8UIsX72XixCXMmjWEoCA//v77do9PpppQlFLn7FhyBt+tPsCyXXFsPpTIoYT0M8YJ8PXi1q6RdIisyuXNa+BbThNHfseOpTJ+/HymTVtDZGQV9uw5QYsW1Tw+mYAmFKVUIRLTM9l0MJF/dh3neEoGWw4n4efjxY7Y5NMSSICvFz0ah3PLRfVoXrsSFXy98fX2uuAWdMsaYwwff7yG8ePnk5iYwWOPdefJJ3sQGFh6X6g8V7rHlVIALNsZx0/rDxETn8qWw0ln3HV4ewl1q1agflhFmtasxDVta3N58+r4+3i7KWLP8/nn62jWLJx33rmS5s2ruTucYqcJRalyIjvHkHIyi9jEDFbsOU5CWmbey4HzNx05Y/w+TavRu2l1omsE07pOlTJRJFPSUlMzefHFJYwe3YE6dSoxa9YQKlcOKLPbUhOKUmXYhgMJ/LT+EPM2HGbXscK/Rd42ogota1fmtm71qV8KWtQtC376aTtjx/7Enj0nqF07mLvu6kjVqiXzbXd30YSiVBmQnpnNvA2H2X0shX92x7H1cNJpNawAekVXIyIkkKjqQYRW9KNjZAghFf3K/DseJS0mJpH77pvHrFmbado0jD/+uJUePeq5O6wSoQlFKQ+VnpnN9/8e4LO/97Lx4KnWdQP9vPH38aJz/RBa1K7Mla1q0kaLrErMxImLmTt3Oy++2IsHH+yKn1/5ecYkxvFTZB6gQ4cOZuXKle4OQym3MMawet8J7v96DfuOp+b1jwwNZEjHugxqW4calQPcGGH5tHz5ASpU8KFly+rExaWSkJBBgwZV3R3WaURklTGmgyuXoXcoSpVyuQ/T35i/nV82HubAiTQAmteqRL8WNRjRNbJUfQSqPElISOfxxxfyv/+t5KqrGjN79jBCQwMJDQ10d2huoQlFqVLmSGI6nyzdw8LNscQmpZ/xLGTspQ1pVacKlzev4aYIlTGGr7/eyP33/0JsbArjxnViwoRe7g7L7TShKFUKGGP4ZuV+Hvt2PTl2KbQIRIQE0iEyhLYRVWgQVpE+TauX26ZKSpPPP1/HLbd8T4cOtZgzZxjt29dyd0ilgiYUpdzEGMPWI0n8sOYg0/7aQ1pmNgAd6lXltm71uax5dW2upBTJyMhi1654mjYNZ8iQ5mRl5XDLLa3x1n2URxOKUiXoUEIaz/+4iZ83HD5jWPdGYbw7vL02WVIKLVq0m7vumktqaibbt4/D39+H225r6+6wSh09cpVyoazsHFbujefDP3fz25ZYsnNO1ars2SScRuFBXNw4nB5RYfo+SCkUG5vCQw/9ymefraNBg6q8996AUvs999JAt4xSxSz3eciLP20hIe3UA3URGNSuNle1qkmv6OpujFA5Y8eO43Tq9D7JySd54omLeeKJi6lQQWvTFUUTilIXKD0zmx2xyfy54xjTl+9jT9yp90OiawRzfYe6XNI4nEbVgtwYpXJWYmIGlSr507BhVUaNasvIkW1p2jTc3WF5BE0oSp2n9TEJvDxvC0t3HsOhJIuqgb4M6VCXcb2jCNLiEY+RknKS55//g/ffX826dXdRp04lXnnlMneH5VH0aFfqHGw/ksQX/+xjXcwJVu87AUAFX28euaIJjaoF07F+VW3O3QP9+ONW7r77Z/btS2DUqLZl6hslJUkTilJnkZKRxed/72XSz1tO6z+4fR1u7RpJi9qV3RSZulBZWTkMGTKD777bQvPm4SxZchvdu0e4OyyPpQlFqUKs3hfPi3M3s9L+ZogI9G9Rk5s6R9A+Uu9EPJkxBhHBx8eLmjWDeOml3tx/f5dy1ZCjK2hCUcpBemY236zczw9rDuZ9fOriqDCubFmTq9vUItBPfzKe7u+/Yxg79ifef38A7drVZOrUK90dUpmhvw6lsBpgfG/xLl6ed6pYq07VCrw1rC3tIkpXq7Hq/MTHp/H44wt5991V1KoVTHx8mrtDKnNcmlBE5ArgTcAb+MAY81K+4RHAJ0AVe5xHjTE/uTImpfKbvfYg93z1b173fX2iGNm9PpW0Bd8y4+uvN3DPPfM4diyV++67iOee60lwsL+7wypzXJZQRMQbmAr0BWKAFSIy2xizyWG0J4FvjDH/E5FmwE9ApKtiUipXXHIGT3y3gXkbTzWBck2bWjwzoDlVK/q5MTLlClu2HCMysgrz5t1E27Y13R1OmeXKO5ROwA5jzC4AEZkODAQcE4oBKtl/VwYOujAeVc4ZY1i2K47xM9blfVME4Lp2dbimbS0ujtKX18qK9PQsXn75T9q1q8mAAU14/PGLefLJHtqQo4u5MqHUBvY7dMcAnfON8yzwq4iMAyoCfQqakYjcAdwBEBGhVfqU8w4npPPmwm1sOpjI2piE04Z9cEsHejetpm1olTELFuxizJi5bN9+nAcf7MKAAU3w9dXaWyXBlQmloF9p/u8NDwOmGWP+IyJdgM9EpIUxJue0iYx5D3gPrE8AuyRaVWYcSkjjqe83snpfPMdTTgIQ7O9DvdBA2tStwv19GhMZVtHNUariduRIMg888CtffrmeRo1C+PXXm+nbt6G7wypXXJlQYoC6Dt11OLNIaxRwBYAxZpmIBABhQKwL41JlTEpGFrNWx7B42zHWxZwgNikjb1i3RqHc2rU+fZtpY4xl3fz5u5g5cxNPP92Dxx67mIAArcRa0ly5xVcAUSJSHzgADAVuzDfOPqA3ME1EmgIBwFEXxqTKkIdmrOXn9YdIOZmd1y8iJJBOkSEMbFuLmzrXc2N0qiSsXXuY7duPM3hwM266qSXdutWlfn2t5u0uLksoxpgsEbkb+AWrSvBHxpiNIvI8sNIYMxt4EHhfRO7HKg671RijRVqqSKv2xvPYt+vYdiQZgN7R1ejdtDoD29TSj1OVE8nJJ3nmmUW8+eY/REZW4ZprovHx8dJk4mYu/fXZ75T8lK/f0w5/bwK6uTIGVXbMWhXD1EU72HUsBYAuDUL5+LaOBOgD13Ll+++3MG7cz8TEJHLHHe2YNKkPPj5ae6s00Ms55RG+XR3DgzPWAnBt29oMaldbq/mWQ+vXH+Haa7+mZctqfP31YLp2rXv2iVSJ0YSiSi1jDBPmbOajv3YDEOTvw5xx3bWGVjmTmZnNkiX76NWrPi1bVmfu3Bvp27eBVgUuhTShqFInKzuHB75Zy+y1pyoFDulQh0f7NSVE32IvV5Yu3c/o0XPYuPEoW7feTaNGIfTvH+XusFQhNKGoUuWblft5eOa6vO4nr2zKbd3q4+2lLx+WJ8ePp/Hoowt4//3V1K1biW+/HUKjRiHuDkudhSYU5XbGGL5ZuZ/X5m/jSKL1DsnANrV47urmVAnUO5LyJj09izZt3uHgwSQefLALzz7bk6AgPQ48gSYU5RZZ2TnExKex9UgSr/6yle2xVhXgiJBAXhncis4NQt0coSppMTGJ1KlTiYAAHyZMuJQ2bWrQunUNd4elzoEmFFWijDGs2BPPkHeXnda/Y2RVPry1ozYZXw6lpWUyadKfvPzyX8yceT0DBjRhxIg27g5LnQenEoqI+AERxpgdLo5HlVGpJ7OYsTKGZ2ZvzOvXum4VxvZsSOPqwVpzq5z69dedjBkzl50747n55lZ06lTb3SGpC3DWhCIiVwKvAX5AfRFpAzxjjLnW1cEpz7fzaDLP/7iJP7adalHnogYh3NmjIZdGV3NjZMrdxo37iSlTVhAVFcKCBcPp3buBu0NSF8iZO5TnsZqdXwRgjFkjIo1cGpUqEybO3cT7S3bndV/WrDovX9dKP2BVjmVnWw2Je3t7cdFFdQgLC+SRR7prQ45lhDN7MdMYcyLfNyO0vS1VqMT0TMZ+sZol248BcEePBjzev6mbo1Lutnr1IUaPnsPw4a0YN64zN93Uyt0hqWLmTELZLCJDAC+75eB7gb9dG5byRBlZ2by1cDtTF+3M6/fdmAQMw0MAACAASURBVK60jdAG+8qzpKQMnn56EW+9tZzw8EBq1gx2d0jKRZxJKHcDTwM5wLdYrQc/5sqglGfJys5h8i9beW/xrrx+t3evz0OXN9GGG8u5X3/dyciRP3DwYBKjR3fgxRd7U6VKgLvDUi7iTEK53BjzCPBIbg8RGYSVXFQ5lZNjWHcggU+W7mH22oNk5xgq+HpzV8+GDOsUQXiwv7tDVKWAn5831apVZNasIXTuXMfd4SgXk7N9fkREVhtj2uXrt8oY096lkRWiQ4cOZuXKle5YdLlmjGHN/hN8vWI/K/YcZ+fRlLxhvt7CkA51GX95E32zvZzLzMzmtdeWkZiYwcSJvQHr4sNLm85xO/u83cGVyyj0DkVELsf6PG9tEXnNYVAlrOIvVU488d16vvhnX1533ZAKNKkeTNdGoVzfvi7NalVyY3SqtPjzz315DTlef32zvESiyaT8KKrIKxbYAKQDGx36JwGPujIoVTqkZGRx7/R/WbA5FoA7ezSgT7PqdIzURvrUKXFxqTzyyAI+/PBfIiIq8+OPw7jqqsbuDku5QaEJxRjzL/CviHxhjEkvwZiUG207ksSdn60i9WRWXkONXgJ/jL+UuiGBbo5OlUZxcWlMn76Bhx/uytNPX0JFfc+o3HLmoXxtEZkINAPyqmcYY/QSpAyJS85g5CcrWbv/RF6/nk3Cubx5DYZ0qKvNx6vTbN58lG++2cgzz/SkceNQ9u27n5CQCu4OS7mZMwllGvAC8CrQD7gNfYZSJhhj+GXjYe6dvoaMLGuX+ngJM+/qSpu6VdwcnSqNUlMzmThxMa+8spSgID9GjWpHnTqVNJkowLmEEmiM+UVEXjXG7ASeFJElrg5MudbkeVv4bNlekjKyALi6dS16N63GwDbaOJ8q2Lx5OxgzZi67d59gxIjWvPJKX8LDtVFPdYozCSVDrHZXdorIaOAAoK36eaBlO+OYuSqG37fGEpdykkA/bx7rF83VbWpRs7JeYarCJSefZPjw7wgNrcCiRSPo2TPS3SGpUsiZhHI/EATcA0wEKgMjXRmUKl4nUk9yx2erWL77OAAhFf0Y2rEuzwxoTgU/fZNdFSw7O4evvtrAsGEtCAryY8GC4URHh+Hvrw05qoKd9cgwxvxj/5kEDAcQEX3l1QNkZucwfcV+Xpy7mbTMbIIDfPh+bDcahge5OzRVyq1adZA775zDqlWHqFDBh+uua6ZfT1RnVWRCEZGOQG3gT2PMMRFpjtUESy9Ak0opNnfdIcZ+uTqv++ErmnBnj4ZaW0sVKSEhnaeeWsTUqSuoVq0i06dfx6BB2lK0ck5Rb8pPAq4D1mI9iP8Oq6Xhl4HRJROeOh/rYxLyksk9vaO4s0cDKmoxhXLCddd9w2+/7Wbs2I688EIvKlfWhhyV84o6ywwEWhtj0kQkBDhod28tmdDUuTLGMH7mOmauigHg8f7R3NGjoZujUqXdrl3xhIcHEhzsz8SJvfDyEjp21Np+6twVlVDSjTFpAMaY4yKyRZNJ6WOMYd6Gw6yJOcG7f5xqPv7NoW20CrAq0smT2bz66lImTFjMPfd04uWX+2qLwOqCFJVQGohIbhP1AkQ6dGOMGeTSyFSREtIyeX3+NqYt3ZPXL6SiHx0jqzLx2paEBWnz8apwixfvZfToOWzefIzBg5txzz2d3R2SKgOKSijX5eue4spAlPP+3hXH0Pesj2YG+nkzrFME/3dxA2poebdywuuvL+OBB34lMrIKc+feSP/+Ue4OSZURRTUOubAkA1FFy84xvD5/Gws2H2HL4SQA2kVUYdZdXbHeO1WqcDk5hpSUkwQH+3PllY05ejSVJ5/sQWCgr7tDU2WIVv0p5XJyDO8s3snkeaceX7WuW4X/XN+KRtX029zq7DZujGX06Ll5X05s3DiUF1/s7e6wVBnk0oQiIlcAbwLewAfGmJcKGGcI8CxggLXGmBtdGZMn2X88lYsnL8rrvrNHA+7r01jfbldOSU3NZMKEP3j11WVUruzPyJFtMMboHa1yGacTioj4G2MyzmF8b2Aq0BeIAVaIyGxjzCaHcaKAx4Buxph4EdE2woC9cSk8+M1aVu6NB6BeaCDz7u2hiUQ57d9/DzFo0Dfs2XOC225rw+TJfQkL0+/ZKNc6a0IRkU7Ah1hteEWISGvgdmPMuLNM2gnYYYzZZc9nOta7LZscxvk/YKoxJh7AGBN77qtQtqzeF8+gt5cC0L5eVW7pUk+r/yqn5d6BRERUJiKiMp98cg09etRzd1iqnHDmDuUt4CrgewBjzFoRudSJ6WoD+x26Y4D8dRMbA4jIX1jFYs8aY+Y5Me8y6ZsV+3l41joARnWvz1NXNXNzRMpTZGXlMGXKcmbP3sr8+cMJDQ3kjz9udXdYqpzxcmYcY8zefP2ynZiuoIJak6/bB4gCegLDgA9E5IwvO4nIHSKyUkRWHj161IlFe56lO4/lJZPnrm6uyUQ5bfnyA3Tq9D733/8LAQE+JCY6XTKtVLFyJqHst4u9jIh4i8h9wDYnposB6jp018FqviX/OD8YYzKNMbuBrVgJ5jTGmPeMMR2MMR3Cw8OdWLRnmbfhEDe+bzXq/O7w9ozoGunegJRHSE4+ydixc7noog84ciSFGTOuZ+7cG6laVb9to9zDmSKvu7CKvSKAI8ACu9/ZrACiRKQ+1ke5hgL5a3B9j3VnMk1EwrCKwHZRTiSmZ3Lj+3+z4UAiAPf2juLy5tpEuHKOr68Xv/++l3HjOjFhQi8qVdLWEZR7OZNQsowxQ891xsaYLBG5G/gF6/nIR8aYjSLyPLDSGDPbHnaZiGzCKkYbb4yJO9dleaLYpHQ6TTz17ugPY7vRWr/jrs5ix47jPP/8H0yd2p/gYH9WrbqDgAB9nUyVDmJM/sca+UYQ2YlVFPU18K0xJqkkAitMhw4dzMqVK90ZwgVzrMk1tGNdJg1qqe8GqCJlZGQxefJfTJy4BD8/b+bOvZGLL9baW8p5IrLKGNPBlcs46zMUY0xD4AWgPbBeRL4XkXO+Y1GWHbFJeclkUNvavHRdK00mqkiLFu2mdet3ePrp37nmmmi2bLlbk4kqlZy6VzbGLAWWisizwBvAF8B0F8ZVJh1JTKfPa4sBmH9/D6Kqa9MpqmjGGCZOXEJmZg7z5t3E5Zc3cndIShXKmRcbg7BeSBwKNAV+ALq6OK4yJSfHMHN1DA/PtKoFV/Tz1mSiCpWTY/jww9VccUUj6tatzGefXUuVKgFUqKANOarSzZk7lA3Aj8BkY8wSF8dT5uyNS2HwO8s4mpRBtWB/xl/ehMHt9SNGqmDr1h1h9Og5LFsWw9NP9+C55y6lZk29+FCewZmE0sAYk+PySMqgJ79fz+d/7wMgukYw34/tRoCvtselzpScfJLnnvud11//m6pVKzBt2kBuuaW1u8NS6pwUmlBE5D/GmAeBWSJyRlUw/WJj4dJOZnPv9H/5ddMRACYPbsWQDnXPMpUqz5599nf+859l3H57W156qQ+hodqQo/I8Rd2hfG3/r19qPAcvzNnEB3/uzuv+9f4eNNbnJaoA+/cnkJKSSXR0GI8+2p1rromme/cId4el1HkrtNqwMWa5/WdTY8xCx39YD+dVPv9duD0vmUwe3IrtE/tpMlFnyMrK4bXXltG06VTuvHMOAGFhgZpMlMdz5hnKSM68SxlVQL9yKzE9kyteX8zBhHQAFjzQQ7+mqAr0998xjB49h7Vrj3DllVFMmdLf3SEpVWyKeoZyA1ZV4foi8q3DoGDghKsD8ySXv76YQwnphAX58cXtF2kyUQWaO3cbAwZ8Ra1awXz77RCuuSZaX2pVZUpRdyjLgTisVoKnOvRPAv51ZVCeZNLPmzmUkM5lzarz7vD2eoJQpzHGcPBgErVrV6JPnwY8//yl3HtvZ4KDtSFHVfYUmlDs5uR3Y7UurAoQm5TOu39YjSO/MbSNJhN1mm3b4hgzZi7btsWxadNYgoL8ePLJHu4OSymXKarI6w9jzCUiEs/pH8YSwBhjQlweXSn25/Zj3PqxVW/htSGtCfTTFl+VJT09i5de+pNJk/6kQgUfJk3qTYUKenyosq+oozz3M79hJRGIJ8nJMdz8ofVBrJHd6jOonb75riyHDyfTo8fHbN9+nGHDWvDaa5dTo0aQu8NSqkQUVeSV+3Z8XeCgMeakiHQHWgGfA4klEF+pk5GVTbeXFgHwaL9oRl/S0M0RqdIgMzMbX19vqlevSI8e9Zg6tT99++qxocoXZz4B/D3W538bAp9ivYPypUujKsWGf7CcY8kZdGsUyh0XN3B3OMrNcnIM77yzkoYN3yImJhER4YMPrtZkosolZxJKjjEmExgEvGGMGQfUdm1YpdOO2CSW7zlOzybhfHH7RXh56UP48mzt2sN07fohd901l6ioUDIzs90dklJu5dQngEXkemA4cI3dr1y2o/34txsAeGZAczdHotzJGMP48fN5442/CQmpwGefXctNN+lXN5Vy9k35MVjN1+8SkfrAV64Nq/T5YMkulu85zjVtalE/rKK7w1FuJCLEx6cxapTVkGPVqhXcHZJSpcJZvykPICI+QO6n4nYYY7JcGlUR3PFN+Y/+3M3zczZRvZI/P47rTrXggBJdvnK/vXtPcO+983j66Uto164mOTlGizyVRykV35QXkYuBHcCHwEfANhHp5sqgSpPsHMPzczYBMP+BSzSZlDOZmdlMnvwXzZq9zfz5u9i69RiAJhOlCuBMkdfrQH9jzCYAEWkKfAa4NNOVBiezcuj8otVQwO3d61MpoFw+Oiq3li7dz513zmHDhlgGDmzCW2/1IyKisrvDUqrUciah+OUmEwBjzGYR8XNhTKXG83M2Ep+aSVS1IB7vry32lzcLFuwiISGd77+/gYEDo90djlKl3lmfoYjINCAD664E4CYg0BgzwrWhFayknqGkZGTR/JlfANj5Yn+8tYijzDPG8Nln6wgPD6RfvygyMrLIzMwhKKhcXD+pMq5UPEMBRgM7gYeBR4BdwJ2uDKo0eH+J1ejjfX2iNJmUA1u2HKNXr08ZMeJ7Pv54DQD+/j6aTJQ6B0UWeYlIS6Ah8J0xZnLJhOR+xhjeWLCd2lUqcG/vKHeHo1woLS2TF19cwssv/0XFin68++5V3H57O3eHpZRHKvQORUQex2p25SZgvoiMLLGo3Oz7NQcAGNqxrr6sVsb9+OM2XnhhCTfc0IItW8Zyxx3ttQaXUuepqDuUm4BWxpgUEQkHfsKqNlymHU3K4P6v1xJS0Y8xlzY6+wTK4xw+nMyaNYe54opGXH99MyIjb6dTp3LZmpBSxaqoZygZxpgUAGPM0bOMW2Z0nGhVEx7ZLVKfnZQx2dk5vP32Cpo0mcLw4d+RlpaJiGgyUaqYFHWH0sDhW/ICNHT8trwxZpBLI3ODtJPZiEDlCr7c3UufnZQlq1cfYvToOaxYcZA+fRrw9tv9qVBB3ytSqjgVlVCuy9c9xZWBlAbTlu7BGHhlcGt3h6KK0e7d8XTq9D5hYYF8+eUghg5toc/GlHKBoj6wtbAkAykNvlq+j/phFenTtJq7Q1EXyBjD+vWxtGpVnfr1q/LxxwMZMKAJVapo0zlKuUq5eC7ijG1Hkth3PJXWdSrr1auH2707nquu+oq2bd9l3bojAAwf3lqTiVIu5tKEIiJXiMhWEdkhIo8WMd5gETEi4rb2wSbP20KArxcPX6FNbHiqkyezeemlP2ne/G3++GMPr77al2bNwt0dllLlhjNteQEgIv7GmIxzGN8bmAr0BWKAFSIy27FdMHu8YOAe4B9n513cfl5/iAWbY3nossbUqqLftvBE2dk5dO36IatWHWLQoKa88cbl1K2rDTkqVZKcab6+k4isB7bb3a1F5L9OzLsT1rdTdhljTgLTgYEFjDcBmAykOx928Xp53hYARnSNdFcI6jwlJlrXON7eXowc2ZYffxzGrFlDNJko5QbOFHm9BVwFxAEYY9YClzoxXW1gv0N3DPm+RS8ibYG6xpg5Rc1IRO4QkZUisvLo0aNOLNp58Skn2ROXytWtaxGszdN7DGMM06atoUGDN/nhB+uCYMyYjlx1VWM3R6ZU+eVMQvEyxuzN1y/biekKerKd17SxiHhhfWvlwbPNyBjznjGmgzGmQ3h48ZaJf/63tWpDO9Ut1vkq19m06Sg9e37Cbbf9QHR0GA0bhrg7JKUUzj1D2S8inQBjPxcZB2xzYroYwPEsXQc46NAdDLQAfrdrVdUAZovI1caYEvnGb1Z2Dl+vtG6iujQILYlFqgs0efJfPPHEb1Sq5M8HHwzgttvaattbSpUSziSUu7CKvSKAI8ACu9/ZrACiRKQ+cAAYCtyYO9AYkwCE5XaLyO/AQyWVTAAm/byFmPg07unVSKsKl3LGGESEGjWCuOmmlrzySl/Cwyu6OyyllIOzJhRjTCxWMjgnxpgsEbkb+AXwBj4yxmwUkeeBlcaY2eccbTH78M/dANx5SUM3R6IKc/BgEvfeO4+LL47gnns6c8strbnlFm3JQKnS6KwJRUTex+HZRy5jzB1nm9YY8xNWK8WO/Z4uZNyeZ5tfcZq/yXrhrUF4RSr6O117WpWQ3IYcn3jiNzIzc+jatY67Q1JKnYUzZ9IFDn8HANdyeu0tj5ORlc3/fWqVrH1zZxc3R6PyW7PmMLffPptVqw5x2WUNefvt/vrgXSkP4EyR19eO3SLyGTDfZRGVgM+WWTW7rmxZk7AgfzdHo/JLSEjn4MEkvv56MNdf30yfbynlIc6nrKc+UK+4AylJn9lVhV+/oY2bI1FgPXCfMWMT27fH8cQTPbjkkkh27bqXgAAtilTKkzjzpny8iBy3/53Aujt53PWhuca2I0nsjUulXUQV/Hy0bUx327nzOP37f8kNN8zkhx+2kplpveKkyUQpz1Pkr1assobWWNV+AXKMMWc8oPckuQ/jR2vNLrfKyMji1VeX8sILS/D19eLNN69gzJiO+GiSV8pjFZlQjDFGRL4zxrQvqYBc7Vc7ofRorK3QutP+/YlMmLCYAQOa8MYbl1O7diV3h6SUukDOXA4uF5F2Lo+kBCzaEsva/Sfo26w6Ab7e7g6n3Dl6NIUpU5YD0KhRCJs2jWXGjOs1mShVRhR6hyIiPsaYLKA78H8ishNIwWqjyxhjPC7JfPDnLgDu7a3fiy9JOTmGjz/+l4cfXkBSUgZ9+zagSZMwGjSo6u7QlFLFqKgir+VAO+CaEorFpYwx/LUjjs71Q2hRW5s2LykbNsRy111z+fPPfVx8cQTvvHMVTZqEnX1CpZTHKSqhCIAxZmcJxeJSE+duBqC2fkCrxJw8mc1ll33GyZPZfPTR1dx6axt9p0SpMqyohBIuIg8UNtAY85oL4nGJlIwsPrDb7Xr+mhZujqbs++233VxyST38/Lz55pvriY4OIyws0N1hKaVcrKiH8t5AEFYz8wX98xjf2E3Uj+vViCBtt8tlYmISue66b+jd+1M+/XQtAN27R2gyUaqcKOrsesgY83yJReJCa/efAODuXo3cHEnZlJWVw5Qpy3nqqUVkZ+cwaVJvbrqplbvDUkqVsLM+Q/F0iemZzF57kJa1K+Pvo1WFXWH48O+YPn0D/fo1YurU/tSvr7W3lCqPikoovUssCheatSqGHAM3dY5wdyhlyokT6fj4eBEU5MfYsR257rqmXHddU33orlQ5VugzFGPM8ZIMxBWycwxTftsBwNVtark5mrLBGMP06Rto2nQqTz31G2A9Jxk8WFsFVqq8K9MNJ206mEhcykmuaF6DQD99GH+hduw4zuWXf86wYbOoU6cSN9+sz0mUUqeU6bPsvI2HAHi0X7SbI/F8X365npEjf8Df34cpU/oxenQHvL3L9PWIUuoclemEsmBTLA3DKxIZVtHdoXiszMxsfH296dChFoMHN2Py5L7UquVRtcaVUiWkzF5iZmRls/VIEo2qBbk7FI8UG5vC8OHfccMNMwFo3DiUzz8fpMlEKVWoMptQYuLTAGhRS9vtOhc5OYb33ltFkyZT+PrrDTRvHk52do67w1JKeYAyW+SV+yGt6JraNLqzdu2K5+abv2XZshh69ozkf/+7kuhobchRKeWcMptQft14GIBe0dXcHInnqFzZnxMn0vnkk2sYPryVVgNWSp2TMlnklZKRxep9J+jZJBxvLz0pFmX27K0MGvQ12dk5hIYGsmHDGG65pbUmE6XUOSuTCeWTZXsA6Now1K1xlGb79iVwzTXTGThwOtu2xXHoUDIAXpqAlVLnqcwVeWVm5/DG/O0EB/hwa9f67g6n1MnKyuGNN/7mmWd+xxjDyy/34f77L8JXP4mslLpAZS6h/Lj2ICezc3ikXzR+PmXyBuyCZGfn8MEHq+nVqz7//W8/IiOruDskpVQZUebOuG8u3A7ow3hH8fFpPPLIfJKSMvD39+Gvv0Yye/ZQTSZKqWJV5hLK3rhU6oZUoL6+HY8xhi++WEd09FT+859lLFq0B4DQ0EB96K6UKnZlqshr//FUAPq1qOnmSNxv27Y4xoyZy8KFu+nUqTa//HIzbdrUcHdYSqkyrEwllNikDADaRWhRzn33zWPlyoO8/XZ/7rijvTbkqJRyuTKVUP7dFw9ASEV/N0fiHvPn7yQ6Ooy6dSvzv/9dib+/DzVqaFtmSqmS4dLLVhG5QkS2isgOEXm0gOEPiMgmEVknIgtFpN6FLO9Xu7mVVnXKV/tdhw8nc+ONs7jsss95+eW/AKhXr4omE6VUiXJZQhERb2Aq0A9oBgwTkWb5RvsX6GCMaQXMBCZfyDKX7z5OvdBAAsrJOxU5OYZ33llJdPQUZs3azDPPXMKrr17m7rCUUuWUK+9QOgE7jDG7jDEngenAQMcRjDGLjDGpduffQJ3zXdi+OGs2PaLCz3cWHmfSpCXcdddc2revxbp1o3n22Z4EBJSpUkyllAdx5dmnNrDfoTsG6FzE+KOAnwsaICJ3AHcAREREFDjx7rgUADrWDzn3SD1IUlIGx46lUr9+VUaP7kD9+lUZNqyFVgNWSrmdK+9QCjrDmQJHFLkZ6AC8UtBwY8x7xpgOxpgO4eEF34HsiLXaompQRt8/Mcbw3XebadbsbW64YSbGGEJDA7nxxpaaTJRSpYIrE0oMUNehuw5wMP9IItIHeAK42hiTcb4LO3TC+qBWWfxC4969J7j66ukMGvQNISEVeOutfppElFKljiuLvFYAUSJSHzgADAVudBxBRNoC7wJXGGNiL2Rh62ISiCyDD+SXLdtPnz6fAfDqq325996L8NE2ypRSpZDLEooxJktE7gZ+AbyBj4wxG0XkeWClMWY2VhFXEDDDvuLeZ4y5+lyXdSw5g+V7jnNr18jiWwE3S0zMoFIlf9q1q8nIkW0YP74bERHlqzq0UsqzuLRKkDHmJ+CnfP2edvi7T3Es5+sV1rP/nk08v4ZXXFwqjz66gF9/3cXGjWMICvLjv//t7+6wlFLqrMpEHdN5G6zP/Xby4Bpexhg++2wdDz74K/HxaTzwQBf0MYlSypOUiYSy/kACXRqEEujnmauTkJDONdd8ze+/76FLlzq8885VtGpV3d1hKaXUOfHMM7CDRVusZ/kNq3ledWFjDCJCpUr+hIUF8t57VzFqVDv9DK9SyiN5fHWhuesPATCmZyM3R3JufvllB+3avUdMTCIiwowZ1/N//9dek4lSymN5fEJJSMvEx0uoVaWCu0NxyqFDSQwdOpMrrviC1NRMYmNT3B2SUkoVC48v8pq/6QhtPeT7J1OnLufxx38jIyOL557rySOPdMPf3+N3gVJKAR6eUDYfSgQgyENOyqtWHaJz59pMndqfqKhQd4ejlFLFyjPOxIWY9tceAF4Z3Nq9gRQiMTGDp59exPDhrWjfvhZvv30l/v7e2myKUqpM8uiEciQpHYAalQPcHMnpjDHMmrWZe++dx6FDSUREVKZ9+1ratLxSqkzz6DPcupgEIkIC3R3GaXbvjufuu3/mp5+206ZNDb79dgidO5/3Z16UUspjeHRCSU7PonYpq931xRfrWbx4L6+/fjl3391JG3JUSpUbHptQEtIyOZmdQ++m1dwdCkuW7CUjI5s+fRowfnxXbr21DXXqVHJ3WEopVaI89vL5y3/2AVAv1H1FXseOpTJy5A/06DGN55//AwB/fx9NJkqpcslj71D2HbdeCLy0ScnfoRhjmDZtDePHzychIYNHHunGU0/1KPE4lGfIzMwkJiaG9PR0d4eiyoGAgADq1KmDr69viS/bYxPK0STr445VAv1KfNk//bSdkSNn061bXd555ypatHB/sZsqvWJiYggODiYyMlKrjCuXMsYQFxdHTEwM9evXL/Hle2yR18ItsVSuUHIZODU1k7/+sorZ+veP4ocfhrJ48W2aTNRZpaenExoaqslEuZyIEBoa6ra7YY9MKHuOpWAMJZZQfv55Oy1avE2/fl9w4kQ6IsLVVzfRhhyV0zSZqJLizmPNIxPKz/YHtV6+rpVLl3PgQCLXXz+D/v2/xN/fhx9/HEaVKqXrJUqllCotPDKhbDuSBED7elVdtozY2BSaNXubOXO28cILl7J27WguuSTSZctTylW8vb1p06YNLVq0YMCAAZw4cSJv2MaNG+nVqxeNGzcmKiqKCRMmYIzJG/7zzz/ToUMHmjZtSnR0NA899JA7VqFI//77L7fffvtp/QYOHEiXLl1O63frrbcyc+bM0/oFBQXl/b1t2zb69+9Po0aNaNq0KUOGDOHIkSMXFNvx48fp27cvUVFR9O3bl/j4+DPGWbRoEW3atMn7FxAQwPfffw/AwoULadeuHW3atKF79+7s2LEDgClTpvDxxx9fUGwuYYzxqH/t27c3V721xLR9/lfjCjExCXl/v/nm32bHjjiXLEeVH5s2bXLr8itWrJj39y233GJeeOEFY4wxqamppkGDBuaXX34xxhiTkpJirrjiTQ1HVAAAEXRJREFUCjNlyhRjjDHr1683DRo0MJs3bzbGGJOZmWmmTp1arLFlZmZe8DwGDx5s1qxZk9cdHx9v6tSpY6Kjo82uXbvy+o8YMcLMmDHjtGlzt01aWppp1KiRmT17dt6w3377zaxfv/6CYhs/fryZNGmSMcaYSZMmmYcffrjI8ePi4kzVqlVNSkqKMcaYqKiovONn6tSpZsSIEcYYa1+1adOm0PkUdMwBK42Lz88eWcsrx+EKqrgkJKTz5JO/8e67q/j779tp164m99zTudiXo8q3537cyKaDicU6z2a1KvHMgOZOjdulSxfWrVsHwJdffkm3bt247LLLAAgMDGTKlCn07NmTsWPHMnnyZJ544gmio6MB8PHxYcyYMWfMMzk5mXHjxrFy5UpEhGeeeYbrrruOoKAgkpOTAZg5cyZz5sxh2rRp3HrrrYSEhPDvv//Spk0bvvvuO9asWUOVKtZnKBo1asRff/2Fl5cXo0ePZt8+qzLMG2+8Qbdu3U5bdlJSEuvWraN161MNxM6aNYsBAwZQvXp1pk+fzmOPPXbW7fLll1/SpUsXBgwYkNfv0ksvdWqbFuWHH37g999/B2DEiBH07NmTl19+udDxZ86cSb9+/QgMtN6vExESE63jJSEhgVq1agHWvoqMjGT58uV06tTpguMsLh6ZUDYeTOTy5sXzzXVjDDNmbOK+++Zx+HAyd9/diYYNXVeUppS7ZGdns3DhQkaNGgVYxV3t27c/bZyGDRuSnJxMYmIiGzZs4MEHHzzrfCdMmEDlypVZv349QIHFOvlt27aNBQsW4O3tTU5ODt999x233XYb//zzD5GR/9/e3UfXfOcJHH9/FiUIq1Q2RitUVR4ammYGHZqWKR3TZTkq9Th6dNtiaiqlu3vsObW10+kMw66lq9b2eDiD0KKKsnR1aEmJFQ8VYzykRjeVUEuqNCSf/eP3y81NciM33Ifc+LzOuefc3/P3fs6993N/39/vfr5xxMTEMHLkSKZMmULv3r05c+YMAwYMIDc3t8J+srOzSUpKqjBv5cqVvP7668TExDBs2DC/EsqRI0eqxMKXoqIi+vTp43PZihUrSEhIqDDv3LlzxMbGAhAbG0tBQcFN979q1SoyMjI804sXL2bgwIFERUXRokULsrKyPMtSU1PZtWuXJZTbcb3EOTsRbv9OBlVl6NDVrF9/jJSUWDZsGEFqarvb3q8x1fH3TCKQrl69Svfu3cnLy+ORRx7hySefBJz3f3V3BNXmTqHt27ezatUqz3SrVjX/IHvmmWdo0KABAOnp6bzxxhs899xzrFq1ivT0dM9+jx496tnm8uXLFBUVER0d7ZmXn5/PPffc45k+d+4cJ06coHfv3ogIDRs25MiRIyQlJfl8TbW9Iyo6OpqcnJxabeOv/Px8Dh8+zIABAzzz5s6dy+bNm+nRowezZs0iIyODxYsXA9C2bVuOHTsWlLbcqoi7KH+9pBTgtmp4Xb9eAjhvpt6972XevKfYu/d5SyamXoqKiiInJ4cvv/yS4uJiFixYAEBiYiLZ2dkV1j116hTNmzcnOjqaxMRE9u/fX+P+q0tM3vMq/y+iWbNmnue9evXixIkTFBYWsn79eoYOHQpAaWkpe/bsIScnh5ycHL766qsKyaTstXnvOzMzk4sXL9KxY0fi4uLIy8vzJLvWrVtXOHv65ptvaNOmjScW/rzWoqKiChfQvR/eya9MTEwM+fn5gJMw2rat/ntr9erVDBkyxPMP98LCQg4ePEiPHk7Xe3p6Ort37/asf+3aNaKi6lZx3IhLKN/fcJLBrZat/+STPJKTF/LBB05mf/XVR3n55R40aBBxoTCmVlq2bMm8efOYPXs2169fZ9SoUXz66ads374dcM5kJk+ezGuvvQbAtGnTePPNNzl+/DjgfMHPmTOnyn779+/P/PnzPdNlX9oxMTHk5uZ6urSqIyIMGTKEjIwM4uPjad26tc/9+joziI+P99z5BE5315YtW8jLyyMvL4/9+/d7Esrjjz9OZmYmxcXFACxZssRznWTkyJHs3r2bTZs2efa1ZcsWTzdembIzFF+Pyt1dAIMGDWLp0qUALF26lMGDB1cbh5UrVzJixAjPdKtWrbh06ZIn/tu2bSM+Pt6z/Pjx41W6+8Iu2Ff9A/34q/sTtMPfbdQzF65Ue4eDLwUF3+rYsesUZmjHjv+iH398quaNjAmAunSXl6rq008/rcuWLVNV1UOHDmlaWpp26dJF77//fp0xY4aWlpZ61v3www81JSVFu3btqvHx8Tp16tQq+y8qKtKxY8dqYmKiJicn6/vvv6+qqmvWrNFOnTppWlqaTpo0yXOHkq+7rfbt26eALlmyxDOvsLBQhw8frg899JDGx8friy++6PP1JSUl6eXLl/X06dParl27Cu1XVX344Yc1KytLVVVnzJihSUlJ2q1bNx06dKgWFBR41svNzdUBAwZo586dNT4+XtPT0/Xrr7++aWxrcv78ee3bt6927txZ+/btqxcuXPC83vHjx3vWK2t7SUlJhe3Xrl2rSUlJmpycrGlpaXry5MkKr6uwsNDnccN1l5doEO6YCqa7O8RrixGzyXvrZ35vs3LlYSZN2sy33xYzbdqjTJ/+GE2bhr5wmrkz5ebmVvhlaQJr7ty5REdHV/kvSn124MAB5syZw/Lly30u9/WeE5H9qpoazHZFXD9PSakS1ahBrba5caOUpKS25OS8xK9+1c+SiTH1yIQJE2jcuHG4mxFS58+fZ+bMmeFuRhURd5fXtRsl9O168wvyV64UM3PmTu67ryUTJ/6Q0aOTGT062eopGVMPNWnShDFjxoS7GSFVdqdeXRNxZygAbVtU/2tk48bjJCa+zW9+8xnHj18AnIt+lkxMOEVa17KJXOF8r0XcGQpAQmzVERHPnr3M5MkfsW7dMRIS7mHnznH06dMhDK0zpqImTZpw4cIFK2Fvgk7VGQ+lSZPwFLGNyITSoXWzKvNOnbrI1q0n+fWv+5GR0Yu77qrddRZjgqV9+/acPXuWwsLCcDfF3AHKRmwMh4hMKE3dZLF371fs2fNnfvnLnjz2WAfOnHmF1mEcY94YXxo1ahSW0fOMCbWgXkMRkadE5I8ickJE/t7H8sYikuku/1xE4vzZ73dXipk4cRM9ey5mzpwsrlxx/qhkycQYY8InaAlFRBoAC4CfAgnACBGp/FfS8cBFVe0MzAWqL8PpZVDf5bzzzn4mT+7B4cMTaNYs9OPKG2OMqSiYXV4/Ak6o6ikAEVkFDAa8C94MBma4z98D5ouI6M1uU1C4NzaazRtGkJISG5SGG2OMqb1gJpQfAH/2mj4LVB5gxLOOqt4QkUtAa+C890oi8gLwgjv5ffa5F474UWn6TtCGSrG6g1ksylksylksyj0Y7AMEM6H4uj+y8pmHP+ugqouARQAikh3s8gGRwmJRzmJRzmJRzmJRTkSya17r9gTzovxZ4F6v6fbA/1a3jog0BFoC3wSxTcYYY4IkmAllH/CAiHQUkbuAZ4ENldbZAPzcfT4M+O+bXj8xxhhTZwWty8u9JvILYCvQAHhXVb8QkTdwyihvAP4TWC4iJ3DOTJ71Y9eLgtXmCGSxKGexKGexKGexKBf0WERc+XpjjDF1U0QWhzTGGFP3WEIxxhgTEHU2oQSrbEsk8iMWGSJyVEQOicjHIlJvyyzXFAuv9YaJiIpIvb1l1J9YiMhw973xhYisCHUbQ8WPz8h9IrJDRA64n5OB4WhnsInIuyJSICJHqlkuIjLPjdMhEUkJaAOCPcbwrTxwLuKfBDoBdwEHgYRK60wEFrrPnwUyw93uMMbiCaCp+3zCnRwLd71oYCeQBaSGu91hfF88ABwAWrnTbcPd7jDGYhEwwX2eAOSFu91BisVjQApwpJrlA4GPcP4D2BP4PJDHr6tnKJ6yLapaDJSVbfE2GFjqPn8P6Cf1c7CJGmOhqjtU9Tt3MgvnPz/1kT/vC4CZwG+Ba6FsXIj5E4u/BRao6kUAVS0IcRtDxZ9YKFA2kFJLqv4nrl5Q1Z3c/L98g4Fl6sgC/lJEAlbDqq4mFF9lW35Q3TqqegMoK9tS3/gTC2/jcX6B1Ec1xkJEHgbuVdWNoWxYGPjzvugCdBGRz0QkS0SeClnrQsufWMwARovIWWAz8HJomlbn1Pb7pFbq6ngoASvbUg/4/TpFZDSQCqQFtUXhc9NYiMhf4FStHheqBoWRP++LhjjdXo/jnLXuEpEkVf2/ILct1PyJxQhgiar+TkR64fz/LUlVS4PfvDolqN+bdfUMxcq2lPMnFojIT4DpwCBV/T5EbQu1mmIRDSQBn4hIHk4f8YZ6emHe38/IB6p6XVVPA3/ESTD1jT+xGA+sBlDVPUATnMKRdxq/vk9uVV1NKFa2pVyNsXC7ed7BSSb1tZ8caoiFql5S1TaqGqeqcTjXkwapatCL4oWBP5+R9Tg3bCAibXC6wE6FtJWh4U8szgD9AEQkHieh3IljMm8Axrp3e/UELqlqfqB2Xie7vDR4ZVsijp+xmAU0B9a49yWcUdVBYWt0kPgZizuCn7HYCvQXkaNACTBNVS+Er9XB4WcsXgX+Q0Sm4HTxjKuPP0BFZCVOF2cb93rR60AjAFVdiHP9aCBwAvgOeC6gx6+HMTXGGBMGdbXLyxhjTISxhGKMMSYgLKEYY4wJCEsoxhhjAsISijHGmICwhGLqHBEpEZEcr0fcTdaNq66yai2P+YlbrfagW6rkwVvYx0siMtZ9Pk5E2nktWywiCQFu5z4R6e7HNq+ISNPbPbYxNbGEYuqiq6ra3euRF6LjjlLVbjhFR2fVdmNVXaiqy9zJcUA7r2XPq+rRgLSyvJ1v4187XwEsoZigs4RiIoJ7JrJLRP7HfTzqY51EEdnrntUcEpEH3Pmjvea/IyINajjcTqCzu20/dwyNw+5YE43d+W9J+Rg0s915M0RkqogMw6mp9nv3mFHumUWqiEwQkd96tXmciPzbLbZzD16F/UTk30UkW5yxT/7JnTcZJ7HtEJEd7rz+IrLHjeMaEWlew3GM8YslFFMXRXl1d61z5xUAT6pqCpAOzPOx3UvAv6pqd5wv9LNumY104Mfu/BJgVA3H/2vgsIg0AZYA6ar6EE5liQkicjcwBEhU1WTgn703VtX3gGycM4nuqnrVa/F7wFCv6XQg8xbb+RROeZUy01U1FUgG0kQkWVXn4dRqekJVn3BLsPwj8BM3ltlARg3HMcYvdbL0irnjXXW/VL01Aua71wxKcOpSVbYHmC4i7YG1qvonEekHPALsc8vSROEkJ19+LyJXgTyc8uYPAqdV9bi7fCkwCZiPM9bKYhHZBPhdKl9VC0XklFtH6U/uMT5z91ubdjbDKTPiPeLecBF5AedzHYszkNShStv2dOd/5h7nLpy4GXPbLKGYSDEFOAd0wzmzrjJ4lqquEJHPgZ8BW0XkeZxy3UtV9R/8OMYo70KSIuJzfB23dtSPcIoNPgv8Auhbi9eSCQwHjgHrVFXF+Xb3u504oxK+BSwAhopIR2Aq8ENVvSgiS3AKIFYmwDZVHVGL9hrjF+vyMpGiJZDvjl8xBufXeQUi0gk45XbzbMDp+vkYGCYibd117haRDn4e8xgQJyKd3ekxwB/caw4tVXUzzgVvX3daFeGU0/dlLfA3OGN0ZLrzatVOVb2O03XV0+0uawFcAS6JSAzw02rakgX8uOw1iUhTEfF1tmdMrVlCMZHibeDnIpKF0911xcc66cAREckBuuIMdXoU54v3v0TkELANpzuoRqp6Daca6xoROQyUAgtxvpw3uvv7A87ZU2VLgIVlF+Ur7fcicBTooKp73Xm1bqd7beZ3wFRVPYgzfvwXwLs43WhlFgEficgOVS3EuQNtpXucLJxYGXPbrNqwMcaYgLAzFGOMMQFhCcUYY0xAWEIxxhgTEJZQjDHGBIQlFGOMMQFhCcUYY0xAWEIxxhgTEP8PCzsv1jsat28AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(xgb, y_test, X_test, \"Decision Tree (XGBoost)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.94 0.88 6729\n", + " 1 0.66 0.37 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.66 0.68 8749\n", + "weighted avg 0.79 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, xgb.predict(X_test)))" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[2] = ([\"Gradient Boosted\" , \n", + " classification_report(y_test, xgb.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From the accuracy and AUROC, we observe that the XGBoost performs similarly to the random forest ensemble. It has a slight bump in AUROC at 0.76, but the accuracy is the same." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ModelRecall-1AUROC
0Decision Tree0.4099010.606997
1Random Forest0.3707920.768455
2Gradient Boosted0.3712870.778407
\n", + "
" + ], + "text/plain": [ + " Model Recall-1 AUROC\n", + "0 Decision Tree 0.409901 0.606997\n", + "1 Random Forest 0.370792 0.768455\n", + "2 Gradient Boosted 0.371287 0.778407" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Logistic Regression\n", + "\n", + "#### Theory\n", + "Logistic regression is a regression technnique used to predict binary target variables. It works on the same principles as a linear regression model. \n", + "\n", + "Our binary target (default vs non-default) can be expressed in terms of odds of defaulting, which is the ratio of the probability of default and probability of non-default. \n", + "\n", + "In the logistic regression model, we log the odds (log-odds) and equate it to a weighted sum of regressors.\n", + "\n", + "![image.png](https://wikimedia.org/api/rest_v1/media/math/render/svg/4a5e86f014eb1f0744e280eb0d68485cb8c0a6c3)\n", + "\n", + "We then find weights for the regressors that best fits the data. Since the binary target (default or not) follows a bernoulli distribution, each data point has the following probability distribution function:\n", + "\n", + "![image.png](https://wikimedia.org/api/rest_v1/media/math/render/svg/614e0c64d59f0ff2e926deafcb2de6e502394fac)\n", + "\n", + "We would like to update p for each data point such that the log product (joint probability) of the above function for all data points is maximised. In other words, we are maximising the log-likelihood function.\n", + "\n", + "The logistic regression equation produces a \"squashed\" curve like the one below. We then pick a cutoff value for the y axis to classify a data point as 0 (non-default) or 1 (default).\n", + "\n", + "![image.png](https://upload.wikimedia.org/wikipedia/commons/thumb/8/88/Logistic-curve.svg/1280px-Logistic-curve.svg.png)\n", + "\n", + "\n", + "#### Training\n", + "We will adopt a top-down approach for training our logistic regression model, i.e. include all regressors first and then remove the most insignificant ones at each iteration to achieve the best fit." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": {}, + "outputs": [], + "source": [ + "import statsmodels.api as sm" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization terminated successfully.\n", + " Current function value: 0.445436\n", + " Iterations 7\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
Logit Regression Results
Dep. Variable: Y No. Observations: 17496
Model: Logit Df Residuals: 17452
Method: MLE Df Model: 43
Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1737
Time: 19:35:44 Log-Likelihood: -7793.4
converged: True LL-Null: -9431.5
Covariance Type: nonrobust LLR p-value: 0.000
\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
coef std err z P>|z| [0.025 0.975]
LIMIT_BAL -0.8354 0.114 -7.302 0.000 -1.060 -0.611
SEX -0.1196 0.041 -2.920 0.004 -0.200 -0.039
AGE 0.0615 0.100 0.614 0.539 -0.135 0.258
PAY_0 0.6546 0.059 11.081 0.000 0.539 0.770
PAY_2 -0.5622 0.098 -5.715 0.000 -0.755 -0.369
PAY_3 -0.1059 0.113 -0.941 0.347 -0.327 0.115
PAY_4 -0.2041 0.159 -1.287 0.198 -0.515 0.107
PAY_5 -0.0925 0.178 -0.521 0.603 -0.441 0.256
PAY_6 0.3289 0.147 2.244 0.025 0.042 0.616
BILL_AMT1 -1.0372 0.528 -1.964 0.049 -2.072 -0.002
BILL_AMT2 0.6000 0.765 0.785 0.433 -0.899 2.099
BILL_AMT3 1.5425 0.732 2.107 0.035 0.107 2.978
BILL_AMT4 0.2846 0.715 0.398 0.690 -1.116 1.686
BILL_AMT5 -1.9241 0.913 -2.107 0.035 -3.714 -0.134
BILL_AMT6 1.3621 0.839 1.623 0.105 -0.283 3.007
PAY_AMT1 -1.1622 0.306 -3.794 0.000 -1.763 -0.562
PAY_AMT2 -2.0159 0.390 -5.172 0.000 -2.780 -1.252
PAY_AMT3 -0.5874 0.310 -1.898 0.058 -1.194 0.019
PAY_AMT4 -0.0638 0.281 -0.227 0.820 -0.614 0.487
PAY_AMT5 -1.0410 0.291 -3.576 0.000 -1.612 -0.470
PAY_AMT6 -0.3858 0.256 -1.507 0.132 -0.887 0.116
GRAD 1.3893 0.230 6.030 0.000 0.938 1.841
UNI 1.3690 0.229 5.975 0.000 0.920 1.818
HS 1.3654 0.233 5.863 0.000 0.909 1.822
MARRIED 0.0176 0.158 0.111 0.911 -0.292 0.328
SINGLE -0.0968 0.159 -0.609 0.543 -0.408 0.215
PAY_0_No_Transactions 0.0316 0.124 0.255 0.799 -0.211 0.274
PAY_0_Pay_Duly 0.2120 0.122 1.734 0.083 -0.028 0.452
PAY_0_Revolving_Credit -0.6866 0.137 -5.022 0.000 -0.955 -0.419
PAY_2_No_Transactions -1.4519 0.236 -6.151 0.000 -1.914 -0.989
PAY_2_Pay_Duly -1.3960 0.225 -6.201 0.000 -1.837 -0.955
PAY_2_Revolving_Credit -0.9936 0.230 -4.318 0.000 -1.445 -0.543
PAY_3_No_Transactions -0.5672 0.275 -2.059 0.039 -1.107 -0.027
PAY_3_Pay_Duly -0.5717 0.250 -2.284 0.022 -1.062 -0.081
PAY_3_Revolving_Credit -0.5379 0.239 -2.249 0.025 -1.007 -0.069
PAY_4_No_Transactions -0.7044 0.356 -1.980 0.048 -1.402 -0.007
PAY_4_Pay_Duly -0.8759 0.338 -2.593 0.010 -1.538 -0.214
PAY_4_Revolving_Credit -0.7888 0.328 -2.404 0.016 -1.432 -0.146
PAY_5_No_Transactions -0.4367 0.393 -1.110 0.267 -1.208 0.334
PAY_5_Pay_Duly -0.4715 0.378 -1.247 0.212 -1.213 0.270
PAY_5_Revolving_Credit -0.3428 0.369 -0.930 0.352 -1.065 0.379
PAY_6_No_Transactions 0.4826 0.323 1.492 0.136 -0.151 1.117
PAY_6_Pay_Duly 0.4427 0.316 1.401 0.161 -0.177 1.062
PAY_6_Revolving_Credit 0.2143 0.308 0.695 0.487 -0.390 0.818
" + ], + "text/plain": [ + "\n", + "\"\"\"\n", + " Logit Regression Results \n", + "==============================================================================\n", + "Dep. Variable: Y No. Observations: 17496\n", + "Model: Logit Df Residuals: 17452\n", + "Method: MLE Df Model: 43\n", + "Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1737\n", + "Time: 19:35:44 Log-Likelihood: -7793.4\n", + "converged: True LL-Null: -9431.5\n", + "Covariance Type: nonrobust LLR p-value: 0.000\n", + "==========================================================================================\n", + " coef std err z P>|z| [0.025 0.975]\n", + "------------------------------------------------------------------------------------------\n", + "LIMIT_BAL -0.8354 0.114 -7.302 0.000 -1.060 -0.611\n", + "SEX -0.1196 0.041 -2.920 0.004 -0.200 -0.039\n", + "AGE 0.0615 0.100 0.614 0.539 -0.135 0.258\n", + "PAY_0 0.6546 0.059 11.081 0.000 0.539 0.770\n", + "PAY_2 -0.5622 0.098 -5.715 0.000 -0.755 -0.369\n", + "PAY_3 -0.1059 0.113 -0.941 0.347 -0.327 0.115\n", + "PAY_4 -0.2041 0.159 -1.287 0.198 -0.515 0.107\n", + "PAY_5 -0.0925 0.178 -0.521 0.603 -0.441 0.256\n", + "PAY_6 0.3289 0.147 2.244 0.025 0.042 0.616\n", + "BILL_AMT1 -1.0372 0.528 -1.964 0.049 -2.072 -0.002\n", + "BILL_AMT2 0.6000 0.765 0.785 0.433 -0.899 2.099\n", + "BILL_AMT3 1.5425 0.732 2.107 0.035 0.107 2.978\n", + "BILL_AMT4 0.2846 0.715 0.398 0.690 -1.116 1.686\n", + "BILL_AMT5 -1.9241 0.913 -2.107 0.035 -3.714 -0.134\n", + "BILL_AMT6 1.3621 0.839 1.623 0.105 -0.283 3.007\n", + "PAY_AMT1 -1.1622 0.306 -3.794 0.000 -1.763 -0.562\n", + "PAY_AMT2 -2.0159 0.390 -5.172 0.000 -2.780 -1.252\n", + "PAY_AMT3 -0.5874 0.310 -1.898 0.058 -1.194 0.019\n", + "PAY_AMT4 -0.0638 0.281 -0.227 0.820 -0.614 0.487\n", + "PAY_AMT5 -1.0410 0.291 -3.576 0.000 -1.612 -0.470\n", + "PAY_AMT6 -0.3858 0.256 -1.507 0.132 -0.887 0.116\n", + "GRAD 1.3893 0.230 6.030 0.000 0.938 1.841\n", + "UNI 1.3690 0.229 5.975 0.000 0.920 1.818\n", + "HS 1.3654 0.233 5.863 0.000 0.909 1.822\n", + "MARRIED 0.0176 0.158 0.111 0.911 -0.292 0.328\n", + "SINGLE -0.0968 0.159 -0.609 0.543 -0.408 0.215\n", + "PAY_0_No_Transactions 0.0316 0.124 0.255 0.799 -0.211 0.274\n", + "PAY_0_Pay_Duly 0.2120 0.122 1.734 0.083 -0.028 0.452\n", + "PAY_0_Revolving_Credit -0.6866 0.137 -5.022 0.000 -0.955 -0.419\n", + "PAY_2_No_Transactions -1.4519 0.236 -6.151 0.000 -1.914 -0.989\n", + "PAY_2_Pay_Duly -1.3960 0.225 -6.201 0.000 -1.837 -0.955\n", + "PAY_2_Revolving_Credit -0.9936 0.230 -4.318 0.000 -1.445 -0.543\n", + "PAY_3_No_Transactions -0.5672 0.275 -2.059 0.039 -1.107 -0.027\n", + "PAY_3_Pay_Duly -0.5717 0.250 -2.284 0.022 -1.062 -0.081\n", + "PAY_3_Revolving_Credit -0.5379 0.239 -2.249 0.025 -1.007 -0.069\n", + "PAY_4_No_Transactions -0.7044 0.356 -1.980 0.048 -1.402 -0.007\n", + "PAY_4_Pay_Duly -0.8759 0.338 -2.593 0.010 -1.538 -0.214\n", + "PAY_4_Revolving_Credit -0.7888 0.328 -2.404 0.016 -1.432 -0.146\n", + "PAY_5_No_Transactions -0.4367 0.393 -1.110 0.267 -1.208 0.334\n", + "PAY_5_Pay_Duly -0.4715 0.378 -1.247 0.212 -1.213 0.270\n", + "PAY_5_Revolving_Credit -0.3428 0.369 -0.930 0.352 -1.065 0.379\n", + "PAY_6_No_Transactions 0.4826 0.323 1.492 0.136 -0.151 1.117\n", + "PAY_6_Pay_Duly 0.4427 0.316 1.401 0.161 -0.177 1.062\n", + "PAY_6_Revolving_Credit 0.2143 0.308 0.695 0.487 -0.390 0.818\n", + "==========================================================================================\n", + "\"\"\"" + ] + }, + "execution_count": 57, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "glm = sm.Logit(y_train,X_train).fit()\n", + "glm.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 13475\n", + " 1 0.67 0.35 0.46 4021\n", + "\n", + " accuracy 0.81 17496\n", + " macro avg 0.75 0.65 0.67 17496\n", + "weighted avg 0.79 0.81 0.79 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train,list(glm.predict(X_train)>0.5)))" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.69 0.35 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm.predict(X_test)>0.5)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The logisitc model with all features performs quite well on both the train and test set with an accuracy of about 0.8. We will now try removing all the insignificant features to see how that affects the model performance." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization terminated successfully.\n", + " Current function value: 0.446914\n", + " Iterations 7\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
Logit Regression Results
Dep. Variable: Y No. Observations: 17496
Model: Logit Df Residuals: 17472
Method: MLE Df Model: 23
Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1709
Time: 19:35:44 Log-Likelihood: -7819.2
converged: True LL-Null: -9431.5
Covariance Type: nonrobust LLR p-value: 0.000
\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "\n", + " \n", + "\n", + "
coef std err z P>|z| [0.025 0.975]
LIMIT_BAL -0.7989 0.111 -7.183 0.000 -1.017 -0.581
SEX -0.1204 0.040 -2.986 0.003 -0.199 -0.041
PAY_0 0.5793 0.037 15.471 0.000 0.506 0.653
PAY_2 -0.6376 0.077 -8.284 0.000 -0.788 -0.487
PAY_6 0.2012 0.030 6.656 0.000 0.142 0.260
BILL_AMT1 -0.8123 0.359 -2.261 0.024 -1.516 -0.108
BILL_AMT3 2.2505 0.535 4.206 0.000 1.202 3.299
BILL_AMT5 -0.9943 0.358 -2.779 0.005 -1.696 -0.293
PAY_AMT1 -1.3333 0.287 -4.642 0.000 -1.896 -0.770
PAY_AMT2 -2.3238 0.373 -6.232 0.000 -3.055 -1.593
PAY_AMT5 -0.9218 0.252 -3.658 0.000 -1.416 -0.428
GRAD 1.2429 0.179 6.961 0.000 0.893 1.593
UNI 1.2478 0.177 7.052 0.000 0.901 1.595
HS 1.2675 0.181 7.014 0.000 0.913 1.622
PAY_0_Revolving_Credit -0.8716 0.092 -9.431 0.000 -1.053 -0.690
PAY_2_No_Transactions -1.6757 0.200 -8.387 0.000 -2.067 -1.284
PAY_2_Pay_Duly -1.4967 0.179 -8.379 0.000 -1.847 -1.147
PAY_2_Revolving_Credit -1.0909 0.182 -6.007 0.000 -1.447 -0.735
PAY_3_No_Transactions -0.3503 0.159 -2.199 0.028 -0.663 -0.038
PAY_3_Pay_Duly -0.3319 0.111 -3.003 0.003 -0.548 -0.115
PAY_3_Revolving_Credit -0.3494 0.081 -4.332 0.000 -0.507 -0.191
PAY_4_No_Transactions -0.2852 0.130 -2.193 0.028 -0.540 -0.030
PAY_4_Pay_Duly -0.5573 0.103 -5.433 0.000 -0.758 -0.356
PAY_4_Revolving_Credit -0.4761 0.075 -6.332 0.000 -0.623 -0.329
" + ], + "text/plain": [ + "\n", + "\"\"\"\n", + " Logit Regression Results \n", + "==============================================================================\n", + "Dep. Variable: Y No. Observations: 17496\n", + "Model: Logit Df Residuals: 17472\n", + "Method: MLE Df Model: 23\n", + "Date: Mon, 18 Nov 2019 Pseudo R-squ.: 0.1709\n", + "Time: 19:35:44 Log-Likelihood: -7819.2\n", + "converged: True LL-Null: -9431.5\n", + "Covariance Type: nonrobust LLR p-value: 0.000\n", + "==========================================================================================\n", + " coef std err z P>|z| [0.025 0.975]\n", + "------------------------------------------------------------------------------------------\n", + "LIMIT_BAL -0.7989 0.111 -7.183 0.000 -1.017 -0.581\n", + "SEX -0.1204 0.040 -2.986 0.003 -0.199 -0.041\n", + "PAY_0 0.5793 0.037 15.471 0.000 0.506 0.653\n", + "PAY_2 -0.6376 0.077 -8.284 0.000 -0.788 -0.487\n", + "PAY_6 0.2012 0.030 6.656 0.000 0.142 0.260\n", + "BILL_AMT1 -0.8123 0.359 -2.261 0.024 -1.516 -0.108\n", + "BILL_AMT3 2.2505 0.535 4.206 0.000 1.202 3.299\n", + "BILL_AMT5 -0.9943 0.358 -2.779 0.005 -1.696 -0.293\n", + "PAY_AMT1 -1.3333 0.287 -4.642 0.000 -1.896 -0.770\n", + "PAY_AMT2 -2.3238 0.373 -6.232 0.000 -3.055 -1.593\n", + "PAY_AMT5 -0.9218 0.252 -3.658 0.000 -1.416 -0.428\n", + "GRAD 1.2429 0.179 6.961 0.000 0.893 1.593\n", + "UNI 1.2478 0.177 7.052 0.000 0.901 1.595\n", + "HS 1.2675 0.181 7.014 0.000 0.913 1.622\n", + "PAY_0_Revolving_Credit -0.8716 0.092 -9.431 0.000 -1.053 -0.690\n", + "PAY_2_No_Transactions -1.6757 0.200 -8.387 0.000 -2.067 -1.284\n", + "PAY_2_Pay_Duly -1.4967 0.179 -8.379 0.000 -1.847 -1.147\n", + "PAY_2_Revolving_Credit -1.0909 0.182 -6.007 0.000 -1.447 -0.735\n", + "PAY_3_No_Transactions -0.3503 0.159 -2.199 0.028 -0.663 -0.038\n", + "PAY_3_Pay_Duly -0.3319 0.111 -3.003 0.003 -0.548 -0.115\n", + "PAY_3_Revolving_Credit -0.3494 0.081 -4.332 0.000 -0.507 -0.191\n", + "PAY_4_No_Transactions -0.2852 0.130 -2.193 0.028 -0.540 -0.030\n", + "PAY_4_Pay_Duly -0.5573 0.103 -5.433 0.000 -0.758 -0.356\n", + "PAY_4_Revolving_Credit -0.4761 0.075 -6.332 0.000 -0.623 -0.329\n", + "==========================================================================================\n", + "\"\"\"" + ] + }, + "execution_count": 60, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#remove all insignificant attributes\n", + "sig = glm.pvalues[glm.pvalues < 0.05]\n", + "glm_2 = sm.Logit(y_train,X_train[sig.index]).fit()\n", + "glm_2.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 13475\n", + " 1 0.66 0.36 0.46 4021\n", + "\n", + " accuracy 0.81 17496\n", + " macro avg 0.75 0.65 0.67 17496\n", + "weighted avg 0.79 0.81 0.79 17496\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_train,list(glm_2.predict(X_train[sig.index])>0.5)))" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.35 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.5)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since there is not much change to the model performance on both the train and test set when we reduce the features, we will use the reduced logistic regression model from this point onwards (Principle of Parsimony). \n", + "\n", + "We now Calculate the AUROC for the train set." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.22554888390620112\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e8lvUEgCTUJSWihtwhSRRARFEGaomIBfopgR8SOioi9vWDB3kFABAEREBBEWhCklxBaSGjpve19/5hNXEIIC2Sz2c35PA8P2ZnZmTNl58zcO3Ov0lojhBBCXEg1ewcghBCicpNEIYQQokySKIQQQpRJEoUQQogySaIQQghRJkkUQgghyiSJogxKqTuUUsvtHYe9KaVClVIZSimXClxmmFJKK6VcK2qZtqSU2q2U6nUZ37usY1Ap5aWU+lUplaqUmnup368oSqnflFJ3X8b3eiil9tsipsrMHr9FAOUo71EopY4AdYBCIANYBjyotc6wZ1zOyLytx2qtV9oxhjDgMOCmtS6wVxzmWDTQRGsdY+PlhFFO66yUGgU8BHQtj+1nTnLfaa2Dr3Rel7n8ctkHSqkXgWeBXKAA2ANM1FpvuOIgnZij3VEM1Fr7Au2A9sDTdo7nstjzKtlZrtAvRRXd3g2BA5eTJKrAMTLHfB4JBFYDNrnjcqrtqLV2iH/AEeA6i89vAEssPnsAbwHHgFPAx4CXxfhBwHYgDTgE3GAeXgP4HEgATgCvAC7mcfcAf5n//hh4q0RMC4HHzX/XB+YDZzCuCh+2mO5FYB7wnXn5Y0tZvxrAN+bvHwWeA6pZxLEe+D8gFdgH9Cnx3bLWYT3wLpBkHtcIWAUkAmeB7wF/8/TfAiYgG+PO7UkgDNCAq3maNcBU83zTgeVAoEU8d5nXIRF4vuS+K7HeXsDb5ulTgb/Mw4qWebd5n54FnrX4XidgA5BiXu8ZgLvFeA1MAA4Ch83D3geOm/fBVqCHxfQuwDPmYyPdPD4EWGueV6Z5e9xqnv4mjOMpBfgbaFPiWJ0M7MC4cnW13Abm2KPNcZwC3jEPP2ZeVob5XxcsjkHzNC2BFeZ9eQp4ppRt+hKQB+Sb5zMG46LwOfN2Po1xrNUwT1+0rceYY1hbyjx7AXEX2IdlHbsu5v17FuN38SDnH0tjzX83Bv40HwdnMU7olLYPSsZj3lc/m2NIBGZcINYXMe6Mij63MM87yGJYWfu2A7DNfIzMBeYAr1huI/O+Pwl8a8X8JmP8ZtOB/Zh/12UcI0X7qmj71QcWmY+HGOB/Jdb1J/O+SQd2A1GXdf6t6BP+5f7j3B9aMLATeN9i/HvmDVYL8AN+BaZbbPRUoC/GD6YBEGke9wvwCeAD1AY2A/dbnGSLEkVPjJNMUXFdTYyTaX3zPLcCLwDuQAQQC/Sz2GH5wGDztF6lrN83GInHz3wwHADGWMRRADwGuGH8UFKBWlauQwFGMYQrxkm4sXlbeABBGD/E90rb1hc4ONdgnFCbmue3BnjN4oeXAXQ3b4u3zOt+oUQx0/z9Bhgnla7muIqW+al5GW0xTrrNzd/rCFxtXqcwYC/wqMV8NcYJtVbR9gbuBALM35mI8WP2NI+bhHFMNQOUeXkBFvNqXOJkcRrobI75bvM287DYftsxTl5eJbcpRoIbZf7bF7i6tO1cyjHoh5EUJwKe5s+drTwhjsY4kUSYl/kz/53Iipb7DcYxVNrx2YsLJ4qyjt1xGMU7wRi/mZVcOFH8iFEsVM28ft1L7M/GpcVj3gf/YlwM+ZT87oW2C8bx+RpGUiqK54L71jz9UeARjN/hEIyEbJkoCoDXzdN7XWR+zTDOKfUt9kOjSzlGMBLrh+Z1boeRKPtYrGsOMMC87OnAxss6/9o7AVgdqLFxMzAyowb+4L+rYIVxtdHIYvou/Hcl+QnwbinzrINx8rG88xgJrC7lR6owrrZ6mj//D1hl/rszcKzEvJ8GvrTYYeddpVlM62KOo4XFsPuBNRZxxGNOUuZhm4FRVq7DsQst2zzNYGBbiW19sUTxnMX48cAy898vAD9ajPPG+DGdlygwTgjZQNtSxhUtM7jEOt92gXV4FFhg8VkDvS+y3slFy8a4mht0gelKnqQ+AqaWmGY/cI3F9htdyvFblCjWYlz1B5aY5pztXMoxONJyP11k3V7k3ETxBzDe4nMzjATuarHciDLm14tSEoUVx+4qzBct5s/XlXIsFSWKb4BZlvu8jH1QHA/Gb/2M5Xa7yHbJw7i6L8S4++hlzb7FuFg8wbm/w784N1HkYb74sGJ+jTGSyHUY9VKW01z0GMG4ECkE/CzGTwe+sljXlRbjWgDZ1hw/Jf85Wh3FYK21H8YOicQoYwTjqtgb2KqUSlFKpWBUdgeZx4dgXAGX1BDjyiDB4nufYFyVn0MbW3o2xo8V4HaMIpui+dQvmod5Ps9gnMSLHC9jvQL572qlyFGMq+wiJ8wxWI6vb+U6nLNspVRtpdRspdQJpVQaRpFYIJfmpMXfWRhXPZhjKl6e1joL48dYmkCMK6HS9k2Zy1FKNVVKLVZKnTSvw6ucvw4l13uiUmqv+UmgFIwik6LvXOgYKU1DYGKJ/R2Cse6lLruEMRh3Y/uUUluUUjdZudxLibGk+px/fLli/TF6IRc7ds85Hi6yjCcxLsg2m58SG21lDCHAUW19fcxPWmt/jHXfhXF3WqSsfVuf83+HJdfnjNY6x5r5aaNi/lGME/pp82+y6Biy5hipDyRprdMthpU8b5T8/XheTt2JoyUKALTWfwJfYRRrgHHrmA201Fr7m//V0EaFFRg7s1EpszqOcTUUaPG96lrrlhdY9I/AMKVUQ4y7iPkW8zlsMQ9/rbWf1nqAZdhlrNJZjKu7hhbDQjGuXoo0UEqpEuPjrVyHksuebh7WRmtdHaNIRpUx/aVIwChmAIzHNDGKe0pzFuPWuLR9czEfYdTVNDGvwzOcuw5gsR5KqR4Y5cEjgJrmE0WqxXcudIyU5jgwrcT+9tZa/1jaskvSWh/UWo/ESOavA/OUUj5lfecyYiwpnvOPrwKM8u/i0C5jvhc7ds85HjBOkqXSWp/UWv9Pa10f467kQ6VUYytiOA6EXuoJUGt91rycF5VS9SzmdaF9m8D5v8OS61NyG5Z5rGitf9Bad8fYfhrjeCjrGLEUD9RSSvlZDCt53igXDpkozN4D+iql2mmtTRhl2e8qpWoDKKUaKKX6maf9HLhXKdVHKVXNPC5Sa52AURH7tlKqunlcI6XUNaUtUGu9DeMW9zPgd611innUZiBNKTXZ/Py6i1KqlVLqKmtWRGtdiFHpNE0p5WdORI9jXOkXqQ08rJRyU0oNB5oDSy91Hcz8MIrxUpRSDTDK5y2dwijLvhzzgIFKqa5KKXeM2+eSJ3AAzPvtC+AdpVR983bropTysGI5fhgVfRlKqUjgASumL8BcRKGUegGobjH+M2CqUqqJMrRRShUluJLb41NgnFKqs3laH6XUjSV+sBeklLpTKRVkXv+iY6jQHJuJC2/7xUBdpdSjSikP87HS2ZplYlzkPKaUCldK+WLcgc25hKvwotg9Lf+Z4y3r2P0JeMT8m/PHSNYXmvdwpVRRUknGOHEWmj+XdUxuxjiJv2beF55KqW7WrI/Weh/wO8bdDJS9bzeY43lQKeWqlBqEUf9ZlgvOTynVTCnV23y852Bc7Baat8WFjhHL2I9jVI5PN69zG4w7ke8pZw6bKLTWZzDKNJ83D5qMUVm3URlFESsxymHRWm8G7sWo7ErFqAAqugK6C+PWeQ/GwTkPKLq6KM2PGGWKP1jEUggMxKhMOoxxlfUZRtGGtR7CqGeJxSj3/AHjJFpkE9DEPO9pwDCtdVGRzqWuw0sYlWypwBKMik1L04HnzLfKT1zCOqC13m1el9kYP950jHLY3At85QmMSuQtGE9uvI51x+UTGMV/6Rg/xjkXmf534DeMitajGD9My2KDdzBOassxEtDnGJWRYBQNfG3eHiO01tEYdVQzMLZ3DEZdgrVuAHYrpTIwnsS6TWudYy6mmwasNy/rassvmYsY+mIcaycxnui61splfoHxRNtajGM0B2M/XYoGGCczy3+NKPvY/RRjm+7AeFpoKUbCPuekZ3YVsMm8XRYBj2itD5vHvYjFPrD8ksXvrzFGPWIcxgMf1noTuE8pVbusfau1zsOowB6DcfK+EyN5X+jY5iLHigf/VaafxLgYfMY8rtRjpJRFjMSot4gHFgBTtNYrLmHdreIwL9xVZUqpezAq/LrbO5ZLZb56TcEoIjp8semFc1NK9Qc+1lo3vOjEDkAptQljfb60dyy25LB3FKLyUkoNVEp5m8tU38K4Yzhi36iEPZiLYgeYi2oaAFMwrnwdklLqGqVUXfP63A20wXhwxqlJohC2MAjjVjgeo7jsNi23rlWVwijqTMYoetqL8Qi1o2qG8c5GKsb7LMPM9YROTYqehBBClEnuKIQQQpTJ4RqtCgwM1GFhYfYOQwghHMrWrVvPaq2DLj7l+RwuUYSFhREdHW3vMIQQwqEopY5efKrSSdGTEEKIMkmiEEIIUSZJFEIIIcokiUIIIUSZJFEIIYQokyQKIYQQZbJZolBKfaGUOq2U2nWB8Uop9YFSKkYptUMp1cFWsQghhLh8tryj+AqjqdwL6Y/RDlAT4D6MjmiEEEKUo0KTJie/tFbdrWezF+601muVUmFlTDII+MbcWNxGpZS/UqpeVWhgSwghykNOfiFn0nPZHZ/KkcQs9iWkYdKQnV/IgVPpHE3MKpfl2PPN7Aac23FMnHnYeYlCKXUfxl0HoaGhFRKcEEJUBlpr4pKzOZ6cReyZTJbuTCD2TCYn00rrx8gQWdcPP1cX/NLyidt1htoB3lcUgz0TRWndY5balK3WehYwCyAqKkqauxVCOB2tNUcSs9h8OJHTabnsik8l5nQGp9NzSc85t8faWj7uDG5XHy93F1rUq46fpxttQ/ypV8MTTzcXtNZERX3K4f1nefnla3n44c64ud152bHZM1HEcW7H5MEY/RcIIYTTybEoDkrKzCM+NZvEjDyOnM0k5kwGKVn5533H3bUaLetX57rmdahXw5OGAd40CvLF39u91GX8/fdxWreujZ+fB599NpDAQG9CQi6lR+bS2TNRLMLopHw20BlIlfoJIYQzyMkvZMuRJDYcSmTLkSS2HEkudTpPt2o0reNH0zp+BPl6ULeGJ9e3qEOj2r5U93TD3dW6540SE7N46qmVfPbZNqZMuYYXX+xF+/b1ym19bJYolFI/Ar2AQKVUHEYXiG4AWuuPMTpZH4DR2XgWcK+tYhFCiPJUaNIcPJ3O7hNp7IpPZV9COh5u1UjOzGPfyXRyC0znTB9Sy4smtf0Y2LYedat7EejrTn1/L3w8ruwUrLXmm2/+5YknVpCcnM2kSV2ZNKnrFc2zNLZ86mnkRcZrYIKtli+EEFfCZNLsjk/jwKl04pKzSck2iom2Hk0mJ99EXuG5ySDQ152IIF/ah/rTKMiXhgHe9G1Rl/BAH5vFOHnySt5882+6dg3h449vpHXrOjZZjsP1RyGEEOVBa01SZh6n0nKJS85i14lU1h48S0pWHqfTc8nKO//dg9Ba3rQN8celmqJX0yAaBvrQqn4Ngvw8Kizu7Ox8MjPzCQz0ZsyY9jRpUosxYzpQrVppzweVD0kUQginlpSZx9ajyUQfSSL6aDKHz2biWk1xOj231Okj6/oxsE19CrUmPNCHDqE1aRTkg7+3u9V1BraybFkMEyYspV27usyfP4JmzQJp1izQ5suVRCGEcBp5BSb2JqSxIy6F33efYtPhRPIL/3ui3tfDleCaXjTw9yKkljduLorgmsaTRHVreNK4tq8do7+w+Ph0Hn10GXPn7qFZswAefPCqCl2+JAohhEMqNGkOn83g138T2B2fyoFTGRxLOvdNZKXguua16deyLldHBBBS68pePLOHP/6I5ZZb5pCXV8jUqdcyaVJXPK6wEvxSSaIQQlR6h85k8FP0cRIz8th1IpW45Gwycs99CS0iyIc7OocSWsubNsH+tA2pgbe7457i8vMLcXNzoW3bugwY0IRXXulN48a17BKL425FIYRTOZ2ew+bDSWw7lkJWXiEr957CRanzmqpoUtsoJmpmfv/gqrCadI4IwMWGlbkVKS0tl+efX8WmTSdYv340gYHezJ49zK4xSaIQQlQIrTX7T6Wz5UgyKZl57D2Zxl8Hz1KtmiItOx9TicZ5wgN9yM4r5N5uYXi6udC3RR06hNa0T/AVQGvNvHl7eOSRZZw8mcH48VeRm1uIt7f9uw2SRCGEsImCQhMbY5P4ZsMRdsSlltqInadbNer7enHrVSH4e7kTEeRDZF0/Qmt5o5Rz3CFY48yZTO6++xd++y2G9u3rsnDhbVx1VQN7h1VMEoUQ4ooVmjQbYxOZtzWOo4mZHDiVcV4dQo8mgbRqUIPrW9ShVYMauLnY/0q5sqhe3YOzZ7N4771+TJjQCVc7P4ZbkiQKIcRlMZk0W44kMXPNIdYeOHPOuIhAH66NrE1ITS+GdgymUVDlfOzUntauPcq0aeuYP38Evr7ubNw41qYvzV0JSRRCCKvk5BeyYs8pVu49xYnkbKKP/tfQXXigD/1a1uWOzqEE1/SqUsVGl+rs2SwmTVrBV19tJyzMnyNHUmjVqnalTRIgiUIIUYZTaTl8/tdh/j50ll0n0s4Z1ym8Ft0bB3LrVSHUqe5ppwgdh9aaL7/czqRJK0hLy+Xpp7vz3HM98fZ2s3doFyWJQgiB1poz6bnEnM5gT0Ia/xxLZuvRZE6l/dfMRddGAfRtUYcb29QjyNdD7houw3ff7aBFiyA+/vhGWrasbe9wrCaJQogq6GRqDvP/iWPfyXQ2H048JyEU8XSrRo8mgYzuFs61kY5zUqtMsrLyefXVdYwbF0VwcHXmzx9BjRqelbqYqTSSKISoIkwmzdcbjjBzdQxnM/KKh/t6uNIh1J9WDWrQukENmtTxI7KuH55uLvYL1gksXXqQCROWcuRICg0a+PHAA1dRs6aXvcO6LJIohHBChSZNXHIWexPS2XnCaCAv5nRG8fgbW9djSIcG9GpW22neaK4s4uLSePTRZcyfv5fmzQP588976Nmzob3DuiKSKIRwEkmZeXy74SibDify96HEc8YF1/Sia6MAosJqMe6aCIduA6mymzZtLUuWHOTVV3szcWJX3N0d/85MGR3NOY6oqCgdHR1t7zCEsLtdJ1KZG32cmDMZbIpNosCiDYz+repSr4YXV0fUom2IvzyVZGObN5/Ay8uV1q3rkJiYRWpqLhERlau5EaXUVq111OV8Vy4rhKjktNYcT8pm78k0ElKyWb7nFNFHks/pirNvizqE1vKmZ9MgejQOdLjKUkeVmprDM8/8wUcfRXPTTU1ZtGgkAQHeBAQ4XnPmZZFEIUQllJVXwJIdCfy6I+G8t5693Fxo1aA63RoH0q9lXVo1qGGnKKsurTVz5uzmscd+5/TpTB56qBNTp/a2d1g2I4lCiEqg0Nwcxj/HktlwKJF1B88Wj2tWx48ODWvSo0kgobW8aVbXT9pJsrPvvtvBXXf9QlRUfRYvHknHjvXtHZJNSaIQwo72nUxj6Y4EPlgVUzzMy82FyLp+DO0QzO2dQ/Gp4N7MROlycwuIjU2mefMgRoxoSUGBibvuaotLFUjacgQKYQfxKdk8s2Ana/YbxUrhgT70bBLIhN6Nqe0nFc+VzerVh3nggSVkZeVz8OBDeHi4cu+97e0dVoWRRCFEBUpIzebdFQeY/88JCk2a8EAfPr6zI83q+tk7NFGK06czeeKJ5Xz77Q4iImoya9bACu+vujKoemssRAUzmTSztxznu41H2ZNgNKzXKMiH929rLxXRlVhMTBKdOn1KRkYezz7bg2ef7YGXV+VvwM8WJFEIYQO5BYX8dfAsmw4nMWttbPHwPpG1ebhPE9qG+NsxOlGWtLRcqlf3oFGjmowZ057Ro9vTvHmQvcOyK0kUQpST3IJCth5NZunOBL7beKx4eGRdP3o1q83YHuEE+nrYMUJRlszMPF5++U8+/fQfdux4gODg6rz55vX2DqtSkEQhxBXKKzCxYFscr/22j+Ss/OLh93QNY3S3cEKd7OUrZ/Trr/t58MHfOHYslTFj2jtEHxEVSRKFEJdpb0Iav+1MYMbqGIpaz3i4TxOGdwyWXt4cREGBiREj5rJgwT5atgxi3bp76d491N5hVTqSKIS4BIUmzbytx3l92X6SMo2mumt4uXF3l4bcd00jfKvgEzGOSGuNUgpX12rUq+fLa6/14bHHujhFA362IEe1EFaIOZ3O138fZfmek5xKy8XDtRpD2jfg9s6hdAitKW0rOZCNG+OYMGEpn346kA4d6jFz5o32DqnSk0QhRBlSsvJ4/Kd/WbXvdPGwx/s2Zdw1jXB3df43cp1JcnI2zzzzB598spX69f1ITs62d0gOw6aJQil1A/A+4AJ8prV+rcT4UOBrwN88zVNa66W2jEkIa/xzLJn/++Mgaw6cQWsY0r4Bo7uH07J+dal7cEBz5uzi4YeXcfZsFo8+ejUvvdQLPz95As1aNksUSikXYCbQF4gDtiilFmmt91hM9hzwk9b6I6VUC2ApEGarmIQoy9mMXD7/6zCfro0t7tuhgb8XT97QjEHtGtg5OnEl9u07S1iYP8uW3UH79vXsHY7DseUdRScgRmsdC6CUmg0MAiwThQaqm/+uAcTbMB4hzrMzLpVftp8gITWbpTtPFg+/tlkQr9zSmgb+jtnHcVWXk1PA66//RYcO9Rg4sBnPPNOD557rWSUa8LMFWyaKBsBxi89xQOcS07wILFdKPQT4ANeVNiOl1H3AfQChofLomrgyqVn5fLDqIMv3nOR40n/l1L0jazO2ezhXRwRI5bQDW7kylvHjl3DwYBITJ3Zh4MBmuLnJ00xXwpaJorRfWsl+V0cCX2mt31ZKdQG+VUq10lqbzvmS1rOAWWB0hWqTaIXTO5maw6tL97Lo3/9uXO+8OpT7ezYipJa8FOfoTp3K4PHHl/PDDztp3LgWy5ffSd++jewdllOwZaKIA0IsPgdzftHSGOAGAK31BqWUJxAInEaIcpCQms3M1TEs3B5Pek4BAO4u1Xj31nbc0KouLnLn4DRWrIhl3rw9vPBCT55+ugeenvJQZ3mx5ZbcAjRRSoUDJ4DbgNtLTHMM6AN8pZRqDngCZxDiCu06kcqHa2KK6x183F24q0tDekfWpkeTIEkQTuLff09y8GASw4a14I47WtOtWwjh4TXtHZbTsVmi0FoXKKUeBH7HePT1C631bqXUy0C01noRMBH4VCn1GEax1D1aaylaEpdFa82fB87w7Yaj/GF+7yGyrh8P92nCgNbypIszycjIY8qU1bz//ibCwvwZPDgSV9dqkiRsxKb3ZuZ3IpaWGPaCxd97gG62jEE4N601u+PTeGXJHjbGJhUPb1bHj2m3tCIqrJYdoxO28Msv+3jood+Ii0vjvvs6MH36dbjKy482JYV4wiElZebx/C+7WLIzoXhYSC0vhrQPZsRVIfJYq5PaufMUt9wyh9atazNnzjC6dg25+JfEFZNEIRzKvpNpzFgVw5r9Z8jILSAiyIdrm9Vm1NUNCQv0sXd4wgby8wtZt+4YvXuH07p1HZYsuZ2+fSPkkdcKJIlCOISkzDx+2XaClxcb72tGNazJpH7N6BwRYOfIhC39/fdxxo1bzO7dZ9i//0EaN67FgAFN7B1WlSOJQlRqJpNm0rwdzP8nDgA/D1e+vPcqqXtwcklJ2Tz11Eo+/fQfQkKq8/PPI2jcWPa5vUiiEJXWyj2neGfFAfYkpNEoyIexPSK4qU09/Dyl9zFnlpNTQLt2HxMfn87EiV148cVe+Pq62zusKk0ShaiUvtlwhBcW7gbggV6NeLJfM2m11cnFxaURHFwdT09Xpk69lnbt6tK2bV17hyWQRCEqmdX7TvPG7/vZm5BG49q+zL2/CzV95GrSmWVn5zN9+l+8/vp65s0bzsCBzbj77nb2DktYsCpRKKXcgVCtdYyN4xFVkNaa33ad5OM/D7EjLhUwKqs/GdVRkoSTW778EOPHL+HQoWTuvLMNnTpJc+6V0UUThVLqRuAdwB0IV0q1A6ZorW+xdXDCuZ1Ky2HR9nim/7YXc/cPdG0UwKu3tJZHXauAhx5ayowZW2jSpBYrV46iT58Ie4ckLsCaO4qXMZoHXw2gtd6ulGps06iEU9t1IpVP18WycLvRRqSPuwt3dmnI7Z1CaRggCcKZFRYaDUO7uFTj6quDCQz0ZvLk7tKAXyVnzd7J11qnlKhIlPaYxCVbe+AMby3fX1y81DbEn4d7N6ZXs9rSSF8V8M8/CYwbt5hRo9rw0EOdueOONvYOSVjJmkSxVyk1Aqhmbgn2EWCjbcMSzuazdbG8smQvHq7VGNK+AQ/1aUK4FC9VCenpubzwwmo++GAzQUHe1KvnZ++QxCWyJlE8CLwAmICfMVqDfdqWQQnnUNSa64uLdnMkMYu2If58c28nanjLexBVxfLlhxg9eiHx8emMGxfFq6/2wd/f095hiUtkTaLop7WeDEwuGqCUGoKRNIQ4R2ZuAV/8dZgF204QezazeHin8Fp8OipKkkQV4+7uQu3aPsyfP4LOnYPtHY64TOpi3T8opf7RWncoMWyr1rqjTSO7gKioKB0dHW2PRYsypOfk8/byA3y38SgF5keYmterTqewmoztESFdjVYR+fmFvPPOBtLScpk2rQ9gNMMifZDbn/m8HXU5373gHYVSqh9GN6UNlFLvWIyqjlEMJQRZeQV8tOYQ/7fKeMUmItCHx/o2pX+ruri6SB8BVclffx0rbsBv+PAWxQlCkoTjK6vo6TSwC8gBdlsMTweesmVQonIzmTQbDyfy1u/7+edYSvHwt4a3ZWiHBtLURhWTmJjF5Mkr+fzzbYSG1uDXX0dy001N7R2WKEcXTBRa623ANqXU91rrnAqMSVRS62PO8tm6WFbv/69bc9dqionXN2Nsj3Dc5A6iSkpMzGb27F08+WRXXnjhGnzkbXqnY01ldgOl1DSgBVD8uILWWi4ZqohCk6DMJ6IAACAASURBVOb+b7eycu8pANoG16Bb40CGdQwmIsjXztEJe9i79ww//bSbKVN60bRpAMeOPUatWtKroLOyJlF8BbwCvAX0B+5F6iiqhNTsfN5beYAfNx8jJ9/Y5euevFYqpquwrKx8pk1by5tv/o2vrztjxnQgOLi6JAknZ02i8NZa/66UektrfQh4Tim1ztaBCfvacCiRkZ8a71X6uLvw0s0tGXV1Q6mYrMKWLYth/PglHD6cwt13t+XNN/sSFCQvTVYF1iSKXGXUTh5SSo0DTgC1bRuWsIe8AhMv/bqb7zcdKx42vlcjJklfEFVeRkYeo0YtICDAi9Wr76ZXrzB7hyQqkDWJ4jHAF3gYmAbUAEbbMihR8fbEpzH++60cScwCYEiHBgzrGEzXRoF2jkzYS2GhiR9/3MXIka3w9XVn5cpRREYG4uEhDfhVNRfd41rrTeY/04FRAEopecXSSeQWFDJjVUzxexATrm3EQ72b4OnmYufIhD1t3RrP/fcvZuvWBLy8XBk6tIX0NleFlZkolFJXAQ2Av7TWZ5VSLTGa8ugNSLJwcOtjzvK/b6LJyisEYN64LkSFSQf2VVlqag7PP7+amTO3ULu2D7NnD2XIkOb2DkvYWVlvZk8HhgL/YlRgL8BoOfZ1YFzFhCfKm8mkWbIzgQ/XHGJvQhoA9/WM4PG+TeUuQjB06E+sWnWYCROu4pVXelOjhjTgJ8q+oxgEtNVaZyulagHx5s/7KyY0UZ4KTZr3/zjI4h3xxJ4xGusb3jGYR/s2pYG/PNpYlcXGJhMU5I2fnwfTpvWmWjXFVVdJl6TiP2UlihytdTaA1jpJKbVPkoRjWrXvFM/8vIuTacYL9vd0DeOp/pFyB1HF5eUV8tZbfzN16loefrgTr7/eV1p4FaUqK1FEKKWKmhJXQJjFZ7TWQ2wambhi6Tn5jP/+H9YdPAvAyE4hTBvcWt6FEKxde5Rx4xazd+9Zhg1rwcMPd7Z3SKISKytRDC3xeYYtAxHlp6DQxEu/7mH2lmPkF2qubRbEW8PbEuDrYe/QRCXw7rsbePzx5YSF+bNkye0MGNDE3iGJSq6sRgH/qMhAxJXLLzTx/sqDzFoXS16BCZdqindGtGVIBylOqOpMJk1mZh5+fh7ceGNTzpzJ4rnneuItHUkJK8ibM05i7YEzPPvLTo4nZRPg485zQ5tzS3tJEAJ27z7NuHFLinuaa9o0gFdf7WPvsIQDsWmiUErdALwPuACfaa1fK2WaEcCLgAb+1VrfbsuYnE1SZh5PzP2XVftOAzC2ezjP3dTCzlGJyiArK5+pU//krbc2UKOGB6NHt0NrLc2xiEtmdaJQSnlorXMvYXoXYCbQF4gDtiilFmmt91hM0wR4GuimtU5WSkkbUlZKycpj2pK9zN0aVzzslwndaBfib8eoRGWxbVsCQ4b8xJEjKdx7bzveeKMvgYHS6q+4PBdNFEqpTsDnGG08hSql2gJjtdYPXeSrnYAYrXWseT6zMd7N2GMxzf+AmVrrZACt9elLX4WqJTO3gGcW7GTh9vjiYW8Oa8PwqBA7RiUqi6I7htDQGoSG1uDrrwfTs2dDe4clHJw1dxQfADcBvwBorf9VSl1rxfcaAMctPscBJZ/BawqglFqPUTz1otZ6mRXzrpJOpubQ7721pGbnA/DqLa257aoQedxVUFBgYsaMzSxatJ8VK0YREODNn3/eY++whJOwJlFU01ofLVGuWWjF90o7e+lSlt8E6IXRdtQ6pVQrrXWK5URKqfuA+wBCQ0OtWLRz0VqzYNsJJs/fQX6hZsrAFtzdJUwShABg8+YTjBu3mG3bTtK/f2PS0nKpWVPethflx5pEcdxc/KTN9Q4PAQes+F4cYFkeEozRDEjJaTZqrfOBw0qp/RiJY4vlRFrrWcAsgKioqJLJxqnlFhQy9uvo4pfmZo3qyPUtpRVPYfQRMXnyCj76KJp69fyYO3c4Q4c2l8pqUe6sSRQPYBQ/hQKngJXmYRezBWiilArH6OzoNqDkE02/ACOBr5RSgRhFUbHWhe78Fm4/waR5O8grMNG9cSDv3NqW2n7SSJswuLlVY82aozz0UCemTu1N9eryQqWwDWsSRYHW+rZLnbHWukAp9SDwO0b9wxda691KqZeBaK31IvO465VSezCKsyZprRMvdVnOptCkmTx/B/O2xhHo686rt7SWuwgBQExMEi+//CczZw7Az8+DrVvvw9NTXocStqW0LrskRyl1CNgPzAF+1lqnV0RgFxIVFaWjo6PtGYJNbYxN5PE524lPzaFzeC3eHtGW4JryWGNVl5tbwBtvrGfatHW4u7uwZMnt9OghTzMJ6ymltmqtoy7nu9b0cNdIKdUVo+joJaXUdmC21nr25SxQXNgnfx5i+m/7AHiqfyT394yQ8mbB6tWHeeCBJezfn8itt7bknXf6Ub++n73DElWIVfesWuu/gb+VUi8C7wHfA5IoyklBoYmnft7JvK1xhAV48+EdHWlRv7q9wxKVgNaaadPWkZ9vYtmyO+jXr7G9QxJVkDUv3PlivCh3G9AcWAh0tXFcVYbWmjs/38TG2CRa1KvOoge74epSzd5hCTsymTSff/4PN9zQmJCQGnz77S34+3vi5SUN+An7sOaMtAu4GnhDa91Yaz1Ra73JxnFVGe+uPMjG2CQGtq3Pkoe7S5Ko4nbsOEX37l9w332L+eyzfwCoV89PkoSwK2uKniK01iabR1IFTf9tL5/8GUuXiAA+uK2d1EdUYRkZebz00hrefXcjNWt68dVXg7jrrrb2DksIoIxEoZR6W2s9EZivlDrv0Sjp4e7yJWfmMWnev6zce5rwQB8+urODJIkq7sUX1/D22xsYO7Y9r712HQEB8qSbqDzKuqOYY/5ferYrRwu2xfHYnH8BaNWgOvPGdZW+q6uo48dTyczMJzIykKee6s7gwZF07171mqgRlV9ZPdxtNv/ZXGt9TrIwv0gnPeBdonHfbmXZ7pMAvH9bOwa1a2DniIQ9FBSY+OCDTbzwwmo6dqzPn3/eQ2CgtyQJUWlZU3M6upRhY8o7EGf37IKdLNt9kjrVPdj54vWSJKqojRvjiIqaxcSJy+nVK4yvvx5s75CEuKiy6ihuxXgkNlwp9bPFKD8gpfRviZIycwt48Id/WL3/DEF+Hvw1uTdu8mRTlbRkyQEGDvyR+vX9+PnnEQweHCl1U8IhlFVHsRlIxGj1dabF8HRgmy2DchapWflc+/YakjLzGNSuPm8PbyuPv1YxWmvi49Np0KA6110XwcsvX8sjj3TGz08a8BOO46JtPVU2jtLW0yd/HmLW2liSsvJ4fUgbRlwlPdBVNQcOJDJ+/BIOHEhkz54J+Pq62zskUYXZpK0npdSfWutrlFLJnNvhkAK01rrW5SywKnhx0W6++vsI9Wt48vndUfSOrGPvkEQFyskp4LXX/mL69L/w8nJl+vQ+eHlJC6/CcZV19BZ1dxpYEYE4g0KTZsQnG9h6NBmANZOuxd1VipqqkpMnM+jZ80sOHkxi5MhWvPNOP+rW9bV3WEJckbIejy16GzsEiNda5ymlugNtgO+AtAqIz2Hk5Bcy4P11xJ7NBGD9U70lSVQh+fmFuLm5UKeODz17NmTmzAH07dvI3mEJUS6sOZP9gtENaiPgG4yGAX+waVQOZlNsIm1eWk7s2Ux6Ng3i8PQBNPCXPourApNJ8/HH0TRq9AFxcWkopfjss5slSQinYk3BqUlrna+UGgK8p7X+QCklTz2ZnUzN4c7PN5FfqJnYtykP9Wli75BEBfn335Pcf/9iNm06Qe/e4eTnF9o7JCFswqquUJVSw4FRQNHbQdKUJZBfaOL+b6MpMGkWTuhG2xB/e4ckKoDWmkmTVvDeexupVcuLb7+9hTvuaC3vRAinZU2iGA2Mx2hmPFYpFQ78aNuwKjetNRtiE5mycDcHT2fw3I3NJUlUIUopkpOzGTPGaMCvZk0pZhTOzar3KJRSrkBR11oxWusCm0ZVBnu/R5GdV0jzF5YB4OfhyvM3tZB3JKqAo0dTeOSRZbzwwjV06FAPk0lTrZrcQQjHcSXvUVy0Mlsp1QOIAT4HvgAOKKW6Xc7CHF2hSXPtW2sAqKZg/dO9JUk4ufz8Qt54Yz0tWnzIihWx7N9/FkCShKhSrCl6ehcYoLXeA6CUag58C1xWZnJkH62J4WRaDgPb1peOhqqAv/8+zv33L2bXrtMMGtSMDz7oT2hoDXuHJUSFsyZRuBclCQCt9V6lVJVriyA1O5/3/zgIwJvD2kiSqAJWrowlNTWHX365lUGDIu0djhB2Y02i+Ecp9QnGXQTAHVSxRgGz8wq5atpK8gs1s0Z1lI6GnJTWmm+/3UFQkDf9+zdh8uRuPP54F2mjSVR51rxwNw44BDwJTAZigfttGVRlorXRLEdegYmb2tTj+pZ17R2SsIF9+87Su/c33H33L3z55XYAPDxcJUkIwUXuKJRSrYFGwAKt9RsVE1LlobWm//vr2HcynTuvDuWVwa3tHZIoZ9nZ+bz66jpef309Pj7ufPLJTYwd28HeYQlRqVzwjkIp9QxG8x13ACuUUqX1dOe0tNa8unQv+06mE+Djzgs3tbR3SMIGfv31AK+8so5bb23Fvn0TuO++jvJEkxAllHVHcQfQRmudqZQKApZiPB5bJUz/bR+frjuMm4uSBv6czMmTGWzffpIbbmjM8OEtCAsbS6dO0jWtEBdS1tkvV2udCaC1PnORaZ3K34fOMmttLF0iAtg/tb9UXjuJwkITH364hWbNZjBq1AKys/NRSkmSEOIiyrqjiLDoK1sBjSz7ztZaD7FpZHYSn5LN6K+2UKe6B28ObyPFEE7in38SGDduMVu2xHPddRF8+OEAvLykyTIhrFFWohha4vMMWwZSWUz/bR85+SY+u6sdwTW97R2OKAeHDyfTqdOnBAZ688MPQ7jttlbyHowQl6Csjov+qMhAKoN9J9P49d94+reqS/cm0rGfI9Nas3Pnadq0qUN4eE2+/HIQAwc2w9/f096hCeFwqky9gzXGfm00Nvh0/+Z2jkRcicOHk7npph9p3/4Tduw4BcCoUW0lSQhxmWyaKJRSNyil9iulYpRST5Ux3TCllFZK2a39qJmrY4hLzkYpCA2QIidHlJdXyGuv/UXLlh/y559HeOutvrRoEWTvsIRweNY04QGAUspDa517CdO7ADOBvkAcsEUptciy3SjzdH7Aw8Ama+dd3uJTspmxKoaa3m6smtjLXmGIK1BYaKJr18/ZujWBIUOa8957/QgJkQb8hCgP1jQz3kkptRM4aP7cVin1f1bMuxNG3xWxWus8YDYwqJTppgJvADnWh12+3l5+gOz8Qj4ZFUVNH2mywZGkpRnXLi4u1Rg9uj2//jqS+fNHSJIQohxZU/T0AXATkAigtf4XuNaK7zUAjlt8jjMPK6aUag+EaK0XlzUjpdR9SqlopVT0mTNnrFi09fafTGf+P3H0aBJIp/Ba5TpvYTtaa776ajsREe+zcOE+AMaPv4qbbmpq58iEcD7WJIpqWuujJYZZ04t8ac8fFnenp5SqhtHXxcSLzUhrPUtrHaW1jgoKKr8y54JCE+O/34q7SzWmDGxRbvMVtrVnzxl69fqae+9dSGRkII0aSYIXwpasqaM4rpTqBGhzvcNDwAErvhcHWHb/FgzEW3z2A1oBa8zPtNcFFimlbtZaV0hfp7/vPsWhM5k8d2NzGtf2q4hFiiv0xhvrefbZVVSv7sFnnw3k3nvby0uRQtiYNYniAYzip1DgFLDSPOxitgBNlFLhwAngNuD2opFa61Sg+GUFpdQa4ImKShIArywx6tXv6NywohYpLpPWGqUUdev6cscdrXnzzb4EBfnYOywhqoSLJgqt9WmMk/wl0VoXKKUeBH4HXIAvtNa7lVIvA9Fa60WXHG05WnvgDAmpOdzYuh5e7tKWU2UVH5/OI48so0ePUB5+uDN33dWWu+5qa++whKhSLpoolFKfYlG3UERrfd/Fvqu1XorR6qzlsBcuMG2vi82vPP22KwGAqYNbVeRihZWKGvB79tlV5Oeb6No12N4hCVFlWVP0tNLib0/gFs59mskhrdp3mqiGNaklj8NWOtu3n2Ts2EVs3ZrA9dc34sMPB0iFtRB2ZE3R0xzLz0qpb4EVNouoAmw/nsKptFzulLqJSik1NYf4+HTmzBnG8OEtpAE/IezM6jezLYQDDn2G/Wq90SHRyM6h9g5FYFRUz527h4MHE3n22Z5cc00YsbGP4Ol5OYenEKK8WfNmdrJSKsn8LwXjbuIZ24dmG2fSc/llezzXNqtNoK+HvcOp8g4dSmLAgB+49dZ5LFy4n/x84xUdSRJCVB5l/hqVcc/fFuPxVgCT1vq8im1H8uGaGAAe6NXIzpFUbbm5Bbz11t+88so63Nyq8f77NzB+/FW4SpezQlQ6ZSYKrbVWSi3QWnesqIBsSWvNl+uP0DbEn/ahNe0dTpV2/HgaU6euZeDAZrz3Xj8aNKhu75CEEBdgzeXbZqVUB5tHUgG2H08B4Gpp08kuzpzJZMaMzQA0blyLPXsmMHfucEkSQlRyF7yjUEq5aq0LgO7A/5RSh4BMjDactNba4ZLHy4v34O5aTYqdKpjJpPnyy208+eRK0tNz6ds3gmbNAomIkLs6IRxBWUVPm4EOwOAKisWmUrLy2HYshQ6h/vh7y7sTFWXXrtM88MAS/vrrGD16hPLxxzfRrJl0MyuEIykrUSgArfWhCorFpuZGxwHwWF9phrqi5OUVcv3135KXV8gXX9zMPfe0k3cihHBAZSWKIKXU4xcaqbV+xwbx2Mymw4n4e7vRo4l0jWlrq1Yd5pprGuLu7sJPPw0nMjKQwEDpXlYIR1VWZbYL4IvRHHhp/xxGRm4BK/eepnez2vYOxanFxaUxdOhP9OnzDd988y8A3buHSpIQwsGVdUeRoLV+ucIisaHZm48B0KOplI3bQkGBiRkzNvP886spLDQxfXof7rijjb3DEkKUk4vWUTiDFXtO4VJNcXPbBhefWFyyUaMWMHv2Lvr3b8zMmQMID5enmYRwJmUlij4VFoUNbT+ewqbDSYzpHo6L9IRWblJScnB1rYavrzsTJlzF0KHNGTq0uVRWC+GELlhHobVOqshAbGXOFqNF9Hu6htk3ECehtWb27F00bz6T559fBRj1EMOGSSuvQjgrp29YZ/7WOLo3DiSkllSoXqmYmCT69fuOkSPnExxcnTvvlHoIIaoCp26i89/jKeQVmmgYIEniSv3ww05Gj16Ih4crM2b0Z9y4KFxcnP46QwiBkyeKFXtOATCme7idI3Fc+fmFuLm5EBVVn2HDWvDGG32pX9+hno4WQlwhp04UX6w/TEgtLyKCfO0disM5fTqTiROXk5mZx88/30rTpgF8990Qe4clhLADpy07yC80kZVXiLebU+fCcmcyaWbN2kqzZjOYM2cXLVsGUVhosndYQgg7ctqzaFGx04O9G9s5EscRG5vMnXf+zIYNcfTqFcZHH91IZKS8pChEVee0iWLLEePp3v6t6to5EsdRo4YHKSk5fP31YEaNaiOPuwohACcuevo7JpHIun64ypM5ZVq0aD9DhsyhsNBEQIA3u3aN56672kqSEEIUc8qzaE5+IftPpVOnuqe9Q6m0jh1LZfDg2QwaNJsDBxJJSMgAoJq8vS6EKMEpi55mrY0FpNipNAUFJt57byNTpqxBa83rr1/HY49djZubi71DE0JUUk6ZKDYfNuonhnQItnMklU9hoYnPPvuH3r3D+b//609YmL+9QxJCVHJOV/SkteavmLN0Dq+Fu6vTrd5lSU7OZvLkFaSn5+Lh4cr69aNZtOg2SRJCCKs43Zl09f7TAPSOlE6KtNZ8//0OIiNn8vbbG1i9+ggAAQHeUlkthLCa0xU9fbbuMAB3dQmzbyB2duBAIuPHL+GPPw7TqVMDfv/9Ttq1kzobIcSlc6pEkZSZx9+HEukcXgsv96pdOfvoo8uIjo7nww8HcN99HaUBPyHEZXOqRLFw+wkAxvaIsHMk9rFixSEiIwMJCanBRx/diIeHK3XrSjtXQogrY9PLTKXUDUqp/UqpGKXUU6WMf1wptUcptUMp9YdSquGVLG/5bqPZjqpWP3HyZAa33z6f66//jtdfXw9Aw4b+kiSEEOXCZolCKeUCzAT6Ay2AkUqpFiUm2wZEaa3bAPOANy53eVprNsQm0sDfq8p0eWoyaT7+OJrIyBnMn7+XKVOu4a23rrd3WEIIJ2PLO4pOQIzWOlZrnQfMBgZZTqC1Xq21zjJ/3Ahc9osP+06mA3Bjm3qXOwuHM336Oh54YAkdO9Znx45xvPhiLzw9nao0UQhRCdjyrNIAOG7xOQ7oXMb0Y4DfShuhlLoPuA8gNDS01C8fTcwE4JqmQZceqQNJT8/l7NkswsNrMm5cFOHhNRk5spU87iqEsBlb3lGUdubSpU6o1J1AFPBmaeO11rO01lFa66igoNITwa4TaSgF7UKc8yUyrTULFuylRYsPufXWeWitCQjw5vbbW0uSEELYlC0TRRwQYvE5GIgvOZFS6jrgWeBmrXXu5S5s85EkGgX54uPhfEUvR4+mcPPNsxky5Cdq1fLigw/6S3IQQlQYW55VtwBNlFLhwAngNuB2ywmUUu2BT4AbtNanL3dB+YUmNh9O4ua29a8k3kppw4bjXHfdtwC89VZfHnnkalylaRIhRAWyWaLQWhcopR4EfgdcgC+01ruVUi8D0VrrRRhFTb7AXPMV8jGt9c2XuqyY00YT2c5U7JSWlkv16h506FCP0aPbMWlSN0JDa9g7LCFEFWTTchqt9VJgaYlhL1j8fV15LGffyTQAOoXXKo/Z2VViYhZPPbWS5ctj2b17PL6+7vzf/w2wd1hCiCrMKQr0txxJxsO1Gk3r+Nk7lMumtebbb3cwceJykpOzefzxLkg1hBCiMnCKRLE3IY02wTUctlnx1NQcBg+ew5o1R+jSJZiPP76JNm3q2DssIYQAnKCZ8YJCE3sT0qjv72XvUC6Z1sbTwtWrexAY6M2sWTfx11+jJUkIISoVh08Uh85kkpNvIqphTXuHckl+/z2GDh1mEReXhlKKuXOH87//dZQ+q4UQlY7DJ4ptx5IBCK7pbedIrJOQkM5tt83jhhu+Jysrn9OnM+0dkhBClMnh6yg2HzH6x27rAI/Gzpy5mWeeWUVubgEvvdSLyZO74eGELwgKIZyLw5+l4pKyAajl427nSC5u69YEOnduwMyZA2jSJMDe4QghhFUcPlEcScystC/apaXl8sILqxk1qg0dO9bnww9vxMPDRZrfEEI4FIdOFNFHkjidnsv/KlmPdlpr5s/fyyOPLCMhIZ3Q0Bp07FhfmgAXQjgkhz5zLd6RAMANreraOZL/HD6czIMP/sbSpQdp164uP/88gs6dL7ubDSGEsDuHThRFTXcE16w871B8//1O1q49yrvv9uPBBztJA35CCIfn0IniREo2kXX97F7mv27dUXJzC7nuuggmTerKPfe0Izi4ul1jEkKI8uKwl7vJmXkcT8rm2sjadovh7NksRo9eSM+eX/Hyy38C4OHhKklCCOFUHPaOYntcCgBNavtW+LK11nz11XYmTVpBamoukyd34/nne1Z4HKLyy8/PJy4ujpycHHuHIqoIT09PgoODcXNzK7d5Omyi2BNv1E90axxY4cteuvQgo0cvolu3ED7++CZatbLfXY2o3OLi4vDz8yMsLMzuRaTC+WmtSUxMJC4ujvDw8HKbr8MWPc3ZchwvNxfqVPeskOVlZeWzfv0xAAYMaMLChbexdu29kiREmXJycggICJAkISqEUoqAgIByv4N12ERxLCmL7PzCClnWb78dpFWrD+nf/3tSUnJQSnHzzc2kAT9hFUkSoiLZ4nhz2ETh4+5Cfxu/P3HiRBrDh89lwIAf8PBw5ddfR+LvXzF3MEIIUVk4ZKJIy8knM6+QlvVt93TR6dOZtGjxIYsXH+CVV67l33/Hcc01YTZbnhC24uLiQrt27WjVqhUDBw4kJSWleNzu3bvp3bs3TZs2pUmTJkydOrW4nxSA3377jaioKJo3b05kZCRPPPGEPVahTNu2bWPs2LHnDBs0aBBdunQ5Z9g999zDvHnzzhnm6/vfwzAHDhxgwIABNG7cmObNmzNixAhOnTp1RbElJSXRt29fmjRpQt++fUlOTj5vmtWrV9OuXbvif56envzyyy8A9OjRo3h4/fr1GTx4MACLFy9mypQpVxTbJdFaO9S/jh076mOJmbrh5MV6zpZjurzFxaUW//3++xt1TExiuS9DVB179uyxdwjax8en+O+77rpLv/LKK1prrbOysnRERIT+/ffftdZaZ2Zm6htuuEHPmDFDa631zp07dUREhN67d6/WWuv8/Hw9c+bMco0tPz//iucxbNgwvX379uLPycnJOjg4WEdGRurY2Nji4XfffbeeO3fuOd8t2jbZ2dm6cePGetGiRcXjVq1apXfu3HlFsU2aNElPnz5da6319OnT9ZNPPlnm9ImJibpmzZo6MzPzvHFDhgzRX3/9tdZaa5PJpNu1a1fqdFqXftwB0foyz7sO+dTTbvMTT55uLuU2z9TUHJ57bhWffLKVjRvH0qFDPR5+uHO5zV+Il37dXfy0XnlpUb86Uwa2tHr6Ll26sGPHDgB++OEHunXrxvXXXw+At7c3M2bMoFevXkyYMIE33niDZ599lsjISABcXV0ZP378efPMyMjgoYceIjo6GqUUU6ZMYejQofj6+pKRkQHAvHnzWLx4MV999RX33HMPtWrVYtu2bbRr144FCxawfft2/P2Nxj0bN27M+vXrqVatGuPGjePYMeMhkvfee49u3bqds+z09HR27NhB27Zti4fNnz+fgQMHUqdOHWbPns3TTz990e3yww8/0KVLFwYOHFg87Nprr7V6F8/i4QAAD45JREFUu17IwoULWbNmDQB33303vXr14vXXX7/g9PPmzaN///54e5/bv056ejqrVq3iyy+/BIx6iF69erF48WJGjBhxxXFejEMmiuNJWQC0C77yVmO11sydu4dHH13GyZMZPPhgJxo1cqze8oSwRmFhIX/88QdjxowBjGKnjh07njNNo0aNyMjIIC0tjV27djFx4sSLznfq1KnUqFGDnTt3ApRavFLSgQMHWLlyJS4uLphMJhYsWMC9997Lpk2bCAsLo06dOtx+++089thjdO/enWPHjtGvXz/27t17znyio6Np1arVOcN+/PFHpkyZQp06dRg2bJhViWLXrl3nbYvSpKen06NHj1LH/fDDD7Ro0eKcYadOnaJevXoA1KtXj9OnT5c5/9mzZ/P444+fN3zBggX06dOH6tX/K26Piopi3bp1kiguZOvRZNxdqhEacGW92mmtGTLkJ375ZR8dOtRj0aKRREXVL6cohTjXpVz5l6fs7GzatWvHkSNH6NixI3379gWM4/9CT8hcypMzK1euZPbs2cWfa9a8+IXW8OHDcXExSgRuvfVWXn75Ze69915mz57NrbfeWjzfPXv2FH8nLS2N9PR0/Pz8ioclJCQQFBRU/PnUqVPExMTQvXt3lFK4urqya9cuWrVqVeo6XeoTQn5+fmzfvv2SvmOthIQEdu7cSb9+/c4b9+OPP55XD1O7dm3i4+NtEktJDlmZnV9oIq/QdPnfNz9Wq5Sie/cQPvjgBjZvHitJQjglLy8vtm/fztGjR8nLy2PmzJkAtGzZkujo6HOmjY2NxdfXFz8/P1q2bMnWrVsvOv8LJRzLYSWf6/fx8Sn+u0uX/2/v7oOrqu88jr8/S4UEi9k1LIwt2sQxhcQQkIcW7VBEWsYKhYVxRJ6UHbuOINIWxWEHZ9ZdH0pbQRfRReo6SWsbIk4FBhAW2SgVeRC2ERRsmtIM4DCaxixLUR4C3/3jnCSXPNwcIvcmN/m+Zu7Mveeeh+/9zr3nd8/vnPP93UhFRQVVVVWsWbOGyZMnA3D+/Hl27NhBWVkZZWVlfPTRRxc0EnWfLXbdJSUl1NTUkJ2dTVZWFpWVlfWNWGZm5gVHO59++im9e/euz0WUz3rixIkLTjzHPmIbtTp9+/bl2LGgyvWxY8fo06fl+65eeeUVJk2a1OSO6urqanbv3s24ceMumH7q1CnS05NTEDUlG4q3yqsYmdO2O7LffLOSgoIVrF37IQAPPngTDzzwTbp1S8lUOBdZRkYGy5Yt46mnnuLs2bNMnz6dt99+mzfeeAMIjjzmzZvHww8/DMCCBQt48sknKS8vB4Id99KlS5usd+zYsSxfvrz+dd3OuG/fvhw8eLC+a6klkpg0aRLz588nNzeXzMzMZtfb3D/53NxcKioq6l8XFxezadMmKisrqaysZO/evfUNxc0330xJSQlnzpwBoLCwsP48xLRp03jnnXfYsGFD/bo2bdpU351Wp+6IorlH424ngAkTJlBUVARAUVEREydObDEPxcXFTJ06tcn01atXM378eNLSLrw0v7y8vEm3W6Kk3N6x9rxRe97Iyry89ZljVFWd5O671zB6dBGnT9fSq1ePBEXoXMd1ww03MGjQIFatWkV6ejpr167l8ccfp3///gwcOJDhw4czd+5cAAoKCnjmmWeYOnUqubm55Ofn1/87jvXII49QU1NDfn4+gwYNorS0FIDFixczfvx4brnllvp++pZMmTKFl19+ub7bCWDZsmXs2bOHgoIC8vLyWLFiRZPlBgwYwPHjxzlx4gSVlZUcPnyYESNG1L+fnZ3NFVdcwa5duxg/fjwjR45k6NChDB48mO3bt9efWE5PT2f9+vU8++yz5OTkkJeXR2FhYdwjgCgWLlzIli1byMnJYcuWLSxcuBAIzq3EdiVVVlZy5MgRRo0a1WQdq1atarYBKS0tbXKUkSiymGumU8GAgYPt1Lgn+MVdw/huXt9IyxQX7+f++zfy17+eYcGCm1i06Nv07HnpCmY515KDBw+Sm5vb3mF0ak8//TS9evVq0offmX388cdMmzaNrVu3Nvt+c987SXvNbFhbtpdyRxRnaoNzE1dlRL9Durb2PPn5fSgru48nnhjjjYRzncjs2bPp0aNr9RAcPnyYJUuWJG17KXfV04lTtWT36hH3ruyTJ8/w2GPbuOaaDObMGc6MGQXMmFHgNXec64TS0tKYOXNme4eRVMOHD0/q9lLuiOLk6VqGZ1/Z4k5//fpyrr/+eX760+2Ul1cDwckybyRce0m17l2X2hLxfUu5I4pzZlxzZdP7J44e/T/mzXud1177kLy8v2fbtlmMHPm1dojQuQZpaWlUV1d7qXGXFBaOR9H4CqkvKuUaCoBeaU3DPnSohs2b/8RPfjKG+fNvpHv3S1few7m26tevH0ePHqWqqqq9Q3FdRN0Id5dSyl311OOqHHtr+05GXJvJ7t0fsWPHEX74w+ByuOrqz8j8gndrO+dcZ9Rhr3qSdKukP0iqkLSwmfd7SCoJ398lKSvKevv0uIw5czYwYsSLLF26k5MngxtovJFwzrlLL2ENhaRuwHPA94A8YKqkxrcu3gPUmNl1wNNAy2UVY9w0dCUvvLCXefO+yf79s7n88u6XMnTnnHMxEnmO4htAhZkdApC0CpgIxBZEmQg8Gj5/FVguSRanP8xqz3N1VgYbN05nyJD4d3s655z74hLZUHwVOBLz+ijQeICH+nnMrFbScSAT+EvsTJLuBe4NX57e85d7349QEbgr6E2jXHVhnosGnosGnosG/du6YCIbiuauBWx8pBBlHsxsJbASQNKetp6Q6Ww8Fw08Fw08Fw08Fw0k7Wl9ruYl8mT2UeDqmNf9gMbF0+vnkfQlIAP4NIExOeecu0iJbCjeBXIkZUvqDtwJrGs0zzrg7vD57cB/xzs/4ZxzLvkS1vUUnnOYC2wGugEvmdkHkv6NYJDvdcB/Ar+SVEFwJHFnhFWvTFTMKchz0cBz0cBz0cBz0aDNuUi5G+6cc84lV8oVBXTOOZdc3lA455yLq8M2FIkq/5GKIuRivqQDkvZJ2iqp05bNbS0XMfPdLskkddpLI6PkQtId4XfjA0m/SXaMyRLhN3KNpFJJvw9/J7e1R5yJJuklSZ9Ier+F9yVpWZinfZKGRFqxmXW4B8HJ7z8B1wLdgfeAvEbzzAFWhM/vBEraO+52zMVooGf4fHZXzkU4Xy9gG7ATGNbecbfj9yIH+D3wd+HrPu0ddzvmYiUwO3yeB1S2d9wJysW3gSHA+y28fxvwOsE9bCOAXVHW21GPKOrLf5jZGaCu/EesiUBR+PxVYIw6Z8H/VnNhZqVm9ln4cifBPSudUZTvBcBjwM+AU8kMLsmi5OKfgOfMrAbAzD5JcozJEiUXBtQNi5lB03u6OgUz20b8e9EmAr+0wE7gbyW1WgupozYUzZX/+GpL85hZLVBX/qOziZKLWPcQ/GPojFrNhaQbgKvNbH0yA2sHUb4XXwe+Lmm7pJ2Sbk1adMkVJRePAjMkHQU2Ag8kJ7QO52L3J0DHHbjokpX/6AQif05JM4BhwKiERtR+4uZC0t8QVCGelayA2lGU78WXCLqfbiY4yvydpHwz+98Ex5ZsUXIxFSg0syWSbiS4fyvfzM4nPrwOpU37zY56ROHlPxpEyQWSvgMsAiaY2ekkxZZsreWiF5APvCmpkqAPdl0nPaEd9Tey1szOmtmfgT8QNBydTZRc3AO8AmBmO4A0goKBXU2k/UljHbWh8PIfDVrNRdjd8gJBI9FZ+6GhlVyY2XEz621mWWaWRXC+ZoKZtbkYWgcW5TeyhuBCByT1JuiKOpTUKJMjSi4OA2MAJOUSNBRdcXzadcBd4dVPI4DjZnastYU6ZNeTJa78R8qJmIufA18GVofn8w+b2YR2CzpBIuaiS4iYi83AWEkHgHPAAjOrbr+oEyNiLh4EfiHpxwRdLbM64x9LScUEXY29w/Mx/wJcBmBmKwjOz9wGVACfAf8Yab2dMFfOOecuoY7a9eScc66D8IbCOedcXN5QOOeci8sbCuecc3F5Q+Gccy4ubyhchyPpnKSymEdWnHmzWqqUeZHbfDOsPvpeWPKifxvWcZ+ku8LnsyR9Jea9FyXlXeI435U0OMIyP5LU84tu23Vd3lC4juhzMxsc86hM0nanm9kggmKTP7/Yhc1shZn9Mnw5C/hKzHs/MLMDlyTKhjifJ1qcPwK8oXBt5g2FSwnhkcPvJP1P+LipmXmul7Q7PArZJyknnD4jZvoLkrq1srltwHXhsmPCMQz2h7X+e4TTF6thDJCnwmmPSnpI0u0ENbd+HW4zPTwSGCZptqSfxcQ8S9KzbYxzBzEF3ST9h6Q9Csae+Ndw2jyCBqtUUmk4baykHWEeV0v6civbcV2cNxSuI0qP6XZ6LZz2CfBdMxsCTAGWNbPcfcC/m9lggh310bBcwxTgW+H0c8D0Vrb/fWC/pDSgEJhiZgMJKhnMlnQlMAm43swKgMdjFzazV4E9BP/8B5vZ5zFvvwpMjnk9BShpY5y3EpTpqLPIzIYBBcAoSQVmtoygls9oMxsdlvJ4BPhOmMs9wPxWtuO6uA5ZwsN1eZ+HO8tYlwHLwz75cwR1ixrbASyS1A/4rZn9UdIYYCjwbljeJJ2g0WnOryV9DlQSlKHuD/zZzMrD94uA+4HlBGNdvChpAxC5pLmZVUk6FNbZ+WO4je3hei8mzssJylXEjlB2h6R7CX7XVxEM0LOv0bIjwunbw+10J8ibcy3yhsKlih8DHwODCI6EmwxKZGa/kbQLGAdslvQDgrLKRWb2zxG2MT22gKCkZsc3CWsLfYOgyNydwFzglov4LCXAHcCHwGtmZgr22pHjJBjFbTHwHDBZUjbwEDDczGokFRIUvmtMwBYzm3oR8bouzrueXKrIAI6F4wfMJPg3fQFJ1wKHwu6WdQRdMFuB2yX1Cee5UtHHFP8QyJJ0Xfh6JvBW2KefYWYbCU4UN3fl0QmCsufN+S3wDwRjJJSE0y4qTjM7S9CFNCLstroCOAkcl9QX+F4LsewEvlX3mST1lNTc0Zlz9byhcKnieeBuSTsJup1ONjPPFOB9SWXAAIIhHw8Q7FD/S9I+YAtBt0yrzOwUQXXN1ZL2A+eBFQQ73fXh+t4iONpprBBYUXcyu9F6a4ADwNfMbHc47aLjDM99LAEeMrP3CMbH/gB4iaA7q85K4HVJpWZWRXBFVnG4nZ0EuXKuRV491jnnXFx+ROGccy4ubyicc87F5Q2Fc865uLyhcM45F5c3FM455+LyhsI551xc3lA455yL6/8BCJegqS+KdRoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7698058845975142" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_roc(glm_2, y_train, X_train[sig.index], \"Logistic Regression\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since the optimal cut off was found to be 0.2697615225249289 using the training data, we will use that as our cut off for our evaluation of the test set." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.86 0.86 0.86 6729\n", + " 1 0.54 0.55 0.54 2020\n", + "\n", + " accuracy 0.79 8749\n", + " macro avg 0.70 0.70 0.70 8749\n", + "weighted avg 0.79 0.79 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.2697615225249289)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Unfortunately, the training accuracy has gone down when we used the optimal cutoff. However, accuracy may be misleading in a dataset like ours where most of the targets are non-defaults. \n", + "\n", + "The recall here is more important - detecting defaulters is more useful than detecting non-defaulters. With a higher recall, our model with lower cutoff is able to correctly catch more defaulters.\n", + "\n", + "\n", + "Calculate the confusion matrices for both cut offs to better compare their performance." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Logistic Regression identified 1104\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PredictedFalseTrue
Actual
05790939
19161104
\n", + "
" + ], + "text/plain": [ + "Predicted False True \n", + "Actual \n", + "0 5790 939\n", + "1 916 1104" + ] + }, + "execution_count": 65, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test,glm_2.predict(X_test[sig.index])>0.2697615225249289, \"Logistic Regression\")" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the Logistic Regression identified 716\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PredictedFalseTrue
Actual
06391338
11304716
\n", + "
" + ], + "text/plain": [ + "Predicted False True \n", + "Actual \n", + "0 6391 338\n", + "1 1304 716" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "confusion(y_test,glm_2.predict(X_test[sig.index])>0.50, \"Logistic Regression\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is evident that the lower cutoff is better able to detect defualts." + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.21841387811546378\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dfA8e9NT0gCKXQICTX0AAGkiihSBEGkiIoKWBDsyA8rKKjYRV9Q7NhBQBBRQaoCghTpvUMIEAjpPdn7/jGbuAkpC2Sz2c35PA8P2ZnZmTNl58zce+eO0lojhBBCFMXF3gEIIYQo3yRRCCGEKJYkCiGEEMWSRCGEEKJYkiiEEEIUSxKFEEKIYkmiKIZS6i6l1B/2jsPelFIhSqlkpZRrGS4zVCmllVJuZbVMW1JK7VVK9biK713VMaiU8lZK/aKUSlBKzb/S75cVpdTvSql7r+J73ZRSB20RU3lmj98igHKU5yiUUieA6kAOkAwsAx7RWifbMy5nZN7W92utV9oxhlDgOOCutc62VxzmWDTQSGt9xMbLCaWU1lkpNRJ4FOhcGtvPnOS+1VrXudZ5XeXyS2UfKKVeAp4HMoBsYB8wQWu98ZqDdGKOdkcxQGvtC0QAbYBn7RzPVbHnVbKzXKFfiQq6vesBh64mSVSAY2Se+TwSDKwBbHLH5VTbUWvtEP+AE8BNFp/fBH61+OwJvA2cAs4DswFvi/EDgR1AInAU6GMeXhn4HDgLnAFeAVzN4+4D1pv/ng28XSCmn4GnzH/XAhYCFzCuCh+zmO4lYAHwrXn59xeyfpWBr83fPwm8ALhYxLEB+D8gATgA3Fjgu8WtwwbgPeCSeVwDYDUQC1wEvgOqmKf/BjABaRh3bv8DQgENuJmnWQtMM883CfgDCLaI5x7zOsQCLxbcdwXW2xt4xzx9ArDePCx3mfea9+lF4HmL73UANgLx5vWeCXhYjNfAeOAwcNw87H3gtHkfbAO6WUzvCjxnPjaSzOPrAn+Z55Vi3h7DzdP3xzie4oG/gVYFjtVJwC6MK1c3y21gjn2rOY7zwLvm4afMy0o2/+uExTFonqY5sMK8L88DzxWyTV8GMoEs83zGYFwUvmDezjEYx1pl8/S523qMOYa/CplnDyCqiH1Y3LHrat6/FzF+F49w+bF0v/nvhsCf5uPgIsYJncL2QcF4zPvqJ3MMscDMImJ9CePOKPdzM/O8q1oMK27ftgW2m4+R+cA84BXLbWTe9+eAb6yY3ySM32wScBDz77qYYyR3X+Vuv1rAEvPxcAR4oMC6/mjeN0nAXiDyqs6/ZX3Cv9p/5P+h1QF2A+9bjJ9h3mCBgB/wCzDdYqMnAL0wfjC1gXDzuMXAx0AloBqwGXjI4iSbmyi6Y5xkcovrAjBOprXM89wGTAY8gPrAMaC3xQ7LAgaZp/UuZP2+xkg8fuaD4RAwxiKObOBJwB3jh5IABFq5DtkYxRBuGCfhhuZt4QlUxfghzihsWxdxcK7FOKE2Ns9vLfC6xQ8vGehq3hZvm9e9qEQxy/z92hgnlc7muHKX+al5Ga0xTrpNzd9rB1xnXqdQYD/whMV8NcYJNTB3ewN3A0Hm70zA+DF7mcdNxDimmgDKvLwgi3k1LHCyiAE6mmO+17zNPC223w6Mk5d3wW2KkeBGmv/2Ba4rbDsXcgz6YSTFCYCX+XNHK0+IozFOJPXNy/yJ/05kucv9GuMYKuz47EHRiaK4Y3csRvFOHYzfzEqKThQ/YBQLuZjXr2uB/dmwsHjM+2AnxsVQpYLfLWq7YByfr2Mkpdx4ity35ulPAo9j/A4HYyRky0SRDbxhnt67hPk1wTin1LLYDw2u5BjBSKwfmtc5AiNR3mixrulAP/OypwObrur8a+8EYHWgxsZNxsiMGljFf1fBCuNqo4HF9J3470ryY+C9QuZZHePkY3nnMQJYU8iPVGFcbXU3f34AWG3+uyNwqsC8nwW+tNhhl12lWUzrao6jmcWwh4C1FnFEY05S5mGbgZFWrsOpopZtnmYQsL3Ati4pUbxgMX4csMz892TgB4txPhg/pssSBcYJIQ1oXci43GXWKbDOdxSxDk8Aiyw+a6BnCesdl7tsjKu5gUVMV/Ak9REwrcA0B4HrLbbf6EKO39xE8RfGVX9wgWnybedCjsERlvuphHV7ifyJYhUwzuJzE4wE7max3PrFzK8HhSQKK47d1ZgvWsyfbyrkWMpNFF8Dn1ju82L2QV48GL/1C5bbrYTtkolxdZ+DcffRw5p9i3GxeIb8v8P15E8UmZgvPqyYX0OMJHITRr2U5TQlHiMYFyI5gJ/F+OnAHIt1XWkxrhmQZs3xU/Cfo9VRDNJa+2HskHCMMkYwrop9gG1KqXilVDxGZXdV8/i6GFfABdXDuDI4a/G9jzGuyvPRxpaei/FjBbgTo8gmdz61cudhns9zGCfxXKeLWa9g/rtayXUS4yo71xlzDJbja1m5DvmWrZSqppSaq5Q6o5RKxCgSC+bKnLP4OxXjqgdzTHnL01qnYvwYCxOMcSVU2L4pdjlKqcZKqaVKqXPmdXiNy9eh4HpPUErtN7cEiscoMsn9TlHHSGHqARMK7O+6GOte6LILGINxN3ZAKbVFKdXfyuVeSYwF1eLy48sN64/RopR07OY7HkpYxv8wLsg2m1uJjbYyhrrASW19fcyPWusqGOu+B+PuNFdx+7YWl/8OC67PBa11ujXz00bF/BMYJ/QY828y9xiy5hipBVzSWidZDCt43ij4+/G6mroTR0sUAGit/wTmYBRrgHHrmAY011pXMf+rrI0KKzB2ZoNCZnUa42oo2OJ7/lrr5kUs+gdgiFKqHsZdxEKL+Ry3mEcVrbWf1rqfZdjFrNJFjKu7ehbDQjCuXnLVVkqpAuOjrVyHgsuebh7WSmvtj1Eko4qZ/kqcxShmAIxmmhjFPYW5iHFrXNi+KclHGHU1jczr8Bz51wEs1kMp1Q2jPHgYEGA+USRYfKeoY6Qwp4FXC+xvH631D4UtuyCt9WGt9QiMZP4GsEApVam471xFjAVFc/nxlY1R/p0X2lXMt6RjN9/xgHGSLJTW+pzW+gGtdS2Mu5IPlVINrYjhNBBypSdArfVF83JeUkrVtJhXUfv2LJf/DguuT8FtWOyxorX+XmvdFWP7aYzjobhjxFI0EKiU8rMYVvC8USocMlGYzQB6KaUitNYmjLLs95RS1QCUUrWVUr3N034OjFJK3aiUcjGPC9dan8WoiH1HKeVvHtdAKXV9YQvUWm/HuMX9DFiutY43j9oMJCqlJpnbr7sqpVoopdpbsyJa6xyMSqdXlVJ+5kT0FMaVfq5qwGNKKXel1FCgKfDbla6DmR9GMV68Uqo2Rvm8pfMYZdlXYwEwQCnVWSnlgXH7XPAEDoB5v30BvKuUqmXebp2UUp5WLMcPo6IvWSkVDjxsxfTZmIsolFKTAX+L8Z8B05RSjZShlVIqN8EV3B6fAmOVUh3N01ZSSt1S4AdbJKXU3Uqpqub1zz2GcsyxmSh62y8FaiilnlBKeZqPlY7WLBPjIudJpVSYUsoX4w5s3hVchefG7mX5zxxvccfuj8Dj5t9cFYxkXdS8hyqlcpNKHMaJM8f8ubhjcjPGSfx1877wUkp1sWZ9tNYHgOUYdzNQ/L7daI7nEaWUm1JqIEb9Z3GKnJ9SqolSqqf5eE/HuNjNMW+Loo4Ry9hPY1SOTzevcyuMO5HvKGUOmyi01hcwyjRfNA+ahFFZt0kZRRErMcph0VpvBkZhVHYlYFQA5V4B3YNx67wP4+BcAOReXRTmB4wyxe8tYskBBmBUJh3HuMr6DKNow1qPYtSzHMMo9/we4ySa6x+gkXnerwJDtNa5RTpXug4vY1SyJQC/YlRsWpoOvGC+VX76CtYBrfVe87rMxfjxJmGUw2YU8ZWnMSqRt2C03HgD647LpzGK/5IwfozzSph+OfA7RkXrSYwfpmWxwbsYJ7U/MBLQ5xiVkWAUDXxl3h7DtNZbMeqoZmJs7yMYdQnW6gPsVUolY7TEukNrnW4upnsV2GBe1nWWXzIXMfTCONbOYbTousHKZX6B0aLtL4xjNB1jP12J2hgnM8t/DSj+2P0UY5vuwmgt9BtGws530jNrD/xj3i5LgMe11sfN417CYh9Yfsni99cQox4xCqPBh7XeAh5USlUrbt9qrTMxKrDHYJy878ZI3kUd25RwrHjyX2X6OYyLwefM4wo9RgpZxAiMeotoYBEwRWu94grW3SoO88BdRaaUug+jwq+rvWO5Uuar13iMIqLjJU0vnJtSqi8wW2tdr8SJHYBS6h+M9fnS3rHYksPeUYjySyk1QCnlYy5TfRvjjuGEfaMS9mAuiu1nLqqpDUzBuPJ1SEqp65VSNczrcy/QCqPhjFOTRCFsYSDGrXA0RnHZHVpuXSsqhVHUGYdR9LQfowm1o2qC8cxGAsbzLEPM9YROTYqehBBCFEvuKIQQQhTL4TqtCg4O1qGhofYOQwghHMq2bdsuaq2rljzl5RwuUYSGhrJ161Z7hyGEEA5FKXWy5KkKJ0VPQgghiiWJQgghRLEkUQghhCiWJAohhBDFkkQhhBCiWJIohBBCFMtmiUIp9YVSKkYptaeI8Uop9YFS6ohSapdSqq2tYhFCCHH1bHlHMQejq9yi9MXoB6gR8CDGi2iEEEKUsvSswnp1t57NHrjTWv+llAotZpKBwNfmzuI2KaWqKKVqVoQOtoQQ4lrlmDRRcakkZ2RzJCYZk9bEJGZwNsF4bcWxiykkp2fx76n4EuZUMns+mV2b/C+OiTIPuyxRKKUexLjrICQkpEyCE0IIe0pIy2Ln6XjSsnLIyjGRmW3i0PlkMrJz2HYyjl1RCUV+18PVBR93VxJSM0k7kYifvzUvjSyaPRNFYa/HLLQrW631J8AnAJGRkdLdrRDCKVxMzuB8YjoZ2SbOJaTz079RHDqfzLnEdDKzTUV+z8/TjaY1/Wlaw4+bmlXH3dWFhtV88XJ3obK3O97urkRGfsqFgxeZOvUGHnusI+6fX32c9kwUUeR/MXkdjPcXCCGEw4tNzmDD0VjSM3NIz85hx+l4An08SMnMYd3hC0TFpRX6PT8vN5pU9yMyNIAGVX1pWtMPfy933F1dcHdzobqfJ26uhVcv//33aVq2rIbycOOzzwYQHOxD3bpX8kbmwtkzUSzBeEn5XKAjkCD1E0IIR2Qyaf7Yd56ouFS+3HCCc4np5JgKL/zwcHXBpDUtavvTqk4Vmtb0p26AN55urtSvWonq/l5XvPzY2FSeeWYln322nSlTruell3rQpk3Na12tPDZLFEqpH4AeQLBSKgrjFYjuAFrr2RgvWe+H8bLxVGCUrWIRQojSciklk+T0bBLTszgRm8LWE3Es3nGG+NSsvGmq+LhzfeOqRNYLoGfT6ni6ueDj4YqPR+mecrXWfP31Tp5+egVxcWlMnNiZiRM7l+oywLatnkaUMF4D4221fCGEuFrZOUbF8Q+bT3EmPo1Tl1JxVYrzSen5EoKlYZF1eLhHQ4J8PfD3ci+TOCdNWslbb/1N5851mT37Flq2rG6T5Tjc+yiEEKK0RMensf9sIusOX+TguSTOJqRxIjb1sumq+nnSqJov4TX98HRzoX1oIF7urni6udCslj+1q3ijVGHtc0pfWloWKSlZBAf7MGZMGxo1CmTMmLa4uNhu+ZIohBBOa/3hi8QkpbPzdDzuri5kZJvYfSaBmMR0os3PG+RyUeDn5c4NTapSv6ovQb4eXFc/iLYhAXaK/nLLlh1h/PjfiIiowcKFw2jSJJgmTYJtvlxJFEIIh6e15mRsKltPxvHlhuOcT8zgYnLGZdMFVvLAzUWRmWPiuvqBRNQNoHWdyjSq7kfDar52iNw60dFJPPHEMubP30eTJkE88kj7Ml2+JAohhMOKSUrnm40n+ervEySmZ+cbN6pLKFrDbW1qUzvAm2Dfa3vozF5WrTrGbbfNIzMzh2nTbmDixM54epbtqVsShRDCYWitOXQ+mT8PxfDxn8eITcnMGzekXR16NatO+9BAAit52DHK0pGVlYO7uyutW9egX79GvPJKTxo2DLRLLJIohBDlgtaa05fSOHQ+iZTMbM4nprP6QAzHLqTg4eZCVo6J84n5i5PqV63E4zc2onfzGni5u9op8tKVmJjBiy+u5p9/zrBhw2iCg32YO3eIXWOSRCGEKDMmkyYj28T5xHQWbT9DamY2W07EsTc6gaycwh9QC6zkQaMgH+pU8cHT3QV3Vxe6NQqmfWgglcq4CMaWtNYsWLCPxx9fxrlzyYwb156MjBx8fOz/2iDn2cpCiHIlPSuHj9YeJSvHxNYTcZxLTOfUpcubngb4uBPg40G7egGEBVciom4ValT2ItjXE39vd3ydKBkU5cKFFO69dzG//36ENm1q8PPPd9C+fW17h5XH+feAEMLmsnNMHLuYwpcbTrDvbCI7T+fv2trHwxV3Vxeuqx/I9Y2r4e3uQt1AH3qGVyuz5w/KM39/Ty5eTGXGjN6MH98BNzf730VYkkQhhLgi6Vk57D+byPurDrPtRBxpWTlkF+jXqENoINX8PWldpwqjuoQW2YldRfbXXyd59dV1LFw4DF9fDzZtut+mD81dC0kUQogiRcWlcjYhndOXUpm/NYqNx2Lzjfdyd6FDWCAdwgJxVYoWtSvTtl4Alb3LpgsLR3TxYioTJ65gzpwdhIZW4cSJeFq0qFZukwRIohBCFGAyaX7eeYapv+wjrkC/Ro2r+xJew5/2YYG0rlOZVnWq2ClKx6O15ssvdzBx4goSEzN49tmuvPBCd3x8yn9SlUQhhCA9K4e/j15k9tpjbD5xKW94rcpeTOobTu0q3tSs4k3tKt52jNLxffvtLpo1q8rs2bfQvHk1e4djNUkUQlRg20/FMWnhLg6dT84b1qS6H2HBlXh3eOtS7xa7oklNzeK119Yxdmwkder4s3DhMCpX9irXxUyFkaNAiAokx6Q5fjGFd/44yD/HL3HJ/GRzJQ9XHr2xEbe2rkUtuWsoFb/9dpjx43/jxIl4atf24+GH2xMQ4JjbVhKFEE4sIzuHVftjWLAtiiMxyZc9x9ClYRCP3NCITg2C7BSh84mKSuSJJ5axcOF+mjYN5s8/76N793r2DuuaSKIQwkmYTJpjF5M5diGFv4/GsjMqnu2n/nuewcvdhXb1AuhUP4j2YYF0bRiMq4MVgTiCV1/9i19/Pcxrr/VkwoTOeHg4ftciynjRnOOIjIzUW7dutXcYQthVQmoW+88lcjI2hUPnjTuFFfvOXzZd7Sre3NKqJv1b1ZQWSja0efMZvL3daNmyOrGxqSQkZFC/fvl5jwWAUmqb1jryar4rdxRCOIi0zByW7T3Lk/N2Fjq+WU1/QoN9GBZZl9pVvGlU3a+MI6x4EhLSee65VXz00Vb692/MkiUjCAryISjIx96hlSpJFEKUYyaT5vc955gwfwfpWaa84bWrePNwjwa0qF2ZmpW9qObnKV1hlCGtNfPm7eXJJ5cTE5PCo492YNq0nvYOy2YkUQhRTiSmZ3H6Uip/H4nlwLkkNh2L5UJyBpnZRoLo36om7UMDubFpNeoEONcVq6P59ttd3HPPYiIja7F06Qjatatl75BsShKFEHaUmW1i9JwtrD9ysdDxbUOqcGvrWvRpUZMalb3KODphKSMjm2PH4mjatCrDhjUnO9vEPfe0xrUC9GMliUKIMpadY2JPdCILt0XxzaaTAPh6utGnRQ06hAVSzc+TLg2Dca8AJyBHsWbNcR5++FdSU7M4fPhRPD3dGDWqjb3DKjOSKISwsZSMbFYdiGHx9jOsP3yRzJz/6hpa1q7MwIha3N+tvh0jFEWJiUnh6af/4JtvdlG/fgCffDKgzN9XXR5UvDUWogzEpWSyeMcZZq05ysXk/17fWTfQ6C/p5mY1aFcvgNZ1pclqeXXkyCU6dPiU5ORMnn++G88/3w3vCtorriQKIUrRin3n2Xk6nplrjuQNCwn0YVSXUHo3ryHdYziAxMQM/P09adAggDFj2jB6dBuaNq1q77DsShKFEFchM9vE+iMXOHYhhc3HL+Hh5sLSXWfzTTOuRwMe6dlQOtZzECkpmUyd+ieffvovu3Y9TJ06/rz11s32DqtckCNYiCuQmJ7Fy0v2sfDfqHzD3VwULWtXprK3O9MGtaBmZS+83B2/64aK4pdfDvLII79z6lQCY8a0cYh3RJQlSRRCWCHHpHnnj4N8uPZo3rC7rwvhvs5hhAVXkj6THFR2tolhw+azaNEBmjevyrp1o+jaNcTeYZU7kiiEKMbh80nc+dk/XEj6r0L6xf7NGNM1zI5RiWultUYphZubCzVr+vL66zfy5JOdnKIDP1uQRCFEAVprtpyI44fNp1i0/Uze8D7Na/DBiDZ4uMnzDY5s06Yoxo//jU8/HUDbtjWZNesWe4dU7kmiEMLC3ugEbvlgfd7nWpW9mDygGX1a1LRjVKI0xMWl8dxzq/j4423UquVHXFyavUNyGDZNFEqpPsD7gCvwmdb69QLjQ4CvgCrmaZ7RWv9my5iEsJSVY2LV/hg+W3eMvdGJpGXlADCiQ13G39BQ+lRyEvPm7eGxx5Zx8WIqTzxxHS+/3AM/P097h+UwbJYolFKuwCygFxAFbFFKLdFa77OY7AXgR631R0qpZsBvQKitYhIiV3R8Gl+sP85n64/nDatV2YvudYIZGFGbfi3lDsKZHDhwkdDQKixbdhdt2si+vVK2vKPoABzRWh8DUErNBQYClolCA/7mvysD0TaMRwi01tz35Rb+PHQhb9iQdnV4uEcDGlT1tWNkojSlp2fzxhvradu2JgMGNOG557rxwgvdK0QHfrZgy0RRGzht8TkK6FhgmpeAP5RSjwKVgJsKm5FS6kHgQYCQEGm6Jq5Odo6JUXO2sO6w0VPrZ/dE0r1xVamcdjIrVx5j3LhfOXz4EhMmdGLAgCa4yzMt18SWiaKwhuUF37s6ApijtX5HKdUJ+EYp1UJrbcr3Ja0/AT4B41WoNolWOB2tNecTM1i0/QzfbjrJmXij8jIk0Ie1T/fARZ59cCrnzyfz1FN/8P33u2nYMJA//ribXr0a2Dssp2DLRBEF1LX4XIfLi5bGAH0AtNYblVJeQDAQY8O4RAWw50wC9325JV+HfO1DA7g1ojZ3dwyRt8E5oRUrjrFgwT4mT+7Os892w8tLGnWWFltuyS1AI6VUGHAGuAO4s8A0p4AbgTlKqaaAF3ABIa6C1prle88xa81Rdp9JAKBvixr0bVmTXk2r4y0PUzmdnTvPcfjwJYYMacZdd7WkS5e6hIUF2Dssp2OzRKG1zlZKPQIsx2j6+oXWeq9SaiqwVWu9BJgAfKqUehKjWOo+rbUULYkrcjI2hTeWHWD7qXjOJqQDRvHStEEtuL5xxe7101klJ2cyZcoa3n//H0JDqzBoUDhubi6SJGzEpvdm5mcifiswbLLF3/uALraMQTgnk0nz4s972H0mgV1RCXnD+7Wswf3d6tM2RE4Yzmrx4gM8+ujvREUl8uCDbZk+/SbcpEGCTUkhnnA48amZRExdkff5lpY1ub5xVYa1r1vMt4Qz2L37PLfdNo+WLasxb94QOneWfV4WJFEIh3ExOYNnFu5i5f7/2jocfa2f9Nzq5LKycli37hQ9e4bRsmV1fv31Tnr1qi9NXsuQJApRrqVn5fD8oj3sjU7gwLmkvOGv3taCEe1DpImrk/v779OMHbuUvXsvcPDgIzRsGEi/fo3sHVaFI4lClFspGdm0mbaCzGzjsZoRHULo1iiYPs1rSIJwcpcupfHMMyv59NN/qVvXn59+GkbDhoH2DqvCkkQhypXo+DT+PhrL4u1nWH/EeIK6dd0qLB7XWZ59qCDS07OJiJhNdHQSEyZ04qWXeuDr62HvsCo0SRSi3FiyM5rHftieb9gLtzRldJcwSRIVQFRUInXq+OPl5ca0aTcQEVGD1q1r2DssgSQKUQ4cOJdInxnr8j5/dFdbrqsfREAluYqsCNLSspg+fT1vvLGBBQuGMmBAE+69N8LeYQkLViUKpZQHEKK1PmLjeEQFsv1UHP9bsIvDMckAeLu7snh8F5rU8LNzZKKs/PHHUcaN+5WjR+O4++5WdOhQ294hiUKUmCiUUrcA7wIeQJhSKgKYorW+zdbBCec19Zd9fLHBeBeEm4viq9Ed6NIw2M5RibL06KO/MXPmFho1CmTlypHceGN9e4ckimDNHcVUjO7B1wBorXcopRraNCrhtLJzTHyw+khekvhzYg/qBVWyc1SirOTkGC3YXF1duO66OgQH+zBpUlfpwK+cs2bvZGmt4wtUJkp/TKJERid95zkTn8a2k5eIjk9nx+n4vPE/jessSaIC+fffs4wdu5SRI1vx6KMdueuuVvYOSVjJmkSxXyk1DHAx9wT7OLDJtmEJR6W15uiFZOZvi+LjP49dNj7Ax51hkXV5uncT3OVtYxVCUlIGkyev4YMPNlO1qg81a0odlKOxJlE8AkwGTMBPGL3BPmvLoIRjGvXlZtYczN9L/OguYYzuGkp1fy9JDBXQH38cZfTon4mOTmLs2Ehee+1GqlTxsndY4gpZkyh6a60nAZNyByilBmMkDVHBmUyaqUv3sf10PDvNxUqT+zejbb0Awmv44SX98VRoHh6uVKtWiYULh9GxYx17hyOukirp9Q9KqX+11m0LDNumtW5n08iKEBkZqbdu3WqPRQsLsckZvLHsAD9ujcobNqRdHR7sXp/G1aVooaLKysrh3Xc3kpiYwauv3ggYFxPS5Yr9mc/bkVfz3SLvKJRSvTFeU1pbKfWuxSh/jGIoUcEcvZDMvC2nmbPhBJk5/x0CD3QLY8LNTeTuoYJbv/5UXgd+Q4c2y0sQkiQcX3FFTzHAHiAd2GsxPAl4xpZBifJl7uZTPPPT7nzDBkXU4obwatzSsiZuUvdQocXGpjJp0ko+/3w7ISGV+eWXEfTv39jeYYlSVGSi0FpvB7Yrpb7TWqeXYUyinIhJSuf33eeYssS4TqhV2YsX+zfjhvBqcvcg8sTGpjF37h7+97/OTJ58PZWk6xWnY01ldm2l1KtAMyCvuYLWWi4ZnNT5xHQG/BilNlYAACAASURBVN96YpIy8oYtfbQrLWpXtmNUojzZv/8CP/64lylTetC4cRCnTj1JYKC3vcMSNmJNopgDvAK8DfQFRiF1FE4nOSObmauPsHzvOY5fTAGgbqA3z/VtSrvQAKr5SZNGAampWbz66l+89dbf+Pp6MGZMW+rU8Zck4eSsSRQ+WuvlSqm3tdZHgReUUutK/JZwKN3fXMOllEwA2tUL4IFu9enTQrp4Fv9ZtuwI48b9yvHj8dx7b2veeqsXVavKk/UVgTWJIkMZ/XccVUqNBc4A1WwbligLWmv+PRXHPZ9vJiUzh2BfT1Y+1Z0qPlLGLPJLTs5k5MhFBAV5s2bNvfToEWrvkEQZsiZRPAn4Ao8BrwKVgdG2DErYXkpGNoNmbcjr4tvDzYW1E3vg6ymdswlDTo6JH37Yw4gRLfD19WDlypGEhwfjKcdIhVPiHtda/2P+MwkYCaCUkkcsHVRqZjZPz9/Jb7vPAVAvyIeZI9rSora/vEVO5Nm2LZqHHlrKtm1n8fZ24/bbm8nb5iqwYhOFUqo9UBtYr7W+qJRqjtGVR09AkoUDWXf4Ai//so8j5juIekE+PH5jI25rU1sShMiTkJDOiy+uYdasLVSrVom5c29n8OCm9g5L2FlxT2ZPB24HdmJUYC/C6Dn2DWBs2YQnrkVCahbzt53ml53R7IxKyBs+dWBz7ukUar/ARLl1++0/snr1ccaPb88rr/SkcmVp7SaKv6MYCLTWWqcppQKBaPPng2UTmrhaWmveWn6QD9cezRsWXsOPh3s0YGCEvGpS5HfsWBxVq/rg5+fJq6/2xMVF0b69HCfiP8UlinStdRqA1vqSUuqAJIny7+3lB5m55r9Xmz/XL5yh7eoSIE/LigIyM3N4++2/mTbtLx57rANvvNFLengVhSouUdRXSuV2Ja6AUIvPaK0H2zQycUVikzN4+Zd9LNkZDcDI6+ox4ebG0tRVFOqvv04yduxS9u+/yJAhzXjssY72DkmUY8UlitsLfJ5py0DE1fv4z6NM//0AAN0aBfPO0NZU85eyZVG4997byFNP/UFoaBV+/fVO+vVrZO+QRDlXXKeAq8oyEHFlMrNNfL3xBHP+PkFUXBoAz/drygPd69s3MFEumUyalJRM/Pw8ueWWxly4kMoLL3THx8fd3qEJByBPzjiYw+eTeOePQyzbey5vWI8mVZl9dzvp0VUUau/eGMaO/TXvTXONGwfx2ms32jss4UBsmiiUUn2A9wFX4DOt9euFTDMMeAnQwE6t9Z22jMmRrT98kbs/N55/rF+1Ej2bVGPCzU3w9pAEIS6XmprFtGl/8vbbG6lc2ZPRoyPQWstzM+KKWZ0olFKeWuuMkqfMm94VmAX0AqKALUqpJVrrfRbTNAKeBbporeOUUtKHVCFMJs3EBbtY+K/x2tEXbmnK/d2kiEkUbfv2swwe/CMnTsQzalQEb77Zi+BgH3uHJRxUiYlCKdUB+Byjj6cQpVRr4H6t9aMlfLUDcERrfcw8n7kYz2bss5jmAWCW1joOQGsdc+Wr4NwOnU/i5vf+AsBFwey723Fzc+lKQRQu944hJKQyISGV+eqrQXTvXs/eYQkHZ80dxQdAf2AxgNZ6p1LqBiu+Vxs4bfE5CijYBq8xgFJqA0bx1Eta62VWzNvpFXxorn+rmswYHiGvHRWFys42MXPmZpYsOciKFSMJCvLhzz/vs3dYwklYkyhctNYnC5Rr5ljxvcIKQnUhy28E9MDoO2qdUqqF1jo+34yUehB4ECAkJMSKRTs2rTWDP/qb7aeMzSBFTaI4mzefYezYpWzffo6+fRuSmJhBQIC8SEiUHmsSxWlz8ZM21zs8Chyy4ntRQF2Lz3UwugEpOM0mrXUWcFwpdRAjcWyxnEhr/QnwCUBkZGTBZONU3l1xiA9WHQYgom4VvhrVgcrShFEUIjk5k0mTVvDRR1upWdOP+fOHcvvtTaWyWpQ6a8oxHgaeAkKA88B15mEl2QI0UkqFKaU8gDuAJQWmWQzcAKCUCsYoijpmXejO590/DvLBqsMoBfd1DmX+2E6SJESR3N1dWLv2JI8+2oH9+8czZEgzSRLCJqy5o8jWWt9xpTPWWmcrpR4BlmPUP3yhtd6rlJoKbNVaLzGPu1kptQ+jOGui1jr2SpflDP46dIEPVht9NB2c1hcPN6mLEJc7cuQSU6f+yaxZ/fDz82Tbtgfx8pLHoYRtKa2LL8lRSh0FDgLzgJ+01kllEVhRIiMj9datW+0ZQqnbcyaB/v+3HoCFD3emXb0AO0ckypuMjGzefHMDr766Dg8PV3799U66dZPWTMJ6SqltWuvIq/muNW+4a6CU6oxRdPSyUmoHMFdrPfdqFij+o7XmzeUH+cjcsmnG8AhJEuIya9Yc5+GHf+XgwViGD2/Ou+/2plYtP3uHJSoQq+5ZtdZ/A38rpV4CZgDfAZIorlJ2jokftpzm7eUHSUjLAmDawOYMaiPvABD5aa159dV1ZGWZWLbsLnr3bmjvkEQFZM0Dd74YD8rdATQFfgY62zgup3XwXBK9Z/yV97l9aAA/PHCdPB8h8phMms8//5c+fRpSt25lvvnmNqpU8cLbWxo2CPuw5o5iD/AL8KbWep2N43FqWmteWLwbgBEdQni2Xzj+XvLjF//Ztes8Y8cuZePGKCZP7s7LL99AzZpSzCTsy5pEUV9rbbJ5JE5u8/FLPPXjDqLi0ujbogbTB7e0d0iiHElOzuTll9fy3nubCAjwZs6cgdxzT2t7hyUEUEyiUEq9o7WeACxUSl3WNErecGe9hNQshn28Me/za7dJkhD5vfTSWt55ZyP339+G11+/iaAg6cBPlB/F3VHMM/8vb7a7Blk5Jnq8vQaAt4a0Ymhk3RK+ISqK06cTSEnJIjw8mGee6cqgQeF07er8XdQIx1NkDarWerP5z6Za61WW/zAqtYUVRs/ZQlxqFpH1AiRJCMDowO/ddzfStOksHnpoKQDBwT6SJES5ZU1Tm9GFDBtT2oE4o9d/P8C6wxcBmPdQJztHI8qDTZuiiIz8hAkT/qBHj1C++mqQvUMSokTF1VEMx2gSG6aU+slilB8QX/i3RK7JP+/h640nAVj3vxtwdZE+eCq6X389xIABP1Crlh8//TSMQYPCpW8m4RCKq6PYDMRi9Po6y2J4ErDdlkE5Mq01o+ZsYe3BC4CRJOoGSsVkRaW1Jjo6idq1/bnppvpMnXoDjz/eET8/T3uHJoTVSuzrqbwpz3097Y1OYOjsjaRmGq/rWD3heupX9bVzVMJeDh2KZdy4Xzl0KJZ9+8bj6+th75BEBWaTvp6UUn9qra9XSsWR/4VDCtBa68CrWaCzSs3M5pYPjI79Rl5Xjxf7N5MeYCuo9PRsXn99PdOnr8fb243p02/E21t6eBWOq7ijN/d1p8FlEYgjy84xMXqO8a6lW1rVZNqgFnaOSNjLuXPJdO/+JYcPX2LEiBa8+25vatSQu0rh2IpMFBZPY9cForXWmUqprkAr4FsgsQzicwiTl+xl07FLvHxrc+7tHGrvcIQdZGXl4O7uSvXqlejevR6zZvWjV68G9g5LiFJhTdnIYozXoDYAvsZ4huJ7m0blILTWTFu6j+//OYWbi+K2ttL7a0VjMmlmz95KgwYfEBWViFKKzz67VZKEcCrWFJyatNZZSqnBwAyt9QdKqQrf6klrzVM/7mTR9jNU9fNkzdM98PWUcuiKZOfOczz00FL++ecMPXuGkZWVY++QhLAJq16FqpQaCowEcp8OqtBdnn6z6SRv/H6A5IxsHugWxrN9m+Iiz0lUGFprJk5cwYwZmwgM9Oabb27jrrtayjMRwmlZkyhGA+Mwuhk/ppQKA36wbVjl1w+bT/Hi4j0APHR9fSbe3ESSRAWjlCIuLo0xY4wO/AICvO0dkhA2ZdVzFEopNyD31VpHtNbZNo2qGPZ6jkJrzew/j/HGsgMAbH+xFwGVpF18RXHyZDyPP76MyZOvp23bmphMWi4QhEO5lucoSqzMVkp1A44AnwNfAIeUUl2uZmGObN3hi3lJYsqAZpIkKoisrBzefHMDzZp9yIoVxzh40Oi7S5KEqEisKXp6D+intd4HoJRqCnwDXFVmclTvrDgEwKoJ19NAnrauEP7++zQPPbSUPXtiGDiwCR980JeQkMr2DkuIMmdNovDITRIAWuv9SqkKdTm9bM9Zdp6OZ0DrWpIkKpCVK4+RkJDO4sXDGTgw3N7hCGE3JdZRKKXmABkYdxEAdwE+Wut7bRta4exRR9Hh1ZXEJGXwz3M3Ut3fq0yXLcqO1ppvvtlF1ao+9O3biIyMbLKyTNJHk3AKNq2jAMYCR4H/AZOAY8BDV7MwR/TzjjPEJGXQtWGwJAknduDARXr2/Jp7713Ml1/uAMDT002ShBCUUPSklGoJNAAWaa3fLJuQyo8tJy7x+FzjpCH9NzmntLQsXnttHW+8sYFKlTz4+OP+3H9/W3uHJUS5UuQdhVLqOYzuO+4CViilCnvTndP6+8hFhs7eCMAtLWsSFlzJzhEJW/jll0O88so6hg9vwYED43nwwXbSokmIAoq7o7gLaKW1TlFKVQV+w2ge6/Q++esor/1mNIX98K629GtZ084RidJ07lwyO3aco0+fhgwd2ozQ0Pvp0EH66RKiKMXVUWRorVMAtNYXSpjWKeSYNO+tOJSXJN6/I0KShBPJyTHx4YdbaNJkJiNHLiItLQullCQJIUpQ3B1FfYt3ZSuggeW7s7XWg20aWRnTWtNiynLSzB27yStMncu//55l7NilbNkSzU031efDD/vh7V2huywTwmrFJYrbC3yeactA7O2FxXtIy8ohvIYfn94TKUnCiRw/HkeHDp8SHOzD998P5o47WkgHfkJcgeJeXLSqLAOxp+2n4vjun1MALB7fBS93VztHJK6V1prdu2No1ao6YWEBfPnlQAYMaEKVKtLEWYgr5fT1Dtb4ZtNJAGbf3VaShBM4fjyO/v1/oE2bj9m16zwAI0e2liQhxFWyaaJQSvVRSh1USh1RSj1TzHRDlFJaKVXm/UeZTJrVB2IA6NNCKq4dWWZmDq+/vp7mzT/kzz9P8PbbvWjWrKq9wxLC4Vn9SjallKfWOuMKpncFZgG9gChgi1JqiWW/Uebp/IDHgH+snXdp+mn7GeJTsxjSro49Fi9KSU6Oic6dP2fbtrMMHtyUGTN6U7eudOAnRGmwppvxDkqp3cBh8+fWSqn/s2LeHTDeXXFMa50JzAUGFjLdNOBNIN36sEuH1pqn5+8EYFIf6fTNESUmGtcurq4ujB7dhl9+GcHChcMkSQhRiqwpevoA6A/EAmitdwI3WPG92sBpi89R5mF5lFJtgLpa66XFzUgp9aBSaqtSauuFCxesWLR13l91GIA+zWtQ1c+z1OYrbE9rzZw5O6hf/31+/tl47mXcuPb079/YzpEJ4XysSRQuWuuTBYZZ8xb5wtof5nVVq5RywXjXxYSSZqS1/kRrHam1jqxatXTKnDOyc5ix8jCuLoqP7pa+fRzJvn0X6NHjK0aN+pnw8GAaNAi0d0hCODVr6ihOK6U6ANpc7/AocMiK70UBdS0+1wGiLT77AS2AteY27TWAJUqpW7XWNu9HfM+ZBADu6hgibeodyJtvbuD551fj7+/JZ58NYNSoNtI3kxA2Zk2ieBij+CkEOA+sNA8ryRagkVIqDDgD3AHcmTtSa50ABOd+VkqtBZ4uiyQB8N0m47mJcT0aljClKA+01iilqFHDl7vuaslbb/WialXpqFGIslBiotBax2Cc5K+I1jpbKfUIsBxwBb7QWu9VSk0Ftmqtl1xxtKUkPSuHn7afIaiSBzUqS9v68iw6OonHH19Gt24hPPZYR+65pzX33NPa3mEJUaGUmCiUUp9iUbeQS2v9YEnf1Vr/htHrrOWwyUVM26Ok+ZWWKT/vBWBQG+kMrrzK7cDv+edXk5VlonNnab4shL1YU/S00uJvL+A28rdmcihaa+ZtNcJ/vl9TO0cjCrNjxznuv38J27ad5eabG/Dhh/2kwloIO7Km6Gme5Wel1DfACptFZGNLdhr16S/c0lQqQcuphIR0oqOTmDdvCEOHNpPGBkLYmdVPZlsIA+qVdiBl5ZuNRkvfgRFS7FReaK2ZP38fhw/H8vzz3bn++lCOHXscL6+rOTyFEKXNmiez45RSl8z/4jHuJp6zfWil70x8GltPxgEQ7Oth52gEwNGjl+jX73uGD1/Azz8fJMv8PhBJEkKUH8X+GpVxz98ao3krgElrfVnFtqOYvHgPADOGR0hxhp1lZGTz9tt/88or63B3d+H99/swblx73NykQ2MhyptiE4XWWiulFmmt25VVQLZ0OCYZTzcXae1UDpw+nci0aX8xYEATZszoTe3a/vYOSQhRBGsu3zYrpZyij4tTl1K5qVl1e4dRYV24kMLMmZsBaNgwkH37xjN//lBJEkKUc0XeUSil3LTW2UBX4AGl1FEgBaMPJ621dqjkcT7R6Jy2nrzitMyZTJovv9zO//63kqSkDHr1qk+TJsHUrx9g79CEEFYoruhpM9AWGFRGsdjUgm1RAHSsH2TnSCqWPXtiePjhX1m//hTduoUwe3Z/mjQJLvmLQohyo7hEoQC01kfLKBabemv5QQBa15H3FJSVzMwcbr75GzIzc/jii1u57z5pRCCEIyouUVRVSj1V1Eit9bs2iMcmYpKMYicPVxeq+EizWFtbvfo4119fDw8PV378cSjh4cEEB0uRnxCOqrjKbFfAF6M78ML+OYxtJ4xnJ166tbmdI3FuUVGJ3H77j9x449d8/bXx5sCuXUMkSQjh4Iq7ozirtZ5aZpHY0Mw1RwDo1kjKxm0hO9vEzJmbefHFNeTkmJg+/UbuuquVvcMSQpSSEusonMHe6ERcFNSVFk82MXLkIubO3UPfvg2ZNasfYWHSmkkIZ1JcorixzKKwodz6ia6NSucVqsIQH5+Om5sLvr4ejB/fnttvb8rttzeVymohnFCRdRRa60tlGYitHIlJBmBAq5p2jsQ5aK2ZO3cPTZvO4sUXVwNGPcSQIdLLqxDOyuk71slNFA2q+do5Esd35Mglevf+lhEjFlKnjj933y31EEJUBE7fReemY7EAhAXJ+5Wvxfff72b06J/x9HRj5sy+jB0biaur019nCCGoAIli9YEYQoN8CKgkz09cjaysHNzdXYmMrMWQIc14881e1KrlUK2jhRDXyKkTxelLqaRnmejSUJrFXqmYmBQmTPiDlJRMfvppOI0bB/Htt4PtHZYQwg6cuuzg+82nAOjcQBKFtUwmzSefbKNJk5nMm7eH5s2rkpNjsndYQgg7cto7ihyT5vP1xwG4IVyaxlrj2LE47r77JzZujKJHj1A++ugWwsMlyQpR0TltovjnWCyZ2Sbu7xqGj4fTrmapqlzZk/j4dL76ahAjR7aS5q5CCMCJi56W7T0HwIiOIXaOpHxbsuQggwfPIyfHRFCQD3v2jOOee1pLkhBC5HHaRHHgbBIADarK8xOFOXUqgUGD5jJw4FwOHYrl7FnjeRMXF0kQQoj8nLZMZm90ArUqe9k7jHInO9vEjBmbmDJlLVpr3njjJp588jrc3V3tHZoQopxyykSRlWMiJTOHNiHSOV1BOTkmPvvsX3r2DOP//q8voaFV7B2SEKKcc8qip7UHLwDQqYG89hQgLi6NSZNWkJSUgaenGxs2jGbJkjskSQghrOKUieKt5QcAGNKujp0jsS+tNd99t4vw8Fm8885G1qw5AUBQkI9UVgshrOZ0RU8mk+bQeaNitrp/xa2jOHQolnHjfmXVquN06FCb5cvvJiKihr3DEkI4IKdLFO+sOAjAhF6N7RyJfT3xxDK2bo3mww/78eCD7aQDPyHEVXO6RDF382mq+nnySM+G9g6lzK1YcZTw8GDq1q3MRx/dgqenGzVqSPNgIcS1sellplKqj1LqoFLqiFLqmULGP6WU2qeU2qWUWqWUqncty9NaE5uSSWS9gApVBn/uXDJ33rmQm2/+ljfe2ABAvXpVJEkIIUqFzRKFUsoVmAX0BZoBI5RSzQpMth2I1Fq3AhYAb17LMtOycgBoXsv/WmbjMEwmzezZWwkPn8nChfuZMuV63n77ZnuHJYRwMra8o+gAHNFaH9NaZwJzgYGWE2it12itU80fNwHX1Expx6l4ALwqyMNj06ev4+GHf6Vdu1rs2jWWl17qgZeX05UmCiHszJZnldrAaYvPUUDHYqYfA/xe2Ail1IPAgwAhIUX33fTeykOAcz8/kZSUwcWLqYSFBTB2bCRhYQGMGNGiQhW1CSHKli3vKAo7c+lCJ1TqbiASeKuw8VrrT7TWkVrryKpVC+8y/PjFFLaciKOanyfNa1W+2pjLLa01ixbtp1mzDxk+fAFaa4KCfLjzzpaSJIQQNmXLRBEF1LX4XAeILjiRUuom4HngVq11xtUu7NQlowRr8oCC1SCO7+TJeG69dS6DB/9IYKA3H3zQV5KDEKLM2LLoaQvQSCkVBpwB7gDutJxAKdUG+Bjoo7WOuZaFpWVmAxAS6HMtsyl3Nm48zU03fQPA22/34vHHr8PNTZ6JEEKUHZslCq11tlLqEWA54Ap8obXeq5SaCmzVWi/BKGryBeabr5BPaa1vvZrlHThndCtew0l6jE1MzMDf35O2bWsyenQEEyd2ISTE+YrUhBDln02byGitfwN+KzBsssXfN5XWsr7ddIqG1Xyp5ufYiSI2NpVnnlnJH38cY+/ecfj6evB//9fP3mEJISowpyjD+GDVYS4mZ9CspuM+P6G15uuvdxIePosvv9zB8OHNkWoIIUR54BSN7t9dYTSLfeW2FnaO5OokJKQzaNA81q49QadOdZg9uz+tWlW3d1hCCAE4QaKIT80EoENYIP5e7naO5sporVFK4e/vSXCwD5980p8xY9rK60iFEOWKwxc9bTwaC8BQB3v3xPLlR2jb9hOiohJRSjF//lAeeKCdJAkhRLnj8Ini9z3nAOOOwhGcPZvEHXcsoE+f70hNzSImJsXeIQkhRLEcuugpIzuHJTujqeHvRb2gSvYOp0SzZm3muedWk5GRzcsv92DSpC54ejr0LhBCVAAOfZY6ftG4Gu/ZtJqdI7HOtm1n6dixNrNm9aNRI+ftj0oI4VwcOlGcjDW67bi5WflsIZSYmMHkyWsYObIV7drV4sMPb8HT01W63xBCOBSHThQnzHcUgZU87BxJflprFi7cz+OPL+Ps2SRCQirTrl0t6QJcCOGQHPrM9c/xSwCEBZef+onjx+N45JHf+e23w0RE1OCnn4bRsaNjtcgSQghLDp0o4szPUPiVo+cnvvtuN3/9dZL33uvNI490kA78hBAOz6ETxdGYZOoGets7DNatO0lGRg433VSfiRM7c999EdSp47jdiQghhCWHvdxNzcwmMT2bpjXsd0K+eDGV0aN/pnv3OUyd+icAnp5ukiSEEE7FYe8oVuw7D0DXRsFlvmytNXPm7GDixBUkJGQwaVIXXnyxe5nHIcq/rKwsoqKiSE9Pt3coooLw8vKiTp06uLuXXpG8wyaKrSfiAOjfqlaZL/u33w4zevQSunSpy+zZ/WnRwjGe4xBlLyoqCj8/P0JDQ6VZtLA5rTWxsbFERUURFhZWavN12KKnZXuNrjvKqmlsamoWGzacAqBfv0b8/PMd/PXXKEkSoljp6ekEBQVJkhBlQilFUFBQqd/BOmSi0FpzISmDYN+ySRK//36YFi0+pG/f74iPT0cpxa23NpEO/IRVJEmIsmSL480hE8WlFKNZ7J0dQmy6nDNnEhk6dD79+n2Pp6cbv/wygipVHPsNekIIcaUcMlGcvGR03VGziu2axsbEpNCs2YcsXXqIV165gZ07x3L99aE2W54QtuLq6kpERAQtWrRgwIABxMfH543bu3cvPXv2pHHjxjRq1Ihp06ahtc4b//vvvxMZGUnTpk0JDw/n6aeftscqFGv79u3cf//9+YYNHDiQTp065Rt23333sWDBgnzDfH198/4+dOgQ/fr1o2HDhjRt2pRhw4Zx/vz5a4rt0qVL9OrVi0aNGtGrVy/i4uIum2bNmjVERETk/fPy8mLx4sUAdOvWLW94rVq1GDRoEABLly5lypQp1xTbFdFaO9S/du3a6UX/Rul6k5bq3VHxurRFRSXk/f3++5v0kSOxpb4MUXHs27fP3iHoSpUq5f19zz336FdeeUVrrXVqaqquX7++Xr58udZa65SUFN2nTx89c+ZMrbXWu3fv1vXr19f79+/XWmudlZWlZ82aVaqxZWVlXfM8hgwZonfs2JH3OS4uTtepU0eHh4frY8eO5Q2/99579fz58/N9N3fbpKWl6YYNG+olS5bkjVu9erXevXv3NcU2ceJEPX36dK211tOnT9f/+9//ip0+NjZWBwQE6JSUlMvGDR48WH/11Vdaa61NJpOOiIgodDqtCz/ugK36Ks+7Dtnq6cC5JACq+5deMVBCQjovvLCajz/exqZN99O2bU0ee6xjqc1fiJd/2cu+6MRSnWezWv5MGdDc6uk7derErl27APj+++/p0qULN998MwA+Pj7MnDmTHj16MH78eN58802ef/55wsPDAXBzc2PcuHGXzTM5OZlHH32UrVu3opRiypQp3H777fj6+pKcnAzAggULWLp0KXPmzOG+++4jMDCQ7du3ExERwaJFi9ixYwdVqlQBoGHDhmzYsAEXFxfGjh3LqVNGI5IZM2bQpUuXfMtOSkpi165dtG7dOm/YwoULGTBgANWrV2fu3Lk8++yzJW6X77//nk6dOjFgwIC8YTfccIPV27UoP//8M2vXrgXg3nvvpUePHrzxxhtFTr9gwQL69u2Lj49PvuFJSUmsXr2aL7/8EjDqIXr06MHSpUsZNmzYNcdZEodMFBuPXgQolcpsrTXz5+/jiSeWce5cMo880oEGDQKutktM5AAADxJJREFUeb5ClDc5OTmsWrWKMWPGAEaxU7t27fJN06BBA5KTk0lMTGTPnj1MmDChxPlOmzaNypUrs3v3boBCi1cKOnToECtXrsTV1RWTycSiRYsYNWoU//zzD6GhoVSvXp0777yTJ598kq5du3Lq1Cl69+7N/v37881n69attGjRIt+wH374gSlTplC9enWGDBliVaLYs2fPZduiMElJSXTr1q3Qcd9//z3NmjXLN+z8+fPUrFkTgJo1axITE1Ps/OfOnctTTz112fBFixZx44034u//38O8kZGRrFu3ThJFUUwa3F3VNdfua60ZPPhHFi8+QNu2NVmyZASRkWX/XIaoGK7kyr80paWlERERwYkTJ2jXrh29evUC/ntne2Gu5Le1cuVK5s6dm/c5IKDkC62hQ4fi6uoKwPDhw5k6dSqjRo1i7ty5DB8+PG+++/bty/tOYmIiSUlJ+Pn55Q07e/YsVatWzft8/vx5jhw5QteuXVFK4ebmxp49e2jRokWh63Sl5xA/Pz927NhxRd+x1tmzZ9m9eze9e/e+bNwPP/xwWT1MtWrViI6OtkksBTlkZXZ0fNo1PWiXlZUDGAdJ1651+eCDPmzefL8kCeGUvL292bFjBydPniQzM5NZs2YB0Lx5c7Zu3Zpv2mPHjuHr64ufnx/Nmzdn27ZtJc6/qIRjOaxgu/5Klf7r8blTp04cOXKECxcusHjxYgYPHgyAyWRi48aN7Nixgx07dnDmzJl8SSJ33SznPW/ePOLi4ggLCyM0NJQTJ07kJbGgoKB8dzuXLl0iODg4b1tYs65JSUn5Kp4t/1kmtVzVq1fn7NmzgJEI/r+9uw+uurrzOP7+LAIJFdIWF3YrKrSmkBjCg9CmdhGRLmM1C6XjiOGh0rHrCCK1KA47OLPuai1tBS2iRdY6SWsbEKcWBhAW2Vgq8iC0Cgo2pTQT2OkozbKsUnn+7h+/X5JrcnPzS+Q+5vuauTP393zynZvfueec3/2efv3a/t3V888/z+TJk1v9orqhoYFdu3Zx0003fWT9yZMnyc9PTa67rKsozpvRcOJ0p1OLv/JKHaWly1mz5h0A7r33Gu6++4t065Z1oXCuQwoKCli6dCmPPvooZ86cYdq0abz66qu8/PLLQNDymDt3Lvfffz8A8+fP55FHHqG2thYIbtxLlixpdd4JEyawbNmypuXGm3H//v05cOBAU9dSWyQxefJk5s2bR1FREX379o173njf5IuKijh48GDTcnV1NRs3bqSuro66ujr27NnTVFFcd911rFq1itOng8frKysrm8Yhpk6dymuvvcb69eubzrVx48am7rRGjS2KeK+W3U4AEydOpKqqCoCqqiomTZrUZhyqq6upqKhotX716tWUl5eTl/fRMdna2tpW3W7JknV3x1NnzwPQp4OTAB09eoLbbvsV48ZVcerUWXr37pmM4jmX0UaMGMGwYcNYuXIl+fn5rFmzhocffpjBgwczdOhQRo8ezZw5cwAoLS3l8ccfp6KigqKiIkpKSpq+Hcd64IEHOHbsGCUlJQwbNoyamhoAFi1aRHl5Oddff31TP31bpkyZwnPPPdfU7QSwdOlSdu/eTWlpKcXFxSxfvrzVcUOGDOH48eO8//771NXVUV9fT1lZWdP2QYMG0adPH3bu3El5eTljxozh6quvZvjw4Wzbtq1pYDk/P59169bxxBNPUFhYSHFxMZWVlQlbAFEsWLCAzZs3U1hYyObNm1mwYAEQjK3EdiXV1dVx+PBhxo4d2+ocK1eujFuB1NTUtGplJIss5pnpbDBwyFDja4tYd/c/UHJpQaRjqqv3cdddG/jgg9PMn38NCxdeS69emTOHhctdBw4coKioKN3FyGmPPfYYvXv3btWHn8veffddpk6dypYtW+Juj/e5k7THzEZ15npZ16L44NRZAAr7X9zOns3Onj1PSUk/3njjTr773fFeSTiXQ2bNmkXPnl2rh6C+vp7Fixen7HpZ99TTiVPn+FyfnvS8qFvb+5w4zUMPbeXyywuYPXs006eXMn16qefccS4H5eXlMWPGjHQXI6VGjx6d0utlXYvivBkTiv+uze3r1tVy1VVP8f3vb6O2tgEIBsu8knDpkm3duy67JePzlnUtCoDxRa0HmI4c+T/mzn2JF198h+Liv2Xr1pmMGXNFGkrnXLO8vDwaGho81bhLCQvno2j5hNTHlZUVRUF+6zGGQ4eOsWnTH/ne98Yzb96X6NGj7a4p51JlwIABHDlyhKNHj6a7KK6LaJzh7kLKyooir3tQCeza9d9s336Yb3+7jGuvvYL6+nvo27dXO0c7lzrdu3e/oDONOZcOSR2jkHSDpN9LOihpQZztPSWtCrfvlDQwynk/eVE3Zs9eT1nZMyxZsoMT4fwUXkk459yFl7SKQlI34Engq0AxUCGp5U8XbweOmdmVwGNA22kVY4wY+mOefnoPc+d+kX37ZvGJFE2H6pxzXVEyu56+ABw0s0MAklYCk4DYhCiTgAfD9y8AyyTJEgzb29nzXDawgA0bpjFyZOJfezrnnPv4kllRXAocjlk+ArSc4KFpHzM7K+k40Bf4S+xOku4A7ggXT+3+yx1vRcgI3BVcQotYdWEei2Yei2Yei2aDO3tgMiuKeM8CtmwpRNkHM1sBrACQtLuzP0PPNR6LZh6LZh6LZh6LZpJ2t79XfMkczD4CXBazPABomTy9aR9JFwEFwP8ksUzOOec6KJkVxetAoaRBknoAtwJrW+yzFrgtfH8z8F+Jxiecc86lXtK6nsIxhznAJqAb8KyZvS3p3wkm+V4L/AT4maSDBC2JWyOcekWyypyFPBbNPBbNPBbNPBbNOh2LrEsz7pxzLrWyLimgc8651PKKwjnnXEIZW1EkK/1HNooQi3mS9kvaK2mLpJxNm9teLGL2u1mSScrZRyOjxELSLeFn421Jv0h1GVMlwv/I5ZJqJP0u/D+5MR3lTDZJz0p6T9JbbWyXpKVhnPZKGhnpxGaWcS+Cwe8/Ap8FegBvAsUt9pkNLA/f3wqsSne50xiLcUCv8P2srhyLcL/ewFZgBzAq3eVO4+eiEPgd8KlwuV+6y53GWKwAZoXvi4G6dJc7SbG4FhgJvNXG9huBlwh+w1YG7Ixy3kxtUTSl/zCz00Bj+o9Yk4Cq8P0LwHjlZsL/dmNhZjVm9tdwcQfBb1ZyUZTPBcBDwA+Ak6ksXIpFicU/A0+a2TEAM3svxWVMlSixMKBP+L6A1r/pyglmtpXEv0WbBPzUAjuAT0pqNxdSplYU8dJ/XNrWPmZ2FmhM/5FrosQi1u0E3xhyUbuxkDQCuMzM1qWyYGkQ5XPxeeDzkrZJ2iHphpSVLrWixOJBYLqkI8AG4O7UFC3jdPR+AmTufBQXLP1HDoj8d0qaDowCxia1ROmTMBaS/oYgC/HMVBUojaJ8Li4i6H66jqCV+RtJJWb2v0kuW6pFiUUFUGlmiyV9ieD3WyVmdj75xcsonbpvZmqLwtN/NIsSCyR9BVgITDSzUykqW6q1F4veQAnwiqQ6gj7YtTk6oB31f2SNmZ0xsz8BvyeoOHJNlFjcDjwPYGbbgTyChIFdTaT7SUuZWlF4+o9m7cYi7G55mqCSyNV+aGgnFmZ23MwuMbOBZjaQYLxmopl1OhlaBovyP/IrggcdkHQJQVfUoZSWMjWixKIeGA8gqYigouiK89OuBb4RPv1UBhw3sz+3d1BGdj1Z8tJ/ZJ2IsfghcDGwOhzPrzeziWkrdJJEjEWXEDEWm4AJkvYD54D5ZtaQvlInR8RY3Av8h6TvEHS1zMzFL5aSqgm6Gi8Jx2P+FegOYGbLCcZnbgQOAn8FvhnpvDkYK+eccxdQpnY9OeecyxBeUTjnnEvIKwrnnHMJeUXhnHMuIa8onHPOJeQVhcs4ks5JeiPmNTDBvgPbypTZwWu+EmYffTNMeTG4E+e4U9I3wvczJX0mZtszkoovcDlflzQ8wjH3SOr1ca/tui6vKFwm+tDMhse86lJ03WlmNowg2eQPO3qwmS03s5+GizOBz8Rs+5aZ7b8gpWwu51NEK+c9gFcUrtO8onBZIWw5/EbSb8PXNXH2uUrSrrAVsldSYbh+esz6pyV1a+dyW4Erw2PHh3MY7Atz/fcM1y9S8xwgj4brHpR0n6SbCXJu/Ty8Zn7YEhglaZakH8SUeaakJzpZzu3EJHST9GNJuxXMPfFv4bq5BBVWjaSacN0ESdvDOK6WdHE713FdnFcULhPlx3Q7vRiuew/4RzMbCUwBlsY57k7gR2Y2nOBGfSRM1zAF+HK4/hwwrZ3r/xOwT1IeUAlMMbOhBJkMZkn6NDAZuMrMSoGHYw82sxeA3QTf/Ieb2Ycxm18Avh6zPAVY1cly3kCQpqPRQjMbBZQCYyWVmtlSglw+48xsXJjK4wHgK2EsdwPz2rmO6+IyMoWH6/I+DG+WsboDy8I++XMEeYta2g4slDQA+KWZ/UHSeOBq4PUwvUk+QaUTz88lfQjUEaShHgz8ycxqw+1VwF3AMoK5Lp6RtB6InNLczI5KOhTm2flDeI1t4Xk7Us5PEKSriJ2h7BZJdxD8X/89wQQ9e1scWxau3xZepwdB3Jxrk1cULlt8B3gXGEbQEm41KZGZ/ULSTuAmYJOkbxGkVa4ys3+JcI1psQkEJcWd3yTMLfQFgiRztwJzgOs78LesAm4B3gFeNDNTcNeOXE6CWdwWAU8CX5c0CLgPGG1mxyRVEiS+a0nAZjOr6EB5XRfnXU8uWxQAfw7nD5hB8G36IyR9FjgUdresJeiC2QLcLKlfuM+nFX1O8XeAgZKuDJdnAL8O+/QLzGwDwUBxvCeP3idIex7PL4GvEcyRsCpc16FymtkZgi6ksrDbqg9wAjguqT/w1TbKsgP4cuPfJKmXpHitM+eaeEXhssVTwG2SdhB0O52Is88U4C1JbwBDCKZ83E9wQ/1PSXuBzQTdMu0ys5ME2TVXS9oHnAeWE9x014Xn+zVBa6elSmB542B2i/MeA/YDV5jZrnBdh8sZjn0sBu4zszcJ5sd+G3iWoDur0QrgJUk1ZnaU4Ims6vA6Owhi5VybPHusc865hLxF4ZxzLiGvKJxzziXkFYVzzrmEvKJwzjmXkFcUzjnnEvKKwjnnXEJeUTjnnEvo/wHwZD1sgbEwAgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(glm_2, y_test, X_test[sig.index], \"Logistic Regression\")" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[3] = [\"Logistic Regression\" , classification_report(y_test,list(glm_2.predict(X_test[sig.index])>0.2697615225249289), output_dict = True)[\"1\"][\"recall\"],auroc]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "iCxBcin11EI8" + }, + "source": [ + "### Support Vector Machine\n", + "#### Theory\n", + "Support vector machines attempt to find an optimal hyperplane that is able to separate the two classes in n-dimensional space. This is done by finding the hyperplane that maximises the distance between itself and support vectors (data points that lie closest to the decision boundary).\n", + "\n", + "SVM is computationally expensive for a dataset with a lot of features. Therefore, it is neccessary at this stage to do some data reduction." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### PCA\n", + "We would like to reduce the dimensionality of our dataset before training an SVM on it. This can be done through Principle Component Analysis (PCA). The idea would be to reduce the number of features without loss of information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 2 Component PCA\n", + "First, we will visualize the information retained after performing a 2 component pca." + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": {}, + "outputs": [], + "source": [ + "#perform pca\n", + "from sklearn.decomposition import PCA\n", + "pca = PCA(n_components=2)\n", + "principalComponents = pca.fit_transform(X_train)\n", + "principalDf = pd.DataFrame(data = principalComponents\n", + " , columns = ['principal component 1', 'principal component 2'])" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Explained variation per principal component: [0.2966725486 0.1736524263]\n" + ] + } + ], + "source": [ + "#amount of information each principal component holds after projecting the data to a lower dimensional subspace.\n", + "print('Explained variation per principal component: {}'.format(pca.explained_variance_ratio_))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows that the information of principal component 1 retained is 28.4% and principal component 2 retained is 17.8%, both of which is quite low" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAH6CAYAAAAeHtXEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOyde5xT1b3ovyuZJMMkQWQQ0AEUnBFlqFh81Tq1WlHq2NqqaNrSFk+xtkdry6PnQnv7sKf3Fg9VW1q0py3tFR9HYlFLKwMoVbSjtr6xgEDwUaCDoANKJjB5rvvHToZMZifZCclMZub35ZPPJnuvvfZvJ5n9W+u3fg+ltUYQBEEQhP6Pra8FEARBEAShNIhSFwRBEIQBgih1QRAEQRggiFIXBEEQhAGCKHVBEARBGCCIUhcEQRCEAYIodUEQBEEYIIhSFwYsSqlapdT1SqlHlFI7lFKHlVIfKKValVKzlVLy+x9gKKUuVEpppdQtRZz7dvLc1CuhlHpfKfWsUuompVRVlvPGKqVuVUq9pJQ6oJSKKqX2KaXWK6W+pZQ6Jsc1Z6Zd79JCZRaETEx/pIIwQLgG+BWwB3gS2AmMAq4ClgGXKaWu0ZKBSejOEuB9wA6Mx/i9nAdcnPx/F0qp64GlgAvYCDwAHABqgSbg58D3gRFZrnUDoAGV/P9jpb0VYbAhSl0YyGwHrgBWa60TqZ1Kqe8CzwNXYzykH+ob8YQK5eda67dTb5RSjcALwJVKqY9rrZ9K7v8C8FsMJX611np1ZkdKqfOBO80uopSaCFwArAeGA1copUZprfeW+H6EQYSYH4UBi9b6Ca31n9MVenL/O8B/J99eWEifSqlTlVK/T5pqw0kz61+VUv9u0vZipdRapdR+pVSnUmp70kzbwxyrlNqQNME6lFI/UEq9kTxnq1Lqq2ntvq6U+kdyKWG3UupHmcsISqmTkn3dnZT3j0kZQsmlB1Mzr1LKpZRaqJR6TSl1SCl1MHlv15q0Tb/GSUqpFUqp95Iyv6iU+lSOz/DzSqknk6bqTqXU60qp7ymlXCZtdfKzGaGU+o1Sak/yc9+slPq3jLZ3Y1hkAH6YYUq/MJs8+dBabwY2JN+ek7yWF/hlct/nzBR68txngHOzdJ36Xv8fcDfgAK4rVk5BAJmpC4OXaHIbs3qCUupy4A8Ypta1GKbWYcAU4H9hmPpTbb+WfB9KnrMPYwCxAPi0Uup8rfX7JpdZgaEEWpIyzgB+o5SKAqcDs4BHgb9gWCF+ABwC/sukr/HAc8Am4NfA8YAPWKOU+oLW2p8mrxNYB3wc2Ioxu6xJXt+vlDpDa/1dk2uciGH1eBO4F2PG6QNWKaWmaa2fTG+slPod8BVgN/Awhpn7I8CPgYuVUpdorTO/k2HAM0AEWAlUJ+X6vVIqobVenmz3x+R2FvAURxQxwNsmsheCSm5TSzUzMO71b1rrnCZzrXW4R2fG5z0LOAg8gvFZ3wZcr5RaLEtCQtForeUlr0H1whjM/gPjAT3d4jkjgA8wFMvHTY6PSfv/iUAY44F9aka7u5LX/U3G/g3J/S8Aw9L2T0he8wDwFlCXdmwY8B7wLlCVtv+kZF8a+GnGdc7CGCwcAIam7f9Osn1LRl8jMRSiBj6a5Ro/zLjG9FRfGfuvS+5/GBiSceyW5LFvZexPXWMZYE/bPwljQLYlo/2Fyfa3FPG7SN3nSRn7GzEGThr4WHLf75Lv/0+Rv8HPJc//ddq+h5L7Lu7rvxF59d9XnwsgL3n19gtjRqQx1tqtnjM/ec4SC23/d7LtT0yOHZtU9ocBV9r+Ddke6MATyWNfMTn2/5LHTkzbl1K47wNek3PuTh6flbYvACTIGIQkj81Otv+9yTXeTle2acf/CbyXse8VjAHFMJP2dowByvMZ+zWGtWOoyTlPJY970/aVQqn/PDnI+DFwX5pCfzitbUty39eL/A2mvtPz0vZ9OrlvRV/+fcirf7/E/C4MKpRS38RQ0FuBLxVw6keS2zUW2k5Nbp/IPKC1PqCUegXDQepUDI/pdF406a8tuX3J5Ni/ktsxGIo0nZe11kGTczZgmH4/DCxPrg/XA//SWm81aZ+6jw+bHHtVax032b8Lw2McAKVUDcYyxXvAHKWUySmEgdNM9ge01gezXAMMi4XZfRbLt5JbDXQAr2Eo9/9Oa5NpjreMUqoeY/CxTWv9XNqhNcBeDIe8EVrr9wrtWxBEqQuDBqXUTRjhSlswZsT7Czh9WHL7r5ytDFKOcHuyHE/tH5Z5QGv9gUn71BpzrmMOk2PZvKjfSW6PydgWLC+GNcCMGN0dcY/FUITHAT/Mck42cl0DjFl+KRmv07zfs5AaaI0pov+vYnwWd6fv1FrHlFL3YQw6r8OwKAlCQYj3uzAoUErNwYgn3gRcpA0P+EJIKZY6C21Tynd0luPHZ7QrF6Oy7E/J9UHGtpzyps59RWutcr2O4hq9SWtye3EhJyml0j3cF2V46GsMhQ5HPOMFoSBEqQsDHqXUAuBnwKsYCn1fEd38Lbm9zELbV5LbC01kGQacAXQCrxchRyFMTZrWM0nJ9QpA0kT/BlCnlGowaX9RcvtysYJorTuAzUCjUmp4sf1YILUUUOrZeyYrgf3AeUqpabkaZoTqfQbD+XAbhrOd2etN4BSl1MfLILcwwBGlLgxolFLfB27FWI+++CjWKZdjOLj9u1LqApPrpJth78NwCLs5uX6azo+BocB92iTUqcQcgxHy1oVS6ixgJsbM+ZG0Q7/HMAn/VCllT2s/AiMjWqrN0XAH4MQIRethyldKHauUmtrztIJoT27HHWU/OUkOhL6ZfOtXSk03a6eU+ghGWGGKG5LbH2itrzd7AT/JaCsIlpE1dWHAopSaBfwnxuztr8A3TRy03tZa352vL631e8rIILYSeFIptQbDgWooRvz4WIy4cLTWbyfN/XcCLyulHsQIO/s4hvPYVox49XLzNEbc87kYcd6pOHUb8LUM57PbMKwQnwE2KqVaMGKnr8GYWS7WWrdyFGitf6+UOhO4EXhDKbUOI3XvcIzP7gIMb/6vH8VltmH4PXxOKRVJ9q+Be7XWmY6ER4XW+n6l1BCMZZ21SqlXgWc5kib2PI44B6KUGg9MS77/o2mnBiswLEtXK6VuLtD3QxjkiFIXBjLjk1s7MCdLm6fIcFjKhtZ6dXKmuwBjLfVSjAf4VmBRRtu7lFI7gG9jpKOtwfDW/ilGqFs2569S8haGgrw1uXVhmND/U2u9LkPeiFLqEmAe8AXgZgxHtI3AHK31A6UQSGt9U3JA9HUMBTcMw4y9E+Ozue8o+48rpa7EuOdrAS+GBaKVntEBR43WellycPIN4BIMK4gbwwdjEzCXIxaO65Oy3Ku1juToM6SUWoGxrj4LQ8ELgiWU1pK4SBAGEkqpkzAU+nKt9XV9KowgCL2KrKkLgiAIwgBBlLogCIIgDBBEqQuCIAjCAEHW1AVBEARhgCAzdUEQBEEYIPT7kLYRI0bok046qa/FKBuhUAi3293XYpSdwXKfMHjudbDcJwyeex0s9wmVfa8vvfTSe1rr48yO9XulftJJJ/Hii2aFrQYGGzZs4MILL+xrMcrOYLlPGDz3OljuEwbPvQ6W+4TKvlelVNacC2J+FwRBEIQBgih1QRAEQRggiFIXBEEQhAFCv19TNyMajbJ79246Ozv7WpSsVFdXM2bMGBwOR1+LIgiCIAwQBqRS3717N16vl5NOOgmTqlx9jtaa9vZ2du/ezfjx4/OfIAiCIAgWGJDm987OTmpraytSoQMopaitra1oS4IgCILQ/xiQSh2oWIWeotLlEwRBEPofA1ap9zVf+cpXGDlyJJMnT+5rUQRBEIRBgih1gGAQli2DBQuMbTB41F1ed911rF27tgTCCYIgCII1BqSjXEG0tkJzMyQSEAqB2w3z5kFLCzQ1Fd3tBRdcwNtvv106OQVBEAQhD4N7ph4MGgo9GDQUOhjb1P6Ojr6VTxAEQRAKYHArdb/fmKGbkUgYxwVBEAShnzC4lXogcGSGnkkoBDt29K48giAIgnAUDG6l3tBgrKGb4XZDfX3vyiMIgiAIR8HgVuo+H9iyfAQ2m3G8SD7/+c9z3nnnsW3bNsaMGcPvfve7ovsSBEEQBCsMbu93r9fwcs/0frfZjP0eT9FdP/DAAyUUVBAEQRDyM7iVOhhha21thlPcjh2Gyd3nOyqFLgiCIAh9gSh1MBT47Nl9LYUgCIJQYQTDQfyb/QTaAzTUNuBr9OF1eftarKyIUhcEQRAEE1p3ttJ8fzMJnSAUDeF2uJm3bh4tM1toGld8crJyMrgd5QRBEATBhIRO0Hx/M8FIkFDUCH0ORUMEI0Ga72+mI1KZyclEqQuCIAhCBvsP7yehzZOTJXQC/6bKTE4mSl0QBEEQMgjHw10z9ExC0RA79ldmcjJR6mVk7dq1TJw4kfr6em699da+FkcQBEGwiMvuwu0wT07mdripH16ZycnEUY7yeDfG43FuuukmHn/8ccaMGcPZZ5/NFVdcwaRJk0oktSAIglAuhg8Zjk2Zz3ttyoZvcvHJycrJoFfq5fJufP7556mvr2fChAkAfO5zn2PVqlWi1AVBEPoBNmWjZWZLD/2Q2u9xVmYuk0Gt1IPhYJd3Y4rUGkrz/c20zW8r+ov717/+xdixY7vejxkzhr///e9HJ7AgCILQazSNa6Jtfhv+TX527N9B/fB6fJN9FavQYZArdf9mf17vxtlTi0tKo7XusU8pVVRfgiAIQt/gcXqK1gN9waB2lAu0B8rm3ThmzBh27drV9X737t2ccMIJRfcnCIIgCPkY1Eq9obahbN6NZ599NoFAgLfeeotIJMKKFSu44ooriu5PEARBEPIxqJW6r9FXNu/Gqqoqli5dyvTp0znttNO49tpraWxsLLo/QRAEQcjHoF5T97q8ZfVubG5uprm5uUTSCoIgCEJuBrVSh/7p3SgIgiCUDrNcJd0bBI3y3IEANDQY5bm9lVmpbdArdeh/3o2CIAhCaciWq+R/zvyfZINWaG6GRAJCIXC7Yd48aGmBpsqr1CZKXRAEQRiU5MpVEtgfoKN9D57mZmOm3tUgGTHV3AxtbeCpLKvugHWUM4sTryQqXT5BEISBTq5cJQD++xYaM3QzEgnDJF9hDMiZenV1Ne3t7dTW1lZkwhetNe3t7VRXV/e1KIIgCIOWXLlKEjrBjne3QShE0An+yRAYDg37wbcJvKEQ7Ki8Sm0DUqmPGTOG3bt38+677/a1KFmprq5mzJgxfS2GIAjCoCWVq8RMsduUjfrjJtJ6ykaar+4kAYRc4A7DvOnQ8lA1TfWVV6ltQCp1h8PB+PHj+1oMQRAEoYLxNfqYt25e1uPN136PiXvvIeg6si+U/H/z1Z20XXk53VbUK8BLfsCuqQuCIAhCLlK5SrxOb1d2UbfDjdfppWF4A6v3PEViiPkyaWJINf63Vx/Z0doKdXUwZw4sXmxs6+qM/b3IgJypC4IgCEI+guEgW9/byvVTr6f9UDu1Q2ppHNmIb7KPF5990VhzT3SanhtKdB6pDxIMGt7wFeAlL0pdEARBGHSYxadnZhNtqG2guqqazlhPxV5dVX2kPojfn99Lfnbv5EIR87sgCIIwqEiPT085yYWiIYIRY39HpAOA5vpmU4UO0Bnr5PJTLjfeBAJdM/OgE5ZNhQXTjG0w2rte8jJTFwRBEAYVueLTEzqBf5OfkzmZlh0tOG1OIolIj3ZOm5PV21cb2UgbGsDtprU2RPNMenrKj9H0Vu45makLgiAIg4pc8emhaKhrrXzTvk2mCh0gkoiw5d0txhufj6BL0TwTgq4jHvIhl/G++f07u2b/5UaUuiAIgjCoSMWnm+F2uLvWyg8cPpCzn/bD7cZ/vF78d91IIkuuswQa/6beyT5XUUpdKTVWKfWkUup1pdRmpdS3+lomQRAEYWDha/RhU+bqz6Zs+CYbVdqGDxmes5/a/UfW2wPDIeQ0b5c++y83FaXUgRgwX2t9GvAR4Cal1KQ+lkkQBEEYQHhdXlo+dCveMLiT1nV3BLxhaPnQrV3e740jG6m2m8epV0dh0rI/QodhVrc6+y83FaXUtdZ7tNYvJ/8fBF4H6vpWKkEQBGFAEQzS9IWFbPsFXL0Fzt1lbLf9Apq+sLBLUfsafdiyhKrZNPhet3cVdbE6+y83qlKrhSmlTgKeBiZrrQ9mHLsBuAFg1KhRZ65YsaLX5estOjo68FRYab9yMFjuEwbPvQ6W+4TBc68D5j7fe4+OvTsJDDP0X8IGtqTubnhf4Rk1jo7qamrcNWx851US9NSTNg1T3gHbqNFG5jigI9JBYH/A6FMnupR8w/CGrtl/Kbjooote0lqfZXasIpW6UsoDPAX8X631w7nannXWWfrFF1/sHcH6gA0bNnDhhRf2tRhlZ7DcJwyeex0s9wmD514Hyn0GF8yhzr6kW073FN4wtOl5vDj90+wYuoM5f/4GIcI92rnDsORJJ7Ovv6tbYpmOSAf+TX527N9B/fB6fJN9JVXoAEqprEq94uLUlVIO4CHg/nwKXRAEQRAKxX/CARL7zI8lFPhHt3MyydA3E4UORrjajmO1UbQlDY/TY8Su9xEVtaaujOLnvwNe11rf0dfyCIIgCAOPQP3w7J7qTtjRUAsYzm81VTWm7dwRqL9uXq/ldLdKpc3Uzwe+BPxDKfVqct93tdYtfSiTIAiCMIBoOL4Rt63atFiL21ZN/ahJcBDGDR3Hodgh0z5sbg++z37vyI5gkOADy/G//SiBWmg471P4zpyF1zWIS69qrVu11kprfbrW+ozkSxS6IAiCUDJ8jT5sVQ7TY7YqB77JPhI6wWf9n83ax0rfQ0fWyltbaT1nFHVv38wc1rG4Yx1zVt9M3U9H07qzd0uvVpRSFwRBEIRyk6uOeqpKW/vhdg7HDmftI9BueLkTDBL8zGU0X3m4e4pYJwTjh2i+77JeSxELlWd+FwRBEISy0zSuibb5bVk91d/vfD/n+asDq7npnJvA78d/SoQshVdJxKL4N/l7zXlOlLogCIIwKDkqT/VUNHggQMAT6ZqhZxLS4V5LEQtifhcEQRCEHgyrHpbz+OUTk7XUGxpo6HDiNo98w61cvZYiFkSpC4IgCEIPaofUUuMwD2ercdQwa8os443Ph2+7M6syTTne9Rai1AVBEAQhA5uy8dNLfmp67KeX/PSI57vXi3fVGloeGWIUiEnO2N0R8NpraPnimpJnlMuFrKkLgiAIQgYJnWDh+oWmxxauX8iXp3z5iLJuaqLphX20PbAc/z9Xs8MD9R+9HN+Zs3pVoYPM1AVBEAShB/sP7yehzX3aEzqBf5O/+06PB/3lL6OvuorE6R9Cu1z0RW0VmakLgiAI/YZgOIh/s59Ae4CG2gZ8jb6yZG0Lx8OEoiHTY6FoqIdHe+vOVprvbyahE4SiIdwON/PWzaNlZgtN45pKLl82RKkLgiAI/YLeVJxOe5bk8EnGHDOm6//BcJDm+5sJRoJd+1IDgub7m2mb39ZrZngxvwuCIAgVT7riTCnMUDREMGLs782sbQDpJdb9m/2FmerLiCh1QRAEoeLpbcUZiUdyHt99cHfX/wPtgYJM9eVElLogCIJQ8fS24nTZXV154TNxO9zdEso01DZYbltuRKkLgiAIJScYDrLs5WUseHwBy15eRjAczH9SDnpbcQ4fMhybMleRNmXrllDG1+iz3LbciFIXBEEQSkrrzlbq7qhjzto5LH52MXPWzqHujrqjKkPa24rTpmx5K7mlsFL1rbcQ73dBEAShZJTLEzylODO931PKtxyKM18lt2LblhNR6oIgCELJ8G/2E01ETY9FE1GWv7ocV5WrqDjzvlCchVRyO6qqbyVClLogCIJQMjbv20xnrNP0WGesk7lr5+KschYdZ14JirOSkTV1QRAEoWTsP7w/5/GojlZGnPkARZS6IAiCUDKOHXJswef0doKWgYyY3wVBEATL5Mu9PnloPc4YRArQLr2doGUgI0pdEARBsERHpIO6O+py5l5vfq2TiD1LBxpQPXf3doKWgYyY3wVBEIS8BMNBAvsDeXOvP/zW6oL77u0ELQMZUeqCIAhCXvybs695p6+JP6q3m87GAVBg1/R5gpaBjJjfBUEQhLwE2gOM1CNNj3VbE3fkViufiI3Dd8UP+jRBy0BGlLogCIKQl4baBoLt5vnb09fEP3Xc+ax7f6f5bF3DZ0Y0SZx5GRHzuyAIgpAXX2P2Ne/0NfGrfLfk7Ofqz/1nKcUSMhClLgiCIOTF6/IyZugY02O3Tru1y4Tesudpqm1O03bVNier2zaUSUIBRKkLgiAIFgiGg+w+uNv02ML1C7u83wPtATp1xLRdp45IPHqZEaUuCIIg5MWq9/vYY8bm7GfMMeazfaE0iFIXBEEQ8hJoD5DQCdNjBWWE0yUUSuiBKHVBEAQhLw21DdiUucpI935/btdzOfvJZsIXSoModUEQBCEvVrzfg+EgD299OGu7mqoaSQdbZkSpC4IgCHnxurw0DG/A4/DgtBve7U67E4/D05URzr/Zj83cQg9AXMclHWyZEaUuCIIgWEeBSmaWUcabLgJ7NnMo0Zn11BkNn5XscWVGlLogCIKQl1RBl45IB+F4GIBwPExHpKOroEvDjv24zaPZcEfgor3VvSjx4ESUuiAIgpAXKyFtvrZjsWXxbrdp8O2pLZN0QgpR6oIgCEJe8oW0/d+//l+WH7+PlatceMPgNibzuMPgDUPLQ9V46if1osSDEynoIgiCIOQlV0EXgLfef4ubeYshn4U/roBdx8CO4VC/H3ybweNygE+c5MqNzNQFQRCEvOQKaUvnsBOu8sHhGicJBdrlRHs80NICHnGSKzcyU89D28E2vvOX77D1va2cOuJUFl28iBOGntDXYgmCIPQqqZA2r9NLJB7pcpYzI+SCb1+SIAy4lWLep6FlHDT1nriDFpmp5+CO5+6g7md13PPaPTzf9jz3vHYPdT+r464X7upr0QRBEHodj9ND2/w2Ljrporxtw8QACOkwwegRD3mhvIhSz8IfX/8j8x+bb3rsppabeKfjnV6WSBAEoe/xOD1cPelqXHZXQeelF30RyocodROC4SBXPnhl9gYa5q/+Zu8JJAiCUEH4Gn047I6Czimo6ItQNKLUTfj+ugW5KwkpeGLrul6TRxAEoZLwurysmbmGGkdNQedJ2dXyI0rdhDtf+XW31Idm7E0c7B1hBEEQKpCmcU3s/fZebrvkNiaPnMz4YeM5c/SZuU+SsqtlR7zfTYjpRF6lrvMcFwRBGOi8+s6r/OipH5HQCULRELvUrpztpexq+RGlLgiCIBRMMByk+f5mgpEjCWliOpa1fXrNdaF8iFLvS4JB8PshEICGBiPbktfb11IJgiDkxb/ZnzVtrBmpmutCeRGl3le0tkJzMyQSEAqB2w3z5hlZl5okRYMgCJVNoD1AKBrKetxhcxBNRHE73NiUravmulBeRKn3BcGgodCDaXmUQ8k/juZmaGuTdIpC3yDWI8EiY48Zm/P4NZOuYdwx46gfXo9vsk8Uei8hSr0v8PuNGboZiYRxfPbs3pUpG/KQHzyI9ahfEQwH8W/2E2gP0FDbgK/Rh9dVOX+bHx33UW46+6a+FmPQIUq9LwgEjszMMwmFYEeFJGiQh/zgQaxH/YrWna0039/c5XXudriZt24eLTNbaBrXO3+buz7I4+n+gXi69wUSp94XNDQYCtIMtxvqK8BDNP0hn3q4h0JH9ndIDucBhRXrkVARpHudp9a0Q9EQwUgwb371YDjIspeXseDxBSx7eRnBcPZSqvloqG3A7TB/jomne98hSr0v8PnAluWjt9kqo+awPOQHF/3FeiTk9DrPlV+9dWcrdXfUMWftHBY/u5g5a+dQd0cdrTtbi5LD1+jDpsyfY+Lp3neIUu8LvF6Cf1rJ0o85+eSXbXxyJiw930lwuIe2h+9h1uM3ce5vz2XWI7OIJqJ9I6M85AcX/cF6JAC5vc6z5Vc/mtl9NrwuLy0zW/A6vV0zdrfDjdfpFU/3PkTW1DM4GnOUVVp3tnLpM5/l8MWRrn3rGiLMm54g+syRQjLPtz3P6aecTuCFADeefWPZ5epG6iFvptjlIT/w8PkMfwkzKsV6JABHzN5mij2b2dvK7H721MKdc5vGNdE2vw3/Jj879u8QT/cKQGbqGSzfuLys/QfDQS677zIOxw73OBZNmGdj6pNSr/1hiUAoHV6v4QDp9R6ZsbvdR/aLk1zFUIzZu5jZvVU8Tg+zp85m0bRFzJ46WxR6HyNKPYNHtz+aN+/70eDf7CcS6yz4vIXrF5ZBmhzIQ37w0dRkeLkvWQILFxrbtjaJdKgwijF7i1Pb4EHM75nE4pabvtPxDqM9owvqPrBnM5Ec+ZGzse29bQWfc9SkHvJ+v7GGXl9vzNBFoQ9cPJ7KyZHQC1R6rHc2CjV7+xp9zFtnvrwiTm0DC1HqGZyzW7NOY2m2fs39V/HXrz1bUP9jA3uN8oOFWAM0TDxmQkHXKRmD7CEvDB5ad7Zy2f2XEY1HCcfDuOwu5q6by5qZa3ot1vtoSJm9rZCa3WfGtkv61oGHKPUMVry3AYZZaKig9Z3nCr/Avn3gLPy0W/fnqVMsCIJlguEgl957aTfflnA8TDgeZvp909n77b0VreiKsTCIU9vgQJR6BoFj4tZn0brw/nfVVkGB0SOjDsLo4L8Kv5ggCKb84MkfmDqrAhyKHmL5xuUVm+L0aLLJFTK7F/on4iiXSRmd5AAazvsUQyL526XjjAPt7WWRRxAGG8FwkF8+/8ucbVZvW91L0hRGOeLNhYGFKPVexnfmLBIuR0Hn7BoGHbWV77zTXyhlqkyh/+HfbCEbYhGD+974XRWbTU4YPIj5vZfxurxMPq6Rl/a9av0kBcvH7acyjYH9i0oohCH0LYH2AHGdO8rl8obLC+qzI9JB3R11Zf9dlTPeXBgYyEy9D+jcv6+wExSsPvbd8ggziBDTpQBGzHaNoybrcYfNwawzZlnuLxgOEtgf6JXflcSbC/kQpd4H1HRaj4VPESvzWv9gQEyXAhgx23Zlz3r80S88WpBHeC5zfup3VSrTvCZc8xIAACAASURBVBRREfIhSr0PeLMqWLDnvMNW2Dq80BMxXQpgnpHNYXNQXVXNui+u49KTL7XUT0pR//al32YdLIaiIX770m854Y4TSlIdTYqoCPmQNfU+4BDRgh1xnv7n0+URZhBRTCEMYWBytDHbmb4Z155ybda2f2/7e7f3qd9f8/3NtM1vK1gRN41rYts3trFw/UK2tW9jYu1Ebp12K8d7jy+oH2FgIko9A1cMwlWUNbStKpHsv4BryHrv0ZMzVWYsju/E5l6WSOgNsiVqKTZmO90342gotjpa5oBi075NPLL1EXH2FABR6uaUUaEHw0GCtiJqpGvgjTfg5JNLLlM6/TUXthW6pcqMRQklOnFHwKah5SHw3DbRKFbTnwuYBINGrv5AwCif6/MZRXgGKeWIdsjlm1EIxSz5mA0ojnbmLwwsRKlnEMnuP1MSLMXImqHgnes/x+gnXziq6+dS2lYfgP1Z8TeNa6Lthm34Pz2eHTVQvx98m8ET6QQ6obnZKGLTH4vWtLYa8icSEAoZVfXmzev/A5UiKZcCzOWbUQjFLPmUqy66MHAQpZ5BlYYi5tGWCbQHij73mye8xoNHce1cSnvKqCmWHoADIc7b88hqZr9WBaFwz4OJhDHT7W9FbIJBQ6EH00zCoaTi6c8DlaPAqgIMvr0d/23XETj4Fg1Dx+P79t14Tzola7+5fDNcdhcaTSSeP21kMd7q4uwp5EO83zPwJMo7zmmobSgqZzwKnhxT/HAjV4z2ZfddxvKNy/M+AAdMnHcgcEThZRIKGWVm+xt+vzEgMSM1UBlkWFGArUvmUfebicwZ+hyLT36HOUOfo+43E2ldYu57AbnDyhx2B05b7opN1VXVRXurS5y6kA9R6hlEKDyGvBCa64t3xuq0FTMaMMg1a+mIdnDfxvvyPgAHTJx3Q4NhmjbD7Tbqxvc3BuJA5SjJpwDHaA/Ne39G0AUhl7E/5IKgC5r3/oyOXW+YnmsWVmZTNrxOL2tmrmHNF9fkVNZaa7bfvL0oy5bEqQv5EKWewSF78YrTCi07Woo7UUNHlgmAlcQW+dYBn297HluWn0NqBjBgTH8+H9iy/PRtNuN4f2MgDlSOknwKUK/6E9EsTrFRBf7/+nLWvlMhcUs+uYSF5y9k7NCxtM1vo2lcE03jmlh08SKcdvM/2CpbFau3F1cwRuLUhXzImnovE2gPFOddn+Wc1Bp3XMc5FD2Ew+bg5jU3s8q3ikvrL+1yatu4dyN2Zc+a81on/5mRmgHc/erdOUUcM3RM/3Ci83oN57FMpzKbzdhfgrXnXv8cfD7DKc6M/jpQOUq6RTuk+YDYlI2WmS2s/P7VdB5nfm6nE7YE38zZf3pI3IYNG7op1F0f7Mq6rn60A2Cpiy7kIqdSV0p9CpgLjAS2AHdqrZ/OaHMu8KzWuiR+40qp3wOfAvZprSeXos9CsEFZDfBda+olCJsz8+6NJqJEE1Gm3z+d2y+9nVs23NL1QCuUGkcNdmW3PAN448AbvVLUoiQ0NRnOY36/YZqurzcUXwkUep84E/bCQKU/kksB/vY4D+h95n+LGtqPS7N8FBgqWO5ER1IXXchGVqWulLoEWAX8DXgKOA94Uin1c+DbWuty2anvBpYC95Sp/5xUazshq2q9CMXsO7GZrxZ+min+zX6iiezOc/Mfm1985xo+NHwSX5w6iz9v+zNb39vKjndez3nK0ueXdpOn4uNnPZ6SebmnZuab9m3i1y/9ms5YZ9exzM+hbJRxoNKfyaYAh5/3CdiUZTauoPb8acb/iwgVzJnoSNa+hTKSa6b+Q+AerfW/pXYopb4C/AKYoJT6vNa6M+vZRaK1flopdVKp+7VKvpKMXWgYVj2s8As89HDh56SzdCnMmgVeL5v3be6mPEqKghf2vsSm9a8bs01bNfFIJ04gYvKrcdgcKGU+yhno8bOZM/NspD6HkyljAqGMgUowHMT/8rLKXg7pIxrHn0v1puV0mgSxVuNg0olnFx0qmM/0X3EDXGHAkEupT8ZQ7F1orX+vlNoIPAo8kTTPDyhiVhsquOq0qwru3//2o5A74iUn7/xgHqO/+11oaWH/4f3Fd5QPDQmlj4SuJTqNX0sW+0wui0G/cqIrkEJShqY+h5OrypsVMMVAyClQTlKz6c5Iz9+uw1ltzKbvWZE/VDCLtUfWvoW+QGWzoiul3gFu0Fr/yeTYBGAdxvLzLcD9pVpTT/Z/EvBotjV1pdQNwA0Ao0aNOnPFihWlujQvtb1k2ax+4jEnMqJmREH9/2tvgHcSBy23H+Maw+7w7q73ww7DyQcAm41dE0aw71CBtdnLhEp+aGbOdjZlY+zQsTk/q46ODjz90Ez83qH32HVwl6W0oanPoTpRXfZ7TegEG/duNJXLpmxMGTUlq2d4qegP3+m7h95l5wc7e+wfd8w4jqs5Dv71L3jnnewdjB4NdXX94l5LwWC5T6jse73ooote0lqfZXYsl1J/DHhDa/3vWY6PBFowZvSO3lTq6Zx11ln6xRdfLNWlsf1Qoa086zQEvxsseNS99OnbuPmJ/7A8cLjtlNv49vZvd733dELwVsDtZtlPruarB/rE9cAUh81hOmP3Or1s+8Y2VgdWZzUDb9iwgQsvvLAXpS0NCx5fwOJnF1tq63V6aZvfxovPvlj2e1328jJuXnOz6fJMdVU1Sy9bWvxyiEWnsVJ/p6WOKAiGg9TdUWdqZUl9V557VsCcOeY5ANxuWLIEZs/ut7/fQhks9wmVfa9KqaxKPZf5/SHgu0qp4VrrHnZerfU+pdTHgUeAaaURte/xRCBYbbFtEWa094/SDSGeGnCEQvj21HJDtcoailYWcnjuX9t4LX/a9ifiiTiHYoeoUlXYbXZuufAWJi6d2Kdm4HKFmOXyck5htpZa7pC3XP4WnbFOtry3pbiO+yi/fG8XZunyA5FQQaGfkVWpa61/Dfw618la6xBwaSkFUko9AFwIjFBK7QZ+qLX+XSmvkYuvvH0sSyYeKFultvs33lt83xoaU9Z2txtvfSP/7v0yd21ZnjUsp+T3kaO/VdtWcd+V9/G5hz7XNWt3KmcPL/xSe8XnU5DlXFv2NfqYu26u6TGX3cUFJ16AXdm5/JTLmTJqCmCU0S136F8+f4v2Q+2Fd9pH+eWtFmYpdKBkKZmShAoK/YyKSz6jtf58X14/cOoI0AfK1v++A7vzN8rBipXJ/yRnCad95zdQm6VxOQYmOWrBJ3SCa/9wLZHEkaQbh2KHsndVAq/4fAq73KUqN+7dSDzRM2KiSlURS8R46p9PEYlH+OvOv/Ldv3yXldeuJLA/UPbSmccOOTbn8doh2X40ObCSX74MhXCszKgnjphY8MDNciy5hAoK/QhJE5vBBt4uaz31rA9FS+fCyVH3kdmDx8MbB/9ZXnkzsZH1eoeih7op9HwcrVe8lQIz5cxXn7r+4djhHsdiOkZcx7uyiqXkuuKBK7L2V8r8+fmSm0wYPqHwTvsov3y+GfUr77xSVKGhgvKop0IFFy0ytqLQhQpFlHoGEW05qK0oQraj6F9hOOa0tXWtX+49bkj2qm+9uNReDO4I1LuN1LLvHXovZ+56M6wo7HLmq891/WzEE/Gs55Qy9C9f/oJw1KTsbD7KkF/eSt2CXIVZAH790q8Jx8zvJxqPZh0oSR51YSBSceb3PkcpyqkNoxzFTB16mDf3NBwP77xt3rY3Z/BFoICxL71B3YY6fjT+Ryzethi3w83ctXO58ZwbQZNzbTSfwt68bzPth9uxYSNh8rlX26uPKl1nviI5ZsR0rCv8L5MaRw1tHW0seHzBUTvPrX9zfe7jb61n3kezlxc1pcROY1Z9HXJlZwOIJbIPlDvjnWx5N7tToMSSCwMNUeoZuKqqieVYB04n2N6Gt/aEgvp32B2E40XMksBUSbdFinB4KjFVMUgoSBQS1Kjh+hdhxnl3ElSRrtlrSkkufsYIE8u1NtpQ24DbVm0kxsmgWjn51Uu/QmttqtDBeOBffsrlBQjdHSue72Zki1Y4FD3Eys0rORQ7VP4IgWLGrSV0GivE1yE1o77k3kuKyqDYfjj334jkURcGEpbM70qpJ5RSp2Y5dopS6onSitV3NB432fIDb+TP63hs1R0F9e9xFD8DcKqeWvP9YryYS4UGewJiVVClKUxRKNg2ykbCltuc0LU2evcldLTv6Xas+fgLiEWyhG0lInTGOvMOoB56/SHT/VbMwrnWZK1QU1VjbB01XftSjoVW1oRz8alTcid7vHxikYOZlNPYkiWwcGGP5SCr5KpbEE30NJk3jWvi62d+vSiRt7y7paBlHUHoz1h9Il0IDM1ybChwQUmkqQDuvepeaw0VdFbB9Ffm89g/Vlnu32UrMkeshlPVqB67O4PvF9dfKVBH4uYjVRRm7tcQtUNIW7NaRKKdXDNvLMv8CwiGg7TubKXhtx8inDnO0eCIQbVF14XV23rWtW7d2UrdHXXMWTuHxc8uZs7aOdTdUUfrztZu7czWZK3idri5pvEaFp6/kBmTZjDEPsS0XbHOc6fUnpL1WI2jhllTZhXcZxclcBorJo6+cWRjwZ8zwAttL2T9DgVhoFHINKPHPEwp5QQ+AeTIo9i/OGXEKQXPOK946GrLsyn74c7iTJ8KPhLtXvy5dWcrB6vKWSg2C4XOyrMwcsxEyw/pcBWsnRBnzmuLOeG247n03ks5pCM9BxLKGCx0OiwKkXG+FY/6dFJrsks+uYSF5y9kwUcX4HF68t5XKBrieM/xLJq2iFg8xuF4Tw/6VLtCneeC4SAzHpyR9fgjvkf6fM24mDh6X6Mva9GgfByt5SOTTEtOoQ6TglAucpVe/SHwg+RbDfwtxx/UT0ssV99S4HMjnojjf2k5s8+7KW/bYDxUXMyBhvDQI4oipXz6xBmuFNdUEIvHCjZfh5xALM8adsrX0YKclzd0N0NbyjKWsf6auSb7vY9/D/8mPyu3rOSJt5/oCmtLJxUHHQwHeXhr9sp9NVU1BTvz5bqHmqoadn2wq6D+ykG+OPrN+zaz7OVl3ZwFs+UEcNldaLTp55xJ5ndYTGY/Mwe/H0/4Mc6dTimUI/Q5uRzlWoD3MB6NvwBuB97OaBMBtmqt/1oW6foIpUEXoLhidtjx7GqwoNTDiVhxSl2B95QPdb0tJpyqIPIpxWzHC8hit8oW4E8n/4QZW/8TWx+MTuzKzqwzupuhSxECl1Ly1zZeS90ddabKJhUHvWLTipwDm7iOF1x7O9c9HIodqoiKeZNHTqbaXk1n3NwE/+KeF3l97etdzoJTRk3JmhOgEMfT9O+wdWcrl91/GdF4lHA8jMvuYu66uayZuSarcs7m4JfQiZImDxKEYsmVJvYF4AUApVQQWK21fq+3BOtL6sIOdldHLSsnWwLqLSahy+aJnRcNwbS88cWEU5WUbJ9NAbpZK9h163dpe3MIaxdpFv6tCq0Ud54DOho1ZuWlli+NKe6TezyALWcZI/8sz0pN7UB7gEPR7NEWMybNKFhJFHIPfUVX2dMsSh2OeMNPu2caHxv3seKjRtJI3X/bwTYuvufibgOucDxMOB5m+n3T2fvtvaafezGWHEHoTSzNGbXWyweLQgdYNXpOQe0V4DvRmjexo9gZqeqe2rOhtgGX3VVcXxavV24iVbBjOHg+OMyIQ7BobYxb10TZ8wsn33jFgT1O0Yl1qlT+aM2Xg9u56Y83dPOMtpplzKozXeaa+5JPLmHKqCldM8FciVXcDjcXnXRR3vvIpKBMaX2E1+Vl5TUr8zfEULbr31pvybyeD5uyMfaYsUz4xYSs/R2KHmL5xuWmx8qZzEgQSoHVkDaHUurbSqlnlVI7lVL7Ml/lFrQ3mTr7+1z7us2aQ5iGb75ox/N5a97EIxzHFudkpmHScZO63voafWQrm9tfsCXg1dGwbKoR555Ca81d5yjidrIOLoZEjVe2z9Jus3eFjOXiro2/7aaQrWQZy+VMN+2eaewJdg+9S5njF01bxOyps7sp3HIo4P6SKW3nwZ2WvqNS4LQ78Tq9rLx2JTMenJF31m8WFQH5B2GVYAURBjdWk8/8DPga8CjwJMZa+sDF68V/81Oc+qML+c+PxvPOWj/5pvVF8mmNn2bZJvNZQD7SE6V4XV7+/ex/Z8nfllR85rhsJBSsbYC/joMfjwLnOGjaCf4Jh0gk7GCWzEaDKwY3Pw8RG2waCU+cDEop4mjcOFEOJ7OnzuZXL/wqtwDJzy0z6Um+LGO5TLDheJgJv5jA41963JLTlBUTfTGUJFOaxbrplrtLLlds3reZ/Yf3s3nf5pwFf0qF0+7kjkvvYNYZs1ixaYU1X5Qsf1O5sttVihVEGNxYVerXAAu11reXU5iKoqmJzrnfhBd+lrudgk/NiLL/geV4vprfUe7c8U38v38sJ16EIl69fXW39bqyzgrKUbY1k5RSdUHCBs0zYdsvYOVpELJnCdVTxmDgzrON84ZEQCXAE9E4EjD53QjPnxDhzr8vJUa8oHtJXxPNlWUsnz9DZ6yzIKepcqUqPapMaUXWTU/3Mzg7ejbBcBCvy9vlMR5NRIvKCmdGTVUNdpudW6fdysL1C4nGo93W6KurqnHYHN2y8ln1RcmMikhRrkGYIJQKq0pdAa+VU5BKpGH0pPyNMOKif/lPP98hv1L3NfqY9+jN3ZzeLKHosV73xv43yqZ47XGw221EVO/F30ZtMH4OxHIZPjREq4wXwOGkM90HydwtG2pIfiZpgwKLn1EoGmLHkyvBvz3nzNRKethCnaYqKlVpMEjwM5fhb+ggMBwa9oNvUwhvhB5104PhIMs3LufR7Y9y4PABNu7diF3ZORQ7xB0T76Dujrouk3e6x/jRYk/ANcddwNLr/oDH6eHLU76Mf5OfLe9uof1wO7U1tUwaManH4MjKdzekakiPqIh0zAZh4z8YL+FsQkVgVan/Fvg88HgZZak4fI0+vvqnr+ZXCgqWOl/jOxb69Lq8tFy7io+tmF5wBrbMmfne0N7yzKg1fG6bnVWnKSIkes28bzlhTC6OQlZ7HNr+9jjBP63F68g+M22ub+YbsRtz9tWfnaZa7/k/NH+tgwSGNcQdhnnToeV+aGo/Uje9dWcrl957qWmYGRgDm1S52SpbnkdNgb/juA2O/+Nf8HwBcFofFOUrDuOucrP2S2vzzrgzr7dhwwarogtCWbGq1PcCM5VST2Io9szcpFprnWcBs//hdXkZFob3q/O3LWTm3XTqpUUp48z1ul0f7CqPwlXwh4lxIr1dmNfKZ1LGAUbcDg+dEufh+XDjCyHCdjjwi4sYfuBrNNadga/Rx8a9G7n0nksJJ3KHPPZXp6lgOEjzuz8nmBZYEUr+v3kmtN0ewrNjB8FwkMvuuyyrQk8nnojnD0cr8Ht1h6H+gOoaYFjFzHyeSl4z99y5fO/j3xMTutCvsarUf57cjgM+bnJcAwNOqQMom51uplwzNDgKdER32uxE8vWbxlnHTenxsHnzwJuFXdQizmiyAm1vUwEOf6nY+MXnk8xMF4OX76T6tWrmrptLLBajM9GZV9b+6jTl3+wnkeXLjwP+qU5m19fj3+wnkrDmLxvTMRw2R9YCLmA4P2oNEYvWGhvgezkCOwq3hki5VWEgY0mpa617e85WMSinC8jjoauMIiKFdWwHbV2pX3fOV3vsq7KXp3KuVskCLb1FiXLJlxTVfWvVucsZBVeVo986TQXaA4Qwn1UfcsGTY+PM9vkIPPdjy3HjNVU1JEjkVOqO5J+CaY8a3FFjwOUOGwq95X7wONxQX5w1pKJ8GAShhEg99Tx0qKglhXPAXliUX7WjmkjE2jkuu8u0qtb5Y89n5/s7SzvD1XDNFlg18YjZtdwcfxCO7aS8HvfaUBzRMv/iT2mHx/6gOH4ahl0rSTav8EqjobaBIfYhWQvMPPQhO//tNNo57U5Lij2mY/z5839mxoMzjni/J7/r6ohh5Wq532jbPJNua/k2YKUfdh1jJCqq3w++zeCJAF6b4cwoCEIXlh9xSqmRwHzgLGAscKXWerNS6lvA81rr58okY58SybN22tVOFTbVPH3U6bTuslYG0m4zC9iGWz5+Cw/844GCrpsXBX+a2IuWcG3EpldNojwXTX0tKtl9InmdMt3g9lqYeEOElq9eStML+8Dj6VEAJOUVnh5qVSrSvdHBqKs+a8osywOIcUPHZVXoYHid+2+5Bt9JlzNXOYhYSFkRiUc4fdTpXSbvLf96lfZ7/pvaD2JMei9NSQNttxsm/h1DY9S/b8P3aqzrWDeqqw0nxiLKvgrCQMZqRrlzgABwNUZRl5OB1DzueAxlPzApk1m4kPVWhTKtqf2n1x4spUhdaAXXv2RkfCu7WVzB2IMYddHLca00BR6pwvjFl3HEEnFA0AXNVx6m44HlptnnUl7hpSoDmqJ1Zyujbx/NzWtuZt0b61j3xjpuXnMzo24bZamOeDAcZMYfspdsBQgRYcfza/HOW8ia+xIMsVkz59y4+kZWbFrB9vbtnFY3hV/etJrb/+Zl9na3obRrasDlwnPNTGZffxeLGr/F7OezKHSAG2/MGS8vCIOVQjLKPQlchfFY/Le0Y88DXyixXJVDAQqgI9JheR31qlOv4uY1N1tqmy086vfPLC2Lggo54a5zwRWHw6UIM8uFhhOChiKsBEe5UhEH/P9cjd7s6pUCIMFwkMvuv8y0OMzh2GEuu+8ytt+8ndWB1VkL0Fip/OeKJVP7Tgzh2wT7fu7hNyt/wH88/f2cxYoe2foIj73xWFeylnnKRsvzK2l6Zpfh7FZfb5jSUzPvZcuMhDchk3hytxsmWcshIQiDDatKfSrwGa11QvUsqt4OjCytWBVEAeu8VmuqA7TsaKG6qtqSA1a28Kh94XYok9KN2DFP01oGXjoBzuydSxVNdRQ6q7D8Wzjkgh0eSPRSARD/Zj/ReHZHtM5YJxN+MQG7sh9RrMmypoVkWwvbj6T2nTcdbn06wi1P/chQ6Hk+m/Q8+QDNj8zInnXP5zPyBJhhk7V0QciGVa/2D4DjshybgBHHPrhRsOWZVZabB9oDlj2qs4ZHJSrNZbwIFDxxEhU7S//omI8yb+o3WPqEi3X3gjdsOHDloyoO9R+9vNcKgATaAzljwWM6Rmess0cBmvQlgFyypvsmgOHIFnTBTdMiBFWkqO8vEo9wzYPXdKuS14XXa6yZe73GzByMbWq/rKULgilWlfoq4EdKqQlp+7RSagTwbeDhkktWKVh9WGlo/2BP/nZJcj5Ak+TLKd3RS97pZUXD3qF9LUR2HvI9xO2f/iWzF6/n0ne9tP2qhiVr4ZI3FdnS0wPYqxwctms27dtEPEvoYilj2YstxZtaAoDcFePKQTgeZu0ba7OWraWpyUhJu2QJLFxobNvaZC1dEHJg9S94IXAQ2AI8ndz338A24DDwg9KL1ve0HWyz3lhB7THHW26e6wHqsruYd968HrW3M0n04gO4bFToDB0MB8XV25MlOJMKxnPbL5j4yZk8d7LTKA2bBZu9ioXrF7Lk70u6ZrnVVUZqQpuylbwMqq/Rh8Ne+FpMKBpiy7tbgOwlW502R/bvycr3l8egZGY16MLjMTLGLVpkbGWGLgg5sZp85oBS6iPAl4CLgRCwH1gG3KO1tmCQ7H985y9Wsrkn0TDp/M9Ybp6v2lNKkT/x5BMse3mZqXPTCdrNP/X7Fa0YS0Kp4tc12JIKJmFhPKTR3de8PR6CX7yW5jvm0hEx/8k7bU7sNnu39KmpymGJRIJLJlyCx+nhJxf/hCmjphR9K9A99r2htoGHrnmIKx+8soeznMPmwG6zZ13uufOFO7nytCtpGtdkmm3tcOwwC9f9B6GEyfklTO1bSsdBQRisWI5T11pHgN8lX4OCze9uLkiZ+M7MXtnJjHzpKlt3trJx70a+//R3CBHGjYt5a+fS8sU1NI1rwrdJsbixoEuak3wwpwYVqVKWHeEguhIGDCWUIWHDcuic0+7sseadz0P8gpMu4Lld5ikbIokIT771JNMbpvPD9T/ku3/5btGx6pmx76nv7hHfIwT2B1i9bTUoo4ToVaddxcSlE+nEXKmH42E+sfwT3DH9jq6Y9nTFGgwH+V9rs0StWpypVyXAVeUilGP835+L4AhCpVBwfi2lVBXgzNyvtc6TS7X/4bQ5rc8SNUWZUrOlqwyGgzQvv5Qf1v+4K21niDBEwzTfM522/7WX5z0HSqLwqhLwzXO+waQTzugaVHx5ype54KeTeCVapqIxhVCqmXpG6lcrZK555/MQX//m+pz9xbSRTzjVxyX3XsKb33yT473Wl27SY99TpPqb8aDhUX7T2d2jMFpmtnDJvZdkna1HE1Hmrp3LvHXzmHveXL73se8dCXd74i+ocMTkrx6csWSkRK7PVMHXn4czzvoEK0/VPPH2E6aZ6PprERxBqCSsJp8ZqpRaqpRqAzqBoMlrwFHvPanPFJr/wR8QCZtn9op2HsL/0nK21ubpJM+MVCXAHYEnz7mT2z/1S2ZPnd01MDnYeZBXYhWg0KHPZJh77tweA7Wxx4wt6TU6Y52MXzK+m5NYMBxk2cvLWPD4AlPP8FzWgnTHt3SaxjXx9TO/nlOWmI4RTURZ/MxiTrjjBP74+h+Z9eAXOH3dlYbiNkEBKk8JA3sMDroV24/VXH7K5TjtJqMD+m8RHEGoJKzO1H8NfApjDX0LWeouDDR2vPF367PEo1Q83dJ7xuLs/dt6wlkmb50OI3wunO/byyGTQ9v4gucjLJ39EJ7ho3scL8ifoJykir30sk9gFTa+9/Hv9cq1wvEwzfc30za/jVffebWbWd1pd3Lj6hu7zZ5zWQtymbBPHn6yZZk6Ih1c+eCVxmd/bA7ZLTxB4nZYearmUMda3Ov/SkInqHHUoFA9fEn6YxEcQagkrCr16cBcrfWycgpTaUQ6Q9YSsGgYUkQ4UYrWna1MYD8oTgAAIABJREFUv296dwennnq22/XaP9iDyp38K+eApNrlZum31mV9iG59b2ueznuPqgTEelOpa5gWP9H0s9n1wS7L3diw5cyylk5CJ1j+6nK+85fvdDOrp8zUi59ZzF0v3MWamWu6wiHNFHvJTdj5BqsWHeUOJSfnKZk9Dg+3XnIruz/YLaVPBaGEWH1UhoDd5RSkEmn0TLDmVKWgOlFc+rWs6T3zrFF6jxnZ9aDM1W5qfCTVVdU4bEa4k9vhthROdeoxJ1dEOVRnrJcVOoCCSceYK8aG2gbD18ICk46b1PW55yMUDfHo9kdzOuF1RDpovr+Zyxsuz9ouHAtzOHa4ZzIXyOrA1xdoNNX2ahZNW9Rt2UcQhKPD6uPyduBGpQZCYLR1Fn3pbsttPyjSTzBfek9TNARPHE3cwrdRO2QE7/7Hu/zq8l+x8PyFLPnkEtrmt+X1uF60f2phMhWDBpeZ91UaEQe9vqbuimYPT/Q1+tDa2mjnug9f1xWbno9UXHjeNK2xMA+//jA9szUbxHSMeevmUbu4lgXrF3Qp92B7Gw9vKl0BIHfYGHAV+92Ip7sglAerSroOmAJsU0r9Rim1OOP1X2WUsc84wXM8NWEszViLzcSVL72nKQpqh46mrmp4btk0HDdhcpeHfSGzohPeepfb11HW2brS8Lkpn+eiEy/K07B8MpgRc9jYfPANUyc1r8vLh0edbulzqbZX0zKzhSpb/lWueCLOtAnT8raLJCI89PpDqBwfSjQR7ebw1vroXfivmIAtEssvtAVGdMCS9XZuP/GrDCnSu0Y83QWhPFjVRDMwKlFXAZcA15i8Bh7LlxsmbgvrivXkc0U3p6G2AZdFc246k46bxD3X/E/uRgpGDxuTu00waFTEWrDA2AaTSqyhgaE4cebxbD4atA2Od9Uy2pvLgaD3iZPgZ3/7mWn60mA4yMvvvGxpoLH74G6axjXx6VM+nbdtggQfdH5gSb6t727NO6NP0RHpoPnZm9jsCedfrkmRZ6D4nWdh9utDmFU/g4Sr8N8uiKe7IJQLS0pdaz0+z2tC/l76H8E/r7Q8S3QVqfx8B8dR1Vn4dMc32cfGYCCnfHZlZ9JxOUpUtrZCXR3MmQOLFxvbujpjv89H4JiYUYO8TNREoD7Qzvb27eW7yFFglr7Uv9lP3KL5fcwxYwiGg0dSzeYgkUjwu1es5XWKJCJ56wZ061vBXg/WrC552tjjcMOLQEcH3itmcPXEz1qSoUoZPySrPh2CIBTHoFojL5Rf17xuraGCt+xFhOoHg3g/fTVN/6QoM/eqrXmqwumeyVPSr01zs7FN1awOhY7sV4qG8WdZqkhWLHYNvj21lh3P+or02O9N+zZZzrIXjobxb+4ZM25GTMdMndvMGOkeiY5bN6WHnPCO1TGAIudA8aw94EmNQRMJLtpbbakw0czTZxbk0yEIQnFYVupKqQlKqV8ppf6hlPpXcntXRuW2AcVdJ1vP2KacRYS0+f0QiXCgmoLXjZe/upw9Hbmrwo0bNi77bMjvh0SCoBOWTYUF04xt0AkkEuD34/vwl8o26rPFoeWhajz1kzjj+DPKdJXSkO7UdeDwAcvnrX9rPYH2AJFEfktMTVUNNq3yD+40fOmUa9Cd1kdb7gjsHkpJfBOGp+dDCoXw7anN609iUzaWNi8VT3dB6AWsZpQ7E3gVuBp4Abgnub0aeEUp1Quu0r3PzhrrXulTRucuzmGaJSwQgEiEkw9Q8Ex91bZVjKgZkbPN+GHjsx8MBHhsVIiR/wE3Xg6Lm+Bbn4S6+dBaG4IdO/B+fhYtjwzBURr/qiNomLoHmvY4wOdj6rGTej18zqpXeooxQw3fhIIUkk6GwGXJoJaO3WYnFOvIq3irlA3v5kBBg60EsCP3T+UIqWQ/WY5NezPtvduNt76Rlun34E1UUR3rLrxSSkztgtDLWF0xvQ14BbgsPce7UqoGaEke/0TpxetbrISMAaDh2snXZj3curOV5vsuIxGNdi/MMu5GmpxOJu0rfE19T8cePnTch3K2Ge3J7oD22KgOpn+Jbkok5Ug17Utw47AXmRTw0/zL/+Gzv5/BHxripVusUZCwK2hpAY+H5tc6eeCY3nNx/7cp/8b/bMrjZJiOhvChgwA9S4Pm4PKJl+Nr9DF3zRwi2ZIwavA43LTMbOGC338sr1L3nXYNu157g1AhKxbVLnTC2szeoRVRlX2E5Uof4Nls8MEHNJ15PW1O8DfClhHQXgO1H/0E44aOo21+myh0QehFrD6mzwEWZxZtSb6/DTi31IJVAjZrycBQwKwp5hXaguEgzfdMJxjt6FaYJRjtoHn/L+lwO2g9kYJNo8d7jmeUZ1TONtm8yoMbHuOK/XdlvWa4Cn4WWs/NLTdT98yV/GFiCRU6gIbqqecYNcqBlo1/6NWZ+ro31hUWRqhg/d+MQcCxQ3LkTE1jSBRmnXw1XpeXNcNuMkK/0u9RG05nC/5exZ6R/0XTuCbs+cIiNUw9bgoNx52K2+I40KltJCz8tlQCPGH4eLs3Z+303ccAbjd4vXDPPTDfqN7micDsV+D2x+HuVXD7gicY4RwmCl0Qehmrj+rDkDVmazhkqenYzxkdxNIa51mR47I+vPwvLSfaaZ6YJho+jH/JDWArfJb6mYmfoXFkY1YzcnVVNZNGmHi+B4P4//cVxPNkrIMjdcBLjoKpY87ukmfTP1/o1Vj0to62wk86ZHyHk0dOpspCet5vvlSF5xHD671pJ/yfJ44cSxG3wbj2GJ43jWSNDguphp/5+x/wfXFRV134fERUIv8ARsOMLfDyMhuttdktETURqK+bDEuWQFsbPPJI7n53D7oklILQ51hV6quBW5VS3VxWk+8XAX8utWCVwMXv5pi1pFDwuutg1sObnn6IziyZQjsdsKXtVT51/eKCZqp2ZWfWGbPwNfqypiF12Bzmnu9+P4GhcWJlDFXLh9Pm5IzRZ3TJs2lknhNKPIvPlbgl2/VPHWr4g/oafdjyna8gGo/BDsO5rq3Oy/zpdPcsT/7/pmZ454ShBMNBDsfNq/Kl97uh83W8tSfw6w//wNLn4rQ5cWUbLKTWzxW0NMCHbkiAI3ta27gC388eh9mzweOBrXnqA3QOyLG+IFQ0VpX6POBN4Cml1DtKqY1KqT3AU8n988slYF+iTj3F0oMzSuxI0pYMtra9ltPxqP2fW5l17tcKmqlOHT0Vj9OD12U4IXmd3q6worxxwIEAQRXr07zuripX14AjGNjEhnH5asSW9vq60JtX8Nvwc3REOvC6vMw5cGreBC3tQ+1Qb2RM+3bojzm7n9/xsOXQt/2qk7teuIsvv/4Tq9LjsGdR1GmDjJALwg7ozDGrnzHm0u4V/U49NfeFqwtzRhQE4eixmnymXWvdBFwO3Ak8A9yF4Tj3Ma11exll7DOGn36upWxytnD8SNKWNILhIE8M259zjbL2/Qh0BC0Vg0sxwn3ElblpXBNt89tY8sklXXHA276xja3vbTWtx/3Y2Ai/OodeMXdX26sZUjUk54DDf8IBy3HfR0VyVjptwjRqHDWFnx6PdcWqTxzVmLuxgtpOG/iMgctznTmSBCn4W2eAQHvAsiw3tdxETMcsfYdTj5/Kmplrug38XDFyDjTNcEfgoilXdN+5aFHui4/Jk81QEISSU5ARVmu9FlhbJlkqjsYxHzZ8/nOhjIxd/5+9M4+Pqrz3//uZNcnMgCQoSwJFTIASFPeljb0gihq8XQSdurS0pZugLUt7icu9rW2tXOtSboX+WrmtWK2MYi0VAijWLeoVl6IFBBJcgAZEEoXJJJn1+f1xZiYzmXPOnAmJBnjevvJKMufMOc8Mcb7Pd/t806Itzc1aaBJNfUwbvGHwSSk1LyrwxdGIC7D8rzGtalrW7yltd9Aq7cfeOzY9j9vj9DB//Xzqr6ln4pCJfOnj3/apQbcJGw6bg3nnzEvPIw9sDtDU2qQ7YrOxspTEa323nkxePm4Bhz43lYsfvLjg54ZkmKbWJoLhIN+J5lEalOCccWX676DIWQJx4xRNEXZGDBxhbYxpgQwqHpTe+AU2B2h6ZiWb3nySdScZFAYY3N8mwb8ZOC/jweHDYckSmDMn9wlLloDjU8zxKBTHKAX9XyeEmIpWCT8M2Au8IqV8qi8W1h+oray1dJ4z9fmYFG1hlmZgG1saiZhVNAm4d8Q+ajuxnON22pzMPNWk0v6h2qx53CmN8NqHavnllF8SNxntebjYsPG1U77GvbX3Zhnu1IZDj6ph1ZSIns+it0pRDF4/Yxg3PjKjR8934aCytJLAlkD+anIBS5pXcUukDa/Ly4TR57Btxyp9gymhevvHMKVv8s/jyrQQeXrjF9jBsi3reKFCC7l3xx0BIcCe0I57wlo4r/4h8F6tU/g2ezZcfjnU1cH27QTHjSbw9dNpjL3PWe0nEAwH8bl9ffLaFApFLlbFZ4YLIV5B89KvB85Pfl8vhNgohCjvwzV+atQ31Vuqfr/g3eTPoVC6OAqSwiN5AutROzyeJzWZdX4iatgrHdgSMJyznZAJ1uxYQzRR4JhXtBGbwsJewOPy5Bj0fPir/didfS8T2+mE1e9vMJ1X7jAR2YkQY9KoSZbD5BKZDtefMLDcNPw+JCTY/erTfRJBue+N+7L/Xqqq8L9TYvg/ftgJf3wcpm+Fc3Zr37f/BmpaPOkagRyGDoX776ch8CvKxzzB3Jd/wh0v3cHuQ7tzBuIoFIq+xWqh3O/RvPMaKeVQKeUpUsqhaMZ9KPC7vlrgp0lji/nAlBT/9n7yB0/2B5+/2p/XM444IFFgD3jdhjrdxxtbGg2nd4WiIQSiR/lkewLWrXTjE/o63x6nB6/Ty3VnXcfPn/u57shSI3xuH4suXFTwmrrjsDmwmfw5pyrAzaabHec+znQT9+Mnf0xVWRVFtvyRhSxp2c6PTHPYq06K4/vI2tS1QonGI+nNBQB+P764nZUB9Nck4Kor4LFqeGUEPDYext4ADSMS6RoBPTKjRKn3OCETOQNxFApF32LVnFwA/IeU8qXMB6WULwJ1QJ6B2Ecm2gd4ngpeAR+m7JzNlvXB53P7qLDnESuRhRv17Qe26z4+YuAI0+dNOWkKdlFISZ5GhxPoDNPcMTtdkHdv7b3ce+m91H2+juvPul4LOW9cwh0v3aE7stSIYDhouEkphDOGnkEC4w2URDJtzDTD4SMep4dDcXOZ1id3PqlFFmL5dXMz54Xv3r3F1FPffRyssG0trCPB4rnhRCS9uQA00Zj6enYNdeMxCtoI0op1ITcE3VB7jaDNJKCSL0qUtbFQKBR9htWc+gdoAjR6dAAHemc5/Qt/tZ/56+fTGTHOd7qiUBGygc+Tlj3NxOE7Dg62Gj5fAM4YBY04HTt4rO7jnTHzvKxAUH9NPZPun0RcFjArVsAXr4IDnxmdkx8PhoOU312e5Yll5vG3X7+dNY1raGxppKqsCn+1tukJbAnQ2NLI7oO7CccOfxTcoOJBeJweQ0983jnzmDlxJjc9fZPucRuCeDyGWbYkkojw5gdvEpHxvEVt6XnhwSA79rwFedLKW0VLYeF3qyOBY6Q3F2lqami8eTahV++xfLuETRDYHND99w9sCXDf6/eZRomyNhYKhaLPsGpKfgn8TAjxupQyXS0jhKgAfgLc1heL+7RJ9YFf+qdLaIuGdD9IIw6ou8TJxD+upGZc7jjJUExfTS7FQOkmEg8XZNT/8wv/qfv4hnc2mD5vXeM6BrgHUDOyhufef876DdHUzwIToHvJm5mHFo1HOXGxNlQmHA/jtrv5wdofAJrRMwuFF4LH6eGyMZfx4u4XdY97nV5u+bdb8Lq81F9TT+1DtVndATZhY6X3W1wcXmx6n9LiUmofqiWaJ9jhlKKrbe+BZRzM166dGs7WB9XvTmy6IkRVQ8ebboK6o2eYG3Y1ZL2XRmRGLRQKRd9iNfA7FU0mdqcQ4mUhxCohxMvAzuTjFwohHkl+HVVxtpqRNez98T4WjroGZ5zcsKeANsLUrvyKbt7QbaD4lqLIM5CYXee6Btiw8ex7z+oey+epb3h3A3PXzS3YoAPE7NAUyq1+Nsvjd8Y7CcfDaZnScDxMR6yDjlhHrxl00KaBzTx1pqEQz9pr16aL9/T6+psXNLNr77auLgY9JHxuxOeIxM1F110xuOf829LzwoONmwlb3bD1skG3x2Ft7Z91Cxf91f68I1Mz6W6Y9XLoRqSjFgqFos+x+nEzGGhMfgEMQNN7T+XYj+/ldfUrvC4vi77xIBUbxjPv+Zs1I9yNzkg7y1/5HXPOzxbX+1xsGO/L3YbtTAc6DhQk2ZogYRjK7Iiay4xKZI+Nqcvm0vW2qsqqCvL4eh0Jc86cg9flze7HNuiLh+y+/hSNZRA1qeUSQhuik09HPe6wMfP8G9K/B4Z/BFZHsBfiqec7NwGL28+n5jx9Y5qKQmV62iWOEtoNIkvdDbNZhCbzOWr0qkLxyWLJnEgpj8pCuEJ5/tVHdQ06aK1p857+MRM/c07aSwP4acvJPFy80fCaTmkjZqVfLIkNm2Eos6A8eaEIdL2tVN3Bp4YgKfCjoWewLTHus/DqekND6bQ5tSr2PMRJ0JbsTwdNXAcr4jqS3p2EJ+C50Q7cbyzDX+3X7RXX2wQdDB9kwZO5qs+LLlyULRpkEqEBOLf8XEYMGKFGryoUnzC9+TFyVBMMB/lLeJPpOVEhqX3w0qww/PPDo7gMbK1NQofIX0md9RyTUGb18XnkSw+DeefM0/1wNtKf70mVfU9wYz/sfG0wHGTpW8tMPV+n3cWHoQ8tXS+zmr9qWDUOqzq4BVS/l4uB5ucLePTDZ5i79oemnQipTdDtF97OldVX8tNnf6p7Xt2Guqy/61SERg+P08O3T/82g0sGK4OuUHzCWDbqSQGa7wkhfiaEuKP7V18usj+w/M3lWPGDo9HOrPadxspSwyK4hA3DKWtGLDhvgeEH5c3n31zQtaySKjQzQi9P/dNJP+2TtXTH6Sy2nK8NhoMse2NZjiZ+YEsAKc0taigawmlzWspDZ7Ycjhw4kpgVa22joJx6yUCjScjZhGLtlnvFC2lLM8vJqxy6QvHpYSn8LoT4KrAc7WPnQ6B7tZAE/qN3l9a/WL1jtaUP3U5ibP1wa/r3qmHVeGxFhBK5RWwlwk3CJiyrvOUzrk++86Sl62SRmZtN2p6iKHS6tCEeNncR9RmFZoaXkRKJJCETSKThnPfUfVzxwtr49K5RYndnFcGZ0b1SO1MTP18oGTTvc9qYaax/Z33ee6VaDoPhIDMemdF7efIMmg6+W9AmIGWUzVIT+cSLMms59HLyqU4ClUNXKD49rH6s3gY8BnxfSmk8mUKhjd08tC/9azrnrNPrbne6ePzKlcx4ZEbWB6NEIqVMe0JWPyxXb12Vd3mporYSZwntkfZsw5Ax63v+izD+APgvno13ZG6rHnT1KD/z7jP8ZdtfsAkb7dF2PE4P0bjJRkXAv70Hz55kI1pAPUEKOwKHw8XjV/0tq37BCCua+PmK/WzCxuWfvZwb1t5geE6KlEKe5vlarHMosJ2t0PGxVnrFzYoe9drSrBYmKhSKTw6rRr0M+N9j2aBfNuYy1jcZF1KlEVD2flfuNZ9HY/TBCNp0s6J/FbH4ksXWPiz37TU9fJpjBHMu+QlNrU00tzWz8s0VtOcEXTRZ2PEHYNYWF8wZr3utlOcbT8RzKqbzer1hGNZpx2VzEJWFC8/EkcTjYWY8MsNSIVa+sLJAmBrJVH/7msY1CCFMQ/XjysYx1KvNHNc8X3Odgiz6oE89hZVecbOiR6OQeo8LExUKRZ9g1aj/BZgEPN13S+nfzJw4kxvq83tpSHB+nL33yefRGH0wzjp9Fs8eepZJp0+ytMbL3nOxfiiG7XNbo82MHTyWWafPYuFTC3UNOmjSoE2lgMulq/et5/kWgg1t3nioBwY9EyshZcgfVt7ZutPQUDuFk8YbGhnqHcoT25/I28Z1/TnXp3+uKqvCE7cTsvduV4Jd2LFLiFiq8tCwkudWIXWF4sjHaqHc9UClEGKZEOJqIURt96++XGR/wPL4SAFL7K9nFSUFw0FWbF7BjpYdnFR6EldWX9n7H5DBIDP/9BYlJnreYRFPF0yZ6sRLqOhwwNq1ObK3YK1HOYUTB57k3sETBl8Y6h8vZvy18wyrp1O47W5OPO5Ew+NW5UfzVWq3dLQYFn25wlHW1P86fR2zgTgu7MwcfXn6d3+1H5GnAK9gJKy8ciUuab1xJRVpsPI3ZyTOYyXNoVAoPn2sfjKMQZuj/i3gQWB1t68n+mR1RygyGiPw+nJAC1OX313O3HVzCx52UhCBAL5OyfoHwR3FsN0p5d12Rk3U5wSEb/sZ1Oh/kFspLEtx6wU/Y/qpV3OO80Sm2yew/TN3UvPqfmov/SGxhHk7n5SScyvONTSkVuVH81Vql5WUGXvyLmi6/25oa9OGuRi16kl44hEH3hPHQoP2b+tz+5jtvcBaq5rV0LuEL4/7Mo+Vzc57XYdwsPDzC9m7YG9BRjmzzW3W6bOUh65QHEFYNep/BA4B04CxwIndvkb3yer6GxadrpAbml5aoyulGYqG+mYcZWMjJBLU7ILrXsXQQISiIbbs38IfNv3B9HIb9rxgeGzEwBHpUab5uPW5W3mscRWvRN/lseJ3Gdt6K0vffoCqe6vyqrNFEhFWbVtFu0Fe2mrrlFEvfUrtbPzx4/FIg9ZCqW0uCASyr+PQNhrOuNYtsP5PMHVrGIJBqK2FtuS/7YQJloy14cS07iSvteuMSuPnSLhk1EV8VPdRjmiMQqE4urGaUx8DXC6lzN/PczSTwNo2SMLxH3Va6vs1ygenKsuLg8UsM1EFS1NVBS4XQSK8fTyGnl8RDpa8tiSvhrmRIWrY1cCNG27Ma5BTZGq/pzY2c+rnWHoukFWEl6rM7kme16yuYaJnNPMiUdDbpwhYclqMW5q24s28zm/n0LTuz1R+EMO/BbyZb2ciAYEAzJpF1bBqrX3PTItHwtToCB53GcgJd1sPQGNod3o8qt45p5afoYy5QnEMYtWobwRG9uVC+jvBcNB0LGd3Xh/QTkUBfb+ZZPZU33rirfxk3U/SPdWGYVS/n4Y7f8Cll6PNvTYwDp0yhpX6qmlV03IeS0Ue2sxE0vuIEkcJM8bPYJh3WI9bp4wKEn2P1zP7DTt3nBvXfd+idlheto/UVsTr8jKreSisM0gfhELQpP3b+qv9zP7rd/KubcJpl/D4jvvynuckv1iRmoqmUBy7WA2/zweuF0Jcm1SWK+n+1ZeL7A8sf3O59ZMFNA6I5i3Q0vvg7WnIPuiC2msFbW6MvT1pbV9S7Chm5qkzcx4vpECut2mPtTPMO6xv8ryNjRDXN+gAYQfMO/Rodh1EVRV4DAr9PB6o1P5tfW4fX953XN7UzeP/DFgK0w8sGajJ2r621PAcgVCKbgrFMYpVo/46cDKaqtxuIKjzdVSzeuuqgpTBnM6iHklp5gvZL9+03FDqNGHLs0ABcQuv4c6pd+oazUIK5AActsORjMump96nkTRsFlVVVLW58JhkFKIymr2p8vvBZvC/j82WbgUMtjTz2NCPzf92BAQ7D1mq2ShxlOSVtZ1z9hwVelcojlGsfup+i4LGTRx9xPbmzhI3RMDJQ07uUd9vvp7q+U/Ox2lz9kjq1BWDhNNOLI/KWd2GOr4+8es56ytkzGqxoxi7sNOW6J1QfU/0xM2kYbPSGH4//oXzmD/ZvM4gqw7C54P6eq0oLpHQQu4ej2bQ6+vTrYC/W/5DEvm2zhKGtgveH5j/f7FhvmF5/61FXynYKBSKfo/V0av39/E6+j2DQnHt3bLYdtQZ01rGCpXSzGc4I/FIusitUKlTl6sY7Pa8OXGjIj4rY1bddjdOu5O116xNr637hmbRhYv48VM/zqlqL7YXc+fFd1K3oe6wxU/yScNmKdH5fPhWraX+O1OZckWHYVFbTh1ETQ00NxN8eDmB99fQOEhSdd40/GdMJFXS+PuPn86f8xAwsc3HK+QXbDyr/CzTv0FPBCpbj+n9t0JxTFNQfFQIMRw4DygFWoGXpZTNfbGw/saBEgEFtB35RFcpdSFSmv5qP/PWzi1obSmp03z57seu/islzhIufvBiwzYxMC7iM4o8CATfOeM7vP3h2wBMGzONiUMm4nP7DDc0laWV/PvD/040Hk1LtNpsNk4Zckqv6IkX3HlQU0PNq/u563+/y4KPHiEicqMZeimAhtZN1B64kURJglBbCM+zLzD/uZvS0YA9dguZqQR0jq+C6Ot5Tx1XNi5rtGt3hAT/dUug9hZd4SCFQnF0Y3VKmx34DfAdsv2OuBDi98ANUn5KFVSfEB/67No2xgoSWre/2aP7vPnUn0h0hLKjAnmESULREO989A5xk7B6sb2Y3Qd3M+v0WXzwow/43hPfY8XmFSTI/Wczy1/rRR5GDByRNZTmhV0vcNPTXYat+4YmGA7y5RVfzmmrC0VDTP3TVPb/eP9h64kXMnEsjdfLzO//jpvuXk1ERwK3ewrASjQgJrCUuCo75Vx4I79Rl8n/DA4yZyN4wzLdUqdQKI4trBbK3YqWV78JGAUUJ7/flHz8p72/tP7FoKJB1k8W8GHbBwXfI9jSTO1Lc2h3oj89zQCP08OBjgOmxVMd8Y60IfO6vPy/y/4fHpd+9Xa+/HXmmNWOWAeXBy4vqFp/+ZvL6Yh16K8z1sHyTQV0Ghhg1nkAxlPO8gnVZEYMAlsChmNzo4kogc0B8qrEShgeL2Z8+anmxl/CmbET2H1wt3GKRST/VDJa6hQKxbGFVaP+deAWKeWvpJS7pJTh5PdfAf8JfKPPVthPaNu/u6BSQUdJ4aHPwIM3ErUu6Z3GFotTWlRqOpfdIRxZ3nchxiuT7rK389fPNzQycRknsDmQ8/jqHatNX8+axjWmx63gr/YjhPEyba2wAAAgAElEQVRuaMnGJYbtgVb1z7fs35KunehOZ6yTrQe2IkWePxoBQ4uOT3ZKmO/ehnuGJjcrBh2kMvknWlKSbqlTKBTHFlZNyAnAWwbH3koeP6p5r7O5oJa2aZ//ZsH32PzhFjrza4tk3QcJi8Lnc1LpSaan2m32HO+70OEdej30ZhuJ9mg7z7z3TAEvKEkv1Hn53D5mnznb5BZSd8ORaoH7+XM/RyK56fybDPviWzvM8zEt7S04RJ4Ml4STx9bgc/vwOE02ggJeFHu0zUrUIM0iYMnZ0GaL6U7XUygURz9WjfoO4KsGx74KbO+d5fRTgkHa8rSBZSFg5jnfLfg2Hw1wFWbQhPZV536BcMxctnX2mbN1DVMhwzt6Ij6zcvOjOR7xZWMuM33OtLG5ana9jV5evdDhO4OKzVMyZcVlTDhhQt61LBqnjfTVq2/IJByPaJuV6ETDvxMJBL52miqSUyiOUawa9V8A3xBCbBBCfF8I8RUhxPeEEBuAmcnjvYIQ4hIhxHYhRJMQwrjM95MkEKCASZeUCk+PxD9Kx51mPRqQQcJhZ8M7G0zPcTusDWAxo1DxGQB7OJKeWJdi5sSZhpPXSpwlzJyYq2bXEwpR9CtEyS/lzb+8+2XDexcJF6MHjebSykuNN2oSrt0EQ7+/AIBh3mGmr2e4b7j2w6BBxgN73NB0UqnpdRQKxdGLJVMlpXwEuATwAIuBx4D/AUqAS6SUj/bGYpJV9kuAS4HxwFVCiPG9ce3DorERu1UHVcINkYk9uk11xWk9el7KCBXZi3SPF9ndhtXsRopreo/nKz7To92lTazLxOf2sf7a9Xid3nR4WiDwOr3a472khlaIop+VFjjo8uZ/uPaHbGzeaHzzcIS6DXX86uVfGW/UBAROhrY97wBw6uBq0w3AqWWa11913mXpGfXd8USg8nN9H+lQKBT9E8v+p5TySSnleWiV70OBYinl56SUT/Xies4GmqSU70gpI8AK4Eu9eP2eUVWF12RGeXfcpUN6dBt/dc/yoB7h5sLRF9IZNyraCjPto8E5jxuFm5e+ulT38ZEDRxoaSSNKItD88W5DmVabkdRqL1BIMaCVFrhMbz5zelwOEhI2aIu25Z1mF7XD8vOKAThuT4vpBmDQvw4A4D9jJrZi/UiHrbgE/xm9E+lQKBRHHqafqEKIk4UQFZmPSSkTUsr9UsqEEKJcCHFyL66nHE1bPsWe5GOfLn4/w4JYC40L+LOrZyUGvgic/AEFF4rZ4nHcDrexpx6DNf/p75rxjXm4eU79HN3HZwSms/LKlVlGMt9a252w0rEja3Pw17f/ykV/uoi2aFu6V10iaYu29eqc+WA4yLYD2/j26d9m+menM++ceYbFgFZC9ZZrCgR5suPZ566Zog1A3BJ8x9RT/2dQqwHwuX3Uf309PqcXT1LkyCPc+Jxe6r/ee5EOhUJx5CGMepuFEFOBVcBZUsrNBudMQBvLepWUctVhL0aIK4CLpZTfTv7+NeBsKeUN3c77LvBdgCFDhpyxYsWKw711XrZ9sJlQwtoMcZfdxcknFL7XSRz4kE2RXciMzUOFu4I9YXPd+ZEHBZHhJ7AvZNwbP7RNUD5oJAzWPPYD7QfYfWh3QYVvNgkjiodSetwwWttbCO/djZSS/Z6kLcpYt5BkvY58pF6nTdgYMWAEg0tyIwuF0BZpo7G1EdDC56kIQ1Vpla7RS8gEb37wpu77YRM2Jg6ZyN62vexr25f33vlee/d/04HugVSWVvL2vn/SLo315z3CxbihXX9XCZmgtaOVcDyM2+6mtLi04EhKX9LW1ob3GCnYO1Ze67HyOqF/v9bJkye/LqU8U++YWb/NXOCPRgYdQEq5WQjxv8D30TYAh8seYETG7xVAjgytlPL3wO8BzjzzTDlp0qReuLU5P7zxK7xV9LGlc88Zfg7/d+X/FXyPZTddwn841hPP0Oy7c8yd/GjHj0yf5wvDLzuv5icfP0pI5LaYecKweB1MurgObr8dgIVPLeSO7XcUvMa6V5zc/kgrrFhBw69upnZ6JwkBIRfYkvbw9GYY87Hgr2Mk7S5r1818nXWfr+P2SbcXvLYUwXCQ8rvLs5TeUvhcvmzd9wxcu1yGw3dqRtaw7I1l/GTdT/IWCxZFtfei3aA2sfu/6b219zLprEn88ZHf88DWh/UjQhJmVl/N9yfdoHOwf/Lss8/ySfy/2R84Vl7rsfI64ch9rWbb+nMBKyog69By4b3Bq0CVEOJEIYQLrV3ub7107R4TDAf5p8uaQQeYODR/oZxeIdrm0liWQbdKAhD79mKz6T/ZBvi3QKa8WU+K3jxhqGwVMGcOwT/8P2qndxJ0awYdtDxywgavlcOj460b9Kx7WBmxGgzCsmWwcKH2PZhtvK0WvXUnX9++WeEdaEVqvjCsetja3HrQBuCkqv1vv+RO05z6okvvsnhVhUJxrGLmqZeAhbFR2jkGEleFIaWMCSGuB9ajfS7+QUq5pTeufTgEtgQKCiXnK45q2NVA7YOXkohGCRHGg5v56+bxueHnQhsFt7WF3LDn4w+p/8Yqah+4mETyMU9YM+j1D4E3AixZArdogz5MJ64ZaM3bAP8/IvD6nwmcEjPOGwuI9nD6Z94Rqw0NueNO58/Xxp3WaMa3R7rvScyG7+gNtCkJQ9wGM7bC5Pe0zZM3or3ntd90kXA5TT37zE3C8AHDWVK7hDn1c3LOW1K7hKHeoYbXUSgUCjA36nuAzwIv5LnGeOBfvbUgKWU9UN9b1+sNGlsaCzK0ZSVlhseC4SC1D1xMMN5VPR0iDNEwf9/1bI/61D1hqHQPo2bcVJrjcwk89WuaSqGytcvIAJqnnhz0oWeg3HY3UkpmuE/lbwc3kkBrSXPGwZ6AlYHUtWI0lmobh96kyFFkPmI1GNQMeqZnHkoazNpaaG4Gr9d0fK2lSIAJWQNtnllJ5bq/438j0vUep85r8dBc/t8Ezihi5daV/P29v+cMsAHNqGdOjJt91mwu/+zl1G2oY/uB7YwdPJZFFy5SBl2hUFjCzKivBhYIIR6SUuq6GkIILzAPeKIvFtdfqCqryjspLY2E0ceNNjy8/JXf0Rlp14/PxmI4HA5iMlbQ+gTgP0nr/PPiYtY/DE7sNugjZaB+8fwvuPvlu5FIIokIq2KbSQiICXDEIOoAZwxm+DUPtGYXVLVqmwkrhr3EUUJ7rJ0SZ4nhyFeB4N0fvmtuvAIBzUPXI5FIb1jMohB5IwEWSHvzVVfCreUQ0Slus9nwXjWTWV4vO1p2sG7nOt1r6UUOhnqHcv+X7z+sNSoUimMTs5z6LwEv8JIQolaIrgHhQgiXEOJSNC/eC/S8qukIoLaytiAP2ij83rCrgfnPLCRqkHCN2kEUOsFWwpxNTrxXJXuTq6q0kLQeHk/OoA8pJUtfXUo0EU17kqFoOx1OzZjHktu+djcE3VB7DbS5wL/ZmsiBy+7iiuorqPt8Hf9zyf+w/tr1un3jY8rG5PdGGxu7PPPuZGxYejqsRg8jcR7tRj6or6d5iIeZM+ycMwtmzrDTPMSjpQOSlbOFKNspFArF4WDoqUsp9wshLgAeQvPaY0KID9F81uMBJ/A6cIGUcv8nsdhPi/qmemzY8mpzAyBgwzsbmH9etqeY6guPml1DwpT4Z3iuaJ/haFK9+4lJk7u0vv1+LccMBF0QmACNpZpn7X9P4Os26KNQPfcEEKiGWf/QvPZLk0beaNMTiUcY5h3G7Rd27fu6z2P3T/Dz2kuv5b95asOiZ9i7bVj05r77J/gLMugNuxpyquHnr5+froYHWOp+iznXhdKRnI0VcR6YEGKJ+y1m01Vg15eRA4VCoUhhOkJKSrkdOFMI8QXgC3QJwfwLeFZKqT/p4iijsaXRmkFP0tqeO71LM555hsIIGOmrwB7/0PriJFQ88SzMbtMMe9J7bPjOVGq/0tFVNBeB+cUJ6ls3UePtEl4pVM895IampLR4zS7Yexdc8VU760brvzY9T9SsGM2UjA1LDjZbzmSyQu7TfKiZG5++kW0HtjFu8DhuPv/mtDhPitT7VPtQLc0LmjnUeairqC21qUl+n1M/h8s/ezlDvUNz6heArHY5JRajUCh6izxzITWklM8Dz/fxWvotZoVXemz6YBNtkTYdGVITaVEACW/4gkRajAVI9AiLrnwyQPCsidR+zU4wo2U95ALi7WmDlF5bgYV5njBUHrQBCfB48NpsPPLNlQx/ebquElyveqLJDUtO9bvNlhXuLpSlry7Nqjjf2LyRB956AJddvycv1Rb3fNPTxrUWEurWLuD+Kx4CsiMHRf8qYvEliwuOHCgUCkU++o/8VD8mX39yd4QQOb3QVWVVeGSeYekCXtu/SbdK2uw5q0fFsgrgAlsCRBP6xXbReDS9tmA4yNKNS63fi2Rb24SvQl0dLF4Mzc34Jk1l7TVreyWHnZeaGq3KffHirDWk2tkKpflQs24LGWD475Aqbnt1xzOmfeXbG7MFiFKRg3Jfed4xtwqFQtETLHnqxzqp8OmFD1yozS3P4912xjrZemBr1mP+aj/zVl9fsK67FbYPJiufvHn/ZuPhLvFOtn6orS2wJYDMt6CkJ5rueX+8GO+rv8vximsGTaS57JcE3ltDUxlUnncZ/jNn9o3h8nrTUYnD5canbyz4OR6nh4oBFeyIfWD8tyBhbEevyDcoFAqFZZRRt0jNyBr+ed0/GfObMZbOb2lvyfrd5/Yx+4zruOPVX/eoF90MV4KsfPJHh/abhoVb3ngRPhe0lE93x+GCd2D6Oy78O1zIxx7j3q33s3rHagAuG3MZM9vH4PviDLyJBLPSIfEXoX5ijz1oQ4JBLdXQ2KgVzvn9Wli+h2w7sK3g59iEjY5YB3FhviFaVPHNni5LoVAoeoQy6gXw3PvPYRc24haK5jqiOtXrLlevG3QkfK5qcpbnXLrrQ9OwcPPbGwmOGk7V0tl5awXCDthWOZCOs6bx0pQr+MoTX8nqNV+/cz3/EYEnB2mFc4CuIEyvYEFNrlDGDR5nPhddh5VXruTul+82PWf8ARj6o+/2aE0KhULRU1ROvQC27N9iyaADPL71LzmFY1VlVYbFV4fDz666L+v36lY7RblzXdI8/xlJ+ffaGPnz32CzsMt413aQGw7+mYv/8hVd8ZgOV0ZrWyYpQZjeIFNNLrVpCIW6Hm/r2bjW26cUJrFQ4ihh98Hdec/7zNhzem8zo1AoFBYxNOpJwRnLX5/koj8tWjtyW9WMiMoYy1/5fdZj/mo/LlvvGnWHsPHse89m32fUZThN9h5hhyYkM+OLHawc+B2K7cWHvY6ITetfz6Kbgt1hYUVNrgek9NazMImqt8faaWpt4rIxl5led9o5X+vRehQKheJwMPPUV6PJv6628HVUy8SmGFQ8yPrJAh5+Pruy3Of2sfaU/6Y4QrbhkFBic3PX1MKncMVI5MiM+q6aSf3jxfjCYDNRnE0A/9y7qaAefCMizq7+9TQ6CnY9xqKaXE84ZcgpeBwe7EKT+hMJDA27RzqpLK1kTKlxbUWxo5iZp87s8XoUCoWip5jl1E/8xFZxhDDhhAkU2YsMK8u782J0Jw27GtLqYwSD1Fxdx/4wLD8F1iTtwrRGmLnTyf3/1jPjWjGwIvsBn4+a+55k/s8mc+vnjK16yA0LeZZ4/PBL8l1RbYBMFjqCMD2mADU5XQwK7FJKf6FY13WlydxUm81ObVUtY+8da3jOX7/6V9WuplAoPhXMZGLf/yQXciSQkvu0atQRZIu9JEPIEnAn4OT9SfnWzeB1Sla/dH+P1hWO5WrNN58ymls/n2cwjCRvBbdVXAnwv1sCtPeKIEwOBarJZWFSYBco2WYskyu16v+wI6Ol7+urWNO4xvA5HqfHUs5doVAo+oKCqt+FEA5gJFDU/ZiUcmvuM44uMuU+O6IdlqappdTHZp0+CxobaSgLUXsNWTPP518M9Q+FoN1gelseNuzM1Zr/0VM/yv/EXqrEL3YUs9b/V7wn7dbC4JWVmpHtBYMeDAcJbAnQ2NJI1X3X4Z+9FF9YWleTyzOutXHFt42r/4XWzjex1UHlQTv+X/4NeeJ5/OLRu3s8r12hUCj6EktGXQjhBP4HmAkYDdvsgTk68kjJfU763/N5/YNNeQ1j5od88KQRWpV4xjuYGl1aew3816AzWX/o3YLXpLe5eHH3iwVfxwpFjiI6Y53p/PMPzv4BP7vgZ1okYlzv3kt3oMoCQf1x11OzW1jbPOQpsKtqajGevS7cTD/xAmZdMB38fhpaN1F7d7mp4p+auqZQKD5NrLa0/RdwGTALzYxdD3wTeBp4D/j3vlhcf8Xr8vLx/j2WPd2KAVrO+xflO3PbvpIkBBSfeQ7FjsIr0fcc2pMzFjQWL2wme16SUfrOmJZ6iMs4cRln2T+W9e59kqRy3cFIMG1wQ9EQwWgbF7b+mvmTwiw7TRJ05kkf5Cmw8+8tM5QAtjld+P/rEZg1i6BTptdjNFoX1NQ1hULx6WLVqF8J/BR4JPn7RinlA1LKqUAD8KU+WFu/Zl/iY+snC81I/fqNpYYbgZAL9oQP8OTXnqTInpPdMOXtA28zd91cyu8up2GXNjjvBM8JBV2jp6TSC72N2UjYcDzMPf93T85r1iXPfHlfZbWl2ev5RtS67K6+0bpXKBSKArBq1EcAO6SUcaATyOztegiY3tsL6+/YbdazDXsO7tF01hPGo1ddMaj0VFAzsoZfTf1VwSI1oWiIYETzbvcG9/L2gbfzP0mCA1vamJnKwBttRvooh2xFwjbzNetNiAO08LzN4M88WWCXSqksvmQxdZ+vY/Eli2le0NzVtWBhPVNGTcl5jkKhUHzSWC2U2wscl/z5XbTZ6huSv5/U24s6EojGIloVQZ4QvAM7laWVbN7zBlGMjboUWhU858HO1p2FTWrLICET1G2o0zYdeca3I+D7sVM59d9n89jjt/GUeJeY0V7FQEs+lUPOKmgrq8Jf7cfn7rkmeyHjbrOKEbtjcVxrvtnrZuvxOD1MHz9deegKheJTx6pRfxY4H01k5j7gTiFEJRAG/MDDfbK6fkxYSEs59ZiMM234JJ7f8L+mQ1ZObwZvdA8Nuxr47Wu/7fG6QtEQ21u268q5dkck4NTjT2bW6bPY8Zffs9ZReJGeTdgYMXAE5XeXZxe0rZ9P/TX1PfZcU+2DVjCLFgTDQQIl22hc8W2qmlrw7y3DV1ldcHW+2XpUHl2hUPQXrBr1m4HBAFLKXwshBDADKAZ+A/ysb5bXf3FIsOpLP7biv/Ac6gCjceoCXh8GeysGU/tQrWkhFhJIaPeP6fzreZwexpaNZdO+TebXAWwJ8F+7CIJBqv7+Jp5JXdX43e9pFzbcziIEIm24bcLGyitWMuORGQQjXUV6KW82q0e/QDLbB1ObBSOMKs51q+cH2Ki/qJ4aE4NuFHXovp7Ue6Dy6AqFor9gyahLKfcB+zJ+vwe4p68WdSQw+pCDbYNi+b11AWta/4/jBwyBdkynp9WVvkGixVxVzomN2wdN59aOtQSjuXlkIQQnDzmZB//5YN7XMNF3Et7SoQR/fy+dtgRRo9C7gDgJbNhYdNEi9hzcQ2VpJf4JflZsXmFYQNYR7WDOmjncW3tvj0LxqVx3YHOALfu38NvXf5uuvs9Ez1POrJ5Pkd5s3H8RzXPewVs2LOdauhuBjKhDaj1NrU3p90AZdIVC0V8oaEqbEOI4IUSNEOIKIcTnhRDH5X/W0UmnE+viLSUllI47zfT8mB22H9yZN4ccFQke9e3iurNn43V6syq2S5wlxBNxfvr0f2qGNk+31xsdO1m1bRXlzQuo+0KUiAPtOQbPk0iK7EXcfuHtzDp9Fl6X17SALCZjPLz54dwK9WAQli2DhQu175nCMN3wurxcWX0l408Yz+XjLqfIUUSJsyT9mo0qzk2r56OdXDG/gmWBhek2wGA4yL0b72XK8im5bXQZxXip3Hvme6BQKBT9BaviMw7gNmAOUJJxqF0IsRS4WUppMuzz6ONfxRb7wCWcfdq/U3H8aBz/sBEzGJ7isrsYWzaWzfs35zXsr/zrFTbv34wQguvPvB4hBBUDKqjbUEdbLMN7t7DpmPHoDGIi1iUpZPIcvdx1voK2aCJKNBLtCsVv3KQv2frnP+s+v7vnXOLQNi7XnHwNk0dNNvSUG/duMVxTxAHrRid44a07mL9zKYsu+m/qNtQRjoeJJPSTKqbFeAqFQtFPsOqp3w38EPglMB4tvz4euB34AVD4eLEjnJjVd07AmvefYuSAkYYGHcBlc7HowkWGQijdCUVDtEXaWPraUm7+ws24HW5kPtdch7hJm1139HLX/mq/pTUnZILAa8uNZ6I3NubMRNcToGmPtROOh/nb9r8Zh74bGqj65W9NZ8qDpg0QjLYxp34OwUjQtONAyb8qFIojAaum6WvATVLKX0opt0kpW5Pfb0Mrojv2hkcXoJu+o2UHMx6dYXi8yF7E2mvXMsw3LC2EYlWAJuVBWunr1qOQjYBe7jpVQOZz+XAI48BPKBqi6eXVxpKtkDMT3SyEbih6k9R6r30rTGdBkw3MUfKvCoXiSMCqUU8AWwyObSZv9vboIhgOFvyCjYyTy+7izovvTLd+pYqx7q29l4tGX4TIs3tIeZCpMHihlDhKDI+l9N1ddhdep9ewyju15qtPvhqnTb/E3+P0UNmCsWRrIpEzE91so2LoOSe13uvHgMt6ECIvqm1NoVAcCVg16n8Cvm1w7DtA/lLro4jAloD1bYwEp81pnN+NR9hzcE/WY6lirNOGnpbXk3bb3Wzat4nOWGfeDUB37pp6l6kyXuqYQOSNTHhdXu6tvZcih36EwSZs+EdNM5ZstdlyZqKbbVQMPeek1vuWwVru/HBx291K/lWhUBwxWDXq7wPnCiG2CCFuF0LMS37fCpwDvCOEmJ38uq7vlts/aGxptP7OCTgUOVS4cQJGDByR9/LheJh1O9excMNCEiQMPeVMLjrxIvYu2Mv88+bn6J5neu6pHHM4HqYt0mYux0pXKN7r8uK2a5V3brsbryvp5V8101iyFXJmopvl6w0956TW+wdeDjt+5LQ5ueviu5T8q0KhOGKw6sukCuHKgc/qHL8742cJ9FwS7QigqqzKWB1Oh3gijs1RoHHKR7f7pxTkrBj1l/e8nPY6u/deNwebeeztx3QjC5YrwGVXrl5qv2iYSbZWVeUovFkRfMkRivlKLW/e9QMeHU/efx9PGGxFbhZdejd1G+py7rHyipXsOrSLnz/3816RvlUoFIq+xqr4TEH97Ec7tcO+UND5bpu7R2pkuw/u5gQMpq2ZGCy7sBOXxgnlzlhnlnHO1D1f+NTCwvPYSVLV6m0ZojiReIRIPMKlD13K7VNuZ3fHbqrW/RL/FoHvnT1dM9Ffe033mmaCL3pCMfOYR+LqBDGTejx7HKbuhOlvg//iOXjPms3XJ3496x4jBo5gxiMzelX6VqFQKPqaXqwPPnb4S+Cn1k+WMKh4UI/UyKrKqjjUcqigtUUT0by59ZiMsfXDrVmPpTzeNz94E7fdrSsxm68C3KxavS3SxoInFxCJRzQDKWzUX2fNQGZuOoLhICs2r9AU5l77bdY6rVb/x23wh1UwFA/MGa97j/K7y3td+lahUCj6GkOjLoQYD+yUUoaTP5sipdya75yjhVUtL1nfDgnojGvSpvkmgXWntrKWh7cXPitn7OCxbDuwzfSclo6W9M/dPV4j8qUK8rXVpXL0OQYyLOHAAU1hrqpK89x9uWFuq+vMhz0BCy+Ee18Q+Py5r8dKK50SoVEoFP0Rs7D6ZmBixs//NPhKHTtm+JfbfFBKd2xmxWEm1G/+S4+KvYZ7h+Owme86ykrKAH2Bl+6YybFmUmhbXUImCPz1F1BeDrt3wx13wNy52u8NDVnnWlmnVeJ2ePgUKJ+XoKF1U87xHrXSKRQKRT/AzNpMBrZm/HyBwVfq2DGDLGBkJxK+MMI8B998qJmZj8/knPvOYebjM2k+1AzA5pf+WpDITQq7zZ43BO8UWkGdmVfqsru49KRLWXzJYksV4FbV5VKEoiGa7r9bE4xJidKkFOZqa7MU5szWmYOFjVDUDsF4u25Ff49a6RQKhaIfYPgJLKV8TkrZlvGz6dcnt+RPn6HthXneZw4+2fDY0leXUn5POQ+89QAbmzfywFsPUH5POUtfXcpHH+8zfJ4Zx5ccn7cKfslrS2iLtJl6pZF4hIlDJ1oeXJKpLpcyiqnWNj08uKn8yGDzkUhkKcwVophXHAWvML5v1m10lOl61EqnUCgU/QBL1kkIMUUI8Q2DY98QQkzu1VX1c0rao9bD4gJeeO0x3UPNh5qZUz9H99ic+jm4BwwqeG0CwQmeE2iPtZueJ6UksDnQ615pqiBw8SWLqft8HXdNvQuvU39DYJMS/xsGeuuhUJbCnGloP/lv4QmDLwxPPgh7O+aw8PMLcdqc2Ez+zPXC6XqbE6spCIVCofg0sepy3gYMMTg2GG3QyzHDK879hYXF2/QFW258+kbTp20+LoooMKee6g/Pl9tOGTMzr1Qg6Ih1sPCphSx7Y1l6TGk+MseTzjl7DmuvXatvII+fi9donR5PlsKc2TqLYjD/RVi8DprvgpoWD97K8Sy6cBGtC1u59pRr05K3Oc+1F+luXLpvTqymIBQKheLTxGoNdzXa4BY9/gH8Z+8s58igFXMvOAsJ0w7q74fyVajHRAIhCk+q//a13+YNv6e8cCOBl4RMkCBB3Ya6w+7TNmznC0u40UCnyGbLUpjTXWcEbBLqH4KaXRnPdXc91+vycvuU23ngrQd0b9MZ72TamGm6xwrtVlAoFIpPG6tGPQaUGhwr66W1HDGIAmabCWCm6yzdY+MSg9hookxXNWAUMt6zcarXnXkdv37l18brEiKdG+5udCsGVFD3dF1WAdnh9mnrGkgXXQpzqRb/yq4AACAASURBVA6BlMJcfX2WwlwwHGTbgW3MOm0WrZ2t+Fw+grt3Urp6A9uG25i4L4LPqf/c+qZ6ihxFdMY6c9ZV5ChizY41yngrFIqjAqtGvQH4sRBilZQynQQVQriABcALfbG4/oonZifosDACTEJlC3hPPTX3WDDI7T9p4IHvGz/99GZpnPQwISZjbD+wHY/TY1hcNufMOVmGOdPoLntjGfKT6tOuqYHmZli3DurquhTmMoxy9/70InsRnfFOzVCfGcOTcDD/Qhv1H9RQc+sfYdiwrFs0tjTqGnTQ1PVUi5pCoThasJpTvxkYBzQJIX4lhJgvhPgV0AiMBer6aoH9keqQxV5sobVOdR9UAkAgwPCDcZbUoxV6pRzy5M9LjruG/S27kD1pacOORJpWi5uF9RvffIZQVD/F0Cd92l4vDB4Mt98Os2bleOjd+9NTYj4pQx2yxQg6EtQOXk/bhDE5Pe6qRU2hUBwrWDLqUsq3gLOAF4GvAf+d/N4AnC2l3NxnK+yHVAwebbn63WsvzhlUAsDmzdDZyexXYe+dMHMTnLtL+773Tpi9awhVx48ruFAOwGF3MG3MtJ4ZsmCQqmWP4THQ1/mkjWAh/ekJIPCZtqwe92A4SGesk2hcv8petagpFIqjCcva71LK7cBVfbiWI4aPhx4Huy2cKOGbrSP1j330UfrHoSG4f1W34y0t1P7oZh5+9omC1xeYEeCCEy/gpqdv0j1uasgCAfxb7cz/N/3Dtlj8EzWChfSnh9zQVEq6x73horFa2D4WJZLIaEMU4IyD3eli5RUrVYuaQqE4alDT13qA3WVN2ATg6uqv6h8o7ao7DLpg2emaHvmy07XfKSujfu/zeZXh9NhzaE/Pe60bG/F93E79Q1rPd8pjT/WA13dO/0SNYCHSs54wVLYCoRDBpi1dYftEMp8ukl9SS4vYOiPMeGQ6DbsaTK6qUCgURw6WPXUhxAzgcqACKOp+XEp5di+uq19z2ZjLWL9zvaVzvxl5hOlvVOTO4q6uhqIiGk7opPYaLXQccmuGaf7FUF/hpLGl0Xj0qglrGtcw5+w5PZoMR1UVeDzU7ArRfBcEqjXvt+IQSJeTJ77SwrY3ln1is8X91X7mr59v6Vwb4N8CeDwEhrWQaDMI2yf3Se0uINqmJq8pFIqjBquKcj8FHgE+ixZ43qLzdcwwc+JMax60gHXRt5m7bi7ld5dne4R+P8ESB7XXQNCtGXTQvgfdUPvxEo73HN+zBWbk4TOFYCzJvfr96fYybwRm/QOmNcKNF0Ldv0W5o22d/uvpI3xuH/VfWYlPFOGRWu+9K9WD3624cNEGbc3YbDRWllkO2+tJxSoUCsWRiNXw+yxgkZTyNCnlNVLKb3b/6stF9jd8bh+1lbWWzw9FQwQjwezhIT4fgaWzSRhJnyN5Y+8bPVrftLH6YiqW8Pm0Pm+fDzwegi66Nh4u7RTd19NXNDRQc/YMmu+xsfhvUeZttGOLRLVjqfcuGVafPxXmX+Zk2X3XMWLwaDzOEku3UJPXFArF0YJVo+4Dnu7LhRxplDkHFDwWtbtH2FjaZSi7E4qGeOejdwpeV7GjmJkTZ5qeEwwHWfbGMmP511Tv+OLFBH58KYki/RqCPvdwU9PagkG8H7cz6x8wfl8cu4FEQNgJ95wZZe7OJdy44UZiUQNd+W6otjaFQnG0YDWnvgK4BGXY05Tu+rDgsajdPcKqsipKHCW6w1c8Tg+jB40Ga3YpzQ/O/oFpiL27kIuh/KvXC7Nm0fjUDkIvrbX0enqLYDhIYEuAxmdWUlUdxv8G+CJaAeHKz3alKoxIh91N1PoyUW1tCoXiaMGqUX8a+G8hxGDgKeDj7idIKet7c2H9nepWO3YnxAvoH+juEY4cMNJwmloiHuOJ7U9wxolnFLQuM1GZTCGXFEbyr8FwkOVvLueJ7eYtdRUDKgpaXz5yNh2TYf5kLV9edyFE9Oey6GP0ViSNvUe4sTldavKaQqE4arBq1FMx1lGAXmxXAoV83B7x+Eddxuw967EgFpsm0yMMhoN8ccUX9U+UQDhMm9NAAcYAFw7TMLKZkEum/GvDrgYufvBi2g1U5bLogeJd2hNvaaSqrAp/ddd7krPpSHrlc2p7di9dBJz7L8G3v3kX/jNmKoOuUCiOGqwa9RP7dBVHIL6rZjLvq/O546xofmMjwRWD+jMWpQ3I7177HeG4sdGO9MCAuRxuwzByMBxk5daVhhXhqVB6MBzk0ocutWbQgT0H9xS0xu6euMvuYvaa2fzx1D+y+c3NltXjDgdPBL594X8w6zz9WfYKhUJxpGJVJvb9fF99vdB+h8/HLT9ejSdq4VwBEQe8cPfctHzpb179jen5ccsKAkkk3HrBz3S9zoZdDZTfXc6z7z1r+PRUaiCwJUA4Zj1CIKX1akE9HfdIPEI0EWVf2z7mr59v3IbWW146YPN48X/5lt67oEKhUPQTDI26EKIk8+d8X5/McvsXvklTWfe19ZQIV/5KeAE3fSHKqvvrCIaD7Dq4y/z8QjXfBdz09E05LWaZhtQsMpBKDWzZv4VowspORWPJa0sst7Xl03Ev5L5muOwufC4fS2qX6CvqXbtWhdwVCsVRiZk/GBRCnCel3Ai0kd/MHFM59RQ146byQV0L5beVcUjkKVUXMOPAUu55c1z+C/fAM40lYjljUfMZUpfdhdvu1orFwpLWt16xXDUOmqce2Bzgyuorc/Lk3RXnCtFxPxymjJrCI1c+gtfl5esTv16Yop5CoVAcwZgZ9W8BO5M/H1PiMoXidXm5ovhM/rfzpbzGMC4ka3as6ZN1xGU8p8UsnyFNG8CNm2BCOYO+0K7N47NIKBrimfeeYd76ebptchOHTEwb+31t+yhxlljO1/cEj9PD9PFd+vSZc+IVCoXiaMfQqEsplwMIIZxAE/CulLL5k1rYkcZjFqfPak6wwIaNBL1bFOayu3Kq36vKqihyFKVnj2dS5CjSDGBYpkVeJuyFogh0GojidKfEWcLKrSuzQvupTcSUB6aQSCSw2WxE4hHDnvzeRPWcKxSKYxkrhXJx4O9ouu8KAz4OH7IUsi62FzNtzDRkwUnz/AhEjkGrrazVNegAnbFOpo2ZBoGANq4U8G8GZwFLS8gEdqGfeYnEIsRkjEhylnlfGnRLE+gUCoXiKCevUZdSJoBGYEjfL+fIJNhiPYBhEzYOhg/2iVGfMX5GjkGrb6qnyJ4zVC/Nr1/+NTQ2Qkjzrn0RdMeuFke0tjxnsjG/JAI+p5fLx11ubKwNNjkljhIuOekSLZ9vsxgSMGH0caNZfMlimhc0Z6viKRQKxTGGVT20m4H/EkKc3JeLOVIJPHij5XPPLj+bm/9+c97z7AXWHZY4Spg8anLO440tjXTG9T11gHteuYe2k0aAp2tmec0uaL4LFq+Duhfg+o1gl+CIa3PIHTGIC1g58DtMPnGy5XnnKdpj7Zw69FRa/qOF27xfKrzSvxsneE6wNoFOoVAojnKsGvVbgDJgkxBilxDiVSHExsyvPlxjv6fxw22Wq8Vfb37NkhETQuCyW/di7Ta7bi65qqzK9DoCQWCCSI9bDbpg2enw8y+AFHDDK7D0bGhzQ3tS3S3m0IanzPhwCdOqpmETBWjlonn5lZ4KpJQ81vpiQc/V4/Xm1/t+WpxCoVAcAVj9NN4CrAYeQNOB38wxPE+9OyMGn2TZ2zwUDVraAFz+2cstCbvkyyX7q/2m1wnHwzS174H6ehrGFVO+AOZeDHfUwNxLYPQPIWrwV5KwCeob66m/pl7rB5fd5pwbEBcw4vWdlN9dziuOvebvRwLceYQPozLK8k3LzW+qUCgUxwCWdMuklN/o43Uc2Zx+Ovz9YWvnWuwBn3rSVEYNGgUf6h932V1MGTWF6eOnm/Ze+9w+5p03jztevMPwOpWllQSrJ1L7NTvBDP0Xo7Gw6eMyTFNrE7NOn0XzgmYCv53D1if/zD1nxpAmr3HaDviS43/oFHmU8yWM/xC2DomZnwesaVzDnLOV7KtCoTi2MfXUhRDFQojpQogFQoirhRCqWE6H3eECxrAK8nqyNgn+CX5uOf8Ww9C22+7mkSsfsZRLnnWqcZ92JB5h0qhJBLYEiCYMjKfBejOnznldXmZ9615++n9FOE069dxR+NtY8hv0JN/a4sAj8sxaBdi2DZYt02awKxQKxTGKmUzsaLSw+qPAr4AHge1CiKmf0NqOGHwun/ViLwvnpbxcn9tHVWmVvtRpKtweDGrGbOFCQ6N22wu3md7v58//nM37NxsX1BlsWBIy0ZXHDwYhECBw5XgcJkY97NBy8qZI7WtJPXx3cxE2hzPv+VM2vAtz50J5OTQ05LmBQqFQHJ2YfbzeASSA84HX0Sa1LQV+h5ralsUTb/+1oPNPKTmRtzreNTwuEGm5V6/Lq4W29aROGxo00ZhEQmtJ83hg/nyor4eartaubR+Ylzxs37+VMccfhgxBxjoazwvRbtZVlieiYUvAubvhsdXFDO10QH09i9xvMafePLT+08lwTnOIml1oa2luBq+qhlcoFMcWZkb9PGCBlDJVnvy2EOJ7ye/DpJR7+355RwbvfGC9+t0pbDgHloKJUU8gs+RedaVOg8G0ClyaZK95d6M27pCLjUa5fAljDzoZNLLU2gvIXKdMEHhtObNqb0yvo6pV62sPWYiY61Fkd7F+yPV4bx8Pfj9Bp6Tu7lrzJwmtOr/2Gq0Vz5tIaII6s5Q8rEKhOLYwy6kPA97p9thONNMwtM9WdATSmYhYC79L8OLiww6D6rcMKgZWmJ+QoQKXQ8qoJbm95VTTSy1qPZ3qE6opchiL1OgRjodpenl11jr8m623VOgh7Xa49VbNIHu9eQfSZJIAAtVom5umpnynKxQKxVFHvs/f3pc9Owpx2ZzWPHUBsrOTEmeeSbUS6Mwz0zxDBS6HbkZteNXpLHnKmc5Vp++RzFsPlR781X4corAh7g5sVLaQtY6UIp2th7L2NmEjsLlrQ1LIZLeQG5pK0dIQlZV5z1coFIqjjXxGfb0QYn/qC0iF3J/OfDx57JhlgCiyvP2JCJhw/ATzkwTs2bjB/JyqqrQKXEowZuGF2vfgcSXZRm3kSGa/FGXvnTBzE5y7S/u+906Y/SqwZAlvvvsy8ajFiENqmRL8o6ZlqdEBTNwHdmvF7TmEoiEe2/oYwXAynF9WZVmxzhOGylY0IR2/GuqiUCiOPcxcs1s/sVUc4ZwQdfGeRSfXF7dTMSBPaF3C8R8ZS7sCmtGaP5+GkVouOYHmqXrCMJ926j8/ghrQct0zZgAwNAT3r8q9VNCRoPaRL9FBxHprnoT5A6bivWom/OimrEOBCZj2qefjqXeeYvhdw1l77VpqK2u5Qd5g6Xk2wP++VysUVEVyCoXiGMRs9Koy6hbZ42y37OGOLS6n+oRq7AjiJk96fUCeiWY+H8G/raT2yYsJZhSlpQrUah+fQfOCZrxmufckgdHtJBIWUwhJXDH44dW/AZ9PM6IZVfhbhjmIOfILxhgRkzHaom1MeWCKJnHb7W1y2V1E4hGK7EV0xjvxCDe2hKS+Yh7e929RBl2hUByzFJZEVegiEwlrBlHC0KAm3fq9J75rfJ6AZ2zvs+yNZYyWow1PCwzYRcLrAZ2cc0ImtLa4ZO496NI86MZSrULdv1nLfwM0DnEQEtGca3Rfe1o4R4DN7WLsH0+n/pp6ampqtGr7QACammgtfQ7aX877duQjEo+kx7ZmEk/EuW3ybQwsGsieQ3uy2/wUCoXiGEYZ9V6gvbMNishv2AXsDbfic/u46KSprN+53vDUvaF9zF03l5+P/jmuXa6skaLBcJDAlgD3vX6fYRFZKBrS2uKqqmgYU0Tt9M7sEP3FWkFbzS6oOmjH43TrXssdg8/vETz/GUGMrs1LZyJCZyRC7UO1WkTA6023kA1aNxdeOXyjbkRcxrn1+Vtx293apkKNW1UoFArg8LqPFEkOubEcut5xnOYR/+GLf8h7bigaIiET1D5Um55C1rCrgfK7y5m7bi4bm82H41UMqCD4lVpqp3cSdHeF5kNuCLrh0mugbZAH/21/M5605nBgO38SNpv+8UQsmlWtDjDhhAnGM9wzK/APg0g8QjASzHpvFAqF4lhHGfVeoJCisLakMPrwAcNZUrvE0nNSofRgWDNiwUjQUptXOB4m8H49iWJ9A9vmgl88+F18k6Z2TVpLVpqnjLJwONiw+xkiUj9HHkp00vTB1qzH/NV+HDb9IFBJbjT9sEi9NwqFQqHoR0ZdCHGFEGKLECIhhDjz015PIRTSvlUU6XJTZ581m70L9jJz4kyGeYcZPicVSi9EiAVgw84NWp93wljT/Z43ltAWaaNmZA3NC5pZfMli5p0zryvMHjOvwrfHYfD23V0PBIO8uewXJCKd2R65hOIIrH9I643vLY89nWZQKBQKRf8x6mgz2i8Hnv+0F1IoJ3QIy4pyXzg4KOuhod6h3P/l+/nZ5J8Z9mOnpqEVIsQCWhV5VVmVVkFuQEpnHrrkaMefMB67sFu6R9wGP/r4UZa+uhQaGgiOGk7tv+6gXcSyUxIC7BJO3af1xu9dUsRnIwMs3cNMFCdzUpxCoVAc6/Qboy6lfFtKuf3TXkdPEALLOfWaUefrPu6v9mNL6O8MbAmJf4K/ICEWJBzn8OKv9iOl8Y4jHA/neLoFbR6Sr33Omjns++IFBEa1YRS4kCRlXIGhBzo5qzFkaTN02rDTcNr0J7XZhK1rUpxCoVAc4/Qbo34kc8hpMY4s4MPqUbqHfBFYtD6WK+WK9rg3kjT8RgVtOvfasfNVfG4f886bZ3ianqdrunkweal1X4jyzChoNxjmEnJD0/FdEYBBwbilzdBbH7zF6qtXm4+gVSgUCgXCzIvr9ZsJsQH9YTA3SylXJc95FviRlPI1k+t8F/guwJAhQ85YsWJFH6zWOq83v27ZU68YUMEQz5CcxxMHPuTN8C4S3Wx2hbuC5o49THSPxDb4eNoibTS2NiJlwlLE/7ShpwHw5gdv6ubjbcLGxCETszYLCZkwPN8MZxxiNuPCQVsCRhyCwZ02SCQ4UAK7BmrnV7gr2BPeo/88YWPEgBGUFpfS2tFKOB7GbXdTWlxqfZPTj2hra9NaAI9yjpXXCcfOaz1WXif079c6efLk16WUurVnn2ifupTywl66zu+B3wOceeaZctKkSb1x2R4z+SeTrcU8JFx98tU8NO2hnEP/v707j4+qvvc//vpMNsiCGpAdpZKIAi0uXNyoWxEwemvrlrZaqZdqW6kF0Vuxdr0/Wy2CV+pS2/Jr1SvVqVpXUBSEtthqtV6rgEuiIigICBYmCWSb7/3jnEAYksmZMJOZTN7PxyOPZM58z5nPmQQ+890XfG8K37cl1MV0f889fC4/Wn0N80NnMu2GxQBsjGzkmkUz+P2bD3b4YWLByAVMO2Ya+evyqVhYQdRFqW2spSiviJCF2p3nnb8un4r7ziTa2Egt9RSQQ4Pz18BrZwvX1ovTtMWiMGeZMfklR0mDt2b9kKu96XVzD5/LNW9f0+59zD5pNjeedmP8m+0mVqxYQbr/ZrtCT7lP6Dn32lPuE7rvvWrxmWQIOqXN4N1PYnez9Sw/8BPqdrZ9Wm0BVPsfGFeuW0nFwgoamxs6TKLA7v7yltHt4VVhqrdVd7gK24R13t7kN4xv5pZx4GjGxftrsZjvbXAh+M+Jjh+dAkvu815j8UJv7XqL0+ygwXAiIsFkTFI3sy8CtwEHA4vM7FXn3OQ0h5VcDkb2G7nP4Uh9hIcb/tnuaUUNUHbiWXvNU98tThLNceyVDFtGt3coEoGKClx9DXceDY3J/CsxqMv3Fr7ZOG/Ph4en5nvryTe08VoaDCciEkzGdEg65x5xzg11zhU45wZ0l4S+YePbCZW/aeJN+xwLrw4TipOdm0NQOfK8hOep51hO55KhvwlMeIy3+1sqNOZA+Bivr6G4AQ6uhWX3Qkm99yEGNBhORCRRGVNT766u+5+vBSvovJrowAee9LZNLSnZ/dTy95azs7n9RV4++75R/Mgiqg5JYKqZg2EHHtK5ZOhvAlNVumdp2WSrz4Xq8z8Hp46FoUOhf38mfGU2Gz41lPAYqK7VRi0iIolSUt9Pb9a8B0HWaTFoyAFmzoRZs7ztSidMIFIf4Q+r/hD31BWHOmrWvEr50WMpyisKltgN3t+xvuNybSkvh6IioLbDPvvOKsgpoOy086ClO2DFCrjxRoqBAB0EIiLShoxpfu+ujij+VGLLndbW7u6zpqaG8OowTcTfe7wxBOH6V6gcXZlQ83uiU9J2q6wkUmDc+W+kJKED5OXkqZ9cRCTJlNT3041fvbtzJ0ajEA5TtbWq47IhqO7j7e5mFjzLHlx08F6PI/URFryygGufvZYFrywgUh9p+8SSEsJ3XpHQRjW7dbSmu//8zyf+XM3qIiJJpub3/VRSOogQwQaU7bVDWW0tVFdTfuSwjpu4HZQNHEV4dRhLoOo8Y/yM3T+3TIVrPU991pJZ7c5TryqF2vaXjG+TReFzvY9kaf0bcQp532Yvnc0lYy9RYhcRSSLV1PdTeHWYUChAp7qDqMG1E2HBMRA5sBDKyqhcBbkd7PKW1wyVF9+U8IYulx5zKUCbW7bWNtbG3Y+8vG85vWKXt+vg/r77Yg6VJ19BUV5hh8W1ZaqISPIpqe+nqq1VNLkAe68a7MqHORNg5mQY8q06Vp40jJJ31jP9JeI2WX/bjaO4dCDlfcspDJAwAfIsl0VvLwKIOxWuOdrcZnKt2NaPXZZYn/xlAyqoPHYqoQA7vGnLVBGR5FNS30/lfcspyAk478tveq4t8JZGrXjkfGpGDOtwl7eWfvTK0ZU0ReMPqmvR6JpYs2UNEH/XtbqmOn7zyA+I/Pp2bwAfQCTC4h99iV7BXmq3H47eTEmBN6+8JL+EglD77fdaJU5EJPmU1PdT5ehK8nLa3ha0I1EXJTzG2FQcfz/2j0r23mwlEAdbd24FYNgBw+IWfTFnI4PXXsnK8QNg5UpvAN8BzexK5LYMXsjbBOxZknbe5Fu0ZaqISBdSUt9PJQUlPHzBw506t7axluq6D1h//Ki4NfX1pd54xvDqMDkBmrYBMOhb2Nf7edeu+CPSDWoKoOKLO6k550xYs4byTU3kBOhVaK2w155Bb8X5xUwfP53npj6nLVNFRLqIRr8nwbod6yjMLaSuqS6h81qaoO/lk7jl3tnxHuA1o9c31we6tgGj+o0CYP1LSwPNN28MQfjwBqZt3UrF+t40h9rZYaYdl469dJ9jiW4kIyIinaekngRVW6sSTugATdEmKsdUMnvx1XHLNTZ7c9TL+5YHXlHOOTj5kJO987ZCfggaOmhO35UHa0oaoG9f/nhEYoPkQg4uH3d5m88F3khGRET2i5rfk6Al2SbqqIFH4SI72Nq4PW65LXVbqGmooaKsIvBAOQwOv+Nw7nzpTiqHn01ekFXvHGwtCcHo0Tx5wdiEVpM7Nm+4at8iImmmpJ4ElaMrCVnib2Vpr1Lu+Z9rOl65zcEPnvsBZbeVBW5+bzF98XRq/30KX38tt+PlbA36RqJw1llQ0ieh19l5QDtT7SIRWLAArr3W+x5pZxU7ERHZb0rqSdB6GlfrAWEdTXU7a+RZPLb1+Y5rxAa3vngrO5sS6+NucemyK/llgHXcezXAqO25sGgRR35MQmvaDzpgyL4HV66EIUO8TWzmzPG+DxniHRcRkaRTn3qStDUgrKK8ghG/GNFmMi7MK2Tq2KnMfeJ7KY/tmXefCTQVLs9B5atNRKpXs6DgTwk1v08ZMWXvAy2b1rSumdf6YwEqKmDDBihWc72ISDIpqSdR7ICwO/9yi5fQY9Z2L8gpYMnFS3CRHay3HSmPK0SIaLzV6R0UN8DihVCcV8SCQVuJfpLYbi4FeTGtEuGwt2lNW/zNbJimwXMiIsmk5vcU2bD0UaYv80e1x+TH+uZ6ykrLCN93HTmd3B01EU0uzuA6BxOr4e3bYMI6IBRi+YBd1NHQ/jlt+Nv6v+19oKpqT808lr+ZjYiIJJeSeipEIlx31/ntP+9g9lNXU7XlTRpS3FZywagL4q8Xb7C0DA6ZCc8cBpHwvTxc9VjCr/PwGw/vvTFMeTkUtTMjoKgIyrRErIhIsimpp8I99/DEYc3t90kbvFX1AsP6jUhoMFqici2XUw49peP+dIOmXJj8VfjhmjuCL0XbSshCTF88fc9e7V+sgFA7f16hEFRqiVgRkWRTUk+BV5bexyfxNlNz4HbWwahRga6X66xTU+aaXBMf7PiAc484N9gJBrdFltEYbUz4teoa6/j9a79nzl/nMPPpmQz59RGs/P1NUFKyp8ZeVOQ9XrxYg+RERFJAA+VS4LyRr8YvYPBizkeM++dzgUaY//yAC5hvL7Bu+7qE4mhZhnbE1iiPNkBd+5um7daMwzBcJ5oQWvruW1a8q3h9NhvWvk3xI4u8PvSyMq+GroQuIpISSuop8EGv+kDJ+qUdb0AOHZa9fOp8Xl4+K+GkHrIQlYdW4M6YyVXfCHiS0amE3paoixJeu4hpGuUuItIl1PyeZJH6CE1BZoMZfJQXYHU4B+G1i5h7xtyE4ijIKfB2QntkESX1jiteIqX9922pbayleptGuYuIdBUl9SQLrw4HLttU2CvQanLV26oZ3GcwVzWM8xJzS3KOk6R/eMoPmXDIhL2nliU29Xy/tTT/i4hI11Dze5JVba0KnDxHHnwEWyIf0RhqPzsbUFZaRqQ+woKCVXsn8jiv84PlP2D7ru18f8QwSoqKKN9WS2HAfvVkCVmIyjEa5S4i0lVUU0+y8r7lwZq5HXx22GfJyYufZR3wzUe/zuCfltIQb1W4GFEXZc5f5zB4y2xWDotSuQqaOvptd7Z53kEBuXute1+S762Hr53bRES6jpJ6klWODl4zXfLWkzz2pcfbL+AnG+gL7AAAF0pJREFU2aYcqAk1Ue8SW+UNoKaploqLjI/6F3bc19/J5vnCRli765vMnzKf2SfNZv6U+Wy4eoPX/C8iIl1Gze9JVlJQss9a720yeP+dV5j06UKWXLyEz9//eZqb6r3E6/aUaV2+sxotyqjLmtpdir1FXjOEcnIhJyfQFq8WhaJGeOrhXgz87lF7rXsvIiJdTzX1FMgP0kruoKARqKhg0oAT+fhb73HXQZcwu/E48puTG8+upl3eHPIOPhgUNMPab7zJvEnzCHXwpxGKwiX/hI3zYMLGPK0QJyKSAZTUU6B3c4BqtcH6A2HloEa44QaKPzWSadc/zPdufpGGjuau+zX5Xrm9khHu7mtOH34hAweOYPr46Tx18VNxixc2wu1/KqS4QCvEiYhkCiX1FMhvDDbiLBqCivN2UXPbPG/f8dpaLp/S8XkAs0+azdwz5lKU286mKYkysKFDdz+cNGISSy5eQl4ob5+ihZbPU1xE8dxfePuiT1DfuYhIJlCfegok0v0dNQiPdkx7yXv8wFEdXyDk4MaJN3L732+nIRpg8FyQPn4H619/Hs7cc2jSiElsu3Yb97x6D4uqFoGDs0aexdSxUzWqXUQkAympp0BjTvCytflQ3adVJ3qATwQHRQtYuW4lVz19Vfy90hO4JgYL615kbs1HDCweCHir44VXh1m3fR3nHnkulaMrvYGAIiKSkZTUU6Cpdz4QbPpZoculrDYUuDxAA02ced+ZwRI6gAMzKKSAWurbr7kbzF46m7u/cDcr162kYmEFUReltrGWorwiZi2ZxeKLFmuqmohIhlKfepJt2LGBSAIJus6aGFaT2K8hEmpmV/Ou4CcYDOFA5p8xj0ER4tbc3/r4LSL1ESoWVhBpiOzeca22sZZIg3e8pqEmoXhFRKRrKKkn2YynZyS8MtsXLnTUlBZ7+40HOdegKRqwlu5rijYy7cTpnFE+Oe5rjOw3kvDqMFHX9ry8qIsSXhV8fXsREek6SupJ9nTVUwkvFLMzWs89j/4E5s9PTVAAOV5H/40jvhm32E0Tb6Jqa9XuGnos7bwmIpK5lNSTKFIfoaadZNiRx9Y+DSncdzy/qBgiEQafewl3LGbf3d4c3HH6PAYWD6S8b/nuddxjaec1EZHMpaSeRIlsuxrrw8iH3g8Ba/l5JDDE3kH/uhwIhyEa5YqXYONcmPoqHL/O+77xzt5c8fYBgLd+fcja/tPQzmsiIplLo9+TKJFtV/eRYD98o2sO/loGvbfVQM2evdUH1sLdj7UutBOqvWb1kgJvh7XY0e8hC2nnNRGRDKaknkS7t13tRGIvzT8gofLmwCXwOp/kNUJ5uTcYr7aNLoKiIijb06w+4ZAJbLh6A+FVYaq3VVNWWkblmEoldBGRDKaknkSVoyu57PHLEj/RwdsbViV2SoIdJwf3GeRtujJrVtsFQqF9NmUpzi/WzmsiIt2I+tSTqKSghILEZpp5DDZHI9767wGFguwE18LBoPKjocTffKWkxKuZg/e9RJuyiIhkA9XUk+yUuv4802dz4k3whjeQLaC8KNQH/UhmUNqnv/fzhAneJizhsNeHXlbm1dCV0EVEuj0l9SRrOKCw8ydXVUHvYEUf33gak4cuD/zhYVPNpj0PiotTOn1ORETSQ83vSbapbnOnz33tkGD7oxeE8pl0/FeY+VJO4FHzr296vdNxiYhI96CknmTNHRdp16TauwKVK8wvgh072FwQfFrbx3Uf70dkIiLSHSipJ1moqJN90wabdgar5Q/s3Y/IdVfz4Ojgl+9f1L9zcYmISLehpJ5kQ4eO6tyJLcu2dlTzdvD1t4r44akJ7NvuYNrR6kMXEcl2SupJds6Y8xJeHS5RX162mV8cT0Iryl0+7vJUhiQiIhlAST3Jpo6dSk6n14oNZsb4rUQTeIkccrQSnIhID6CknmQlBSWc25DCXcwcPDiiPqF58L1yg42qFxGR7k1JPQUmDTiB/MYUXbwTjQA7m2q56umrWPDKAiL1wVetExGR7kVJPQUqL76RgkSWcU1Ugok96uDWF29l5tMzGXLLEFauW5mauEREJK2U1FOgpO9gFp94ByX1UFif7mjY/SGgtrGWSEOEioUV1DTUpDcmERFJOiX1FJlw9hVsuGYjv/jkuOSPht/P60VdlPCq4OvMi4hI96CknkLFpQOZ1ueU5F94PwfX1zbWUr2tOjmxiIhIxlBST7Xy8qRfMi+R+WxtKMoroqw0hSP0RUQkLZTUUywy8eTghQPm6gIXdCm5toUsROWYyv26hoiIZB4l9RS79g8JrOQWpK/cwa/eOjx4v7qDHH+XmaK8IkryS1h80WItRiMikoW0n3qK3VX7p+AfnQLU1POicMuw9QnFMOczV7PlwDzKSsuoHFOphC4ikqWU1FPMJdL9HWBDl0PqclndZ2fg6/1s0xhm/XhuAkGIiEh3peb3TBJgh7Zj60vJyc3vsBwOfjbqSq775evJik5ERDKcknp3YjB02GhOGX5qh+VOP+x0rrvwF10SloiIZAYl9UyQwKC3USefx2nDT+uwaEGoYP9iEhGRbkdJPVMETOyVx07lw8iHHZY7a+RZ+xmQiIh0NxoolwkCDqbLtxDF+cVs27ktbrkcy2Hq2KlJCExERLoT1dQzRYCaupn3Geyg3gfFLXf+kedr2pqISA+kpJ4pAtTWm5y3SfuY/mPoldOrzTIFoQLOGHFGMiMTEZFuQkm9G2l2XnW+cnQleTl5bZbJz83XErAiIj2UknqGMH9ueRAlBd5SryX5JRTlFQFaAlZERDRQLjM4OCCay79ym+KWKXR7PoNNOGQCG67eQHhVmOpt1VoCVkRElNQzxRnNw3kwp7r9vnWD83oft9eh4vxiph0zLfXBiYhIt6Dm9wzxvcFfjvu8ReGUky/qomhERKQ7UlLPELcOeId5K/Lb7Vcvzu1N5bGaey4iIu1TUs8EBm9tf5dZNyzj0Ud6kdvs1cwBQg5KQr1ZPPUZ9ZeLiEhc6lPPBA5G9hsJEyZwzvNb+OT+ewi/v4jqPjDsgEPYcO1mJXQREemQknqGuGniTd4PxcUUXzadaUwHYMWKFUroIiISSMY0v5vZzWb2ppm9ZmaPmNmB6Y5pf0W2bghcdmDxwBRGIiIiPUHGJHXgWWCMc+4zwNvAdWmOZ7/NufuydIcgIiI9SMYkdefcM865ltVXXgCGpjOeZLhtx9LAO7CJiIjsL3Mu4NqkXcjMngDCzrn72nn+cuBygAEDBhz7wAMPdGV4gb2y8R9BV37l2EHHtnm8pqaG4uLs71PvKfcJPedee8p9Qs+5155yn5DZ93raaaf9wzk3rq3nujSpm9lSoK3O4+udc4/5Za4HxgHnugDBjRs3zr388svJDTRJ7Efm1dQ7qq07cD9u+1ZXrFjBqaeemuzQMk5PuU/oOffaU+4Tes699pT7hMy+VzNrN6l36eh359zEeM+b2VTgbOBzQRJ6xguY0ANX50VEROLImCltZjYFuBY4xTlXl+54uoz63EVEJEkyZqAccDtQAjxrZq+a2V3pDkhERKQ7yZiaunOuLN0xiIiIdGeZVFMXERGR/aCknkpBBsA56G0Z02AiIiLdmJJ6CvWtIVBif/K8R1Mei4iIZD8l9RSavD430Oj208eclfpgREQk6ympp9B7/XM0B11ERLqMknoKNRT11jx0ERHpMkrqKTQ6Z1DHNXUlfRERSRIl9RS68ZD/6LBMr5xeXRCJiIj0BErqKTT4om9w6Hbi1tbHDhjbZfGIiEh2U1JPpZISnpn8+7hFFp63sIuCERGRbKeknmKHT/kyVx073XsQU2O/6virGFE6ouuDEhGRrKSk3gVu+ffbqb6ymhOHncjg4sGcOPREqq+s5pbJt6Q7NBERySJan7SLjCgdwfPTnk93GCIiksVUUxcREckSSuoiIiJZQkldREQkSyipi4iIZAkldRERkSyhpC4iIpIllNRFRESyhJK6iIhIllBSFxERyRJK6iIiIllCSV1ERCRLKKmLiIhkCSV1ERGRLKGkLiIikiWU1EVERLKEkrqIiEiWUFIXERHJEuacS3cM+8XMtgDvpzuOFOoHfJzuILpAT7lP6Dn32lPuE3rOvfaU+4TMvtdDnXMHt/VEt0/q2c7MXnbOjUt3HKnWU+4Tes699pT7hJ5zrz3lPqH73qua30VERLKEkrqIiEiWUFLPfL9OdwBdpKfcJ/Sce+0p9wk95157yn1CN71X9amLiIhkCdXURUREsoSSeoYysylm9paZVZvZ7HTHkypmNszMlpvZG2a22sxmpDumVDKzHDP7XzN7Mt2xpJKZHWhmD5nZm/7v9oR0x5QKZnaV/3e7yszuN7Ne6Y4pWczst2a22cxWtTpWambPmlmV//2gdMaYLO3c683+3+9rZvaImR2YzhiDUlLPQGaWA9wBnAmMAr5sZqPSG1XKNAFXO+eOBI4HpmfxvQLMAN5IdxBdYD7wtHPuCGAsWXjPZjYE+A4wzjk3BsgBvpTeqJLqbmBKzLHZwDLnXDmwzH+cDe5m33t9FhjjnPsM8DZwXVcH1RlK6plpPFDtnHvXOdcAPACck+aYUsI5t9E594r/cwTvP/8h6Y0qNcxsKHAWsCDdsaSSmfUBTgb+P4BzrsE596/0RpUyuUBvM8sFCoENaY4naZxzfwa2xRw+B7jH//ke4AtdGlSKtHWvzrlnnHNN/sMXgKFdHlgnKKlnpiHA+laPPyBLE11rZjYcOBp4Mb2RpMytwHeBaLoDSbHDgC3A7/yuhgVmVpTuoJLNOfchMBdYB2wEtjvnnklvVCk3wDm3EbwP5ED/NMfTVf4DeCrdQQShpJ6ZrI1jWT1NwcyKgYeBmc65HemOJ9nM7Gxgs3PuH+mOpQvkAscAv3TOHQ3Ukj3NtLv5/cnnAJ8CBgNFZnZxeqOSZDOz6/G6CRemO5YglNQz0wfAsFaPh5JFzXqxzCwPL6EvdM79Md3xpMhJwOfNbC1ed8rpZnZfekNKmQ+AD5xzLS0uD+El+WwzEXjPObfFOdcI/BE4Mc0xpdomMxsE4H/fnOZ4UsrMpgJnAxe5bjL/W0k9M70ElJvZp8wsH2/wzeNpjiklzMzw+l7fcM7dku54UsU5d51zbqhzbjje7/M551xW1uqccx8B681spH/oc8CaNIaUKuuA482s0P87/hxZOCAwxuPAVP/nqcBjaYwlpcxsCnAt8HnnXF264wlKST0D+YMzvg0swftP4g/OudXpjSplTgK+ildzfdX/qkh3ULLfrgQWmtlrwFHAz9IcT9L5LREPAa8Ar+P9f9otVyFri5ndD/wNGGlmH5jZNOAm4AwzqwLO8B93e+3c6+1ACfCs///SXWkNMiCtKCciIpIlVFMXERHJEkrqIiIiWUJJXUREJEsoqYuIiGQJJXUREZEsoaQuPYKZ/djMXKuvDWb2sJmNCHDu1/xzipMc06n+dcck87r+tYf71z47QNkBZnarmb1jZvVm9omZPWVmk5MdVzYys/Fm9uOAZceZ2d3+DoxRM7s7tdFJT6OkLj3JduAE/+savPnTywKsS77IPyfZC1C84l/3nSRfNzB/gZj/xdtoZi4wCbgEWAs8bmZj0xVbNzIe+FHAsicBE/AWmPooZRFJj5Wb7gBEulCTc+4F/+cXzGwd8BegAngwtrC/BW6Oc24L3gYlSeWvcf9ChwVTayHe7lQnxqy5/4SZ/RLI1t3V0uU259x8ADN7Od3BSPZRTV16spbNVYYD+M2iL5vZF8xsNbALOC62+b1V0/aFZvYrM9vur0L1EzPb69+UmX3GzJ4ws3+ZWY2Z/d3MzvCf26f53X88y8zmm9k2/7zb/OWCW8oMMrPfmtm7ZrbTzN42sxtalwnCzE4GjgWua2sTHefca865da3KX2hmr/tN9OvN7Kf+lqMtz7e8T8eY2Qozq/NX4jrGzIrM7Hf+e/WumX05JpYVZvaQmV1uZmv9+1pk3p7lrcv1M7N7zGyrf/0VZjYupsxaM5trZlf5v5dPzOwBMzswplyp//vbZGa7zOyvZnZcTBlnZjPM7GdmtsXMNpvZHWZW0HLPwG2tyjozW9Hee+6cy/Yd+iTNlNSlJxvuf/8o5tgc4Ea8Gvx7cc6fA9QA5wP3AT/0fwbAzI4AngcGAd8Evgg8wt6b9bTlarxNfC4CbgAuB37a6vl+eLXrWcAU4GbgUvzkkoBTgGZgaUcFzWwSEMbrMjjHf61r8JbSjHUPcD9wHt6Ogw/hre+/Ae/9eRG417z95Vs7AW952VnANOAzwKMxZR4FJvuvXYn3f9hyMyuLKXch3lrsl+Ot3302rZaq9ZPyUrylTv8Tb1/wLcBSMxsYc62r8XZhuxjvvf4GMMN/bhEwr1X8JwBXtPGeiHQN55y+9JX1X8CPgY/xupxygcOB5cAOYJBf5m68LW6Pijn3a/7xYv/xcP/xvTHlXgUeaPX4frwdy3q3E9Op/nXGtDrmgDeBUKtj1+P155e2c51c4Ct4LQv5MTGeHec9uQvYGPD9ewFYHnPsu3gfCobGvE9TW5Wp8I/9ttWxA4BG4Futjq3wjx3a6thJ/rlT/MdT/MentCpThJeMf9Xq2Fq8cQq5rY7dCnzU6vE0oAEoj3kf3wFujvl9/Dnmvh8FXmj1+Nvef6UJ/02+DNyd7n8b+squL9XUpSfpi5c4GoG3gMOASufcxlZlPnTOvRrwes/EPF6DV8NucToQds7tTDDOx9zezbR/BHoDY8Db2c7MZprZGjPbiXc/C4EC4JAEX6vDzR/8sQXHsO+4gzBeTfmEmOPLWv1c7X9/bvcLOrcdLxHv1bQOvOKce79VuefxtvYc7x8aD2xxzv2pVZla4Em8wWetLXfexkgt1gD9W3VRTMTrfnnPzHJbdSP8CdirOZ+Of88iGUMD5aQn2Y73n7nDa3Lf4JyLTWqbErhe7CCyBqBXq8d9gY0kLnaP6pbHg/zvM/FGqt+El4Q+Af4NuCPm9TvyIXCwmfVyzu2KU64fkMe+703L49KY463fl4Y2jrUcj421rb25N7Pnvge1EUNLHPFiaHk9A/L9n/sBx+N9IIoVOxshSOwiGUFJXXqSJudcRyOOk7lt4Vb2JKRE9G/nccsHhAuAB51z17cUMLNRnXidFcB/4fU9L4pT7mO85Bcb1wD/+7ZOvHZbYq/fcqzlvje2U2ZAJ2LYhtf8/a02nqtP8FoiGUPN7yKpswy40MwSrdWdEzOK/lxgJ7DKf9ybfRPPRYkG55z7C14T9M/MrCT2eTP7tJkNc841++UuiClyIRDF24c6GY4xs93dB2Z2El4S/7t/6EW8JvSTW5UpxJtjvzLB11oGlAHrnHMvx3y9nuC1GvxYVHuXtFNNXSR1foK3yMifzWweXs39aGCrc+63cc4rAR40s98Ao/FG1d/unGupjT4LfMfMXsRrKr4IL0F1xkV4AwZfNrP/xusv7oM3wvwy4DhgPd7iKkvM7HfAA8Cngf8H/MY590EnXzvWZuBJ81Zn6wX8HK+f/WkA59wSM3seCJvZbLz38xq8Dzk3J/ha9+LNSFhhZnOBd/G6S8bjDaj77wSu9ab/fYaZPQfscM691VZBMzsYb9YBwEHAoWZ2PoBz7qEE70FkH0rqIininHvLzCbg9X0v8A+vAb7Xwanz8Abx3Y/XmrYg5pz/Ag7Gm+4G3kC67wBPdDLGY4Dr8EazD8Ebaf934CvOuX/65Z4xsy8B38f7ILDZjzPoSmpB/A1vmtmtePe3Am9KWmtf9F/3VrzE/3fgdOdcNQlwzu0ys9Pw3suf4DXhb/av93iCcf8F70PFDLypkH/Gm9nQltHsPeDwsFZlLcHXFdmH7TtOSETSxcwccKVzrq3531nLX7DlY+fc+R2VFZH2qU9dREQkSyipi4iIZAk1v4uIiGQJ1dRFRESyhJK6iIhIllBSFxERyRJK6iIiIllCSV1ERCRLKKmLiIhkif8DxCjHnwm2fxIAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#visualizing pca\n", + "pca_visualize_df = pd.concat([principalDf, y_train], axis = 1)\n", + "fig = plt.figure(figsize = (8,8))\n", + "ax = fig.add_subplot(1,1,1) \n", + "ax.set_xlabel('Principal Component 1', fontsize = 15)\n", + "ax.set_ylabel('Principal Component 2', fontsize = 15)\n", + "ax.set_title('2 component PCA', fontsize = 20)\n", + "targets = [1,0]\n", + "colors = ['r', 'g']\n", + "for target, color in zip(targets,colors):\n", + " indicesToKeep = pca_visualize_df['Y'] == target\n", + " ax.scatter(pca_visualize_df.loc[indicesToKeep, 'principal component 1']\n", + " , pca_visualize_df.loc[indicesToKeep, 'principal component 2']\n", + " , c = color\n", + " , s = 50)\n", + "ax.legend(targets)\n", + "ax.grid()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, there is no clear linear separation for the target attributes for 2 pca components, justifying the above percentages. Nonetherless, we will continue to use PCA by finding the optimmum number of PC components which retains 90% of information" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Perform PCA to retain 90% of information\n", + "perform PCA to reduce components so we can run SVM model" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": {}, + "outputs": [], + "source": [ + "#setting pca threshold to 90%\n", + "pca = PCA(0.9)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PCA(copy=True, iterated_power='auto', n_components=0.9, random_state=None,\n", + " svd_solver='auto', tol=0.0, whiten=False)" + ] + }, + "execution_count": 73, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pca.fit(X_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "No. of components before pca: 44\n", + "No. of components after pca: 13\n" + ] + } + ], + "source": [ + "#get number of components after pca\n", + "print('No. of components before pca: {}'.format(len(X_train.columns)))\n", + "print('No. of components after pca: {}'.format(pca.n_components_))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, the number of components is reduced from 26 components to 13 components" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": {}, + "outputs": [], + "source": [ + "#perform pca on training and test attributes\n", + "X_train_pca = pca.transform(X_train)\n", + "X_test_pca = pca.transform(X_test)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Apply SVM model\n", + "Next, we will used the reduced attributes train set to train our SVM model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First, we train our SVM model without parameter tuning\n", + "nor pca reduction" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n", + " max_iter=-1, probability=True, random_state=None, shrinking=True, tol=0.001,\n", + " verbose=False)" + ] + }, + "execution_count": 76, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn import svm\n", + "#train svm model without standardization and no parameter tuning\n", + "clf_original = svm.SVC(kernel = 'rbf', probability = True, gamma = 'scale')\n", + "clf_original.fit(X_train, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.1660749149902864\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAEWCAYAAAAkUJMMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gUVffA8e8hgYSQhBZaCKH33kWKKKCAoLyCCCIW9KeIYn0VK9hRxA72V7GCCqIIKkVRQFSK9N4htFDSQ0LK/f0xQ1xCygaymezmfJ6Hh+zUM7Ozc2bu3LlXjDEopZRSTinldABKKaVKNk1ESimlHKWJSCmllKM0ESmllHKUJiKllFKO0kSklFLKUV6TiERkhIgscDoOp4lIpIgkiohfEa6zjogYEfEvqnV6kohsEpGe5zHfeR2DIlJWRH4QkTgR+aag83urghyrhXlci8g0EXnOzWl96th2ZW9Xg2IQR77fx3klIhHZKyKn7APniL2i4PML0z3GmC+MMZd7ch3Fkb2ve5/5bIzZb4wJNsZkOBmXUwrjx2WMaW6M+S2f9ZxzgrqAY3AIUA2obIy59jzmzym+x0Rkj/0bjBKRr+zh74nIpzlM30pEUkWkkog8ZW/bPdmmuc8e/lRhxFiQY9UbjmsR6SkiURcw/5ljKtH+t1dEHsk2jeu5NUZE5olILZfx00TktMsyEkXkugvZruLgQu6IBhpjgoE2QFvg0cIJqWg5eSXki1dh+Smh+7s2sN0Yk17QGXOKWURuAkYCve3fYAfgF3v0NOAaESmXbbYbgbnGmJP25+3ATTlMs72gMbobt8pSwf7ehgBPikifbOPPnFtrAEeBt7KNn2Qn7TP/vvJksEVR+nLBRXPGmCPAfKyEBICIBIjIZBHZLyJHReRdESnrMv5qEVkrIvEisktE+trDy4vI/0TksIgcFJHnzuwEEblZRJbZf78rIpNd4xCR70XkAfvvcBGZJSLH7KvGe1yme0pEZorI5yISD9ycfZvsOD61598nIk+ISCmXOP4QkbfsopatItIr27x5bcMfIvKaiJwEnhKR+iLyq4icEJHjIvKFiFSwp/8MiAR+sK98Hs5+pS4iv4nIs/ZyE0RkgYiEucRzo70NJ0TkScl2h5Vtu8uKyCv29HEissz1ewNG2N/pcRF53GW+TiLyp4jE2ts9RUTKuIw3InKXiOwAdtjD3hCRA/YxsFpEurtM7yfWFf8ue5tWi0gtEVliT7LO9UpQRAbYx1OsiCwXkVYuy9orIuNEZD2QJCL+rvvAjn2VHcdREXnVnvXMumLtdXVxPQbteZuLyEIROWnP+1gO+/RpYDxwnb2cW0WklH1M7RORaPtYK29Pf+b7vVVE9gO/5vBVdQTmG2N2gfUbNMa8b//9J3AQGOy6P4HrgU9clrESCBKR5me2BShrD89RQePO4VitKyJL7O90kYhMFZHPs83v7nH9jVilMXH2MpvnFne2bfAT69x0XER2A1dmG3+LiGyx17lbRO6wh5cDfgLC5d87kfD8jv28GGNWAZtwOXdmG58CzASaubO8vIhIN/v3dqn9uYnLsbtNRIa6TDtNRN4RkR9FJAm41B42Vaw7tAQR+VtE6rvMk+vy3GKMKfA/YC/W1RhABLABeMNl/OvAHKASEAL8AEy0x3UC4oA+WImwJtDEHvcd8B5QDqgKrADusMfdDCyz/+4BHADE/lwROAWE28tcjfXjLwPUA3YDV9jTPgWkAYPsacvmsH2fAt/bsdfBukq81SWOdOB+oDRwnb09ldzchnRgLOCP9cNvYO+LAKAK1gnw9Zz2tf25DmAAf/vzb8AuoJG9vN+AF+1xzYBEoJu9Lybb2947l+91qj1/TcAPuNiO68w6P7DX0RpIBZra87UHLrK3qQ6wBbjPZbkGWIh1PJS1h90AVLbneRA4AgTa4x7COqYaA2Kvr7LLshq4LLsdEA10tmO+yd5nAS77by1Qy2XdWfsU+BMYaf8dDFyU037O4RgMAQ7bsQfanzvnsl+fAj53+TwK2Il1bAYD3wKfZVvvp1jHUE7H5w3ASXs/dQD8so1/HFjk8vkK4BhQ2jUe4DHgJXvYJKxSjc+Bp3LZjgLFnX0f2vt6Mtax2A2IP7Nfcpj2N3I5rl1iCcE6Pl8H1rqMmwY8l8s2jAa22sdDJWBxtvVeCdTHOu4uAZKBdva4nkBUtuXleexnmzb7Nl5kL/8/uZxbg7AuHj51Z9tyWJ/BOr9cgXW+7GQPL2d/vsWOux1wHGjuso44oCvWOTLQHnYS6/ztD3wBzCjA8vKM+UISUSKQYG/sL1i3m9hfYBJQ32X6LsAe++/3gNdyWGY1rJNbWZdhw4HFOZwEBNgP9LA//x/wq/13Z2B/tmU/Cnzs8iNckse2+dlxNHMZdgfwm0sch7CToD1sBVZRiTvbsD+3ddvTDALW5HRg5vGDfcJl/BjgZ/vv8cB0l3FBwGlySET2AXcKaJ3HDygi2zYPy2Ub7gNmZ/tBXJbPdsecWTewDbg6rx+Xy+d3gGezTbMNuMRl/43K4fg982NfAjwNhOWyzbklouGu31M+2/YUZyeiX4AxLp8bY10g+Lust14+yxwBLML6rZ0AHnEZF2kvL8L+/AVnXyg+hZVwIrF+R6Xt/2uRdyIqUNyu+9BeVzoQ5DL+c/JORDke1znEVcGet7z9eRq5J6JfgdEuny/P/j1nm/474F77755kS0T5Hfu5HFOxWL81g5WYXc8le7HOrbH2/joEtHQZPw1IscfHAsfziMVgnfv2ZVvGdcDSbNO+B0xwWcen2cZPAz50+dwf2FqA5eWZiC6kaG6QMSbE/nKaAGdum6tgnfBW27erscDP9nCwDvZdOSyvNtYP4rDLfO9h3VWcxVhbNwPrZABWscMXLssJP7MMezmPYSWJMw7ksV1hWFds+1yG7cO6SzjjoB2D6/hwN7fhrHWLSFURmSFWMV481o8zjII54vJ3MtbVKnZMWeszxiRjnbRyEoZ15ZPTd5PnekSkkYjMtYtK4oEXOHcbsm/3g3YRSJy9n8q7zJPbMZKT2sCD2b7vWljbnuO6s7kV66p7q4isFJEBbq63IDFmF865x5c/7h+jGKviRG+sk/Bo4BkRucIetx8rwd4gViWiQZxdLIfLdDuxvq8dxpg813mBcYcDJ+1jML9pz8jtePMTkRfFKrqNxzp5g3u/m7N+E5y9PYhIPxH5yy5iisU64ea6XDeP/ezC7G35L9b5s3S28YOMMRWw7vbuBn4Xkeou4ycbYyrY//Jb133A18aYDS7DagOds/1mRgCu68jpu8ntPOPO8vJUGM+IfsfKeGee2RzHyvbNXXZWeWM9fANrA+ufuyQOYN1NhLnMF2qMya3sdzowRERqY90FzXJZzh6XZVQwxoQYY/q7hp3HJh3Husqr7TIsEqvc/YyaIiLZxh9ycxuyr3uiPayVMSYUq9hF8pi+IA5jFZ0C1jMgrOKwnBzHutLK6bvJzztYxR0N7W14jLO3AVy2Q6znQeOAoUBF+0cX5zJPbsdITg4Az2f7voOMMdNzWnd2xpgdxpjhWBcLLwEz7ecB+e33gsSY3SHOPb7SsR5MZ4XmzoKMMWnGmG+A9UALl1GfYFU+GIz1e/gnl0V8ilW8eE5Nu0KO+zBQSUSCXIbVymXa/FwPXA30xrqAqWMPz37M5RaH63ojz/whIgFY55HJQDX7uPzRZbk5bZs7x/45jDEZxphXsH5zY/KY5lsgA6so83xcCwwSkftchh0Afs/2mwk2xtzpuvoCrMOd5eWpsN4jeh3oIyJtjDGZWM8SXhORqgAiUvPM1RrwP+AWEekl1sPPmiLSxBhzGFgAvCIiofa4+iJySU4rNMaswSr3/hDrwW2sPWoFEC/WA+qy9tVTCxHp6M6GGKv66NfA8yISYie6B7DuVM6oCtwjIqVF5FqgKfBjQbfBFoJ9Ky4iNbHK/V0dxSqTPx8zgYEicrH9APVpcvmR2N/bR8Cr9kNYP7Ee0Ae4sZ4QrPL+RBFpAuR3AIZgncCOAf4iMh4IdRn/IfCsiDQUSysROZNAs++PD4DRItLZnraciFwpIiFuxI2I3CAiVeztP3MMZdixZZL7vp8LVBerynOAfax0dmedWBdR94v18D4Y6yr6K+NmrTqxKk1caa+zlIj0A5oDf7tMNgvrhPs0OdwNufgKq3jqa0/GbYzZB6zCqqBTRkS6AAPdWGdOQrAu+E5glb68UIB5v8b67UaISEXAtfp0Gay7kGNAur1fXavrHwUqi11BwyWWghz72b0IPCwigdlH2Mfz1VjPwLcUcLlnHAJ6YW3zmYQ3F2gkIiPtc1hpEekoIk3Pcx0XvLxCSUTGmGNYV1RP2oPGYd3y/2Xfri7CKk/GGLMC66HWa1hXwb/z71XWjVgHw2asZwYzsaow5mY61lXRly6xZGAd4G2APVhX+h9iXTm5ayxW2ftuYJm9/I9cxv8NNLSX/TwwxBhzpsiroNvwNNbDvThgHtYDYFcTgSfsW97/FmAbMMZssrdlBtaVYALWg/3UXGb5L1YlgZVYDyZfwr1j5L9YV6kJWIkhv+qk87FqIG3HKhpJ4eyigFexThgLsH7k/8N6YA3W841P7P0x1Fg1j/4PmIK1v3eSQ03IPPQFNolIIvAG1nOvFLsI6XngD3tdF7nOZIxJwKpkMhCryGIHcKmb6/wI+Ayr+GwP1vaPLUDM8VhX3vuxkuck4E5jTFaNPmNMEv8moy9yWog93SljzCJjzKkiiHsE1vPiE8BzWMdJbsdiXj7FOm4OYv3O/irAvB9gHX/rgH9w+b3Z3+k9WMdeDNYxPcdl/Fasc85u+5gIp+DHfnbz7HX9n8uwH+zjMR7rGLzJ/i2fF7sIthcwTkRus7fzcmAYVqI6gvVbd+eiM6flX/DyztQ6U24SkZuB24wx53ur7Bj7KjYWqxhhj9PxqJJNrJdwtxpjJjgdi3KW1zTxo86PiAwUkSD7ucdkrDuevc5GpUoiu7imvl2c2BfrOc93TselnKeJyPddjXW7fAirOHGY0dtg5YzqWNWyE4E3sYoT1zgakSoWtGhOKaWUo/SOSCmllKO8rmHCsLAwU6dOHafDUEopr7J69erjxpgq+U9Z9LwuEdWpU4dVq1Y5HYZSSnkVEdmX/1TO0KI5pZRSjtJEpJRSylGaiJRSSjlKE5FSSilHaSJSSinlKE1ESimlHOWxRCQiH4nVr/3GXMaLiLwpIjtFZL2ItPNULEoppYovT94RTcNqYj83/bDaPmsI3I7VwZRSSqlClplZvJty89gLrcaYJSJSJ49JrsbqF91g9VtUQURq2J3LKaWUukCHY08x+M1lHEo+7XQoeXKyZYWanN0ZWpQ97JxEJCK3Y901ERkZmX20UkopF3HJaVz91lL2xlj9HUqyW53/OsbJygo5dVmd4/2jMeZ9Y0wHY0yHKlWKZVNJSilVLOw+lkjrZxawN+YUJj2TS0OC2PHq+fbKXjScvCOKwurG+IwIrD5zlFJKFVBKWgaDXl/K1hNJALSuFsKUIa2pVau8w5Hlz8lENAe4W0RmAJ2BOH0+pJRSBXfseBI9XlrMKT8hAHj35o5c2qSq02G5zWOJSESmAz2BMBGJAiYApQGMMe8CPwL9gZ1AMnCLp2JRSilfZIzhw2lreObXHfjVDKaKERY/2Yvg4ACnQysQT9aaG57PeAPc5an1K6WUrxsybj6rS2XgVzMYgAXjexNcrozDURWc1/VHpJRSJdmpU2nsOZLI60t3sbpUBgAP9G7EHT3rEeDv53B050cTkVJKeYFjCam8MH0ts5fvh+pBWcPfvaEdfVvUcDCyC6eJSCmliiFjDKfSMpj441YWbzlKVFyKNaJqWdpUCeb6S+pxbfsIRHJ6E8a7aCJSSqli5r3fdzHxp61nDUvdn8BVjaryxiPdCQws7VBknqGJSCmlioHohBRmro7ivd93E3cqDYBOdSvRuVYF/vpsExOfvZwGDSo5HKVnaCJSSikHxaek8eqC7UxbvjdrWNUMkD8OM/35fvj5lYL+TZ0LsAhoIlJKqSK25XA8k+dv45et0WcNH1SrEl+/sJxVhxMZM6YjqakZBAX5frdxmoiUUsrDjDG8v2Q3P286wpr9sWeNu6ZtTbrUqsDHz/3BGy+tom3b6nz/3TA6dqzpULRFTxORUkoVsuj4FLYdTSAlLZNv/4nip41Hssa1jihP0xqhDO1Yi3aRFQFITU3n5ePJvP76Fdx1Vyf8/X3/LsiVJiKllLoAp05n8N3ag+w5nsSCTUc4GHuKtIxzOxIY1bUu9/ZuSPmyVo23JUv2ccX/zWPWrKEEB5fhr79uo1Qp76+KfT40ESmllJui41OYveYgcafSOJWWwYJNRzkYe+qsacKCy3B959o0Dw8lvHxZAkuXok5YOUr7WXc5x48n89BDC5k2bS116lRg795YWrSoWmKTEGgiUkqpPKVlZLJw81Emz9/G7uNJWcNDAv0JCfCne8MwBrYOZ2CrcMqWyb2JHWMMH3+8loceWkh8fCqPPtqNJ57oQVCQb70TdD40ESmllIuk1HTeX7Kb+ZuOEJN8mqPxqVnjWtQM5e5LG3Bpk6rn1a7b55+vp1mzKrz77pU0b+493TR4miYipVSJd+BkMm/8soM5aw9xOiMza3jDqsG0qVWBljXLM7B1OLUrlyvQcpOT03jhhaWMHt2BiIhQZs0aSvnygSW6GC4nmoiUUiVKWkYmc9cfYvHWYxyNT2HDwTiST2dkje/dtBo9G1dhaIdalLmA2ms//riDu+76kb17Y6lZM4Q77+xIxYplC2MTfI4mIqVUiXAiMZWHZ64/5yXSjnUqEhzgz/Wda9OjUdgFd6UQFRXPfff9zKxZW2jaNIzff7+ZHj1qX9AyfZ0mIqWUzzDGkJCazsaoOE6lZXAw9hSJqeks23Gc5btOZE13S9c63NurIRWCCr8TueefX8K8eTt44YXLePDBiymTRwUGZRGro1Tv0aFDB7Nq1Sqnw1BKOcgYw+bD8ayPiuOrlQfYfDiezExDhjHkdkoL8C/Fi4NbMrBVOP5+hfvC6IoVBylb1p+WLatx4kQycXGp1KtXsVDXcaFEZLUxpoPTceRE74iUUl7jWEIqN3+8gk2H4s8aXrtyEJGVgmhbqwIBpf0oX7Y0rSLKE+DvR8Wg0gQH+hPg74dfIVcSiItL4bHHfuGdd1YxYEAj5swZTuXKQVSuHJT/zCqLJiKlVLGWlJrOJ3/u5ZPle7OqUtcoH0iriPKM6lqX1rUqEFi6aIu/jDF89dUm7r9/PtHRSYwd24lnn72sSGPwJZqIlFLFzuG4U7yyYDtbDsefdffTLrICg9tHMKKzsw//P/98PTfe+B0dOoQzd+5w2rcPdzQeb6eJSClVLMQlp7HlSDwfLt3Noi3/1mzrXLcSlzapys0X1ynyOx9Xqanp7N4dQ9OmVRg6tDnp6ZnceGNrq78gdUE0ESmlHBGdkMK6A3H8uOEwi7YcJSEl/azxL17TkmGdIh2K7myLF+/hzjvnkZycxo4dYwkI8OeWW9o6HZbP0ESklCoSGZmGv/ec4JkfNrP1SMJZ40IC/GlYNZhhnSLpUq8yzcJDHYrybNHRSfz3vwv47LP11KtXkfffH0hAgJ42C5vuUaVUoTt1OoN/9sewPiqO5buOs3TH8bPGVw8NpFPdSvRpVo2mNUJoUDXEoUhzt3PnSTp1+oDExNM8/nh3Hn+8O2XLagOlnqCJSCl1waITUvjy7/3MXB2FMZzTNQJAtwZh9GgURvvalWhfu3i9Y+MqPj6V0NAA6tevyK23tmXUqLY0bVrF6bB8miYipVSBRSekcNsnq0hMST+rawSAsOAAbuxSm2qhgfRuWo3alYMcrWTgrqSk0zzzzO988ME/rF9/JxERobz88uVOh1UiaCJSSuXJGENqeibvL9nN6n0x/L792Fnjr20fQXqmoX/LGvRuWhUR72tZ+ocftnH33T+xf38ct97aVvsIKmKaiJRSZ5m7/hC/bolm7YFYDsQkn9Ptdac6lQgoXYoh7SMY2Crcq7s0SE/PZOjQb5g9eyvNm1dh6dJb6NateNTUK0k0ESlVgkUnpPDNqihW74vhYMwpth39tzZbuTJ+NK0RSlhwAB3qVCQksDTXto/wimK2/BhjEBH8/UtRo0YwL77Yi/vv76INlDpEE5FSJYgxhvVRccxYeYB56w8R7/LuTrcGYdSoEEiFsqUZ168JNcr7Zt85f/0VxV13/cgHHwykXbsaTJ16pdMhlXiaiJQqIaITUrh6yh8cjkvJGta9YRjXd4qkb4vqXvlspyBiYk7x2GO/8N57qwkPDyEm5tyafcoZHk1EItIXeAPwAz40xryYbXwk8AlQwZ7mEWPMj56MSSlfZozhYOwpZv9zkM2H40lNz+R0eiZ/7zmR9aynb/PqjLm0Pi1rlvf55HPGV19t5J57fub48WTuu+8inn66JyEhAU6HpWweS0Qi4gdMBfoAUcBKEZljjNnsMtkTwNfGmHdEpBnwI1DHUzEp5UsyMg3bjiSwaMtRft1qVS7IrmmNUILK+NGhdiX8/YRhHSO5slUNB6J11tatx6lTpwI//zyCtm1L3vYXd568I+oE7DTG7AYQkRnA1YBrIjLAmbY8ygOHPBiPUl7PGMM/+2N5fPaGc5rJaVI9hLph5WgeHkrj6qH0alLVq2u0XYiUlHReemkZ7drVYODAxjz2WHeeeKKHNlBaTHkyEdUEDrh8jgI6Z5vmKWCBiIwFygG9c1qQiNwO3A4QGalVK1XJYYxh1b4Ypq/Yz8LN5zYMemu3uvRoVIXOdSv5RG22wrBo0W7GjJnHjh0nefDBLgwc2JjSum+KNU8mopwuxbJ34jscmGaMeUVEugCfiUgLY0zmWTMZ8z7wPlhdhXskWqWKkW1HEvho2R6+XRN11ns8PRtXoVXN8vRtUaPYNAxaXBw9msgDDyzgyy830KBBJRYsuIE+feo7HZZygycTURRQy+VzBOcWvd0K9AUwxvwpIoFAGBCNUiXMos1H+XZNFHuOJ7PlsNUZXIWg0gxoVYMbLqpNk+qaePKycOFuZs7czPjxPXj00e4EBmqlYG/hyW9qJdBQROoCB4FhwPXZptkP9AKmiUhTIBA4hlIlRGJqOl+vPMD7S3ZzJN6qVh1YuhQ3XBTJ4HYRtI0svo2DFgfr1h1hx46TDBnSjBEjWtK1ay3q1tV95m08loiMMekicjcwH6tq9kfGmE0i8gywyhgzB3gQ+EBE7scqtrvZGKNFb8onZWYajiakEBVzim//iWLN/tizKhz0a1Gdp65qTrXQQAej9A6JiaeZMGExb7zxN3XqVGDQoCb4+5fSJOSlPHrvar8T9GO2YeNd/t4MdPVkDEo5xarhFkN0fCpLdhxn+or950wzuF0EXepXpluDMKqX1wTkju++28rYsT8RFRXP7be3Y+LE3vj7a204b6aFqEoVkrSMTLYfTWD+xiNsOZLA2gOxHEtIzRof4F+KSxtXpX+rGkRULEs7LXYrsA0bjvKf/3xFy5ZV+eqrIVx8ca38Z1LFniYipc7T0fgUvvh7P3/tPsHO6EROJp0+a3z9KuVoU6sC91zWkKqhAVrkdp7S0jJYunQ/l11Wl5YtqzFv3vX06VNPq2T7EE1EShXApkNx/LIlmgWbj7DxYHzW8MrlytC9YRgX1atMz8ZVaFYjtMQ0n+NJy5cfYPTouWzadIxt2+6mQYNK9O/f0OmwVCHTRKSUG+JT0nhz0Q4+XLYna9gljapww0W1S3QLBp5y8uQpHnlkER988A+1aoXy7bdDadCgktNhKQ/RRKRULpJPpzN5/nY++uPf5FOrUlneHNaWRtVCKBegPx9PSElJp02bdzl0KIEHH+zCU0/1JDi4jNNhKQ/SX5JS2SzZfoxPlu/ll63/vlfdvWEYQzvUon/LGvjp3Y9HREXFExERSmCgP88+eylt2lSndevqToelioAmIlXinU7PZPmu47z7+y52RidyPNGqdNClXmUaVw/hkX5NtB03Dzp1Ko2JE5fx0kt/MHPmtQwc2JibbmrjdFiqCLmViESkDBBpjNnp4XiUKhKxyadZuPkoryzYntWiwRlDO0TwcN8mhAVrfzWetmDBLsaMmceuXTHccEMrOnWq6XRIygH5JiIRuRJ4FSgD1BWRNsAEY8x/PB2cUoVtwaYjfLh0Dyv2nswaFlGxLNd1qMXA1uHUrhyktd2KyNixPzJlykoaNqzEokUj6dWrntMhKYe4c0f0DFb3DYsBjDFrRaSBR6NSqpAcT0zluzUHmfTzNk5n/Nuoe3CAP+P6NaFLvco0qBrsYIQlS4b9Hfj5leKiiyIICwti3Lhu2kBpCefOt59mjInNdpWo7cGpYssYwyOzNjB7zcGzkk+FoNLc3qMe17SN0OZ0HPDPP4cZPXouI0e2YuzYzowY0crpkFQx4U4i2iIiQ4FSdkva9wJ/eTYspQom7lQae48nMXf9IT5Y+m9164Gtw+nVpCqXNa1KaGBpByMsuRISUhk/fjFvvrmCKlWCqFEjxOmQVDHjTiK6GxgPZALfYrWm/agng1LKHfEpaSzeGs3sNQf5bdvZvYe0r12RL27rrLXdHLZgwS5GjfqeQ4cSGD26Ay+80IsKFfRuVJ3NnUR0hTFmHDDuzAARuQYrKSlVZDYfimdHdAJr9scybfnes8ZVCw3glq516VC7Im1qVcDfT1tjLg7KlPGjatVyzJo1lM6dI5wORxVTkl/3PyLyjzGmXbZhq40x7T0aWS46dOhgVq1a5cSqlQNW7j3J9L/38+2ag+eMK+NXivv7NGJg6xqEly+rzewUA2lpGbz66p/Ex6fy/PO9AKsfJv1unGeftzs4HUdOcr0jEpErsLrxrikir7qMCsUqplOq0KVlZPLbtmP8ti2aOesOkZCSDkApgdt71KdbgzAaVQumSkiAVrMuZpYt25/VQOm11zbLSkCahFR+8iqaiwY2AinAJpfhCcAjngxKlTwbD8bx0Mz1bDn8b4vW9cLK0bRGKE9e2Yxm4aHatE4xdeJEMuPGLeJ//1tDZGR5fvhhOAMGNHI6LOVFck1Expg1wBoR+cIYk5LbdAamQ68AACAASURBVEqdr40H45g0fxu7ohM5GHsKgNBAf27tVo/hnWpRVfvv8QonTpxixoyNPPzwxYwffwnlymkDpapg3KmsUFNEngeaAVlnBmOMXvKo87LjaAIPz1rPmv2xWcMub1aNMZc2oE2tCg5Gpty1Zcsxvv56ExMm9KRRo8rs338/lSqVdTos5aXcSUTTgOeAyUA/4Bb0GZEqoPiUNFbvjWHq4p2s2hcDQNMaoTw3qAXta2uX2d4iOTmN559fwssvLyc4uAy33tqOiIhQTULqgriTiIKMMfNFZLIxZhfwhIgs9XRgyvut2HOSj5btYXt0AruPJWUNDyrjxyejOtGxjnZ05k1+/nknY8bMY8+eWG66qTUvv9yHKlXKOR2W8gHuJKJUsaon7RKR0cBBoKpnw1LeJjE1ncVbo/lz9wnW7o9ls0ulgwD/UgzvFEn72hVpHVGeBlWDtcabl0lMPM3IkbOpXLksixffRM+edZwOSfkQdxLR/UAwcA/wPFAeGOXJoJT3yMg0vPTzVt5fsvus4RWDSjOgVTgjLoqkcbUQTTxeKCMjk+nTNzJ8eAuCg8uwaNFImjQJI0B7plWFLN8jyhjzt/1nAjASQET0FekSbPW+k7y2cAd7jidl1XYDeHlIK3o3rUZFrTXl9VavPsQdd8xl9erDlC3rz+DBzbS3VOUxeSYiEekI1ASWGWOOi0hzrKZ+LgM0GZUw248mcN+MtVnFbhWDStO/ZXVaRVTgxi61CSqjV8reLi4uhSefXMzUqSupWrUcM2YM5pprmjodlvJxebWsMBEYDKzDqqAwG6vl7ZeA0UUTnnJaRqbhuzUHefqHTcTbrRw0rBrMm8Pb0rRGqMPRqcI2ePDX/PrrHu66qyPPPXcZ5bW7DFUE8rqEvRpobYw5JSKVgEP2521FE5pyUkJKGrd9soq/9/zbk2mriPI8dVVz2kVqdWtfsnt3DFWqBBESEsDzz19GqVJCx47aZbcqOnklohRjzCkAY8xJEdmqScj3pWdksmTHMUZN+7dh2Zu61ObOng20Mzkfc/p0BpMnL+fZZ5dwzz2deOmlPtpCtnJEXomonoic6epBgDounzHGXOPRyFSR23ciiQFvLctqaPSq1uG8Obytw1EpT1iyZB+jR89ly5bjDBnSjHvu6ex0SKoEyysRDc72eYonA1HO2XQojsdnb2TtAavJncbVQvjolo7UrKBvy/ui1177kwceWECdOhWYN+96+vdv6HRIqoTLq9HTX4oyEFX0UtMzuGf6GuZvOgpA1ZAA3hzelovqVXY4MlXYMjMNSUmnCQkJ4MorG3HsWDJPPNGDoCDtPl05T+vbllAnElNp/9wiwEpA741sT1uthOCTNm2KZvToeVk9pTZqVJkXXujldFhKZfFof8oi0ldEtonIThHJsQ8jERkqIptFZJOIfOnJeJR1Zfzy/K1ZSah1RHmWjbtMk5APSk5O49FHF9GmzXts2XKMAQMakl+PzEo5we07IhEJMMakFmB6P2Aq0AeIAlaKyBxjzGaXaRoCjwJdjTExIqJt2HnQ/5bt4dm5Wbufide0ZHinSAcjUp6yZs1hrrnma/bujeWWW9owaVIfwsKCnA5LqRzlm4hEpBPwP6w25iJFpDVwmzFmbD6zdgJ2GmN228uZgfVu0maXaf4PmGqMiQEwxkQXfBNUfg6cTGbs9DVZlRFGXlSbx69sSmBpP4cjU4XNGIOIEBlZnsjI8nzyySB69KjtdFhK5cmdO6I3gQHAdwDGmHUicqkb89UEDrh8jgKy1xFtBCAifwB+wFPGmJ/dWLZyQ1pGJg98vY4f1h3KGvb5rZ3p1jDMwaiUJ6SnZzJlygrmzNnGwoUjqVw5iN9/v9npsJRyizuJqJQxZl+21pMz3Jgvp+aWsxdQ+wMNgZ5YbdctFZEWxphY14lE5HbgdoDISC1KyosxhlX7YvhuzUG++Ht/1vA3h7flqtbhDkamPGXFioOMHj2XNWuO0K9fA+LjU6lYUaveK+/hTiI6YBfPGfu5z1hguxvzRQG1XD5HYDUTlH2av4wxacAeEdmGlZhWuk5kjHkfeB+gQ4cO+rQ1F5mZhnbPLSQ2OQ2wKiIMbB3OjV3qUMbfo/VSlAMSE08zbtxC3nlnFTVqhPDNN9cyeHBT7XJDeR13EtGdWMVzkcBRYJE9LD8rgYYiUherM71hwPXZpvkOGA5ME5EwrKK63agCO3AymSvfXEp8Sjo1K5Tlk1EdaVA1xOmwlAeVLl2K337bx9ixnXj22csIDQ1wOiSlzos7iSjdGDOsoAs2xqSLyN3AfKznPx8ZYzaJyDPAKmPMHHvc5SKyGau47yFjzImCrqskOxh7ilfmb+PbNQcBGNCqBm8Nb6tXxT5q586TPPPM70yd2p+QkABWr76dwEB9HVB5N8nvvQIR2QVsA74CvjXGJBRFYLnp0KGDWbVqVf4TlgAbD8Yx4K1lWZ+nXN+WAa30OZAvSk1NZ9KkP3j++aWUKePHvHnX07271oZT7hOR1caYDk7HkRN3emitLyIXYxWtPS0ia4EZxpgZHo9O5Sgj0zB2+j/8uOEIANe2j+DFwa3wK6V3Qb5o8eI93HnnPLZtO8F11zXn1VevIDxci12V73Drnt4YsxxYLiJPAa8DXwCaiIrY3uNJjJ+ziSXbj2UNmzu2Gy1qlncwKuVJxhief34paWmZ/PzzCK64ooHTISlV6Nx5oTUY60XUYUBT4HvgYg/HpVwkpabTZeIvWT2kgtVH0O2X1NcWsn1QZqbhf//7h759G1CrVnk+++w/VKgQSNmy2kCp8k3u3BFtBH4AJhljlno4HmU7GHuKndGJRMUk8/jsjQBUDw3klaGt6dpAX0j1VevXH2X06Ln8+WcU48f34OmnL6VGDS2GU77NnURUzxiT6fFIFGA9/3lk1nq+WR111vCWNcvz1R0XEVRGa0j5osTE0zz99G+89tpfVKxYlmnTrubGG1s7HZZSRSLXs5qIvGKMeRCYJSLnVK3THloL3z/7Yxj92WqiE6y2Zd8Y1oZG1UKoGhJA5WB9R8SXPfXUb7zyyp/cdltbXnyxN5UrawOlquTI6/L6K/t/7ZnVw9IzMnnwm3V8v9ZqeKJ306q8el0bQgP1mYAvO3AgjqSkNJo0CeORR7oxaFATunXTJqxUyZNXD60r7D+bGmPOSkb2i6rag2shWLw1mlumWS0a+ZUSfr63Ow2r6TMBX5aensmbb/7N+PGLad8+nN9/v5mwsCBNQqrEcueBwyjOvSu6NYdhqgA2Hozjri//Yd+JZAC6Nwzj45s74u+nbcL5sr/+imL06LmsW3eUK69syJQp/Z0OSSnH5fWM6DqsKtt1ReRbl1EhQGzOc6n8pGdk8vKCbbz3u9WkXveGYdzZsz4X19eacL5u3rztDBw4nfDwEL79diiDBjXRppiUIu87ohXACaxWs6e6DE8A1ngyKF9ljOGOz1bzy1ar/79593Sjebi+jOrLjDEcOpRAzZqh9O5dj2eeuZR77+1MSIhWPlHqjHzbmituvLGtOWMMD369LqthUoBNT19BuQCtiu3Ltm8/wZgx89i+/QSbN99FcHAZp0NSJZhXtjUnIr8bYy4RkRjO7tBOAGOMqeTx6HyAMYbRn69m/qajVC5Xhi71KzNpSCt9H8iHpaSk8+KLy5g4cRlly/ozcWIvypbV71up3OT16zjTHbg+vDhPmZmG+75ay/xNRwFY+XhvSmnDpD7tyJFEevT4mB07TjJ8eAteffUKqlcPdjospYq1vKpvn2lNoRZwyBhzWkS6Aa2Az4H4IojPa63Yc5Kh7/0JQGigP8seuUyTkA9LS8ugdGk/qlUrR48etZk6tT99+tR3OiylvII7dYW/w+omvD7wKVbDp196NCovFZ+SxvQV+2n8xE9ZSah1RHmWP9pLX071UZmZhnffXUX9+m8SFRWPiPDhh1dpElKqANwpuM40xqSJyDXA68aYN0VEa83ZPv9rH2/+sgMDHLOb5jljzt1daRVRwZnAlMetW3eEO+6Yy99/H+Syy+qSlpbhdEhKeSW3ugoXkWuBkcAge5he3gP93ljKlsNWCWW3BmEMaFWD2pWC+E/bCMoH6S7yVcYYHnpoIa+//heVKpXls8/+w4gRLfWdIKXOk7stK4zB6gZit4jUBaZ7NqziLSbpNA98vTYrCS24vweNtFmeEkNEiIk5xa23Wg2UVqyofUIpdSHceo9IRPyBM11D7jTGpOc1vSc5/R7R/hPJDH53OccSUqlVqSyfjepMnbByjsWjisa+fbHce+/PjB9/Ce3a1SAz02jlE+VVivN7RPlWVhCR7sBO4H/AR8B2Eenq6cCKo+iEFHq8vJhjCalc0bwaSx++TJOQj0tLy2DSpD9o1uxtFi7czbZtxwE0CSlViNwpmnsN6G+M2QwgIk2Bz4BimVk9JSPT8N9v1gNwb6+G3N+nkcMRKU9bvvwAd9wxl40bo7n66sa8+WY/IiO1SSalCps7iajMmSQEYIzZIiIlqq2SpNR0uk9azMmk00RWCuK+3g2dDkkVgUWLdhMXl8J3313H1Vc3cTocpXxWvs+IRGQakIp1FwQwAggyxtzk2dByVtTPiA6cTKb7pMUAhJcP5Nf/9iSwtF+RrV8VHWMMn322nipVgujXryGpqemkpWVqG3HKJ3j1MyJgNLALeBgYB+wG7vBkUMXFt/9EZSWhPs2qsfzRXpqEfNTWrce57LJPuemm7/j447UABAT4axJSqgjkWTQnIi2B+sBsY8ykogmpeEhJy+CBr9cB8Mq1rRncPsLhiJQnnDqVxgsvLOWll/6gXLkyvPfeAG67rZ3TYSlVouR6RyQij2E17zMCWCgio4osqmLg3hlW4xHjBzTTJOTDfvhhO889t5TrrmvB1q13cfvt7bVGnFJFLK87ohFAK2NMkohUAX7Eqr7t8xZuPsr8TUfxLyXcdHEdp8NRhezIkUTWrj1C374NuPbaZtSpcxudOtV0OiylSqy8nhGlGmOSAIwxx/KZ1mccjjvF/31qVYZY9MAl+OnVsc/IyMjk7bdX0rjxFEaOnM2pU2mIiCYhpRyW1x1RPRH51v5bgPounzHGXOPRyByQmJpOl4m/AvD8f1roy6o+5J9/DjN69FxWrjxE7971ePvt/pQtq+0BKlUc5JWIBmf7PMWTgTgtPSOTFhPmA1YNuRGdazsckSose/bE0KnTB4SFBfHll9cwbFgLbaBUqWIkr47xfinKQJw2drpVOSEsOID3R7Z3OBp1oYwxbNgQTatW1ahbtyIff3w1Awc2pkKFQKdDU0plUyKe++QnNvk0P208QkiAPyse66VXy15uz54YBgyYTtu277F+vdVN+8iRrTUJKVVMeTQRiUhfEdkmIjtF5JE8phsiIkZEivyt3/SMTNo8sxCA165ro1V3vdjp0xm8+OIymjd/m99/38vkyX1o1qyK02EppfLhTltzAIhIgDEmNf8ps6b3A6YCfYAoYKWIzHFtt86eLgS4B/jb3WUXpnVRsQB0rluJ3s2qORGCKgQZGZlcfPH/WL36MNdc05TXX7+CWrW0gVKlvIE73UB0EpENwA77c2sRecuNZXfC6rtotzHmNDADuDqH6Z4FJgEp7oddeOatPwLAY/2bOrF6dYHi461rIz+/Uowa1ZYffhjOrFlDNQkp5UXcKZp7ExgAnAAwxqwDLnVjvprAAZfPUfawLCLSFqhljJmb14JE5HYRWSUiq44dO+bGqt2z5XA8H/2xB4Bm4aGFtlzlecYYpk1bS716b/D991sBGDOmIwMGaPccSnkbdxJRKWPMvmzDMtyYL6eHLVlNfYtIKay+jh7Mb0HGmPeNMR2MMR2qVCm8Mv/J87cB8PUdXSjtp/U2vMXmzcfo2fMTbrnle5o0CaN+/UpOh6SUugDuPCM6ICKdAGM/9xkLbHdjviiglsvnCOCQy+cQoAXwm11LrTowR0SuMsZ4vJ+HPceT+GVrNB1qV6RTXT2ReYtJk/7g8cd/JTQ0gA8/HMgtt7TVCiZKeTl3EtGdWMVzkcBRYJE9LD8rgYYiUhc4CAwDrj8z0hgTB4Sd+SwivwH/LYoktDM6gd6vLgFgUFtt3sUbGGMQEapXD2bEiJa8/HIfqlTRli+U8gX5JiJjTDRWEikQY0y6iNwNzAf8gI+MMZtE5BlglTFmToGjLSRP/2BV3JswsBk3XKQtKBRnhw4lcO+9P9O9eyT33NOZG29szY03tnY6LKVUIco3EYnIB7g82znDGHN7fvMaY37EarXbddj4XKbtmd/yCkNCShpLdxwnLLgMt3StWxSrVOfhTAOljz/+K2lpmVx8sXbFoZSvcqdobpHL34HAfzi7NpxX+XpVFACjL6nvcCQqN2vXHuG22+awevVhLr+8Pm+/3V8rJCjlw9wpmvvK9bOIfAYs9FhEHnQ6PZNn51rFckM71spnauWUuLgUDh1K4KuvhnDttc20ySWlfJzbLSu4qAt45YOVyQus6tr39W5IaKB2AVBcGGP45pvN7Nhxgscf78Ell9Rh9+57CQw8n8NTKeVt3GlZIUZETtr/YrHuhh7zfGiFKz4ljfeX7KZ5eCj39mrodDjKtmvXSfr3/5LrrpvJ999vIy3NekVNk5BSJUeev3axykRaY1W/Bsg0xpxTccEbvPf7LgCGdaylRT3FQGpqOpMnL+e555ZSunQp3nijL2PGdMTfX18sVqqkyTMRGWOMiMw2xnh9Bz1z1h0iwL8U13WMdDoUBRw4EM+zzy5h4MDGvP76FdSsqU0sKVVSuXP5uUJE2nk8Eg/aezyJAydPMaZnA8roFbdjjh1LYsqUFQA0aFCJzZvv4ptvrtUkpFQJl+sdkYj4G2PSgW7A/4nILiAJqw05Y4zxmuR09/R/AGitLTI7IjPT8PHHa3j44UUkJKTSp089GjcOo169ik6HppQqBvIqmlsBtAMGFVEsHpGZaTgYc4qypf3o2biq0+GUOBs3RnPnnfNYtmw/3btH8u67A2jcOCz/GZVSJUZeiUgAjDG7iigWj3jr153EJKfx9FXNnQ6lxDl9OoPLL/+M06cz+Oijq7j55jZaUUQpdY68ElEVEXkgt5HGmFc9EE+hSsvI5LVF26kYVJobu3jlq09e6ddf93DJJbUpU8aPr7++liZNwggLC3I6LKVUMZXXk3s/IBiru4ac/hV7248mAHBtB62yXRSiouIZPPhrevX6lE8/XQdAt26RmoSUUnnK647osDHmmSKLxANe/MnquXNAqxoOR+Lb0tMzmTJlBU8+uZiMjEwmTuzFiBGtnA5LKeUl8n1G5K3SMjJZuuM4IQH+tIqo4HQ4Pm3kyNnMmLGRfv0aMHVqf+rW1dpwSin35ZWIehVZFB7w3RqrMYjRPbWVbU+IjU3B378UwcFluOuujgwe3JTBg5tqEahSqsByfUZkjDlZlIEUtn/2xwJwnbayXaiMMcyYsZGmTafy5JO/AtZzoCFDtJVspdT58dlmBvadSAIgLDjA4Uh8x86dJ7niis8ZPnwWERGh3HCDPgdSSl04n23iePmuE7SsqS0pFJYvv9zAqFHfExDgz5Qp/Rg9ugN+fj57HaOUKkI+mYiW7zwOQNMaXlHLvFhLS8ugdGk/OnQIZ8iQZkya1IfwcN2vSqnC45OJ6KeNRwB48PLGDkfivaKjk3jwwQUkJZ3m22+vo1Gjynz++TVOh6WU8kE+Wbbyz/4YAKqFBjociffJzDS8//5qGjeewldfbaR58ypkZGQ6HZZSyof55B3R0fgUIiqWdToMr7N7dww33PAtf/4ZRc+edXjnnStp0kQbKFVKeZbPJaKVe09yPPE0wztpte2CKl8+gNjYFD75ZBAjR7bS6thKqSLhc0Vzs1ZHAXBly3CHI/EOc+Zs45prviIjI5PKlYPYuHEMN97YWpOQUqrI+FwimrHyAAAX1avkcCTF2/79cQwaNIOrr57B9u0nOHw4EYBSpTQBKaWKlk8VzZ1MOg1Ah9oV8dd3XHKUnp7J66//xYQJv2GM4aWXenP//RdRurSf06EppUoon0pEv22LBrRZn7xkZGTy4Yf/cNlldXnrrX7UqaMNwiqlnOVTtw17j1vN+lzSuIrDkRQvMTGnGDduIQkJqQQE+PPHH6OYM2eYJiGlVLHgU4lo9/EkwoLLUDVE3x8Cq4HSL75YT5MmU3nllT9ZvHgvAJUrB2llBKVUseFTRXNLth+jcXVtfgZg+/YTjBkzj19+2UOnTjWZP/8G2rSp7nRYSil1Dp9JRAdOJhOfkk7zcG3oFOC++35m1apDvP12f26/vb02UKqUKrZ8JhEtsxs67dui5F71L1y4iyZNwqhVqzzvvHMlAQH+VK8e7HRYSimVJ49eJotIXxHZJiI7ReSRHMY/ICKbRWS9iPwiIrXPd13Ld50AoFl46AVE7J2OHEnk+utncfnln/PSS38AULt2BU1CSimv4LFEJCJ+wFSgH9AMGC4izbJNtgboYIxpBcwEJp3v+o4npAIQGlj6fBfhdTIzDe++u4omTaYwa9YWJky4hMmTL3c6LKWUKhBP3hF1AnYaY3YbY04DM4CrXScwxiw2xiTbH/8CIs53ZWsOxNCrSdXzDtYbTZy4lDvvnEf79uGsXz+ap57qSWCgz5S2KqVKCE+etWoCB1w+RwGd85j+VuCnnEaIyO3A7QCRkZHnjF+9L4aUtEwuqlf5vIP1FgkJqRw/nkzduhUZPboDdetWZPjwFlodWynltTx5R5TTmdHkOKHIDUAH4OWcxhtj3jfGdDDGdKhS5dyXVXcfs9pJ6+zD7csZY5g9ewvNmr3NddfNxBhD5cpBXH99S01CSimv5slEFAW4trUTARzKPpGI9AYeB64yxqSez4pS062O26r7aEd4+/bFctVVM7jmmq+pVKksb77ZT5OPUspneLJobiXQUETqAgeBYcD1rhOISFvgPaCvMSb6fFe0aMtRAALL+F7DnX/+eYDevT8DYPLkPtx770X4++s7QUop3+GxRGSMSReRu4H5gB/wkTFmk4g8A6wyxszBKooLBr6xr/D3G2OuKui6ytotR/tSjbn4+FRCQwNo164Go0a14aGHuhIZqS/rKqV8j0erWBljfgR+zDZsvMvfvQtjPccTU2lZ0zdO0idOJPPII4tYsGA3mzaNITi4DG+91d/psJRSymO8vozHGMPKvTEEeHlxlTGGTz9dR5MmU/n447Vcd11z9DGQUqok8PqXTs60qBBZOcjhSM5fXFwKgwZ9xW+/7aVLlwjefXcArVpVczospZQqEl6fiD79cy8AT13V3NE4zocxBhEhNDSAsLAg3n9/ALfe2k6761ZKlSjeXZ4FxJ9KJyTQ3+sqKsyfv5N27d4nKioeEeGbb67l//6vvSYhpVSJ4/WJaP/JZGp7UbHc4cMJDBs2k759vyA5OY3o6CSnQ1JKKUd5ddGcMYaj8Sk0reEdLW5PnbqCxx77ldTUdJ5+uifjxnUlIMCrvwKllLpgXn0W3Hw4nvRMQ6sI76i6vXr1YTp3rsnUqf1p2ND328VTSil3eHUi+mWL1RhD57rFs425+PhUxo9fzMiRrWjfPpy3376SgAA/bZ5HKaVceHUi2hFtNXbaoU7xSkTGGGbN2sK99/7M4cMJREaWp337cO2iQSmlcuDVZ8Y9xxOpEFQav2JU02zPnhjuvvsnfvxxB23aVOfbb4fSufN5d7OklFI+z2sTUXpGJjuOJnJNu+J1kv/iiw0sWbKP1167grvv7qQNlCqlVD68NhElpKSTmp5J/SrlnA6FpUv3kZqaQe/e9XjooYu5+eY2RER4R00+pZRymtderqekZwBQzsHqz8ePJzNq1Pf06DGNZ575HYCAAH9NQkopVQBee0d0NN7qQ8+Jp0PGGKZNW8tDDy0kLi6VceO68uSTPRyIRBV3aWlpREVFkZKS4nQoqoQIDAwkIiKC0qW9p7UZr01E248mAM40dvrjjzsYNWoOXbvW4t13B9CiRdUij0F5h6ioKEJCQqhTp45W21ceZ4zhxIkTREVFUbduXafDcZvXFs1tORwPQP0qwUWyvuTkNP74Yz8A/fs35Pvvh7FkyS2ahFSeUlJSqFy5siYhVSREhMqVK3vdHbjXJqL1UXEAVC5XxuPr+umnHbRo8Tb9+n1BbGwKIsJVVzXWBkqVWzQJqaLkjceb1yYi/1JC2dJ++Pt5bhMOHozn2mu/oX//LwkI8OeHH4ZToUKgx9anlFIlkdcmovRMQ7vaFTy2/OjoJJo1e5u5c7fz3HOXsm7daC65pI7H1qeUp/j5+dGmTRtatGjBwIEDiY2NzRq3adMmLrvsMho1akTDhg159tlnMcZkjf/pp5/o0KEDTZs2pUmTJvz3v/91YhPytGbNGm677bazhl199dV06dLlrGE333wzM2fOPGtYcPC/Rfvbt2+nf//+NGjQgKZNmzJ06FCOHj16QbGdPHmSPn360LBhQ/r06UNMTMw50yxevJg2bdpk/QsMDOS7774DYMSIETRu3JgWLVowatQo0tLSAJg7dy4TJky4oNiKFWOMV/1r3769McaY2uPmmhs+/MsUtqiouKy/33jjL7Nz54lCX4cqOTZv3ux0CKZcuXJZf994443mueeeM8YYk5ycbOrVq2fmz59vjDEmKSnJ9O3b10yZMsUYY8yGDRtMvXr1zJYtW4wxxqSlpZmpU6cWamxpaWkXvIwhQ4aYtWvXZn2OiYkxERERpkmTJmb37t1Zw2+66SbzzTffnDXvmX1z6tQp06BBAzNnzpyscb/++qvZsGHDBcX20EMPmYkTJxpjjJk4caJ5+OGH85z+xIkTpmLFiiYpKckYY8y8efNMZmamyczMNMOGDTNvv/22McaYzMxM06ZNm6zpssvpuANWmWJwDs/pn1fWmku13yFKSk0vtGXGxaXwxBO/8t57q/nrr9to164G99zTgp2FggAAEK1JREFUudCWr9TTP2xi86H4Ql1ms/BQJgx0v3fiLl26sH79egC+/PJLunbtyuWXXw5AUFAQU6ZMoWfPntx1111MmjSJxx9/nCZNmgDg7+/PmDFjzllmYmIiY8eOZdWqVYgIEyZMYPDgwQQHB5OYaLUHOXPmTObOncu0adO4+eabqVSpEmvWrKFNmzbMnj2btWvXUqGCVcLRoEED/vjjD0qVKsXo0aPZv9+qJPT666/TtWvXs9adkJDA+vXrad26ddawWbNmMXDgQKpVq8aMGTN49NFH890vX375JV26dGHgwIFZwy699FK392tuvv/+e3777TcAbrrpJnr27MlLL72U6/QzZ86kX79+BAVZtYH79++fNa5Tp05ERUUB1nOgnj17MnfuXIYOHXrBcTrNKxPRicTTAPRqWu2Cl2WM4ZtvNnPffT9z5Egid9/difr1K17wcpUqbjIyMvjll1+49dZbAatYrn379mdNU79+fRITE4mPj2fjxo08+OCD+S732WefpXz58mzYsAEgx+Kn7LZv386iRYvw8/MjMzOT2bNnc8stt/D3339Tp04dqlWrxvXXX8/9999Pt27d2L9/P1dccQVbtmw5azmrVq2iRYsWZw2bPn06EyZMoFq1agwZMsStRLRx48Zz9kVOEhIS6N69e47jvvzyS5o1a3bWsKNHj1KjRg0AatSoQXR0dJ7LnzFjBg/8f3t3HxxVfS5w/PvIWwIEqrxdKSp0pJgXQ4TQggUUUbTKhcKoMbxUHK0DiNSiONzBmStoLW0FNIIFLtebVNsQdCowoFDgxoLIi+EaCQJNU7ryYm5MMRdTCiaQ5/5xTjabsCEbZffsJs9nZmd2z8ueZ3+z2Se/3znn+c2Zc9Hy6upqXn/9dV5++WX/svT0dHbu3GmJyCu7Sv4O8I3nIVJVJk5cy7p1Rxg06Go2bMgkPb335QjRmIs0p+dyOZ09e5a0tDR8Ph+DBw/mjjvuAJzvf2NXWDXnyqtt27axZs0a/+srr2z6H7n77ruPNm3aAJCRkcHChQt56KGHWLNmDRkZGf73PXTokH+fL7/8ksrKShISEvzLSktL6dGjh/91WVkZJSUlDB8+HBGhbdu2HDx4kJSUlKCfqblXmCUkJFBYWNisfUJVWlpKUVERd95550XrZs6cyciRI+slwZ49e/LZZ5+FJZZIi8mLFWpvZh3wLwlNbBlcdbUztCciDB9+DVlZd7Fv3yOWhEyLFB8fT2FhIZ9++ilVVVUsX74cgOTkZAoKCupte/ToUTp37kxCQgLJycns37+/yfdvLKEFLmt4X0unTnU1IocNG0ZJSQnl5eWsW7eOiRMnAlBTU8Pu3bspLCyksLCQkydP1ktCtZ8t8L3z8vKoqKigX79+9O3bF5/P50+S3bp1q9db++KLL+jevbu/LUL5rJWVlfUuLAh8BCbNWr169aK0tBRwEk3Pno3fd7h27VomTJhwUUWEBQsWUF5ezpIlS+otP3fuHPHx8U3GHAtiMhHV3kPUo3OHZu/73ns+UlNXsH79EQCefPJmHn/8+7QJ42XgxkSDrl27kpWVxYsvvkh1dTWTJ0/m/fffZ9u2bYDTc5o9ezZPP/00AHPnzuWFF16guLgYcBJDwx9DgDFjxrBs2TL/69of+169enH48GH/0FtjRIQJEyYwZ84cEhMT6datW9D3DdYTSUxMpKSkxP86NzeXzZs34/P58Pl87N+/35+Ibr31VvLy8qiqcob2s7Oz/eeBJk2axAcffMCmTZv877V582b/cGOt2h5RsEfDYTmAcePGkZOTA0BOTg7jx49vtB1yc3PJzMyst2z16tVs2bKF3Nxcrrii/m9UcXHxRcOSsSomf32LTp7m6q5xzepWl5ef4cEH1zFqVA5ffXWehITmJzFjYt1NN93EwIEDWbNmDfHx8axfv57nn3+eAQMGcOONNzJkyBBmzZoFQGpqKi+99BKZmZkkJiaSkpLi/+8+0DPPPENFRQUpKSkMHDiQ/Px8ABYtWsTYsWO57bbb/OdJGpORkcEbb7zhH5YDyMrKoqCggNTUVJKSklixYsVF+91www2cPn2ayspKfD4fx44dY+jQof71/fr1o0uXLuzdu5exY8cyYsQIBg8eTFpaGrt27fJfOBAfH8/GjRt55ZVX6N+/P0lJSWRnZ1+yBxOKefPmsXXrVvr378/WrVuZN28e4JzbCrzk3Ofzcfz4cW655ZZ6+0+fPp2ysjKGDRtGWloaCxcu9K/Lz8/nnnvu+UbxRQvRgHsGYsHg9HQ9dfsC7kzuxcqp6SHtk5tbxGOPvcM//lHF3Lk3M3/+SDp2jJ2CgCZ2HT58mMTERK/DaNGWLl1KQkLCRfcStWRlZWVMmjSJ7du3B10f7HsnIvtVNbQfzQiLuR5RTY2TOAf0Cv380PnzNaSk9KSwcDo///loS0LGtCAzZsygQ4fWNcJx7NgxFi9e7HUYl03MXTV33k1Evb/V+Em6M2eqeO65HVx7bVdmzhzClCmpTJmSGpM1mIwxlxYXF8fUqVO9DiOihgwZ4nUIl1XM9YguuIkorl2boOs3biwmOflVfvnLXRQXnwKck6GWhIxXYm3428S2WPy+xVyP6PTZatoBfa6s3yM6ceJLZs9+l7ffPkJSUg927JjGiBHXeROkMa64uDhOnTplU0GYiFB3PqK4uNgqzhxziaj6Qg3tgNQ+9QueHj1awZYtf+UXvxjNnDnDaN8+eI/JmEjq06cPJ06coLy83OtQTCtRO0NrLIm5RHS26gKj+l1F+7ZXsG/fSXbvPs5PfzqUkSOv49ixJ+jmwYytxjSmXbt2MTVTpjFeCOs5IhG5S0T+LCIlIjIvyPoOIpLnrt8rIn2bes+qCzVc360TM2duYujQ1SxZsoczZ5wb1CwJGWNM7AlbIhKRNsBy4IdAEpApIg1vPX4YqFDV64GlQONlaQP819K9rFy5n9mzv09R0Qw6RWCWVmOMMeERzqG57wElqnoUQETWAOOBwIJM44Fn3edvActERLSJyz56V8HGD3/CoEGXvlvbGGNM9AtnIvo2cDzg9Qmg4QQ//m1U9byInAa6AX8P3EhEHgUedV9+tf9/Hz0YQsX21qA7DdqqFbO2qGNtUcfaos4ArwNoTDgTUbBrVRv2dELZBlVdBawCEJGCaC1TEWnWFnWsLepYW9SxtqgjIgVNb+WNcF6scAK4JuB1H6Dh5Bn+bUSkLdAV+CKMMRljjIky4UxEHwL9RaSfiLQHHgA2NNhmA/Cg+/xe4L+bOj9kjDGmZQnb0Jx7zmcWsAVoA7ymqp+IyEKgQFU3AP8JvC4iJTg9oQdCeOtV4Yo5Bllb1LG2qGNtUcfaok7UtkXMTQNhjDGmZYm5oqfGGGNaFktExhhjPBW1iSgc5YFiVQhtMUdEDonIARHZLiIttux4U20RsN29IqIi0mIv3Q2lLUTkfve78YmI/D7SMUZKCH8j14pIvoh85P6d3O1FnOEmIq+JyOcicrCR9SIiWW47HRCRQZGOMShVjboHzsUNfwW+A7QHPgaSGmwzE1jhPn8AyPM6bg/bYhTQ0X0+ozW3hbtdArAD2AOkex23h9+L/sBHwJXu655ex+1hW6wCZrjPkwCf13GHqS1GAoOAg42svxt4F+cezqHAXq9jVtWo7RH5ywOpahVQWx4o0Hggx33+FjBaWuaEL022harmq+o/3Zd7cO7ZaolC+V4APAf8CjgXyeAiLJS2+AmwXFUrAFT18wjHGCmhtIUCXdznXbn4nsYWQVV3cOl7MccDv1XHHuBbIuJ5rbRoTUTBygN9u7FtVPU8UFseqKUJpS0CPYzzH09L1GRbiMhNwDWqujGSgXkglO/Fd4HvisguEdkjIndFLLrICqUtngWmiMgJ4B3g8ciEFnWa+3sSEdE6H9FlKw/UAoT8OUVkCpAO3BLWiLxzybYQkStwqrhPi1RAHgrle9EWZ3juVpxe8k4RSVHV/wtzbJEWSltkAtmqulhEhuHcv5iiqjXhDy+qROXvZrT2iKw8UJ1Q2gIRuR2YD4xT1a8iFFukNdUWCUAK8J6I+HDGwDe00AsWQv0bWa+q1ar6N+DPOImppQmlLR4G1gKo6m4gDqcgamsT0u9JpEVrIrLyQHWabAt3OGolThJqqecBoIm2UNXTqtpdVfuqal+c82XjVDVqiz1+A6H8jazDuZAFEemOM1R3NKJRRkYobXEMGA0gIok4iag1zt++Afixe/XcUOC0qpZ6HVRUDs1p+MoDxZwQ2+LXQGfgTfd6jWOqOs6zoMMkxLZoFUJsiy3AGBE5BFwA5qrqKe+iDo8Q2+JJ4D9E5Gc4Q1HTWuI/riKSizMU2909H/bvQDsAVV2Bc37sbqAE+CfwkDeR1mclfowxxngqWofmjDHGtBKWiIwxxnjKEpExxhhPWSIyxhjjKUtExhhjPGWJyEQdEbkgIoUBj76X2LZvY5WGm3nM99zqzR+7JXEGfI33mC4iP3afTxOR3gHrVotI0mWO80MRSQthnydEpOM3PbYx4WKJyESjs6qaFvDwRei4k1V1IE4x3V83d2dVXaGqv3VfTgN6B6x7RFUPXZYo6+J8ldDifAKwRGSiliUiExPcns9OEfkf93FzkG2SRWSf24s6ICL93eVTApavFJE2TRxuB3C9u+9odw6bIneulw7u8kVSNwfUi+6yZ0XkKRG5F6fm3+/cY8a7PZl0EZkhIr8KiHmaiLzyNePcTUDBShH5jYgUiDP30AJ32WychJgvIvnusjEistttxzdFpHMTxzEmrCwRmWgUHzAs97a77HPgDlUdBGQAWUH2mw68rKppOInghFvOJQP4gbv8AjC5ieP/K1AkInFANpChqjfiVCKZISJXAROAZFVNBZ4P3FlV3wIKcHouaap6NmD1W8DEgNcZQN7XjPMunDI+tearajqQCtwiIqmqmoVTS2yUqo5yS/08A9zutmUBMKeJ4xgTVlFZ4se0emfdH+NA7YBl7jmRCzh10xraDcwXkT7AH1T1LyIyGhgMfOiWP4rHSWrB/E5EzgI+nGkCBgB/U9Vid30O8BiwDGeuo9UisgkIecoJVS0XkaNuna+/uMfY5b5vc+LshFPOJnCGzftF5FGcv+urcSaAO9Bg36Hu8l3ucdrjtJsxnrFEZGLFz4AyYCBOT/6iSe9U9fcishe4B9giIo/glL3PUdV/C+EYkwMLpIpI0Pmt3Npm38MpovkAMAu4rRmfJQ+4HzgCvK2qKk5WCDlOnFlIFwHLgYki0g94ChiiqhUiko1T2LMhAbaqamYz4jUmrGxozsSKrkCpO3/MVJzeQD0i8h3gqDsctQFniGo7cK+I9HS3uUpErgvxmEeAviJyvft6KvAn95xKV1V9B+dCgGBXrlXiTEsRzB+AH+HMkZPnLmtWnKpajTPENtQd1usCnAFOi0gv4IeNxLIH+EHtZxKRjiISrHdpTMRYIjKx4lXgQRHZgzMsdybINhnAQREpBG7AmRL5EM4P9h9F5ACwFWfYqkmqeg6nOvGbIlIE1AArcH7UN7rv9yec3lpD2cCK2osVGrxvBXAIuE5V97nLmh2ne+5pMfCUqn4MfAR8AryGM9xXaxXwrojkq2o5zhV9ue5x9uC0lTGeserbxhhjPGU9ImOMMZ6yRGSMMcZTloiMMcZ4yhKRMcYYT1kiMsYY4ylLRMYYYzxlicgYY4yn/h+f8Vryc7oj6AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7220704605012441" + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#plot roc for svm\n", + "get_roc(clf_original, y_test, X_test, \"SVM original data RBF kernel\")" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM original data RBF kernel identified 733\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06391338
11287733
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6391 338\n", + "1 1287 733" + ] + }, + "execution_count": 78, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_original.predict(X_test), \"SVM original data RBF kernel\")" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.36 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.66 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_original.predict(X_test)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Evidently, SVM model fit with no tuning or feature reduction with RBF kernal shows low performance. Now, we will fit SVM model with reduced standardized features and access accuracy" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "note that the default values of gamma = 1/13 and c= 1" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=1, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma=0.07692307692307693,\n", + " kernel='rbf', max_iter=-1, probability=True, random_state=None,\n", + " shrinking=True, tol=0.001, verbose=False)" + ] + }, + "execution_count": 80, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#train svm model with feature reduction and no parameter tuning\n", + "clf_reduced = svm.SVC(kernel = 'rbf', probability = True, gamma = 1/13, C = 1)\n", + "clf_reduced.fit(X_train_pca, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.16374485135727915\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAEWCAYAAACUr7U+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdZ3QV1deA8WeTDiShQyAJIQiGXgwdxQKKCIooioqKyquoiIq9K4pdLH9RsFfAhoKIIqA0pUqTLp3QW0II6Tnvh5mEm5AKSSb33v1bi0Xu1D11zzlzZkaMMSillFLKM1RyOgCllFJKlR5N7EoppZQH0cSulFJKeRBN7EoppZQH0cSulFJKeRBN7EoppZQHqZCJXURuEJHfnY7DaSISKSLHRcSnHOcZJSJGRHzLa55lSUTWisj5pzHeae2DIhIkIj+LSIKIfFfS8T2JiMwRkaHlPM/PROSFQvq/ICKHRGRfecblrUTkcRH5yOk4zpSIbBeRnhUgjiEisqCo4YpM7PYCJdsJZp994FQtnTDzZ4z52hhzcVnOoyLKu/MYY3YaY6oaYzKdjMsp9gXGWWcyDWNMC2PMnCLmc8rFzBnsg1cDdYGaxpiBpzF+fvE9LiLb7GMwTkS+sbuPF5Ev8hm+tYikikgNEXnWXrYReYa5z+7+bGnE6A5EJAJ4AGhujKl3htM6X0TiSieyisPeX74qrekZY140xpTJxZ29/ybZx8VuERnjWgiyLyxT7P4JIjJPRFq59H9WRNLt/tn/Hi6LWMtbcUvs/YwxVYG2QDvgsbILqew4WQr1lBJwSXjp+m4IbDLGZJR0xPxiFpGbgRuBnvYxGAvMtnt/BgwQkSp5RrsJmGaMOWL/3gTcnM8wm84kNjfUEDhsjDngdCAesj4rgjb2cdEDuBa4NU//4Xb/msAc4Ms8/b+xC0/Z/14ty2DLa7uXqCreGLMPmIGV4AEQkQAReV1EdorIfhEZJyJBLv2vEJGVInJMRLaISG+7e6iIfCwie+2rrReyr7Zcqxvs6b3uGoeITBGRkfbf9UXkBxE5aJdqRrgM96yIfC8iX4nIMWBI3mWy4/jCHn+HiDwpIpVc4vhLRP5nX/FtEJGL8oxb2DL8JSJvisgR4FkRaSwif4jIYbs68GsRqWYP/yUQCfycfeWYtyRpX4E+b083UUR+F5FaLvHcZC/DYRF5SgqpPhKryvgNe/gEEVngut2AG+xtekhEnnAZr6OILBSReHu53xURf5f+RkTuFpH/gP/sbm+LyC57H/hHRM51Gd5HrBLpFnuZ/hGRCBGZZw+yyl4f19rD97X3p3gR+VtEWrtMa7uIPCIiq4EkEfF1XQd27MvsOPaLyBh71Ox5xdvz6iJ5qrxEpIWIzBSRI/a4j+ezTp8Dngautadzm4hUsvepHSJywN7XQu3hs7fvbSKyE/gjn03VAZhhjNkC1jFojPnA/nshsBu4ynV9AtcDn7tMYylQWURaZC8LEGR3z1d++6/d/VYRWS8iR0Vkhog0dBmnl32MJIjIu4C49MtVEsxn364hIp+KyB572j+5DFvYNm8nIsvtfecbILCA5ekJzATq29vmM7t7Z3ua8SKySlxu24jILfayJorIVhG5w+5eBfjVZVrHxToP5boNIHlK9QXsn4WdvwraX/Mu2/li1eQ8YO9je0XkFpf+BZ7j8kynN/A4J/ffVS5x93QZLmdbumzHmyX/80VJhg0Skc/t7b9erHNgsWpFjDGbgb9wyU15+mcAk4DmxZleYUQkxt5Wg+zfJcpBdrdv7W2SKNbtwliXcR6Vk+fDdSJyZYmDNMYU+g/YjlVaAAgH/gXedun/FjAVqAEEAz8DL9n9OgIJQC+si4gGQIzd7ydgPFAFqAMsAe6w+w0BFth/nwfsAsT+XR1IBurb0/wH62TqD0QDW4FL7GGfBdKB/vawQfks3xfAFDv2KKxSzG0ucWQA9wN+WFeECUCNYi5DBnAP4It1Ij3LXhcBQG2shPJWfuva/h0FGMDX/j0H2AI0tac3B3jZ7tccOA50t9fF6/ay9yxgu461x28A+ABd7biy5/mhPY82QCrQzB7vHKCzvUxRwHrgPpfpGqwTaI3s9Q0Mxrpi9sWqCt0HBNr9HsLap87GSgRtsKqxs6d1lsu02wMHgE52zDfb6yzAZf2tBCJc5p2zToGFwI3231WBzvmt53z2wWBgrx17oP27UwHr9VngK5fftwKbsfbNqsBk4Ms88/0Cax/Kb/8cDByx11Ms4JOn/xPALJfflwAHAT/XeLBO2K/Y3V7FqnX7Cni2gOUYwqn7b397WZrZ3Z4E/raHrwUcw7oV4Yd1zGQAQwtYL7nWOfAL8A3W8e0H9Chqm2Pt5zs4eXxejbXPv1DAMp0PxLn8bgAcBvpgnR962b9r2/0vAxpj7Zc9gBNA+/ymZXf7zHXe+cxvOy77J0Wfv/LdXwtYrgxglL0e+tixVi/qHFfU/lvAeSlnGIo+X5Rk2JeBufY+EA6szruO88SVc34AYrCO0ftd+s/h5P7nD4wG5hW2rIXMazvQE2t/3An0tbuXOAfZ3VLs7eQDvAQscpnXQE7mt2uBJCAs73mp0HiLuUDHgUR7Rc4Gqtn9xJ5pY5fhuwDb7L/HA2/mM8269gYNcul2HfBnPidVsVfkefbv/wP+sP/uBOzMM+3HgE9dVuq8QpbNx46juUu3O4A5LnHswb6osLstwaoaLc4y7Cxo3vYw/YEVhRxAUZya2J906X8X8Jv999PARJd+lYE08kns9g6TjFWNlbdf9jzD8yzzoAKW4T7gxzwH24VFLPfR7HkDG4Erijpw7d/vA8/nGWYjJ5PAduDW/A5I++95wHNArQKWuaDEfp3rdipi2Z4ldwKbDdzl8vtsrAPd12W+0UVM8wZgFtaxdhh41KVfpD29cPv31+S+8H4WK4FHYh1Hfvb/ERSd2PMeW7/ikhDs/egEVhX3TeQ+OQkQRzESOxAGZGEnojzzLHCbY1305z0+/6b4if0R7Issl24zgJsLGP8n4N78pmV3+4yiE/utLr+LOn/lu78WsFzJ5N5/D2BdgBd6jitq/817DOUdhiLOFyUcNich2r+H5l3HeeIyWBeTSfbfE7Ev8u3+c7D2z3isc2ECcFGe5Uiz+2f/q1/AvLbb2yIOuKAE2/BZ8uQgu5vrxXhzILmQ5VyJfY6kmIm9uFXx/Y0xwVg7UAzW1TlYpc7KwD92VVY88JvdHayTx5Z8ptcQ6wSz12W88Vil3lyMtTSTsE6uYFUzfu0ynfrZ07Cn8zhW0s22q5DlqsXJq/5sO7Cu5LPttmNw7V+/mMuQa94iUkdEJolVbX8M68Rai5Jxbc17AutKHjumnPkZY05gJYH81MIqeea3bQqdj4g0FZFpYjWkPAa8yKnLkHe5H7Cr1hLs9RTqMk5B+0h+GgIP5NneEVjLnu+887gNq7Zjg4gsFZG+xZxvSWLMqz6n7l++FH8fxVgN+XoC1YBhwCgRucTutxMrAQwWq1Frf3JXw+My3Gas7fWfMabQeRYQV0PgbZd1fwQrgTfg1P3PFLVcLiKAI8aYo/n0K2yb1yf/47O4GgID80y7O9aFBiJyqYgsEuv2SzxWCaukx2teruukqPNXSfbXwyZ3u47sY7Y457jSUNB5qSTD5tqHKN7+094e/1qsJJu3vckIY0w1rPNdX+B7cbmVA3xrjKnm8m9PIfMahlVD9adLt9PNQXnXQaCcvC11k5y89RQPtKSE+11J77HPxboqzb7nfQjrSrGFy4oJNVZjBbAWqHE+k9qFdRVZy2W8EGNMiwJmPRG4Wqz7eZ2AH1ymsy3Phgk2xvRxDbuQRTqEVdpp6NItEuu+ZbYGIiJ5+u8p5jLknfdLdrfWxpgQrGpWKWT4ktiLVX0FWPersKq/83MIqyoov21TlPeBDUATexkeJ/cygMtyiHU//RHgGqwSWTWsK+fscQraR/KzCxidZ3tXNsZMzG/eeRlj/jPGXId18fUK1kFepbBxTiPGvPZw6v6VAex3Da04EzLGpBtjvsOqomzp0utzrBLzVVjHw/ICJvEF1u2EU1rSFzTLPL93Yd1qcl3/QcaYv7H2v4jsAe1jJsJl3CSsQkA211bpu4AaYrc3yWeeBW3zveR/fBbXLqwSu+u0qxhjXhaRAKzzzOtAXXu/nc7J/Ta/bVbYMmZzHa/Q81ch+2tJFOccV1B82YqzXKUh1zmM3PtPgYzlW6xbF08XMEyWMWY+1sXt6T5xNQyIFJE3XbqdaQ7Kxc5xHwLDsW5JVgPWcOo5tlCn8xz7W0AvEWlrjMmyg3hTROrYgTXILk0AHwO3iMhFYjUiaiAiMcaYvcDvwBsiEmL3aywiPfKboTFmBdZ9w4+wGhLF272WAMfEapASJFZDrJYi0qE4C2Ksx8i+BUaLSLC9UkdilaSz1QFGiIifiAzEur84vaTLYAvGuq0RLyINsO6butqPdY/mdHwP9BORrmI1ZnuOAnYGe7t9AoyxG374iNVgLKAY8wnGqv46LiIxwJ3FGD4Da/v5isjTQIhL/4+A50WkiVhai0j2BUne9fEhMExEOtnDVhGRy0QkuBhxIyKDRaS2vfzZ+1CmHVsWBa/7aUA9sR4RC7D3lU7FmSfWRen9ItLILlG/iNUSt1it5sVqxHaZPc9KInIp0AJY7DLYD1gnwefIp7Tu4husk9q3xYw9r3HAY3KyEV6ofUyAdY+8hYgMsEseI8idAFYC54n1boZQXJ6ssY+lX4H3RKS6faydZ/cubJsvxNq3RojVEG0AVrue4voK65i5xD4GAsVqiBaOVcoNwNo3Muz17poQ9gM17WVxXcY+YjUErId1m6owhZ6/Ctlfi62Y5zhX+4Eoyd24biUwyN4usVhtGcrCt1j7V3X7/Di8hOO/DNxur/tTiEgXrGrvtacZXyLQG2s/ftnudkY5KB/ZBY2Ddsy3kPsivlhKnNiNMQexrvifsjs9gnUVtEisqtlZWPcRMcYsAW4B3sQqpc3l5JXjTVgHzzqse67fY1eBFWAiVuOFCS6xZAL9sFpCbsO6Ov0Iq6q3uO7BuiLdCiywp/+JS//FQBN72qOBq40x2VXcJV2G57CqjhKwToST8/R/CXhSrCqYB0uwDBhj1trLMgnryjcR6z5bagGjPIjVaG0pVpXqKxRvf3gQ63ZIItZJ95sihp+BddLehFUFmELuqqkxWAf071gXDB9jNTAB617U5/b6uMYYswyrjcW7WOt7M/k86VCI3sBaETkOvI11by/Fvm0xGvjLnldn15GMMYlYDav6YVWh/QdcUMx5foL1iM08rH00BWs7FdcxrFqRnVgn91eBO40xOS32jTFJnEzuX+c3EXu4ZGPMLGNMcgnm7zr+j1j7yST7WF8DXGr3O4TV6OdlrFtATbBaKWePOxNrX1mN1dhoWp7J34hVstyAtd/eZ49X4DY3xqQBA+zfR7GqY/MeU4Utzy7gCqz1exBrv3wIqGRv8xFY++ZRrH1+qsu4G7DOSVvtfaY+1nZehXU/9neKODaKcf7Kd38t7vK5KOoc5yr7pUqHRSS75ucprBqro1jnsAn5jVgKRmHdw96GlUe+p+Dz1ymMMf9i5RjXAtO7Yj+5gLV9njTG/Hq6AdqFyl7ApSLyfCnlINfprwPewLpo3Q+0wuU4Kq7sluYqHyIyBKvxT3enYykpu3QYj1Vlvs3peJRSqiRE5E6si5nCakFVPirkK2XV6RGRfiJSWaz7cK9jlci3OxuVUkoVTUTCRKSbfcvpbKz2ID86HZc70sTuWa7Aaqy1B6sqdJDRKhmllHvwx3qyKBHrZU1TgPccjchNaVW8Ukop5UG0xK6UUkp5EP0QQSmrVauWiYqKcjoMpZRyK//8888hY0ztoodURdHEXsqioqJYtmyZ02EopZRbEZGSvDVQFUKr4pVSSikPooldKaWU8iCa2JVSSikPooldKaWU8iCa2JVSSikPooldKaWU8iBem9hF5BMROSAiawroLyLyjohsFpHVItK+vGNUSimlSsprEzvwGdZnEQtyKdb71psAtwPvl0NMSinldQ4cO52v0aqCeO0Laowx80QkqpBBrgC+sD+iskhEqolImDFmb7kEqJRSHu7EiXSuHDWbjWQ6HYpH8drEXgwNgF0uv+PsbqckdhG5HatUT2RkZLkEp5RS7uzLH9by1NLtTofhkby5Kr4okk+3fD+FZ4z5wBgTa4yJrV1bX3WslFKFeXTCipykXivIj3+e7OlsQB5GS+wFiwMiXH6HY33nXCmlVAllZGQxZ842fj50jCmrrVPp/Rc14d5eTR2OzPNoYi/YVGC4iEwCOgEJen9dKaVKbsmS3dxx9y8c6loXnyBfAnwrMWtkDyJqVHY6NI/ktYldRCYC5wO1RCQOeAbwAzDGjAOmA32AzcAJ4BZnIlVKKfe0KS6eW9/8ix3p6fhc1AAfu/uqZy4m0M+n0HHV6fPaxG6Mua6I/ga4u5zCUUopj2CMYcHmQzw++V92HU0GPwjw9SMmLIQB54QzuHND/H21eVdZ8trErpRSqnQYY/h4wTb+2XGUX9fsy+ke5FOJB7o2YuhlMQ5G5300sSullDptXy7awdNT1mDsZ4ZMZhZ1KwfwzT3diKpZGZH8HjBSZUkTu1JKqRLZuC+Rt2ZtylU6l8Q0dnzwL9dc1ZwxY86nfq0qDkbo3TSxK6WUKtKJtAx+WL6bsX9sZp/LK2DDs4SF760gqm4w03++nt69z3IwSgWa2JVSSuUjK8uwdPsRJizZyfZDSayKS8jp17lRDYb1aEyPs2szefJ6zvcP5IknziUoyM/BiFU2TexKKaVyfL14B5//vZ1N+4/n6t6nVT2ah4XQoVoVRt7zG4uOZnL+o3W46qrmXHVVc4eiVfnRxK6UUorktEzOffVPDh1PBeCKtvWpXy2IPi3DaFE/hOTkdJ57bi73jVlI9epBREaGOhyxKogmdqWU8lJr9yQwbu5Wth06zprdxwDw8xFmjexBw5onG7/9+ec2hgyZws6dCQwd2o6XX+5JzZr61riKShO7Ukp5kaTUDF74ZR0Tl+zK1b1nszp0jq7Jbd0bnfKIWuXKfoSGBjB//i10765fsKzoNLErpZSXMMbQ8tkZOc+cX9Y6jNu6N6JdRLVcyTwjI4t33lnM7t3HeOONS+jUKZyVK4dRqZI+k+4ONLErpZQH23YoiQ17jzHvv0N8t2wXxkCX6JpMvL1zvsMvWhTHsGHTWLVqP/36NSUjIwtf30qa1N2IJnallPIwGZlZrIqL55Ef/mXzgdyt2/u1qc/rA1ufMk58fAqPPTaL8eP/oX79YCZPvob+/WP0zXFuSBO7Ukp5CGMMXy3eyaif15KeadW3X9YqjCvbNeDsesGEhQbi65P/B1ji41P4+ut/uf/+zjz77PkEBweUZ+iqFGliV0opN7f/WAqf/b2d75bt4tDxNACubNeAQR0i6BRds8DxNm06zJdfrmLUqAuIiqrG9u33UaNGUHmFrcqIJnallHJDx1Mz+Hj+NrYcPM7UVXtyut/WvRF3nd+YmlULLnGnpGTw8ssLeOmlBQQF+XLLLe2Ijq6uSd1DaGJXSik3EX8ijQ/nb2VvQgqTl+/O6X5Ow+pcGFOH28+Lxq+AqvZss2Zt5a67fuG//45w/fWteOONi6lXr2pZh67KkSZ2pZSq4IwxPPHTGiYs3pnT7ey6wfRtHcaw8xsXmcyzJSenc9NNP1Klij+//z6YXr0al1XIykGa2JVSqgJKTstkysrdzFy3n9kbDuR0v7VbIx7vE1NgI7i8srIMEyb8y7XXtiAoyI8ZMwbTpElNAgP19O+pdMsqpVQFkpiSzpiZm/j0r+053YIDfOnepBZvXtuWQD+fYk9r5cp9DBs2jcWLrWr7wYNb06pV3dIOWVUwmtiVUsphSakZfLloB7+t2cfKXfEA+FYShl94FiMubFLil8McP57GM8/8ydtvL6ZGjSC+/PJKbrihVVmEriogTexKKeWQw8dTeej71fzhUtXeJbomV7ZrwID2DYpd3Z7Xddf9wLRpm7jjjnN46aWLqF5dW7t7EzHZLw1WpSI2NtYsW7bM6TCUUhWUMYbF245w8ydLSM3IAiCmXjA3dmnIwHMi8Pc9vWS+Y0c8oaGBVKsWyPLle0lNzaBLl4jSDL1Micg/xphYp+PwBFpiV0qpcjB300EmL49jysqTz5xXDfDlpQGt6NMqDJ/TfBd7enomb765iOeem8ttt7XjnXcupX37sNIKW7khTexKKVXKNh9IZNrqvSzaepjElAzW7jmW069Jnao0qlWFZy5vQYNqZ1ZF/tdfOxk27BfWrDnA5ZefzYMPdj3T0JUH0MSulFJnKCk1g1/X7GPWuv2sjotnT0IKAJX9fagkQseoGjQLC+amrlE0rl06L4N5//2l3HXXdCIiQvjpp2u54oqYUpmucn+a2JVS6jR9/08c3y7bxZJtR3K6BfpV4oKzazOsR+NC39N+OowxJCamERISwKWXNuHhh7vy1FM9qFrVv1Tno9ybJnallCqBQ8dTmbJyD18v3sHWg0kA1AkOoG/r+gw7P5o6wYFlMt8NGw5x552/EBjoy/Tp1xMVVY1XXulVJvNS7k0Tu1JKFUNqRiav/baRjxZsy+kWFhrIlLu7USekbJI5WK+BffHF+bzyyl9UqeLPK6/0LLN5Kc+giV0ppQqRlWV4dcZGxs3dktNtcOdInunXotjvaD9da9YcoH//SWzZcpTBg1vz+uu9qFtXP9iiCqeJXSml8khJz2TtnmO8NWsT6/Yc43CS9Y3zns3q8uFN5yByeo+mFZcxBhEhIiKE8PAQxo/vy0UXRZfpPJXn0MSulPJ6xhiS0jKZt+kg90xcQWZW7hd3Xd6mPqOvbElwoF+ZxpGZmcW4ccuYOHENf/55M6GhgcyZM6RM56k8jyZ2pZTXmrfpIC9OX8+GfYmn9Hvu8hac07A6LRuElkssy5fvZdiwaSxduoeePaOJj0+hdu0q5TJv5Vm8OrGLSG/gbcAH+MgY83Ke/pHA50A1e5hHjTHTyz1QpVSp2ZuQzDuz/+PnVXs5npoBWK3aL25Rl6Z1g7mkRT3qlmFjuLySk9N57LHZ/O9/S6hduzITJgxg0KCWZV7drzyX1yZ2EfEBxgK9gDhgqYhMNcascxnsSeBbY8z7ItIcmA5ElXuwSqkzMnPdfn5etYcdh5NYFZeQ0/2a2HDu6NG41F4aczp8fSsxb94Ohg07h9GjL6JatfK7qFCeyWsTO9AR2GyM2QogIpOAKwDXxG6AEPvvUGAPSqkKL+7oCVbtSmBvQjI/rdzNmt3WK11j6gXTJbom13WK5OLmdUv0bfPStG3bUZ55Zg5vv92b6tWDWLjwNgICvPl0rEqTN+9JDYBdLr/jgE55hnkW+F1E7gGqAPk+QCoitwO3A0RGRpZ6oEqpomVkZjF+3lZem7HxlH7NwkL44taO1A4OcCCyk9LSMhkzZiGjRs2lUiXh5pvbcNFF0ZrUVany5r0pvxtYeb9hex3wmTHmDRHpAnwpIi2NMVm5RjLmA+ADsD7bWibRKqVOkZKeyUvT1/PTyj0kJKfndI+oEcTDl8TQqkEo4dWDTvu75qVp/vwdDBv2C+vWHWTAgGa89dYlRESUT8M85V28ObHHAa4fKw7n1Kr224DeAMaYhSISCNQCDpRLhEqpfP29+RBfLNzBb2v35XRrH1mN2KgaDOvRmBpVKt6701999W+SktL4+efr6Nu3qdPhKA/mzYl9KdBERBoBu4FBwPV5htkJXAR8JiLNgEDgYLlGqZQXM8Zw9EQ6mw8c5+dVe0hMSWfFrnh2HD6RM8zAc8J5+arWp/0987JijOHzz1dx3nkNiY6uzkcf9aNqVX+qVMCLDuVZvDaxG2MyRGQ4MAPrUbZPjDFrRWQUsMwYMxV4APhQRO7HqqYfYozRqnalykBmlmHlrqOs3JXAwi2H2bQ/kZ1HTpwyXLOwEJqHhfBw77M5/+w6DkRatHXrDnLnnb8wb94OHnqoK6++qq+CVeXHaxM7gP1M+vQ83Z52+Xsd0K2841LKk2VmGY6eSCM5LZM/Nhxgy8Hj/L52P/uOpZwybNfGNYmoXpk2EdVoHR5K87AQKlWwkrmrEyfSeeGFebz22t+EhATw0Uf9uOWWdk6HpbyMVyd2pVTZMcaw9VASU1bsZu5/h6gkkJqexbq9x04ZtmqAL4M6RNAqPJQW9UOJqRfs2KNoZ+Kll+bz0ksLuPnmNrz2Wi99c5xyhCZ2pVSpMMYwZeUe9h1LYeO+RH5csTtX/3ohgbRsEEKjWlUI9POhU3QNgvx8uKhZHSr7u++paM+eRI4cSaZlyzo8+GBXevaMpkePKKfDUl7MfY8mpZRjjialkZCczq9r9jFr/X427kvMeT1rtph6wbSNqMZlrcPo1Kgm/r7OP3JWmjIzsxg7dilPPvkHMTG1WLx4KKGhgZrUleM0sSul8pWZZXh79n+s3BXPyp1H8fetRHqmyfW8eLaompVpExFK18a1uLlrFJX9fCr0vfAztWzZHu64YxrLl+/l4osb8957ffTd7qrC0MSulMpxMDGVSUt2MnbOZlLST76HKTjAl8a1qxJTL5gAPx+SUjNoH1mdapX96HZWLbe8H366Zs/eSq9eX1K3blW++eZqBg5srkldVSia2JXyYhmZWfy4Yjff/RPHmt0JnEjLBCDIz4d2kdW4NjaCgbERFe4Z8fJmjGH37kTCw0M477yGPP/8BQwf3pHQUP1gi6p4NLEr5YUOHU/lo/nbGDd3S063QL9K9Gpel76tw+jXur5HV6WXxNatR7n77umsWLGXDRuGU61aIE88cZ7TYSlVII9I7CLiD0QaYzY7HYtSFVFWlmHTgURe+XUDh5PSWG1/ujTAtxIXt6jHC1e0JLSyn8NRVixpaZm8/vrfPP/8PPz8KvHCCxcSHKxvjVMVn9sndhG5DBgD+AONRKQt8Iwx5kpnI1PKeZv2J/Lyrxv4Y0Puzxt0jq7BVe3DubxtfQJ8vef+eHEdPnyCc8/9lPXrD3H11c15661LaNAgpOgRlaoA3D6xA6OwPrf6J4AxZqWInOVsSEqVP2MMM9buY5t9riAAACAASURBVO6mg/y95TBJqZkcOp6a039w50j6tAojtmENj3v0rLSkp2fi5+dDjRpB9OjRkNdfv5g+fZo4HZZSJeIJiT3dGBOfp1Wqvs9deQVjDJOW7uK7ZbtYvjM+p3utqgG0CQ+lXmgg/drUp3N0TQejrPiysgyffrqCZ5+dy9y5Q4iOrs777/d1OiylTosnJPb1InINUMn+Utu9wCKHY1KqTBhj+HPjAVbHJbB+7zFmrN0PgL9PJeqGBNCjaW1u6daIZmFabVxca9Yc4M47f2HBgp2ce24kWVlaLlDuzRMS+3DgaSALmIz1tbbHHI1IqVI0/7+DfPbXdkRg1vrc98rDqwdxZbsG3Nezqdc/klZSxhiefPIPXn31b0JDA/jkk8sZMqStPpOu3J4nJPZLjDGPAI9kdxCRAVhJXim3MmPtPv7ZcZSVu+JZv+cYiS6vaQ3y86FjVA2qV/Hjkd4xRNSojJ+P3is/XSJCQkIqN93Umlde6UWtWpWdDkmpUiHu/nlxEVlujGmfp9s/xphznIgnNjbWLFu2zIlZKzeSmpHJxn2JJKVmsnZPAjsOn+DXNftyGruFBvlRSaBlg1C6nVWL3i3qEVVLvxR2puLijnHffb8xcmQXunaNICvL6PP6FYR93o51Og5P4LYldhG5BOgNNBCRMS69QrCq5ZWqcDbuS6Tf/xaQlnnqLtq4dhVa1A/hxQGtaFAtyIHoPFdGRhb/+99inn56DpmZWfTr15SuXSM0qSuP5LaJHTgArAFSgLUu3ROBRx2JSKl8bD+UxKSlu5i2eg9xR5Nzuj92aQytwkOp7O/L2XWDCfLX58nLwpIlu7njjmmsXLmPSy89i7Fj+9CoUXWnw1KqzLhtYjfGrABWiMjXxpgUp+NRCqx3r8cdTeb3dfvYfTSZSUt3kZpxsnTeon4Ij14aQ/ezamkjrXIyb94ODhxI4rvvBnLVVc10vSuP5wn32BsDo4HmQM4XGYwxTZ2IR++xe5+sLMOs9ft5a9Z/rNt7LKe7byWhbkggETWCGNajMec1qa1Vv+XAGMOkSWsICvKjf/8Y0tMzSU7OICQkwOnQVCH0HnvpcdsSu4vPgBeA14FLgVvQe+yqHBxJSuOBb1fy1+bDOffMa1X154ZODTmnYXXOa1rb4Qi9z+bNR7jrrl+YOXMr/fo1pX//GPz8fPDzos/KKuUJib2yMWaGiLxujNkCPCki850OSnkeYwxLtx9l++Ek/lh/gN/W7svpd3/PpvRrE0Z07aoORui9UlMzeOWVv3jxxfkEBPjy7ruXMmyYFv6Ud/KExJ4q1k2zLSIyDNgN1HE4JuVhTqRl0PzpGbm6+ftWom+rMF4f2Ear2B32++9beOaZOQwa1JIxYy4mLCzY6ZCUcownJPb7garACKx77aHArY5GpDzG2j0JTFi8kwlLdgIQFhrIeze0p0ndYKoGeMLh474OHEhiyZLd9O3blL59m7JkyVA6dGjgdFhKOc7tz0zGmMX2n4nAjQAiEu5cRMrdHT6eyntztvDxgm25ug/t3ogn+zZ3KCqVLSvL8NFHy3nkkVkYY9i1636CgwM0qStlc+vELiIdgAbAAmPMIRFpgfVq2QsBTe6q2Ob/d5B3/9jMil3xpLk8nnZdxwiu7RBJm/BQfUyqAli9ej/Dhk1j4cI4evRoyPvvX0ZwsLZ2V8qV2yZ2EXkJuApYhdVg7kesL7u9AgxzMjblHlLSM5mweCejpq3L6dYxqgYhQb4M7tyQc5vU1g+rVCB79iTSocOHhIQE8Pnn/bnxxtZ6saVUPtw2sQNXAG2MMckiUgPYY//e6HBcqoJKSs1gy8HjzFi7j68W7SQhOT2nX3CgL5Nu70yL+qEORqjys3r1flq3rkv9+sF8+ukVXHJJY2rW1A+2KFUQd07sKcaYZABjzBER2aBJXeW1+cBxFm09zITFO3O9PAYgulYVbuzSkGs7RFDZ350PBc+0c2cCI0b8ypQpG3Maxl1/fSunw1KqwnPns1m0iGR/mlWAKJffGGMGOBOWqihem7GBsX9uAaCSQNuIalzWKoy2kdXoEFXD4ehUQdLTM3nnncU888wcsrIMr7zSk7Zt6zkdllJuw50T+1V5fr/rSBSqwtl68DhPT1nLgs2HAHhjYBsuax1GoL59rMLLyjL06PEZCxfG0bdvU/73v0uJiqrmdFhKuRW3TezGmNlOx6AqhrSMLNbsSWDSkp18uywup3tMvWAm/l9nqlfxdzA6VRzHjqUSHOxPpUrCLbe05aGHutK/f4w2jlPqNLhtYlcKrAZx/d5dwNaDSQD4+QgXxdTltnMbaXW7GzDG8PXX//LAA7/zzju9ufbalvzf/53jdFhKuTWvTewi0ht4G/ABPjLGvJzPMNcAzwIGWGWMub5cg1T5Ss/M4kBiKl8v2sF7c6x76BfF1OH+Xk1pHhair3d1Exs3HuKuu6bzxx/b6NixAWefXcvpkJTyCB6T2EUkwBiTWsxhfYCxQC8gDlgqIlONMetchmkCPAZ0M8YcFRF9/7zDNh9I5MN52/hm2a5c3WMbVufjIR0cikqdjnfeWcxDD80kKMiX99+/jP/7v/b4+FRyOiylPILbJ3YR6Qh8jPWO+EgRaQMMNcbcU8hoHYHNxpit9jQmYT0Xv85lmP8DxhpjjgIYYw6URfyqcMlpmTz43Sp+W7uPzCwDWJ9Gvap9OG0iqnFx87r4akJwG8YYRISwsKpcfXVz3njjYurV0y/iKVWa3D6xA+8AfYGfAIwxq0TkgiLGaQC4FvvigE55hmkKICJ/YVXXP2uM+a1UIlZF+nPjAR76bjWHjp+shLk2NoLrO0XSJkJbSbubffuO88ADv9OqVR0efbQ7Awe2YODAFk6HpZRH8oTEXskYsyNP69nMIsbJ7yasyfPbF2gCnI/13vn5ItLSGBN/ysREbgduB4iMjCxm2Cqv7YeSmLh0J3M3HmTDvkQAu2QeyrUdIgjw1cfV3E1WlmH8+GU89thskpMzaNVK72gpVdY8IbHvsqvjjX3v/B5gUxHjxAERLr/DsV5Jm3eYRcaYdGCbiGzESvRL807MGPMB8AFAbGxs3gsEVYSJS3by2OR/c3ULDfLj3evbcW6T2g5Fpc7UmjUHGDp0KosX7+bCCxvx3nt9tIGcUuXAExL7nVjV8ZHAfmCW3a0wS4EmItII2A0MAvK2eP8JuA74TERqYVXNby3FuL3StkNJrNtzjKTUDFbFxbNg8yF2HD4BQJuIatx3URPOa6ofX/EESUlp7NiRwJdfXskNN7TSZ9KVKieekNgzjDGDSjKCMSZDRIYDM7Dun39ijFkrIqOAZcaYqXa/i0VkHVbV/kPGmMOlHbwnS0nPZMXOeNbvPcayHUdYuTOePQkppwzXtG5VPrwploY1qzgQpSotxhh++mkDK1fu47nnLqBTp3C2bbuXwEBPOM0o5T7EGPeuORaRLcBG4BtgsjEm0cl4YmNjzbJly5wMwXGpGZncM2EFv6/bn6u7v08lmtaryogLmxBTL4SQIF+qVda3wnmCHTviGT78V6ZN20TbtvVYuPA2TeiqRETkH2NMrNNxeAK3P/KMMY1FpCtWdfpzIrISmGSMmeRwaF5nX0IK3y3bxRszTzZxGHHhWXRpXIv2Datp4zcPlJ6eyZtvLuK55+YiAq+/3ot77+2Mr68+gqiUU9y+xO7K/i77W8ANxhhHsog3ltj3H0vhmvELc+6V+/tU4vpOkTxxWTP89Blzj7ZzZwLNmo3l4osb8/bbvYmM1O/Zq9OjJfbS4/YldhGpivVymUFAM2AK0NXRoLzI5gOJ9BwzD4DawQE8f0ULLmlRTxtKebDDh0/w5ZeruffeTkRGhrJmzZ00alTd6bCUUja3T+zAGuBn4FVjzHyng/F0WVmGD+dvZXVcAgs2HyIhOR2wqtxHXny2w9GpsmSM4YsvVvHggzM5ejSZCy9sROvWdTWpK1XBeEJijzbGZDkdhKdLSc/kmSlrc72nPSw0kJYNQniqb3Ni6oU4GJ0qa+vXH+TOO39h7twddOkSzrhxfWnduq7TYSml8uG2iV1E3jDGPAD8ICKnNBQwxgxwICyPcyAxhQ/mbuWjBdsA6NioBpe2rMf1nSK1MZyXyMjIok+fCcTHpzB+fF+GDm2vX9BTqgJz28SO9XgbwLuORuGhth48zsPfr2bZjqM53Z7q25zbujdyMCpVnubM2U63bhH4+fkwYcIAGjeuQZ06+q4BpSo6t03sxpgl9p/NjDG5krv98pnZ5R+VZ9h84Dg9x8zN+T32+vZ0jq5BzaoBDkalysvevYncf/8MvvlmLWPH9uGuuzrQpUtE0SMqpSoEt03sLm7l1FL7bfl0U8Xg+t72Ry+N4fZzo7Xa1UtkZmbx/vvLeOKJP0hNzWDUqPO57bZ2ToellCoht03sInIt1iNujURkskuvYOCUL7Cpgp1Iy+DNmZv4cL51Hz040Je7LziLYT0aOxyZKk9Dh/7MZ5+tpGfPaN57rw9NmtR0OiSl1Glw28QOLAEOY32ZbaxL90RghSMRuRljDAu3Hub6DxcDUNnfhw5RNfjgpnO0YZyXOHYsFWMMoaGB3HlnLBdfHM2gQS31PQRKuTG3TezGmG3ANqyvuakSWLLtCC9OX8/KXScrNtpFVmPynV31hO4ljDH88MN67r33N/r1a8q4cX3p2LEBHTs2cDo0pdQZctvELiJzjTE9ROQo4Pq4mwDGGFPDodAqrF9W7+XuCctzdbu1WyOu7RDB2fWCHYpKlbdt244yfPivTJ/+H23b1uOWW9o6HZJSqhS5bWIHLrD/r+VoFG7AGMMlb81j0/7jAMTUC+Z/17WjSV1N5t5m8uT1DB48GR+fSrz55iUMH95RP9iilIdx28Tu8ra5CGCPMSZNRLoDrYGvgGOOBVeBGGNo9Nj0nN/zH76AiBqVHYxIOSE9PRM/Px/atw+jf/8YXn21F+Hh+rZApTyRJ1yq/wQYEWkMfIH1IZgJzoZUMexNSM6V1Dc831uTupc5dOgEt902hcsvn4QxhqioakyYcJUmdaU8mCck9ixjTDowAHjLGHMP4PUtgIwxXPbOAgB6NqvLv89eTKCftnT3FsYYPv10BTEx7/LFF6tp06YumZme84lmpVTB3LYq3kWGiAwEbgT62938HIzHUWkZWTwzdQ3TVu0lMTWD5mEhfHSzfuLYm+zcmcDgwZOZP38n3bpFMG5cX1q2rON0WEqpcuIJif1W4C6sz7ZuFZFGwESHY3JEVpbhnBdmkpiSAcCAdg0YfWUrh6NS5S00NIAjR5L56KN+3HJLO31zoFJexu0TuzFmjYiMAM4SkRhgszFmtNNxOWHEpBU5SX37y5c5HI0qT9On/8f77y9j8uRrCA0NZPXqOzWhK+Wl3P4eu4icC2wGPgY+ATaJSDdnoyp/v/67l2mr9wKw8YXeDkejysvu3ccYOPA7LrtsAps3H2H37kQATepKeTG3L7EDbwJ9jDHrAESkGfAl4DU3lhNT0rnza+vFM7+M6K6vg/UCmZlZjB27lCef/IP09CxGj76QBx/sir+/bnulvJ0nJHb/7KQOYIxZLyL+TgZUnjIys2jz3O8AXN8pkhb1Qx2OSJWHrCzDxx+voFu3SMaO7UN0dHWnQ1JKVRCekNiXi8h4rFI6wA14yUdg/tufSL93F5BlILpWFUb3b+l0SKoMJSSk8OKL83n00e5Urx7En3/eTPXqgfp+f6VULm5/jx0YBmwBHgYeAbYCdzgaUTlYtv0Ivd6cR0p6FrefF83sB3roCd5DGWP45ps1xMSM5bXX/mbmzK0A1KgRpNtcKXUKty6xi0groDHwozHmVafjKS+ZWYarxy0E4IMbz+HiFvUcjkiVlS1bjnD33dOZMWML7duH8fPP1xEbW9/psJRSFZjblthF5HGs18neAMwUkVsdDqncPDVlDQBtI6ppUvdwjzwyi7//3sXbb/dmyZKhmtSVUkVy5xL7DUBrY0ySiNQGpmM97ubRjiSlMWHxTgAm/l9nh6NRZWHu3O1ERIQSHV2dt97qjQg0aKDvdldKFY/bltiBVGNMEoAx5iDuvSzFciItg46jZwEw6ooWBOmjTR7l4MEkhgz5ifPP/5zRo+cBEB4eokldKVUi7lxijxaRyfbfAjR2+Y0xZoAzYZWdTxZsIyPL0Kt5XW7qEuV0OKqUZGUZPvlkBQ8/PJPjx9N4/PHuPPHEeU6HpZRyU+6c2K/K8/tdR6IoJ3sTknn9900AvH51G4ejUaXprbcW8cADv3PuuZGMG9eX5s1rOx2SUsqNuW1iN8bMdjqG8tTlpT8AuCY2nNDKXvvxOo+RlJTG3r3HOeusGtx2Wztq167M4MGt9fE1pdQZ8/j70p5gzEyrpF41wJdXtbTu9qZN20SLFu8xYMA3ZGUZQkMDufHGNprUlVKlwqsTu4j0FpGNIrJZRB4tZLirRcSISLm/f37XkRO8M/s/AOY/fEF5z16Vori4YwwY8A39+k2kShV/xo7tox9rUUqVOretis9LRAKMMaklGN4HGAv0AuKApSIy1fW98/ZwwcAIYHFpxltc906y3o774U2xVK/iNa/A9zjLl++lR4/PyMzM4qWXLmLkyC76wRalVJlw+xK7iHQUkX+B/+zfbUTkf8UYtSPWt9u3GmPSgEnAFfkM9zzwKpBSWjEX176EFJbvjKd2cAC9mtct79mrUnDsmHWt2apVHW69tS1r197Fo49216SulCozbp/YgXeAvsBhAGPMKqA4ddYNgF0uv+PsbjlEpB0QYYyZVtiEROR2EVkmIssOHjxYktgLdf83KwF4/ooWpTZNVT7i41O4665fiIl5l/j4FPz8fHj77Utp1Ei/wqaUKluekNgrGWN25OmWWYzx8ru5aXJ6ilTC+tb7A0VNyBjzgTEm1hgTW7t26TyqtHZPAgu3HiY40JdL9LWxbsMYw8SJ/xIT8y7jx//Dtde2wMdH76MrpcqPJ9xj3yUiHQFj3ze/B9hUjPHigAiX3+HAHpffwUBLYI7dWrkeMFVELjfGLCuVyAsxaPwiAN4e1FZbS7uJ48fTGDDgG2bO3EpsbH2mT7+B9u3DnA5LKeVlPCGx34lVHR8J7Adm2d2KshRoIiKNgN3AIOD67J7GmASgVvZvEZkDPFgeSX3jvkQSUzNoVKsKF8bovfWKzhiDiFClih+1alXm3XcvZdiwWHx8PKFCTCnlbtw+sRtjDmAl5ZKOlyEiw4EZgA/wiTFmrYiMApYZY6aWcqjFdveE5QC8dnVrp0JQxfTHH9sYOXIGkydfS3R0dSZMyPtCRKWUKl9un9hF5ENc7o1nM8bcXtS4xpjpWF+Fc+32dAHDnn+aIZbI6F/WsfnAcQDOaagNrSqq/fuP8+CDM/nqq9VER1fn0KETREfr9lJKOc/tEztW1Xu2QOBKcrd2dxvGGD6cvw2AX0Z013vrFdRHHy3noYdmkpSUxlNPncdjj3UnKEhf86uUqhjcPrEbY75x/S0iXwIzHQrnjExbvReAQR0iaFE/1OFoVEFWrNhL27b1eP/9y4iJqVX0CEopVY7cPrHnoxHQ0OkgTsfv6/YTHODLi1e2cjoU5eL48TSee24OV17ZjK5dIxgz5hL8/X20RkUpVSG5fWIXkaOcvMdeCTgCFPje94oqM8vw86o9DGjfQN8fXoFMmbKBe+75lV27jlG9ehBdu0YQEOD2h41SyoO59RlKrCJTG6zH1QCyjDGnNKRzB+/9uRmAs+pUdTgSBbBjRzwjRvzG1KkbadmyDhMnXkW3bpFOh6WUUkVy68RujDEi8qMx5hynYzlTk5Za7f2Gdo92OBIF8P3365g1ayuvvtqT++7rjJ+fvttdKeUe3Dqx25aISHtjzHKnAzld+xJS2B2fzFl1quLvqy81ccqiRXHEx6fQu/dZjBjRiYEDWxAZqY0YlVLuxW2ziIhkX5R0x0ruG0VkuYisEBG3SvLfLbNK6y/0b+lwJN7p6NFkhg2bRteuH/P0039ijMHPz0eTulLKLblziX0J0B7o73QgZ+qjBduo7O9D5+iaTofiVYwxfP31v4wcOYMjR5K5//7OPPfcBdraXSnl1tw5sQuAMWaL04GcicPHU0lITqdZWIjToXidP//czo03/kinTg34/fcbadtWv6KnlHJ/7pzYa4vIyIJ6GmPGlGcwp+ufHUcBuP28Rg5H4h1SUjJYunQ3557bkAsuiGLKlEFcdlkT/WCLUspjuPPZzAeoivV51fz+VXjGGG7/8h8Azm9ax+FoPN/MmVto1ep9LrnkKw4eTEJEuPzyszWpK6U8ijuX2PcaY0Y5HcSZePzHf3P+rlZZ3zVeVvbtO87IkTOYOHENTZrUYOrU66hdu4rTYSmlVJlw58Tu1i2cDhxLYeISqzX8ulGXaIOtMnL0aDItWrzH8eNpPPNMDx59tDuBge682yulVOHc+Qx3kdMBnIkZa/cBMOaaNlT2d+fNUDHt3n2MBg1CqF49iOefv4CePaNp2lSfOlBKeT63vblojDnidAxn4oP5W/GpJPRv28DpUDxKYmIqI0fOICrqbf7+26oRueuuDprUlVJeQ4uKDjDGEH8inaoBvvrBl1JijOHHHzcwYsSv7NmTyB13nEOzZvpJVaWU99HE7oCFWw+TmJLByF5NnQ7FY1x//WQmTVpD69Z1+f77a+jcOdzpkJRSyhGa2B1w48dLAOjRtLbDkbi39PRMfH0rISJ06xZBbGwY997bGV99375SyovpGbCcGWPIzLK+LNsmoprD0bivBQt20q7deL79di0Aw4d35IEHumpSV0p5PT0LlrM9CSkADDxHq4pPx+HDJxg6dCrnnvspx46lEhoa6HRISilVoWhVfDn7etEOAIZ0i3I2EDf0ww/rGDbsF44eTeahh7ry9NM9qFrV3+mwlFKqQtHEXs72HbNK7DH19KMvJWUMNGlSg3Hj+tK6dV2nw1FKqQpJE3s523XkBCGBvvjoY25FSk5OZ/To+dSoEcTIkV246qpmDBjQTB8RVEqpQug99nK2dPtRzm2ireGLMmPGZlq2fJ/Ro+ezadNhAEREk7pSShVBS+zlaHVcPAA1quh94YLs3ZvIfffN4Ntv19K0aU1mz76JCy/UT9oqpVRxaWIvR5OWWq84valLQ4cjqbh27kzg5583MmrU+Tz8cDcCAnQXVUqpktCzZjmKP5EGQJO6bvG5+HKzfPle/vhjGw8+2JVOncLZtet+atas7HRYSinllvQeeznJyjIs2nqEuiEBTodSYRw7lsp99/1Ghw4fMmbMQhLsZ/w1qSul1OnTxF5OZm84wJGkNG7opNXwxhi+/34dzZqN5Z13FjNs2DmsW3e3vmxGKaVKgVbFl5Ml26yW3YM6RDgcifP270/i5pt/omnTmvz447V07KifrlVKqdKiib2cbDt0AoCaVb2zKj4tLZPvvlvL9de3ol69qsydO4S2bevpu92VUqqUee1ZVUR6i8hGEdksIo/m03+kiKwTkdUiMltEzqgOfdP+RBrVquKVL6aZP38H7dqNZ/DgH5k/fycAsbH1NakrpVQZ8Mozq4j4AGOBS4HmwHUi0jzPYCuAWGNMa+B74NXTnZ8xhp1HTnC2l7WGP3ToBLfeOoXzzvuMpKQ0fv75Os47T9sYKKVUWfLWqviOwGZjzFYAEZkEXAGsyx7AGPOny/CLgMGnO7P0TOszrTFh3pPYjTFcdNEXrFt3kEcf7caTT55HFX0xj1JKlTlvTewNgF0uv+OAToUMfxvwa0E9ReR24HaAyMjIU/qnZ2YBUNnfp+SRupkNGw4RHV0df38f3nzzEurUqULLlnWcDksppbyGV1bFA/nd6Db5DigyGIgFXitoYsaYD4wxscaY2Nq1T30PfFqGldj9fDx3dZ84kc7jj8+mVav3eeedxQBceGEjTepKKVXOvLXEHge4PncWDuzJO5CI9ASeAHoYY1JPd2ar7HfEe2qzuenT/+Puu6ezfXs8Q4a0ZciQtk6HpJRSXstzi5CFWwo0EZFGIuIPDAKmug4gIu2A8cDlxpgDZzKzqSuta4bOjWueyWQqpCef/IPLLptAYKAvc+bczKefXkGtWvrmOKWUcopXltiNMRkiMhyYAfgAnxhj1orIKGCZMWYqVtV7VeA7EQHYaYy5/HTmdzw1A4CYeiGlEb7jMjKySE3NoEoVf/r1a0pQkC8PPdQNfy9oQ6CUUhWdVyZ2AGPMdGB6nm5Pu/zds7TmNWfTQUICPWNVL126m2HDfqFDh/qMG9eXTp3C6dQp3OmwlFJK2by1Kr5cVfb3IdLNP2ySkJDC8OHT6dTpI/buTeSii/Qb6UopVRF5RjGygktMyaBLtPveX583bweDBn3P/v1JDB/ekRdeuJAQ/UqdUkpVSJrYy1j8iTQysww+ldyvcsQYg4jQsGEoTZrUZOrU64iNre90WEoppQqhib2MLd52BID61dznk6SpqRm89trf/PPPXiZPvoaGDasxd+4Qp8NSSilVDO5XjHQzq3ZZz7D3blHP4UiKZ86c7bRtO56nnvoTP79KJCdnOB2SUkqpEtDEXsamrtrDWXWqUiekYpfYjxxJ5uabf+KCCz4nNTWD6dOv59tvB1K5sp/ToSmllCoBTexlbE98MqFBFT85+vgIc+du5/HHu7NmzV1cemkTp0NSSil1GvQeexnKyMwiy0BkjYr5qNuaNQcYM2Yh48f3JTQ0kA0bhhPoIc/bK6WUt9ISexnan2i9Xj6mXsX6XGtSUhqPPDKTdu3GM3XqRjZsOASgSV0ppTyAnsnL0L6EFADqhVac++vTpm1i+PDp7NiRwK23tuXVV3tR081fnqOUUuokTexlaO5G69sxZ1eQEntmZhZPPvkHVar4M2/eEM49t6HTISmllCplmtjL0JaDSQA0qlXFsRgyMrIYN24ZN9zQiurVg5g69Trq1auqH2xR1H1MTgAAFDZJREFUSikPpYm9DK3fe4yOjWoQ4OtMEl28OI477pjGqlX7EYG77+5IZGSoI7EopZQqH5rYy9DWQ0m0Di//RBofn8Ljj89m3LhlhIUF8/33AxkwoFm5x6EqvvT0dOLi4khJSXE6FOUlAgMDCQ8Px8+v4j8G7K40sZeRJPsb7JUDyn8V33vvb3z11WruvbcTo0ZdQHCwfrBF5S8uLo7g4GCioqIQEafDUR7OGMPhw4eJi4ujUSP9QmRZ0cReRlbHJQBwdt3yaTj333+H8fPzISqqGqNGnc+993aiffuwcpm3cl8pKSma1FW5ERFq1qzJwYMHnQ7Fo+lz7GVk8bbDAGVeFZ+amsGoUXNp1ep9HnzwdwAaNqymSV0VmyZ1VZ50fyt7WmIvIxv2JgLQJrxamc1j9uyt3HXXdDZtOsygQS0ZM+biMpuXUkop96Al9jLiU0nw8xEqVSqbq9MvvlhFz55fkpmZxYwZg5k48SrCwirG8/JKlYSPjw9t27alZcuW9OvXj/j4+Jx+a9eu5cILL6Rp06Y0adKE559/HmNMTv9ff/2V2NhYmjVrRkxMDA8++KATi1CoFStWMHTo0FzdrrjiCrp06ZKr25AhQ/j+++9zdatatWrO35s2baJPnz6cddZZNGvWjGuuuYb9+/efUWxHjhyhV69eNGnShF69ev1/e/ceHVV1L3D8+4OgIYA8JYXGGkOwBAKEhw+Ellfl4W20YCQKlcelFZDqqijea+1apGLro76qwYvcXhvQJiBQEMVqi2C1CEooMUGCGCGl0AghBkkgSEh+949zMpmECZlAMgOT32ets9bMmX32+c2PYfacfXb2pri4+IwymzZtIiEhwbOFh4ezdu1aAPbt28d1111Hz549SU5O5tSpUwCkpqbyhz/84bxiM+dBVW1rxG3QoEGqqnrlf72piS98oI2poqJSDx48pqqqR4+W6a9//b6eOHGqUc9hmpddu3YFOwRt06aN5/HUqVP10UcfVVXVEydOaExMjL7zzjuqqnr8+HEdN26cpqamqqpqTk6OxsTEaG5urqqqlpeX66JFixo1tvLy8vOuIykpSbOysjzPi4uLNSoqSnv16qV79+717J82bZquXLmyxrFVuSkrK9PY2Fhdt26d57WNGzdqTk7OecU2f/58feyxx1RV9bHHHtMHH3zwrOWLioq0Y8eOevz4cVVVve222zQjI0NVVWfNmqUvvviiqjr/VgkJCXXW4+tzB2TqBfAdHgqbdcU3AeczCkdPlDdanZ988iWzZ6+npOQbduyYRfv24fziF99rtPqN+dUbn7Lr38catc7e3S9jQWIfv8sPGTKE7OxsANLT0xk6dChjxji3mCIiIkhNTWXEiBHMnTuXJ598kocffphevXoBEBYWxt13331GnaWlpdxzzz1kZmYiIixYsIBbb72Vtm3bUlpaCsCqVat48803SUtLY/r06XTq1IkdO3aQkJDAmjVryMrKokMH57ZabGwsmzdvpkWLFsyePZv9+/cD8NxzzzF06NAa5y4pKSE7O5v+/ft79q1evZrExEQiIyNZvnw5Dz30UL15SU9PZ8iQISQmJnr2jRw50u+81uX111/nvffeA2DatGmMGDGCJ554os7yq1atYvz48URERKCqbNy4kfT0dM/xKSkpzJkzh4iICKKjo/n444+59tprzztO0zDWsDeB/KITACRfc8V511VaeoqUlPd47rmtdOrUmqefHkNYmN1BMaGnoqKCd999l5kzZwJON/ygQYNqlOnRowelpaUcO3aMnTt3cv/999db78KFC2nfvj05OTkAPruba9uzZw8bNmygZcuWVFZWsmbNGmbMmMFHH31EdHQ0kZGRTJ48mfvuu49hw4axf/9+xo4dS25ubo16MjMziY+Pr7EvIyODBQsWEBkZSVJSkl8N+86dO8/IhS8lJSV873u+f/Cnp6fTu3fvGvsOHTpEt27OQNtu3bpx+PDhs9a/fPly5s2bB0BRUREdOnQgLMxpRqKiojh48KCn7ODBg/nggw+sYQ8Ca9ibwJYvnBHxV5zncq2ff17E6NHL+Ne/jvHTnw7k8cd/QKdOrRsjRGPO0JAr68ZUVlZGQkIC+fn5DBo0iBtvvBFwer7qGkHdkJHVGzZsYPny5Z7nHTt2rPeY2267jZYtnRkjk5OTeeSRR5gxYwbLly8nOTnZU++uXbs8xxw7doySkhLatase61JQUMDll1/ueX7o0CHy8vIYNmwYIkJYWBg7d+4kPj7e53tq6Ajydu3akZWV1aBj/FVQUEBOTg5jx44FqnsmvXnH27VrV3bv3t0ksZizs0u/JrC30Oneuyn+W+d0fHl5BQDR0R0YNuw7bN78nyxZkmiNuglJrVu3Jisri3/+85+cOnWKRYsWAdCnTx8yMzNrlN27dy9t27alXbt29OnTh+3bt9dbf10/ELz31Z55r02b6vUdhgwZQl5eHoWFhaxdu5aJEycCUFlZyZYtW8jKyiIrK4uDBw/WaNSr3pt33StWrKC4uJirrrqK6Oho8vPzPT86OnfuXKM34auvvqJLly6eXPjzXktKSmoMdPPevH+EVImMjKSgoABwGu6uXbvWWfdrr73GhAkTPDPGdenShaNHj3L6tDMZ14EDB+jevbun/MmTJ2nd2r6zgsEa9ibwRWEpl7RsQVjLhqW3vLyCp576kF69FlFcXEarVi1JT7+VG244/y59Yy507du35/nnn+epp56ivLycKVOm8Pe//50NGzYAzpX9vffey4MPPgjA/Pnz+c1vfsOePXsAp6F95plnzqh3zJgxpKamep5XNZ6RkZHk5uZ6utrrIiJMmDCBefPmERcXR+fOnX3W6+tKOS4ujry8PM/zjIwM3n77bfLz88nPz2f79u2ehn3EiBGsWLHCM7I8LS3Ncx998uTJfPjhh6xfv95T19tvv+25vVCl6ord11a7Gx7g5ptvZunSpQAsXbqUW265pc48ZGRkcMcdd9TIy8iRIz0j+Wsfv2fPnjNuQ5gACfbovVDbBg0apD0eWq/jn3tfG2Lz5v3at++LCimamJiuBQUlDTremHNxoY2KV1X94Q9/qMuWLVNV1ezsbB0+fLheffXV2qNHD01JSdHKykpP2TfeeEMHDhyovXr10ri4OH3ggQfOqL+kpESnTp2qffr00X79+unq1atVVXXlypUaExOjw4cP17lz5+q0adNU1ffo9G3btimgaWlpnn2FhYU6adIk7du3r8bFxemsWbN8vr/4+Hg9duyY7tu3T7t3714jflXVAQMG6NatW1VVNSUlRePj47V///46ceJEPXz4sKdcbm6ujh07VmNjYzUuLk6Tk5P1yy+/PGtu63PkyBEdNWqUxsbG6qhRo7SoqMjzfmfOnOkpVxV7RUVFjeO/+OILveaaa7RHjx6alJSkJ0+erPG+CgsLfZ7XRsU37SZOPk1j6dN/gB4f/yizvh/DQzfVv/DKqVMV3HPPWyxZ8g+ioi7jhRfG86Mf9QpApMZAbm4ucXG2QFBTevbZZ2nXrt0Zf8seynbs2MEzzzzDK6+84vN1X587EdmuqoMDEV+os674RlZ2yrk/Ps7P++utWrXg8OETzJt3Pbm5c61RNybEzJkzh0svbV4LMR05coSFCxcGO4xmy0bFN7KKSkWAb3eoe9DIZ58dYd68v/DCC+OJienI6tWTmmyGOmNMcIWHh3PnnXcGO4yAqvrLBhMcdsXeyEpOnqZjRCsu97FU6smTp1mwYBP9+i1m8+b97N59BMAadRNUdjvOBJJ93pqeXbE3sm9OVxBxSdgZf16zYcNe5sxZT17eV0yZ0pennx5DZGTbOmoxJjDCw8MpKiqic+fOtuqWaXKqznrs4eHhwQ4lpFnD3sgqKpWrurQ5Y//rr+9GBDZsuJPRo2OCEJkxZ4qKiuLAgQO2PrYJmPDwcKKiooIdRkizUfGN7NJuPfUXS9byy5vieOml7QwY8C2GDLmC0tJThIW1IDzcfksZY0xtNiq+8TTre+wiMk5EPhORPBH5bx+vXyoiK9zXPxKRaH/q7VgJN9zwMnPnvsWrrzoLWrRte4k16sYYY5pcs21pRKQlsAi4ETgAbBORdarqPe/iTKBYVWNF5HbgCSC5vrrvS/oTXTq15tVXJzB5ct+mCN8YY4zxqTlfsV8L5KnqXlU9BSwHas+neAuw1H28Chgt9Ywwqiyv5K6fDGD37rlMmdLPBiQZY4wJqGZ7xQ58G/iX1/MDwHV1lVHV0yLyNdAZOOJdSETuAu5yn36zeHHizsWLmyTmi00XauWqGbNcVLNcVLNcVPtusAMIFc25Yfd1KV17JKE/ZVDVJcASABHJtAEgDstFNctFNctFNctFNRHJrL+U8Udz7oo/AHgvmxYF/LuuMiISBrQHvgpIdMYYY8w5aM4N+zagp4hcJSKXALcD62qVWQdMcx8nARvV/j7QGGPMBazZdsW798x/BrwDtAReVtVPReQRnOUD1wH/B7wiInk4V+q3+1H1kiYL+uJjuahmuahmuahmuahmuWgkNkGNMcYYE0Kac1e8McYYE3KsYTfGGGNCiDXs56CppqK9GPmRi3kisktEskXkXRG5MhhxBkJ9ufAqlyQiKiIh+2dO/uRCRCa5n41PRSQ90DEGih//R74jIptEZIf7/+SmYMQZCCLysogcFpGddbwuIvK8m6tsERkY6BhDgqra1oANZ6DdF0AMcAnwCdC7Vpm7gcXu49uBFcGOO4i5GAlEuI/nNOdcuOXaAe8DW4HBwY47iJ+LnsAOoKP7vGuw4w5iLpYAc9zHvYH8YMfdhPn4PjAQ2FnH6zcBf8aZQ+R64KNgx3wxbnbF3nBNMhXtRareXKjqJlU94T7dijNfQCjy53MBsBB4EjgZyOACzJ9c/BRYpKrFAKp6OMAxBoo/uVDgMvdxe86cTyNkqOr7nH0ukFuAZerYCnQQkW6BiS50WMPecL6mov12XWVU9TRQNRVtqPEnF95m4vwaD0X15kJEBgBXqOqbgQwsCPz5XFwNXC0im0Vkq4iMC1h0geVPLlKAH4vIAeAt4J7AhHZBauh3ivGh2f4d+3lotKloQ4Df71NEfgwMBoY3aUTBc9ZciEgL4FlgeqACCiJ/PhdhON3xI3B6cT4QkXhVPdrEsQWaP7m4A0hT1adFZAjO3BnxqlrZ9OFdcJrLd2eTsiv2hrOpaKv5kwtE5AfAw8DNqvpNgGILtPpy0Q6IB94TkXyc+4frQnQAnb//R15X1XJV3Qd8htPQhxp/cjETeA1AVbcA4TiLwzRHfn2nmLOzhr3hbCraavXmwu1+fgmnUQ/V+6hQTy5U9WtV7aKq0aoajTPe4GZVDcWFL/z5P7IWZ2AlItIFp2t+b0CjDAx/crEfGA0gInE4DXthQKO8cKwDprqj468HvlbVgmAHdbGxrvgG0qabivai42cufgu0BVa64wf3q+rNQQu6ifiZi2bBz1y8A4wRkV1ABTBfVYuCF3XT8DMX9wP/KyL34XQ7Tw/RCwFEJAPn9ksXd0zBAqAVgKouxhljcBOQB5wAZgQn0oubTSlrjDHGhBDrijfGGGNCiDXsxhhjTAixht0YY4wJIdawG2OMMSHEGnZjjDEmhFjDbkwDiUiFiGR5bdFnKRtd10pWDTzne+4KYZ+407B+9xzqmC0iU93H00Wku9drvxeR3o0c5zYRSfDjmJ+LSMT5ntsY47CG3ZiGK1PVBK8tP0DnnaKq/XEWGPptQw9W1cWqusx9Oh3o7vXaT1R1V6NEWR3ni/gX588Ba9iNaSTWsBvTCNwr8w9E5B/udoOPMn1E5GP3Kj9bRHq6+3/stf8lEWlZz+neB2LdY0e763jnuGtdX+ruf9xd6zxbRJ5y96WIyAMikoQzb/8f3XO2dq+0B4vIHBF50ivm6SLywjnGuQWvBTxE5H9EJFOc9dd/5e67F+cHxiYR2eTuGyMiW9w8rhSRtvWcxxjjxRp2YxqutVc3/Bp332HgRlUdCCQDz/s4bjbwO1VNwGlYD7hTiCYDQ939FcCUes6fCOSISDiQBiSral+cmSTniEgnYALQR1X7AY96H6yqq4BMnCvrBFUt83p5FTDR63kysOIc4xyHM3VslYdVdTDQDxguIv1U9XmcucBHqupId3rZXwI/cHOZCcyr5zzGGC82pawxDVfmNm7eWgGp7j3lCpy5z2vbAjwsIlHAn1T1cxEZDQwCtrlT7rbG+ZHgyx9FpAzIx1na87vAPlXd476+FJgLpOKs9/57EVkP+L1MrKoWished57uz91zbHbrbUicbXCmUB3otX+SiNyF873TDegNZNc69np3/2b3PJfg5M0Y4ydr2I1pHPcBh4D+OD1hJ2sXUNV0EfkI+A/gHRH5Cc4ylUtV9SE/zjHFe9EYEensq5A7P/m1OAuL3A78DBjVgPeyApgE7AbWqKqK08r6HSfwCfA4sAiYKCJXAQ8A16hqsYik4Sx2UpsAf1XVOxoQrzHGi3XFG9M42gMF7hrad+JcrdYgIjHAXrf7eR1Ol/S7QJKIdHXLdBKRK/08524gWkRi3ed3An9z70m3V9W3cAam+RqZXoKzlKwvfwJ+hLNO+Ap3X4PiVNVynC71691u/MuA48DXIhIJjK8jlq3A0Kr3JCIRIuKr98MYUwdr2I1pHC8C00RkK043/HEfZZKBnSKSBfQClrkj0X8J/EVEsoG/4nRT10tVT+KsfrVSRHKASmAxTiP5plvf33B6E2pLAxZXDZ6rVW8xsAu4UlU/dvc1OE733v3TwAOq+gmwA/gUeBmne7/KEuDPIrJJVQtxRuxnuOfZipMrY4yfbHU3Y4wxJoTYFbsxxhgTQqxhN8YYY0KINezGGGNMCLGG3RhjjAkh1rAbY4wxIcQadmOMMSaEWMNujDHGhJD/B1A8TuGEh9jrAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "0.7040433457077317" + ] + }, + "execution_count": 81, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#plot roc for svm\n", + "get_roc(clf_reduced, y_test, X_test_pca, \n", + " \"SVM reduced features no tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM reduced features no tuning RBF kernal identified 738\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06387342
11282738
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6387 342\n", + "1 1282 738" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_reduced.predict(X_test_pca), \"SVM reduced features no tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.37 0.48 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.76 0.66 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_reduced.predict(X_test_pca)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, by reducing features through pca, Although the AUROC did not change much (0.001 increase), the number of correctly idendtified defaulters increased from 297 to 307, suggesting a better recall." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will now try to find best parameters for SVM model" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'C': 1, 'gamma': 0.1}" + ] + }, + "execution_count": 84, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sklearn.model_selection import GridSearchCV\n", + "def svc_param_selection(X, y, nfolds):\n", + " Cs = [0.01, 0.1, 1,10]\n", + " gammas = [0.01, 0.1, 10]\n", + " param_grid = {'C': Cs, 'gamma' : gammas}\n", + " grid_search = GridSearchCV(svm.SVC(kernel='rbf'), param_grid, cv=nfolds)\n", + " grid_search.fit(X, y)\n", + " grid_search.best_params_\n", + " return grid_search.best_params_\n", + "svc_param_selection(X_train_pca, y_train,2)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With 2 folds, it can be found that C = 10, and gamma = 0.01 will have the best svm model with RBF kernel" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,\n", + " decision_function_shape='ovr', degree=3, gamma=0.1, kernel='rbf',\n", + " max_iter=-1, probability=True, random_state=None, shrinking=True, tol=0.001,\n", + " verbose=False)" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#train svm model with feature reduction and cost = 10, gamma = 0.01\n", + "clf_reduced_tuned = svm.SVC(kernel = 'rbf', probability = True, C= 10, gamma = 0.1)\n", + "clf_reduced_tuned.fit(X_train_pca, y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimal Threshold: 0.16537143473786733\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAf4AAAEWCAYAAACHePXKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gV1dbA4d9KCJCQECCElkJC7zUCiiAoXRHsqOjFhiAoqNxPRb32eu1XsCt2LFgp0kRQUKnSe08IhJZCetnfHzMJJyEVkkxyst7n4SHT18yZmTWz954ZMcaglFJKqarBw+kAlFJKKVV+NPErpZRSVYgmfqWUUqoK0cSvlFJKVSGa+JVSSqkqRBO/UkopVYVUyMQvIjeKyAKn43CaiISKyCkR8SzHZYaJiBGRauW1zLIkIptFpN9ZTHdW+6CIeIvIzyISJyLflHR6dyIiv4nI7eW8zBki8nQhw58WkWMicrg846pMRGSfiAwop2W9LSKPlseyypJ9zmxRAeJ4XEQ+K2q8IhO/vRMk2wnosH1g+ZZOmPkzxnxujBlUlsuoiPIecMaYA8YYX2NMppNxOaU0DiZjTHtjzG9FLOeMi51z2AevBhoCAcaYa85i+vzimyoie+1jMFJEvrL7vyMin+QzficRSRWRevaJwIjIPXnGmWz3f7w0YqwMRCQEuB9oZ4xpdI7z6icikaUTWeVR1IVVSRljxhljniqt+WVzOaZP2f/2iciDecZxzW0nRWSOvY9kD58hImku8zglIteVdqxOKO4d/3BjjC/QBegKPFR2IZUdJ+9i3eUOuiSq6PZuCuwwxmSUdML8YhaRfwE3AQPsYzACWGwPngFcKSK18kx2MzDbGHPC7t4B/CufcXacS2yVUFPguDEmxulA3GR7VgZ17OPmauBRERmYZ3h2bmsMHAH+l2f4i/bNV/a/r8oy2PIq3S1RUb8x5jAwH+sCAAARqSEiL4nIARE5YhfdeLsMHyEi/4hIvIjsFpEhdn9/EflARKJFJMougvO0h40RkT/sv98WkZdc4xCRH0XkPvvvJiIyS0SO2ndF97iM97iIfCsin4lIPDAm7zrZcXxiT79fRB4REQ+XOJaLyP/sotttInJJnmkLW4flIvKqiJwAHheR5iLyq4gcF6u48XMRqWOP/ykQCvxsX1n+X947UbGKTp+y55sgIgtEpL5LPDfb63BcRB6VQorsxCqSftkeP05E/nD93YAb7d/0mIg87DJdDxH5U0Ri7fV+U0Squww3IjJBRHYCO+1+r4vIQXsfWCMifVzG9xTrjna3vU5rRCRERJbZo6x3vdIWkcvs/SlWRFaISCeXee0TkQdEZAOQKCLVXLeBHftqO44jIvKKPWn2smLtZZ3vug/a07YXkYUicsKedmo+2/QJ4D/AdfZ8bhMRD3uf2i8iMfa+5m+Pn/373iYiB4Bf8/mpzgPmG2N2g3UMGmPetf/+E4gCrnLdnsANwMcu81gF+IhI++x1Abzt/vnKb/+1+98qIlvFukOaLyJNXaYZaB8jcSLyJiAuw3IVQeazb9cTkY9E5JA97x9cxi3sN+8qImvtfecroGYB6zMAWAg0sX+bGXb/XvY8Y0VkvbhUC4nILfa6JojIHhG50+5fC5jnMq9TYp2Hct0NS55SgQL2z8LOXwXtr3nXra6IzLbncdL+O9hleFHnjZvk9Hnj4fyWYY83FrgR+D97nX+2++cqmXPdDtnbQETut/f/aBG55SzHDRCrGi1eRFaJdb7NOUYLY4xZDWzGJXflGZ4CfAu0K878CiMiF4p1vutvd7eR0+eO7SJyrcu4M0TkLRGZKyKJQH+73zSxSiASRORvEWnuMk2B59NiM8YU+g/Yh3W3ARAMbARedxn+GvATUA/wA34GnrOH9QDigIFYFxlBQBt72A/AO0AtoAGwErjTHjYG+MP+uy9wEBC7uy6QDDSx57kG62RbHWgG7AEG2+M+DqQDI+1xvfNZv0+AH+3Yw7Dugm5ziSMDuBfwAq6z16deMdchA7gbqIZ1om1hb4saQCBWwnktv21td4cBBqhmd/8G7AZa2fP7DXjeHtYOOAVcaG+Ll+x1H1DA7zrNnj4I8AQusOPKXuZ79jI6A6lAW3u67kAve53CgK3AZJf5GqwTbL3s7Q2MBgLsae4HDgM17WH/xtqnWmMlis5YxeTZ82rhMu9uQAzQ0475X/Y2q+Gy/f4BQlyWnbNNgT+Bm+y/fYFe+W3nfPZBPyDajr2m3d2zgO36OPCZS/etwC6sfdMX+A74NM9yP8Hah/LbP0cDJ+ztFAF45hn+MLDIpXswcBTwco0HmAq8YPd7EavU7jPg8QLWYwxn7r8j7XVpa/d7BFhhj18fiMe6s/LCOmYygNsL2C65tjkwB/gK6/j2Ai4q6jfH2s/3c/r4vBprn3+6gHXqB0S6dAcBx4FhWOeHgXZ3oD38UqA51n55EZAEdMtvXna/Ga7Lzmd5+3DZPyn6/JXv/prPegVgXfz5YO2b3wA/uAz/jaLPG33tbfqK/bsVdN7ItY4FHKc549jbIAN40v6Nhtnbse5ZjDvT/udjx30Q+xjNJ84wcu9fvex5XVFAbvPBulj+pLB1Lehf9jbAOv4OAj3s/rXs7luwjpluwDGgvcsy4oDe9v5Q0+53Ait/VgM+B2bmOScUdD59HJfjrMB4i7FC++wdI8FeucVYxSdgHRCJQHOX8c8H9tp/vwO8ms88G2IlE2+XftcDS/I56QpwAOhrd98B/Gr/3RM4kGfeDwEfuWyEZYWsm6cdRzuXfncCv7nEcQj7osPutxKr6LU463CgoGXb44wE1uW3Ixaw8/4GPOIy/C7gF/vv/wBfugzzAdLI5wC2d7BkoHMhB0xwnnUeVcA6TAa+z3MAXFzEep/MXjawHRhR2MHk0v0W8FSecbZzOknsA27NZ//NPriXAU8A9QtY54IS//Wuv1MR6/Y4uRPcYuAul+7WWMmpmstymxUxzxuBRVjH2nHgQZdhofb8gu3uz8l9Yf44VoIPxTqOvOz/Qyg68ec9tuZhXxS77EdJWEXoNwN/uQwTIJJiJH6sYtYs7BN8nmUW+JtjJau8x+cKip/4H8C+CHPpNx/4VwHT/wBMym9edr8ZFJ34b3XpLur8le/+Wox9sAtw0qX7Nwo/b7gmlVoUcN7Ibx0LOE5zxrG3QTK5j60YTl90F2tcrHN1OtDaZdjTFJ34Y+15GqybIdd9ZR9WbovFuuA4BHTMsx4p9vBY4Fgh29zYv93+PPO4Dvg9z7jvAI+5LOOTPMNnAO+7dA8DthWybNfz6eMUI/EXt6h/pDHGz/5h2mBd3YN11+oDrLGLymKBX+z+YJ1cduczv6ZYJ6Bol+newbprzsVYazMT6+QLVjHm5y7zaZI9D3s+U7GScraDhaxXfU7fNWTbj3UnkC3KjsF1eJNirkOuZYtIAxGZKVa1QDzWibc+JePaGjkJ604AO6ac5RljkrCSRH7qY11Z5vfbFLocEWllFyUettfhWc5ch7zrfb9dZBpnbyd/l2kK2kfy0xS4P8/vHYK17vkuO4/bsO56ttlFhZcVc7kliTGvJpy5f1Wj+PsoxmpoOACoA4wDnhSRwfawA1gJYrRYjW5HkruYH5fxdmH9XjuNMYUus4C4mgKvu2z7E1gJPogz9z9T1Hq5CAFOGGNO5jOssN+8Cfkfn8XVFLgmz7wvxLoQQUSGishfdhFtLNYJuKTHa16u26So81ex9lcR8RGroed++5hcBtSR3PXFxT1vZF9clqbjJnebF9flF3fcQKzjxnX7FWf/qm9PPwUrf3nlGT7SGFMHq7RjIrBURFwbfr5kjKlj/yvqt58MfG2M2ejSrynQM89vfCPguoz81qOg36uo82mxlLSOfynW1Uh2nfsxrKup9i4bx99YjSXAWqHmZ86Jg1h3y/VdpqttjGlfwKK/BK4Wqz6xJzDLZT57XeZRxxjjZ4wZ5hp2Iat0DOsqsqlLv1CsetNsQSIieYYfKuY65F32c3a/TsaY2lhFNlLI+CURjVUVA1h1+FjFQfk5hnUlm99vU5S3gG1AS3sdppJ7HcBlPez6pweAa7Hu6OpgFW1lT1PQPpKfg8AzeX5vH2PMl/ktOy9jzE5jzPVYF2cvAN+KVV9b1HYvSYx5HeLM/SsDqyFRTmjFmZExJt0Y8w2wAejgMuhjrDvuq7COh7UFzOITrKLBM54EKGiReboPYlVluW5/b2PMCqz9z7VFtLh2Y5VW+Lh05z3x1RO7vUs+yyzoN48m/+OzuA5i3fG7zruWMeZ5EamBdZ55CWho77dzOb3f5vebFbaO2VynK/T8Vcj+mtf9WCVJPe1jsq/dP+9xmZ+8v5sPBZ838safLYmi1/tcHcU6boJd+oUUMG4uxphMY8zLWOe8uwoZ5zsgE+vi72xcA4wUkcku/Q4CS/P8xr7GmPGuiy/uAopxPi2Ws3mO/zVgoIh0McZkYdUFvyoiDezAgrLvRoAPgFtE5BKxGjkFiUgbY0w0sAB4WURq28Oai8hF+S3QGLMO64d/H6uhU6w9aCUQL1aDGW+xGop1EJHzirMixnpM7mvgGRHxsy8s7sO6E8/WALhHRLxE5Bqs+s25JV0Hmx920ZKIBGHV27o6glXPdza+BYaLyAViNbZ7ggJ2Bvt3+xB4RazGRZ5iNWirUYzl+GHV5Z4SkTbA+GKMn4H1+1UTkf8AtV2Gvw88JSItxdJJRLJPPHm3x3vAOBHpaY9bS0QuFRG/YsSNiIwWkUB7/bP3oUw7tiwK3vazgUZiPQJXw95XehZnmVgXrfeKSLh9R/4s8JUpZqt/sRrZXWov00NEhgLtgb9dRpuFdRJ8gnzu9l18BQzC2ufPxtvAQ3K6kaC/fUyAVUffXkSuFKvB3j3kTgD/AH3FejeFPy5PBtnH0jxgulgN1bxEJDt5Ffab/4m1b90jVkO5K7HqRYvrM6xjZrB9DNQUq4FZMFZJYA3shGNvd9fHO48AAfa6uK7jMLEaKjbCugMsTKHnr0L217z8sG7AYkWkHvBYCbbBt8BlYjVIq45Vv15YXsjvHPUPcIMd/xCsaphSZZ+rv8NqJO1jn3tuLuFsnsdqmHhGA1B73xqB1cZk61mGeQi4BGt/zL7AmA20EqsBpZf97zwRaXuWyyjqfFosJU78xpijWHcM2S9deACrCPEvsYqZFmFdfWKMWYnVqOFVrKuSpZy++7kZ6+DaglVH8S12EVsBvgQGAF+4xJIJDMeq09qLdSf7PlbRR3HdjXWlvgf4w57/hy7D/wZa2vN+BrjaGJNdFFbSdXgCq3FHHNaJ8rs8w58DHhGrSGhKCdYBY8xme11mYl3FJ2DVj6UWMMkUrEZ1q7CKbF+gePvDFKzqlgSsk3JRj7fMxzqp78Aqhk0hd9HWK1iJaAHWBcUHWA2QwKqv+tjeHtcaq2XuHcCbWNt7F/k8qVGIIcBmETkFvI7VbiHFrhZ5BlhuL6uX60TGmASshl/DsYrgdgL9i7nMD4FPsYpf92Kt/90liDkeq1TlANbJ/0VgvDEmpzWzXTybnfw/z28m9njJxphFxpjkEizfdfrvsfaTmfaxvgkYag87hnXH8zxWUXFLYLnLtAux9pUNWA3aZueZ/U1YpW/bsPbbyfZ0Bf7mxpg04Eq7+yRWfWreY6qw9TkIjMDavkex9st/Ax72b34P1r55Emuf/8ll2m1Y56Q99j7TBOt3Xo9Vd7yAIo6NYpy/8t1f85nVa1jHzDHgL6zq1uJug83ABKzzXrS9roW9n+ADoJ29ztlPXkyy1yO7GPuHgiY+RxOxts1hrG39JQWf3/IzB2v97nDp97O9feOxzgH/srfJWbGr1C4BHhCR2+39aBAwCuvC4DDWMVScm6z8FHU+LZbslvIqHyIyBqtx0tkW/TjGvruMxSqS3+t0PEopVZpE5AWgkTHmX07HUtlUyFf2qrMjIsPtYrBaWHWTG7HuPpRSqlIT63n4TnaxfA+sxo/fOx1XZaSJ372MwCpOOoRV1DrKaJGOUso9+GFV5SRiVcG8jPUOFlVCWtSvlFJKVSF6x6+UUkpVIfqhiDJWv359ExYW5nQYSilVaaxZs+aYMSaw6DHV2dDEX8bCwsJYvXq102EopVSlISIleQOjKiEt6ldKKaWqEE38SimlVBWiiV8ppZSqQjTxK6WUUlWIJn6llFKqCtHEr5RSSlUhmvhtIvKhiMSIyKYChouIvCEiu0Rkg4h0K+8YlVJKqXOlif+0GVifwSzIUKz337cExgJvlUNMSimlVKnSxG8zxizD+i59QUYAnxjLX0AdEWlcPtEppZT7i4tLYeh9c50Ow+1p4i++IOCgS3ek3e8MIjJWRFaLyOqjR4+WS3BKKVVZGWP45pvNtO0wnS2emU6H4/Y08Ref5NMv308bGmPeNcZEGGMiAgP1ddNKKVWYNWuiuXbULKrf2Brx1LRU1nQLF18kEOLSHYz13XullFIllJaWyZIlewHo3r0xLR6IAGBElyZOhlUlaOIvvp+Am+3W/b2AOGNMtNNBKaVUZbNs2X66dHmbQYM+Y//+WLo/vYj0LKsA9eVrOjscnfvTr/PZRORLoB9QX0QigccALwBjzNvAXGAYsAtIAm5xJlKllKqcjh1L4v/+byEfffQPYWF1+Oa7axjw7grSM62kv/HxQVTTov4yp4nfZoy5vojhBphQTuEopZRbSUpKp1Ontzh6NImHHrqQQaM7MOYT65PlnYL9+frO86np5elwlFWDJn6llFJlJjIynuDg2vj4ePHMMxfTo0cQAcG16fXcYgCu7xHCc1d2cjjKqkXLVJRSSpW6pKR0HnpoEeHhrzN//i4AbrmlK8HN6vLQdxsAeO/mCE36DtA7fqWUUqVqzpwdTJw4j337YhkzpgvduzchMTWDL1ce4Ok5WwGoV6s6/Vvr485O0MSvlFKq1Nxxx0+8//462ratz9yFo9mYkc5tX61j/cHYnHHuHdCKOy9qpg35HKKJXyml1DnJyMjCw0Pw8BB69w6laVgd5nmmM37RNgAC/WrQIag2vZvX59YLw2lYu6bDEVdtmviVUkqdtVWrohg3bg7X39KZ+JBazD90jCMJqQB4CLx4dWeu7h7scJTKlSZ+pZRSJRYXl8KURxYz8/d91O3agDcjj0HkMQD6tQ6ke2hdbu/TDO/q+oheRaOJXymlVLFlZhle/3I9r6/cj9SqRsCQMABC6/kweUBLrugahEh+nzZRFYUmfqWUUsWy/mAsI6YtB0C8rfTx+qgu9GvVAH8fLydDUyWgiV8ppVSBMjKzeHPxTt5dspskY71ad0j7Rtw7oCWtG9d2ODp1NjTxK6WUyteumFMMeGVpTnfNDMMXd19At6b1HIxKnStN/EoppXKJjkvmoW828Nsuq7EeR5J469ouDB3a0tnAVKnQxK+UUgqA1IxMJn6xjoVbjuT0G+BRnf9NH4S3t9bhuwtN/EoppcjIzKL1I7/kdP/v+q70DqlLvXreDkalyoImfqWUqqKMMcxaG8X8jdEs2hoDApmHEvnjucE0DfJ3OjxVRjTxK6VUFWKMYc+xRB6atZGV+07k9E/eH0/3+r58+uplBAT4OBihKmua+JVSys2lpGey+VAcP6+PZvG2Ixw8kZwzLG3rCepEJfHZa0O58MJQB6NU5UUTv1JKuaGTiWlMW7KL2RuiORyfkmtYh9rePHRNJ3q3rM8//xymfftAvLz01bpVhSZ+pZRyI9+vi+Szvw6wZv/JnH7X9wghLKAWNU9l8MojS5jzzxHGdQqGlvXp0qWRg9EqJ2jiV0qpSurA8SRenL+NY6dSWbP/JOmZJmdYt9A6dAquw0PD2pCUkMbUqYt55501NGnix6xZ13LppfpMflWliV8ppSqJlPRM9hxNZFNUHG8v3c2eY4k5w3qE1yPQrwYhdX24NiKYZoG+OcMGXD6TFSsOMnlyL554oh9+fjWcCF9VEJr4lVKqEvh+XST3frX+jP4vXZP/9+537jxOUFBtfHy8eOGFAXh7V6Nr18blEaqq4DTxK6VUBRWblMZv24/y+uKd7LXv7sdcEEa/1oE0D/QlpN6Zj92lpGTwwgt/8Oyzf/DAA7158sn+XHBBSHmHriowTfxKKVVBGGOISUjl0z/38+P6qFyP3QHMm9SHtoV8EW/Roj3cddccdu48wahRHRg/PqKsQ1aVkCZ+pZRyUHpmFt+sjuSVhds5diot17CB7RpyXlhdru4eQr1a1Qudz/PP/8FDDy2mefO6zJ8/mkGDmpdl2KoS08SvlFIO2BWTwOM/beGP7C/gARe2qE+rhn50b1qXXs3qEeBbeCO8rCxDUlI6vr7VGT68FUlJ6Tz00IX6QR1VKE38SilVxmISUvjy74MkpKSzct8JMrMMmw/F5wwf0aUJjw1vX+Rdvav16w8zbtwcwsLq8OWXV9G+fQOefLJBWYSv3IwmfqWUKkWZWYat0fFEnkxm0dYjbIqKY9vhhJzhAbWqIyIMaNuAi1o34KZeTUs0/1On0nj88d947bW/qFfPmwkTzivtVVBuThO/UkqVgoMnkrj/6/W5PnwDIAI9w+txQ89QLu/cBBE562WsXBnFVVd9TWRkPGPHduO55wboZ3NViWniV0qpEoqKTWb1vhOsOxDL/uOJ7DhyiqjY0y3wr40IZmSXIILqetM0oNY5L88Yg4jQtKk/zZrV5auvrtZH9NRZ08SvlFL5MMYQFZtMclomu4+e4nhiGnM2RLMxMo6E1Ixc4/ZtFcjAdg3pHOLPFV3PfJnO2UpPz+S11/5iwYI9zJ8/moYNfVm6dEypzV9VTZr4lVIK67G6mIRUvvz7AOsOnmTVvpOkZWSdMZ6HwMT+LWjZ0JfzmwcQ6FvjnIrvC7JixUHGjZvNxo0xXH55axISUvH3r1nqy1FVjyZ+FyIyBHgd8ATeN8Y8n2d4KPAxUMce50FjzNxyD1QpdU4yswx/7j7O3E3RHI5L4ddtMWeM4+/tRb9WgQzv3AQvT6FZoC8Na9fEv4wflYuPT2XKlAW8995aQkJq88MP1zFiRJsyXaaqWjTx20TEE5gGDAQigVUi8pMxZovLaI8AXxtj3hKRdsBcIKzcg1VKlcip1AwWbz3Ckm0x7Iw5letROoAuIXVo08iP5oG++NWsxlXdg/Hy9HAkVi8vD5Yt28+UKefz2GP98PUt/iN+ShWHJv7TegC7jDF7AERkJjACcE38Bsh+X6Y/cKhcI1RKFSk5LZMl22P4ef0h/t57goSU9FyfqwVo17g2rRr6MqF/C1o08C2TovqS2L79GM888ztvv30ZPj5erF8/jho19PSsyobuWacFAQdduiOBnnnGeRxYICJ3A7WAAfnNSETGAmMBQkNDSz1QpdRpx0+l8tZvu9l6OJ6/9lgvx8nWu0UAnYLrULOaJ20a+9ElpA4Na1ecevLk5HSee+4PXnhhOT4+XkyYcISePYM16asypXvXafld8ps83dcDM4wxL4vI+cCnItLBGJOrBZAx5l3gXYCIiIi881BKnSVjDF+uPEhMQgp7jyWyfNexXO+37xBUm5C6PkSE1eOyTo0rVJLPa8GC3dx11xx27z7JjTd25OWXB9Gwoa/TYakqQBP/aZGA64OxwZxZlH8bMATAGPOniNQE6gNntgxSSpWKrCzD1sPxPD17K3/uOZ7Tv5qHIALnhdVldK+mDO/UBA8PZ4vsi8sYw1NPLcPDQ1i06CYuuaSZ0yGpKkQT/2mrgJYiEg5EAaOAG/KMcwC4BJghIm2BmsDRco1SqSpgU1QcCzYfZuHWGLZGn26IF1CrOsM6NubBoW2oVcmKwzMzs3j33TWMHNmGxo39+Oqrq6lXz5uaNSvXeqjKT/c4mzEmQ0QmAvOxHtX70BizWUSeBFYbY34C7gfeE5F7saoBxhhjtChfqVKQkp7Jqn0nuOuztTkvyKnr40UT/5oMaNeQyzs3oXvTuo43xDsba9dGM27cbFatOsTJkylMndqHJk38nA5LVVGa+F3Yz+TPzdPvPy5/bwF6l3dcSrmzlPRMxn66hmU7cheezZ/cl9aNKndyTEhI5T//WcIbb6wkMNCHL764klGjOjgdlqriNPErpcpVYmoGB08msWTbUeZsPMSmqNNF+aN7hfKv88No2bByJ/xsU6cuZtq0VYwbF8Gzz15CnToVt7Ghqjo08SulylRqRiZbDsXz3doo5m6M5nhiWq7hEU3rMqh9Q67uHlKi79FXVPv2xZKWlkmrVgE88khfRo/uRM+epff+fqXOlSZ+pVSpOpqQyh+7jhKfnMFXqw6yJTr3W/Ju6BlKeEAtWjfyo3vTupWukV5B0tIyeeWVP3nyyaVceGEoCxbcRMOGvvqInqpw3OOIU0o56nBcCu8u28O8TdFEx6XkGhZSz5s7+jSjQ5A/XUPqVMrGeUX5/ff9jB8/h82bj3LFFW14/fUhToekVIE08SulSiw5LZNpS3axfPcxNkbGkeHytryuoXWY0K8FrRv50aB2DWpU83Qw0rL33XdbueqqrwkN9eenn0YxfHhrp0NSqlCa+JVSxWKM4bO/D/DsnK0kp2fm9B/UriEicE33EAa0a+hghOXHGMOhQwkEBdVm6NAWPP10fyZP7kUtN2ijoNyfJn6lVIEyMrNYtvMoL8zbzvYjCTn9ezWrx8guQQxu34i6VSzZbd16lPHj53DgQBybNt2Fj48XDz/c1+mwlCo2t0z8IlIdCDXG7HI6FqUqk/TMLOZsiOaPXcf4dk1krmFNA3y4qlswt/cJx6e6W546CpWcnM4zz/zOiy8ux9e3Oi++OFDfuqcqJbfba0XkUuAVoDoQLiJdgMeMMVc4G5lSFdcP66J4eeF2Dp5IzukXVMeb+n41uKJLEwa1b0STOt4ORuisyMh4LrpoBnv2nOTmmzvz3/8OpEGDWk6HpdRZcbvEDzyJ9TndJQDGmH9EpIWzISlV8ew7lsj6yFgmzfwnp1/7JrUZ1rExt/QOq5J39Xmlp2fi5eVJkyZ+9O3blPffH07//uFOh6XUOXHHIzvdGBOb55EhfZ++UsCh2GSembuVXUdO5aqzB1h030W0aKDPnIP1QZ3p01fx3/+uYOXKO2jUyJePPhrhdFhKldlvaV0AACAASURBVAp3TPxbReRawMP+0t4k4C+HY1LKMYdik/ni7wN88uc+4lMycvqP6NKEi1oFclGrQAJ8azgXYAWzZs0h7rxzNmvWRDNoUHPSXZ5gUModuGPinwj8B8gCvsP62t5DjkakVDmLS05nU1QcS3cc5d1le3L6X9KmAdedF8KAtg0rzbfry0tWlmHy5F+YNm0VDRrUYubMq7j22vZu+cIhVbW5Y+IfbIx5AHggu4eIXIl1EaCUW0pJz+SPncd49/c9rNx7Itew2jWr8cSI9ozsEqRJrBAeHkJsbAp33RXB009fjL+/flBHuSdxt8/Ji8haY0y3PP3WGGO6OxFPRESEWb16tROLVm7uSHwKS7cf5evVB1m9/2SuYd2b1uWqbsF0CalDs8Ba1PRy77fnna09e04yadIvPPvsxXTs2JCsLKMlIRWAfc6OcDoOd+U2d/wiMhgYAgSJyCsug2pjFfsrVamdTExj2c6j/LAuiiXbc3+7/rywuozu1ZRuoXUJqefjUISVR1paJi+9tIKnnlqGl5cHO3Ycp2NHrf5QVYPbJH4gBtgEpACbXfonAA86EpFSZykjM4u/9pxgxe5jJKZm8NP6Q5xMSs81zqWdGjOkfSMubtPAbb5wVx6WLt3H+PFz2Lr1GFdf3Y7XXhtMUFBtp8NSqty4zdnCGLMOWCcinxtjUoqcQKkKJCU9k01Rcby5ZBer9p4gMS13S/Im/jWJ98jgqREdGNCuAQ38tP75bC1cuIfk5AzmzLmBYcNaOh2OUuXOHev4mwPPAO2AnLOjMaaVE/FoHb8qyL5jiTw1ewvbjyQQefL0G/PC69fi0o6NqVWjGhe1CqRVQ1+qeXo4GGnllpVl+OijdYSG+jNwYHOSk9MxBnx8vJwOTRVA6/jLltvc8buYATwNvAQMBW5B6/hVBZGemcWUb9Yzb9Nh0jKs3bKJf00ubFGfnuH1uDoimMb+VffVuKVt8+YYxo2bwx9/HOCmmzoxcGBzvL014auqzR0Tv48xZr6IvGSM2Q08IiK/Ox2Uqpr2Hkvk1YU7SErL4K89JziVevoFOld2C+Ka7iGc3zzAwQjdU1JSOk89tZSXXvoTf/8afPjh5YwZ08XpsJSqENwx8aeK9bDybhEZB0QBDRyOSVUx03/bxed/HSAq9nQR/nlhdalezYP+rRsw5oIwLb4vQ7NmbeH555dz661deOGFgdSvr086KJXNHRP/vYAvcA9WXb8/cKujEakq4dipVD5ZsY83fj39NegrugZxeecm9G+j155lLTIyns2bYxg8uAU33tiJtm0DiYho4nRYSlU4bpf4jTF/238mADcBiEiwcxEpd5aRmcWirTE8MGsDccmnH7drVr8WX487n/r6Dvwyl5GRxZtvruTRR5fg61udffsmUaNGNU36ShXArRK/iJwHBAF/GGOOiUh7rFf3Xgxo8lelIjYpjT93H+ezv/ezfNfxnP6N/WvywJA2DOvYmOrVtBi/PKxcGcW4cbNZt+4wQ4a0YNq0YdTQdxooVSi3OUJE5DngKmA9VoO+77G+zPcCMM7J2JR7SM3IZPxna/l1W0yu/rf0DuPOvs1ppO92L1c7dhynV6/3adzYj2++uYarrmqr3yJQqhjcJvEDI4DOxphkEakHHLK7tzscl6qEUtIzWbIthhNJaazce4Lo2BRW7rM+fuNboxq39A7jhp6h+uhdOTPGsHFjDJ06NaRVqwBmzBjJyJFtqF1bq1SUKi53SvwpxphkAGPMCRHZpklflUTkySTeW7aHr1dHkpzPN9iHd25Cj7C6XN8jVFvkO2DXrhNMmDCXX3/dy4YN42jbNpCbb+7sdFhKVTrulPibiUj2p3cFCHPpxhhzpTNhqYoqKjaZ/ccSeWnBdjZExpGRZb3FskUDX1o38qNjkD8Xt2lAHW8v6vvW0A+4OCQ1NYMXX1zOM8/8TvXqnrz66mBatdJ3Hyh1ttwp8V+Vp/tNR6JQFV5qRiYj3lzOtsMJOf18a1RjWMdG3NQrjI7B/g5Gp1ylp2dy3nnvsXFjDNdd155XXhlMkyZ+ToelVKXmNonfGLPY6RhUxXYiMY0Zy/fmes7+7dHdadnQl+aBvg5GpvKKj0+ldu0aeHl5ctttXWnduj5DhrRwOiyl3ILbJH6l8rPjSAJvLN7Jyr0niElIzek/6ZKWTB7QUluBVzBZWYYPPljLAw8s4osvrmLIkBZMmtTL6bCUciua+G0iMgR4HfAE3jfGPJ/PONcCjwMGWG+MuaFcg1TFtvbASZ6ds5XV+0/m9OvXOpAbeoQSEVaPerWqOxidys+GDUcYN242f/4ZyUUXNSUsrI7TISnlltw28YtIDWNMatFjgoh4AtOAgUAksEpEfjLGbHEZpyXwENDbGHNSRPQdrBVMemYW/1u8k69XR3I4PgUAn+qevHdzBL1b1Hc4OlWYZ55ZxmOP/Ubdut58/PFIbrqpk5bGKFVG3C7xi0gP4AOsd/SHikhn4HZjzN2FTNYD2GWM2WPPYybWewG2uIxzBzDNGHMSwBgTc8ZcVLnLzDI8PWcLS7cfZc+xxJz+bRvX5u3R3WgaUMvB6FRRjDGICA0b+nLLLV14/vkBBAToB3WUKktul/iBN4DLgB8AjDHrRaR/EdMEAQdduiOBnnnGaQUgIsuxqgMeN8b8UioRq7Py2/YYxny0CoBqHsLoXqF0CanLVd2C9G6xgjt4MI577vmFwYObM25cBLff3o3bb+/mdFhKVQnumPg9jDH785z4z3wbS275ZQmTp7sa0BLoh/Xe/99FpIMxJvaMmYmMBcYChIaGFjNsVVxr9p/g399uYM9R6w6/S0gdvr/rAk32lUBGRhZvvPE3//nPErKyDAMGhDsdklJVjjsm/oN2cb+x6+7vBnYUMU0kEOLSHYz1yt+84/xljEkH9orIdqwLgVV5Z2aMeRd4FyAiIiLvBYQ6B+//voen52wFrI/ivDW6O11CtBFYZbB69SFuv/0n1q8/wqWXtuTNN4dpAz6lHOCOiX88VnF/KHAEWGT3K8wqoKWIhANRwCggb4v9H4DrgRkiUh+r6H9PKcatChGXnM5zc7cyc5VVI/PWjd0Y2rGxw1Gpkjh5Mpljx5KYNetarriijZbQKOUQd0z8GcaYUSWZwBiTISITgflY9fcfGmM2i8iTwGpjzE/2sEEisgWr6uDfxpjjBc9VnYv0zCz+3nOCGSv2EXkyKddb9r6+83x6hNdzMDpVHMYYvvhiI5GR8TzwwIUMHNicXbvuoWZNdzztKFV5iDHuVRItIruB7cBXwHfGmIQiJilTERERZvXq1U6GUKkkpWXw3/nb+Wj5vlz9R3RpQscgf27tHa7vzK8Eduw4zl13zWHx4r307h3C0qVj8NQPG6liEpE1xpgIp+NwV2536W2MaS4iF2AV1z8hIv8AM40xMx0OTeWRnpnFxqg4/t5zggMnEvlrzwn2ujySd2nHxtw/qBXN9HW6lUZKSgbPP/8Hzz33B97e1Zg+fRhjx3bXpK9UBeJ2iR/AGLMCWCEijwOvAZ8DmvgriOOnUnlg1kYWbT1yxrB2jWvTIag2Dw9rh7+PlwPRqXOxb18szz77O9dc056XXx5Eo0Z60aZUReN2iV9EfLFevjMKaAv8CFzgaFAKgGlLdvHDuih2xpzK6Te6VyhXdQumfRN/qlfTu8LK6PDhU8yatYUJE3rQpk19tm+fSHh4XafDUkoVwO0SP7AJ+Bl40Rjzu9PBKEhJz2Twa8vYfzwJgGEdG3FeWD3GXBCmLbsrsawswzvvrOahhxaTnJzBsGEtCQ+vq0lfqQrOHRN/M2NMltNBKKtV98sLdvDmEuszuOH1a/HDXb21CN8N/PPPYcaNm83ff0dx8cXhTJ8+TBO+UpWE2yR+EXnZGHM/MEtEznhUwRhzpQNhVUmpGZm8unAnby/dndNPP4PrPpKT0xk06FMAPv30Cm68saP+rkpVIm6T+LEe3wN409EoqrCU9EwmzVzH/M2nG+0F1fFm7qQ++HvrXX5lZozh11/30r9/ON7eXsyadS0dOjSgbl1vp0NTSpWQ2yR+Y8xK+8+2xphcyd9+Oc/i8o+q6ohLSmfwa8tyPod7Z99m3DeoFTWqeTocmTpX+/fHcvfd8/j55x18/vmV3HBDR/r0aep0WEqps+Q2id/FrZx5139bPv1UKdgUFcdri3bmPJrXMcifWeMv0Bb6biA9PZNXX/2LJ55YCsBLLw3kmmvaORyVUupcuU3iF5HrsB7hCxeR71wG+QFnfEFPnZu9xxIZ9+kath85/WLERy5ty20Xhmt9r5u49tpv+eGHbVx+eWv+97+hhIb6Ox2SUqoUuE3iB1YCx7G+rDfNpX8CsM6RiNyQMYYPl+/jqdlbAOuTuE+OaE/HIH9N+G7gxIlkatasho+PF5Mm9WTMmM6MGNHG6bCUUqXIbRK/MWYvsBfra3yqlKWkZzL+szUs2X40p9+9A1oxaUBLB6NSpcUYw2efbeD++xdw661def75AfTrF+Z0WEqpMuA2iV9ElhpjLhKRk4Dr43wCGGOMfs7tLCSlZTDxi3X8ui0mp99V3YJ5+NK21KtV3cHIVGnZtu0Y48fP4bff9tGrVzDXX9/B6ZCUUmXIbRI/0N/+v76jUbiJORuimfDF2lz97h/YiokXt9AifTfy8cf/cMcdP1OrVnXeeecybr+9m379UCk35zaJ3+VtfSHAIWNMmohcCHQCPgPiHQuuEsnMMkz5Zj3fr4sCoFOwP9edF8KVXYPxrq6P5rmL9PRMvLw8iYhowvXXd+TFFwfQsKF+UEepqkCMOeMld5Wa/Rne84BQYCEwBwg3xlzmRDwRERFm9erVTiy6xHbFJDDglWU53V/feT49wrWGxJ1ERydw330LMMYwc+bVToejVL5EZI0xJsLpONyV29zxu8gyxqSLyJXAa8aYN0REW/UXIiU9k4lfrGXRVqseP7iuN4vvv0hfvuNGMjOzePvt1Uyd+iupqRlMndoHY4xW2yhVBblj4s8QkWuAm4CRdj99X2wB1uw/yVVvrcjpnnHLefRr3cDBiFRp27HjOKNHf8eqVYcYMKAZ06cPo2XLAKfDUko5xB0T/63AXVif5d0jIuHAlw7HVCF9vfog//ftBgDuG9iKu7Xhnlvy969BQkIaX3xxJaNGddDfWKkqzu3q+AFEpBrQwu7cZYzJcCqWilbHn56ZxdyN0Uya+U9Ov3du6s7g9o0cjEqVJmMM3323lS+/3MTXX1+Dh4eQlWW0tb6qNLSOv2y53R2/iPQBPgWisJ7hbyQiNxljljsbmfPiU9Lp9PiCnO5GtWsy554LCfCt4WBUqjTt3XuSiRPnMXfuTrp0aURMTCKNGvlq0ldK5XC7xA+8CgwzxmwBEJG2WBcCVfrq8eCJJPq8uCSne+2jA/UFPG4kLS2TV175kyefXIqnpwevvjqYiRN7UE0/lqSUysMdE3/17KQPYIzZKiJVNsNlZRleX7yT1xfvBODq7sH89+pOWs/rZjIzs3j//bUMHdqS118fQnBwbadDUkpVUO6Y+NeKyDtYd/kAN1JFP9JzNCGVK6YvJ/JkMgAT+jfn34P1gyvu4tixJF544Q+eeKI/Pj5erFx5B/XqeTsdllKqgnPHxD8OuAf4P6w6/mXA/xyNyAELNh9m7KdrAOjXOpAP/nUenlrP6xaMMXz88XqmTFlAXFwql1zSjCFDWmjSV0oVi1slfhHpCDQHvjfGvOh0PE6JiU/JSfovXdOZq7sHOxyRKi1bthxl/Pg5LFu2nwsuCOHtty+lY8eGToellKpE3Kblj4hMBX7AKtpfKCK3OhySI4wxXP/eXwDc0Sdck76bueeeeWzceIT33hvO77/foklfKVVi7nTHfyPQyRiTKCKBwFzgQ4djKldLtsfwyPebiIpNpm+rQKYOa+t0SKoU/PLLLjp3bkjjxn68995wfH2rExhYy+mwlFKVlNvc8QOpxphEAGPMUdxr3Yq0ZFsMt3y0iqjYZK7qFszbo7tpy/1K7tChBK699huGDv2cl16yXqscHl5Xk75S6py40x1/MxH5zv5bgOYu3RhjrnQmrLKVmWUY+8lqFm+zPrDz3s0RDGynxb+VWWZmFtOnr+Lhh38lPT2Lp5/uz5QpFzgdllLKTbhT4r8qT/ebjkRRzl5duCMn6f84oTedQ+o4HJE6V088sZSnnlrGoEHNmT59GM2b66eRlVKlx20SvzFmsdMxlLf0zCzeXLILgJ3PDMXLs0rVbriVuLgUTp5MISysDhMmnEf79oFce217ra5RSpU6zRSV2G0fWx//Gd0rVJN+JWWM4euvN9O27TRuuul7jDE0bOjLddfpV/SUUmVDs4ULERkiIttFZJeIPFjIeFeLiBERx97/Hx2XzLIdRwF4fHh7p8JQ52DPnpMMG/YF1133LY0b+/Hqq4M12SulypzbFPXnJSI1jDGpJRjfE5gGDAQigVUi8pPre//t8fyw3gz4d2nGWxJbo+MZ+vrvAEwd1oZqerdf6Sxduo8hQz7Hy8uD118fwoQJ5+Gpv6NSqhy43ZlGRHqIyEZgp93dWUSK88reHsAuY8weY0waMBMYkc94TwEvAimlFXNJ3TpjFQAPDW3D2L7NnQpDnYX4eOtatEePIG6/vStbt07gnnt6atJXSpUbdzzbvAFcBhwHMMasB/oXY7og4KBLd6TdL4eIdAVCjDGzC5uRiIwVkdUisvro0aMlib1Ip1IziI5LoZqHcOdFmvQri6NHExkz5ge6dn2HpKR0vL29+N//hhEUpF/RU0qVL3dM/B7GmP15+mUWY7r8KldNzkARD+BV4P6iZmSMedcYE2GMiQgMDCzGootn+a5jdHhsPoC+la+SyMoyfPDBWtq0mcYXX2xk1Kj2aDW+UspJ7ljHf1BEegDGrre/G9hRjOkigRCX7mDgkEu3H9AB+M1ugNUI+ElELjfGrC6VyAthjOHG961mBRP6N+fWC8PLepHqHB0/nsTIkV/xxx8H6NMnlLffvox27UrvQlAppc6GOyb+8VjF/aHAEWCR3a8oq4CWIhIORAGjgBuyBxpj4oD62d0i8hswpTySPsCK3ccBaOBXg38PblMei1RnyRiDiFC3rjcBAd58+OHljBnTRVvsK6UqBLdL/MaYGKykXdLpMkRkIjAf8AQ+NMZsFpEngdXGmJ9KOdRi2xAZm3O3P/ueC50KQxXDnDk7ePjhX/nll9E0auTLDz+UeFdUSqky5XaJX0Tew6VuPpsxZmxR0xpj5mJ91c+1338KGLffWYZYYpe/uRyARy5tSwO/muW1WFUCkZHxTJr0C999t5W2betz9GgijRr5Oh2WUkqdwe0SP1bRfraawBXkbq1fqeyKOQXABc0DuL1PM4ejUXkZY3j99b959NElZGZm8dxzl3DffedTvbqn06EppVS+3C7xG2O+cu0WkU+BhQ6Fc84mfrEWgLF9NelXRCLCqlWH6NMnlGnThhEeXtfpkJRSqlBul/jzEQ40dTqIs/Hz+kNsO5wAwEWttDV4RREbm8Ijj/zK2LHd6dSpIR98cDk1anhq4z2lVKXgdolfRE5yuo7fAzgBFPje/Yrsx3+iAFh0X19NKhWAMYaZMzdx773zOXo0iTZt6tOpU0Nq1nS7w0gp5cbc6owlVnbsjPU4HkCWMeaMhn6VQVJaBou2xhBevxYtGvg5HU6Vt3PncSZMmMvChXuIiGjCnDk30L17E6fDUkqpEnOrxG+MMSLyvTGmu9OxnKt3l+0B4PoeIUWMqcrDJ5+s5++/o3jzzaGMGxeh79ZXSlVabpX4bStFpJsxZq3TgZytrCzDjBX76N60rn6Ex0FLluxFROjXL4ypU/swfvx5NGmipS9KqcrNbRK/iFQzxmQAFwJ3iMhuIBHrHfzGGNPN0QBL4Id/oohNSmdI+0ZOh1IlxcQkMmXKAj79dAMDBjSjX78wvL298Pb2cjo0pZQ6Z26T+IGVQDdgpNOBnKv7vl4PwOVdtA65PGVlGd5/fy0PPLCIxMQ0HnmkD1On9nE6LKWUKlXulPgFwBiz2+lAzsWTP28BoGmADw1r61v6ytMPP2zjzjtnc9FFTXn77cto06Z+0RMppVQl406JP1BE7itooDHmlfIM5mwkp2Xy4fK9ACy67yKHo6kaEhPTWL/+CBdcEMLIkW348cdRDB/eSh+fVEq5LXdK/J6AL/adf2U0Z2M0AFMGtcJLW42XuZ9+2s7dd88jPj6VAwcm4+dXg8svb+10WEopVabcKfFHG2OedDqIs5WVZXh14Q4ALu2kdftl6eDBOO6+ex4//ridDh0a8MUXV+LnV8PpsJRSqly4U+KvtHf6AG/8upOo2GR6NatHeP1aTofjtqKi4mnXbjqZmVm88MIA7r23F15e+kEdpVTV4U6J/xKnAzgXm6LiAJhxSw+HI3FPUVHxBAXVJiioNk891Z+RI9sQFlbH6bCUUqrcuU1FsjHmhNMxnIu/9pzgguYB1NS7z1J18mQy48bNplmzN9i48QgAkyf30qSvlKqy3OmOv9LKyMziVGoGPtX15ygtxhg+/3wj99+/gOPHk5g0qacme6WUQhN/hRCXnA5Am0b6OtjSkJVluOyyL5g3bxc9egQxf/5ounTRtyAqpRRo4q8Q1kfGAmijvnOUnp6Jl5cnHh5C375NGT68FWPHdtcP6iillAs9I1YA05ZYLxsc1L6hw5FUXgsX7qZdu+n88ssuAB588ELGjz9Pk75SSuWhZ0WHHYpNZs3+k9T3rYFfTf0ITEkdPnyKG26YxaBBnwHg46PbUCmlCqNF/Q77fl0UAE9c3t7hSCqfTz5Zzz33zCM5OYPHHruIBx+8kJo1dZdWSqnC6FnSYTuOJAAwrKM2PiuptLRMundvwvTpw2jdWj+oo5RSxaFF/Q7bczQRQD8KUwwJCancf/983ntvDQC33daVRYtu0qSvlFIloInfYfuPJxJUx9vpMCo0Ywzff7+Vdu2m88orf7Fzp/WuJhHRCyallCohLep3kDGG+JQMOofoi2UKsn9/LBMnzmP27B107NiAr7++mvPPD3E6LKWUqrQ08Tto2c5jAHQI8nc4koprx47jLFmyl5deGsg99/TUD+oopdQ50sTvoHkbowG4oUeow5FULCtWHGTt2mgmTuzBwIHN2b9/MgEBPk6HpZRSbkHr+B00c9VBAmpVJ6SeJjWAEyeSGTv2Z3r3/pBXX/2LlJQMAE36SilVijTxOyQ2KQ2A5oG+DkfiPGMMn3yyntat3+TDD9cxZcr5rF8/Tp/JV0qpMqBnVodsORQPwMB2+prevXtjue22n4iIaMI771xGp066TZRSqqzoHb9DDpxIAqB7WF2HI3FGcnI6M2duAqBZs7osX34ry5ffqklfKaXKmCZ+m4gMEZHtIrJLRB7MZ/h9IrJFRDaIyGIRaXouyzuVatVfN62C9fsLFuymY8e3uP76WaxffxiAHj2C8PDQZ/KVUqqsaeIHRMQTmAYMBdoB14tIuzyjrQMijDGdgG+BF89lmYu2HgGgVo2qU9sSHZ3AqFHfMnjwZ3h6erB48c107qyvKlZKqfJUdbJO4XoAu4wxewBEZCYwAtiSPYIxZonL+H8Bo892YemZWWyNtt7RX7OKPJeekZHFBRd8SHR0Ak8+2Y//+7/e1KhCFz1KKVVR6JnXEgQcdOmOBHoWMv5twLyCBorIWGAsQGjomc/oz95wiLjk9CrxRb7Nm2No2zaQatU8mD59GC1a1KNlywCnw1JKqSpLi/ot+VUum3xHFBkNRAD/LWhmxph3jTERxpiIwMDAM4Znf5hnZJegswq2MoiPT2Xy5F/o1OltPvxwHQBDh7bUpK+UUg7TO35LJOD6Avhg4FDekURkAPAwcJExJvVsF7bzyCkA/H28znYWFZYxhlmztjJp0i9ERycwfnwEV1+dt7mEUkopp2jit6wCWopIOBAFjAJucB1BRLoC7wBDjDEx57KwTGOoXs09C1vuvnse06atokuXRnz33bX07BnsdEhKKaVcaOIHjDEZIjIRmA94Ah8aYzaLyJPAamPMT1hF+77AN/anYA8YYy4/m+XtOJLA+c3cp8g7LS2TrCxDzZrVuOKKNjRvXpe77+5JNTe9uFFKqcpME7/NGDMXmJun339c/h5QGss5mZjG/uNJ9G/doDRm57jff9/PuHFzGD68Fc8/P4BLLmnGJZc0czospZRSBdBbsnK2M8aq3+/etHK/se/YsSRuvfVH+vadQWJiGn366BcGlVKqMtA7/nJ2JD4FgPD6tRyO5Oz98ssuRo/+jri4VB54oDePPtqXWrWqOx2WUkqpYtDEX84WbrHe2NfYv6bDkZScMQYRITTUn06dGvLGG0Pp0ME9qiyUUqqq0MRfzk4mpVGjmgcBvjWcDqXYkpLSefrpZURGxvPJJ1fQrl0gv/76L6fDUkopdRa0jr+c/XMwtlLV78+bt5MOHabz3HN/4OEhZGRkOR2SUkqpc6B3/OXoaEIqCSkZNPb3djqUIh05coqJE+fx7bdbaNOmPkuW/It+/cKcDksppdQ50sRfjvYft17V27tFxX+GX0RYseIgTz/dn3//uzfVq1eNjwkppZS708RfjlbvPwlAuya1HY4kf6tXH+K999bw1luX0aBBLXbvvoeaNXUXUUopd6J1/OVoQ2Qs1T09aNOoYiX+uLgU7r57Lj16vMfPP+9g717rAkWTvlJKuR89s5ejuOR02gdVnKRvjOGbb7YwefIvHD58igkTzuPppy/GvxI+aqiUUqp4NPGXo+2HT9GmkZ/TYeRIS8tk6tTFNG7sx08/XU9ERBOnQ1JKKVXGNPGXo9T0TBLTMpyNITWDt95azdix3fHx8WLRopsJDq6tH9RRSqkqQhN/OYlJSCEhNYOuIc49w//bb/sYP34O27YdIzDQhxtv7ERYWB3H4lFKKVX+NPGXBwBoOwAAFAFJREFUk49X7AOggwN1/EePJvLvfy/k44/XEx5ehzlzbmDYsJblHoeq+NLT04mMjCQlJcXpUFQVULNmTYKDg/Hy8nI6lCpFE3852X44AYCRXYLKfdm33/4z8+btZOrUC3n44b74+OhBpvIXGRmJn58fYWFhiIjT4Sg3Zozh+PHjREZGEh4e7nQ4VYpW7JaT6LgUwuvXwsOjfE6mmzbFEB1tXWz8978D+eefcTzzzCWa9FWhUlJSCAgI0KSvypyIEBAQoKVLDtDEX05S0jMpj5yfmJjGgw8uomvXd3j00SUAtGoVQLt2gWW/cOUWNOmr8qL7mjO0qL8cZGRmsftoIjf0DC3T5cyevYOJE+eyf38ct97ahRdeGFCmy1NKKVX56B1/OdgQFQdA80DfMlvGG2/8zfDhX1KrVnWWLRvDBx+MICDAp8yWp1RZ8fT0pEuXLnTo0IHhw4cTGxubM2zz5s1cfPHFtGrVipYtW/LUU09hjMkZPm/ePCIiImjbti1t2rRhypQpTqxCodatW8ftt9+eq9+IESM4//zzc/UbM2YM33777f+3d+/RUVX3Ase/PwmSIAEFSi4tlYBESAghBGjhUm0gJloeQVGJaBW8lJYgaPGSFpesRYsuGxCFInKp7XUFLyWJL4QbWysvLUVeAUKISY25mAaUhhQCBElCHvv+cU4mQzKYCcnMaOb3WWvWmnNmn3N+81uT7Nn77Nn7in3dujX+DyksLGTixIkMGjSI8PBwpk+fTmlpaZtiO3v2LPHx8YSFhREfH095ebnLciUlJSQkJBAeHk5ERATFxcUA7Ny5k5iYGCIjI5k5cya1tdbPl7Oysli6dGmbYlPtRyt+L/jwkzIAJkf1bdfz1tbW889/XgQgKWkoK1bcwZEjP+O22/q363WU8qagoCBycnLIy8ujZ8+evPzyywBUVlaSmJjI4sWLKSws5OjRo3z00UesW7cOgLy8PObPn8/GjRspKCggLy+PgQMHtmtsDRVZWzz33HMsWLDAsX3u3DkOHz7MuXPn+Oyzz9w6R1VVFZMmTSI5OZmioiIKCgpITk6mrKysTbGlpqYSFxfHp59+SlxcHKmpqS7LPfLII6SkpFBQUMCBAwfo06cP9fX1zJw5k4yMDPLy8ujfvz8bNmwAYNKkSWzdupVLly61KT7VPrSr3wv+dbGanjdcT0j39psK98CBz/nZz7IIDAxgz57/ICSkGykp49rt/Er9+n8/Jv+LC+16zohvd2fplKFulx87diy5ubkAbNq0iXHjxpGQkABA165dWbt2LbGxsTz22GOsWLGCp59+miFDhgAQEBDAvHnzmp3z4sWLLFiwgOzsbESEpUuXcu+999KtWzcuXrS+SL/55ptkZWWRlpbGrFmz6NmzJ0eOHCE6OprNmzeTk5PDjTdac2AMGjSIPXv2cN111zF37lxKSkoAWL16NePGXfk3WVFRQW5uLsOHD3fse+utt5gyZQohISFkZGTw1FNPtZiXTZs2MXbsWKZMmeLYN378eLfzejVbtmzhgw8+AGDmzJnExsayfPnyK8rk5+dTW1tLfHw80NgLUVZWRpcuXbj11lsBiI+P5ze/+Q2zZ89GRIiNjSUrK4vp06e3OU7VNtri94KPv7hA93Za8ObcuSrmzXuXMWP+wOnTX7Jo0Vh0fIzqiOrq6tixYweJiYmA1c0/cuTIK8rccsstXLx4kQsXLpCXl9fsdVeeeeYZevTowbFjx8jNzWXChAktHlNYWMj27dtZtWoVU6dOZfPmzQDs37+f0NBQQkJCeOKJJ1i4cCEHDx7krbfeatadD5CdnU1kZOQV+9LT05kxYwYzZswgPT29xVgAt99rRUUF0dHRLh/5+fnNypeWltK3r9Uz2bdvX06fPu0yFzfeeCPTpk1jxIgRpKSkUFdXR+/evampqSE7OxuwvjydOHHCcdyoUaPYvXu3W+9PeZa2+L2grKK6XSbuOXr0n9x550bKyi7x+OPfZ9my8XTv3qUdIlSquda0zNtTZWUl0dHRFBcXM3LkSEfL0hhz1VHgrRkdvn37djIyMhzbN93U8mya999/P506dQIgKSmJZcuW8eijj5KRkUFSUpLjvM6V6YULF6ioqCA4uHF9jlOnTvGtbzX+wqa0tJSioiJ+8IMfICIEBASQl5dHZGSky/fU2lHwwcHB5OTktOqYltTW1rJ7926OHDnCzTffTFJSEmlpacyePZuMjAwWLlxIdXU1CQkJBAQ0VjF9+vThiy++aNdY1LXRFr8XfH6ukm8FX3sFXVNTB1g/y7v99v4cPDiH1avv0kpfdUgN9/j/8Y9/cPnyZcc9/qFDhzpakw2OHz9Ot27dCA4OZujQoRw6dKjF81/tC4Tzvqa/Lb/hhhscz8eOHUtRURFlZWW88847TJs2DYD6+nr27t1LTk4OOTk5fP7551dU+g3vzfncmZmZlJeXM2DAAEJDQykuLnZ8KenVq9cVg+vOnj1L7969Hblw5722tsUfEhLCqVOnAOtLSp8+fZqV6devHyNGjGDgwIEEBARw9913c/jwYUdudu/ezYEDB7j99tsJC2ucIbSqqoqgoKAWY1aepxW/h1Vetirt6+3WQmtUV9eybNmHREWt59KlGoKCOvP66/cTE9O+gwSV+jrq0aMHa9asYeXKldTU1PDQQw/xt7/9je3btwNWz8Djjz/OL37xCwBSUlJ47rnnKCwsBKyK+MUXX2x23oSEBNauXevYbqhcQ0JCKCgooL6+3tGV74qIcM899/Dkk08SHh5Or169XJ7XVUs7PDycoqIix3Z6ejrvvfcexcXFFBcXc+jQIUfFHxsbS2ZmJpcvXwYgLS3NcR//wQcf5KOPPuLdd991nOu9997j2LFjV1yvocXv6hEREdEsvsTERMeAvA0bNjB16tRmZUaPHk15ebljIOHOnTsd52q4NVBdXc3y5cuZO3eu47jCwsJmtzmUb2jF72HnKmsAmBbTuql6d+78jKio9Sxd+gHR0f9GVZVvV/VTyhdGjBjB8OHDycjIICgoiC1btvDss88yePBghg0bxujRo5k/fz4AUVFRrF69mhkzZhAeHk5kZKSj9epsyZIllJeXExkZyfDhw9m1y5roKjU1lcmTJzNhwgTHfe6rSUpKYuPGjY5ufoA1a9aQnZ1NVFQUERERrF+/vtlxQ4YM4fz581RUVFBcXExJSQljxoxxvD5gwAC6d+/O/v37mTx5MrfddhsjR44kOjqaPXv2OAbaBQUFkZWVxUsvvURYWBgRERGkpaW5bKG3xuLFi9m2bRthYWFs27aNxYsXA9bYhIYxC506dWLlypXExcUxbNgwjDHMmTMHgOeff57w8HCioqKYMmXKFeMndu3axaRJk9oUn2of4vwbWNX+eoWGm5FPrOf9hT90q/yXX15m7tx32bgxl4EDb2LduonceecgD0eplKWgoIDw8HBfh9GhrVq1iuDgYJeD/zqq0tJSHnzwQXbs2NHsNVefORE5ZIwZ5a34/I22+D3sYnUtYX2CWy5oCwrqzKlTFSxZcht5ecla6SvVwSQnJ9Oli3+NzykpKeGFF17wdRjKpqP6vaClhXlyc0v55S+38+qrifTtG8z77z/stcV8lFLeFRgYyMMPP+zrMLxq9OjRvg5BOdEWvxcMDnE9Ve/Fi5dJSXmfmJjfkZ39BYWFZ4CWvygo5Ul6+095i37WfENb/F6QNLr54jxbt37C/Pl/4sSJC8yZE0Nq6h307Kk/dVG+FRgYyJkzZ3RpXuVxxhjOnDlDYGD7zWiq3KMVvxfc1LVzs31vvJFPjx6BpKffy7hxnl21Tyl39evXj5MnT7Z5znel3BEYGEi/fv18HYbf0VH9Htalb5ipPvUpNTV1rFmzn/j4W4iKCuHChWqCggLo3Ln1v+9XSqmOTEf1e5be43ciIneJyCciUiQii1283kVEMu3X94tIaEvn7Hzddezde4JRo37PokXbyMzMA6B79y5a6SullPI6rfhtItIJeBn4ERABzBCRplNbzQbKjTGDgFXAclpQV1fPuHGvcvZsJW+/PZ1nn215QRCllFLKU7Tib/Q9oMgYc9wYcxnIAJrOVzkV2GA/fxOIkxZGQNVcqmHhwjHk58/jnnvCdcCUUkopn9LBfY2+A5xw2j4JfP9qZYwxtSJyHugF/Mu5kIj8FPipvVn94ot35bmYMtwf9aZJrvyY5qKR5qKR5sIy2NcBdGRa8Tdy1RRvOvLRnTIYY14BXgEQkWwdpGLRXDTSXDTSXDTSXFhEJLvlUupaaVd/o5PAd522+wFNF492lBGRAKAHcNYr0SmllFLtQCv+RgeBMBEZICLXAw8AW5uU2QrMtJ/fB+w0+ntIpZRS3yDa1W+z79nPB/4CdAJeNcZ8LCLLgGxjzFbgv4H/EZEirJb+A26c+hWPBf3No7lopLlopLlopLmwaB48SCfwUUoppfyIdvUrpZRSfkQrfqWUUsqPaMXfDjwx1e83lRu5eFJE8kUkV0R2iEh/X8TpDS3lwqncfSJiRKTD/ozLnVyIyHT7s/GxiGzydoze4sbfyM0isktEjth/JxN9Eac3iMirInJaRPKu8rqIyBo7V7kiEuPtGDskY4w+2vDAGgj4f8BA4HrgKBDRpMw8YL39/AEg09dx+zAX44Gu9vNkf86FXS4Y+CuwDxjl67h9+LkIA44AN9nbfXwdtw9z8QqQbD+PAIp9HbcH83E7EAPkXeX1icCfseZQGQPs93XMHeGhLf6288hUv99QLebCGLPLGHPJ3tyHNV9CR+TO5wLgGWAFUOXN4LzMnVzMAV42xpQDGGNOezlGb3EnFwbobj/vQfP5RDoMY8xf+eq5UKYCrxnLPuBGEenrneg6Lq34287VVL/fuVoZY0wt0DDVb0fjTi6czcb6Nt8RtZgLERkBfNcYk+XNwHzAnc/FrcCtIrJHRPaJyF1ei8673MnFr4Afi8hJ4E/AAu+E9rXU2v8pyg36O/62a7epfjsAt9+niPwYGAX80KMR+c5X5kJErsNa4XGWtwLyIXc+FwFY3f2xWL1Au0Uk0hhzzsOxeZs7uZgBpBljXhCRsVhzh0QaY+o9H97Xjr/87/QqbfG3nU7128idXCAidwBPA4nGmGovxeZtLeUiGIgEPhCRYqz7l1s76AA/d/9GthhjaowxnwGfYH0R6GjcycVs4HUAY8xeIBBr8R5/5Nb/FNU6WvG3nU7126jFXNjd27/DqvQ76n1caCEXxpjzxpjexphQY0wo1niHRGNMR1ycxJ2/kXewBn4iIr2xuv6PezVK73AnFyVAHICIhGNV/GVejfLrYyvwiD26fwxw3hhzytdBfdNpV38bGc9N9fuN42Yunge6AW/Y4xtLjDGJPgvaQ9zMhV9wMxd/ARJEJB+oA1KMMWd8F7VnuJmL/wR+LyILsbq1Z3XQhgIiko51e6e3PaZhKdAZwBizHmuMw0SgCLgEPOqbSDsWnbJXKaWU8iPa1a+UUkr5Ea34lVJKKT+iFb9SSinlR7TiV0oppfyIVvxKKaWUH9GKX6k2EpE6EclxeoR+RdnQq61E1sprfmCv8HbUnuZ28DWcY66IPGI/nyUi33Z67Q8iEtHOcR4UkWg3jvm5iHRt67WVUq5pxa9U21UaY6KdHsVeuu5DxpjhWAtAPd/ag40x640xr9mbs4BvO732E2NMfrtE2RjnOtyL8+eAVvxKeYhW/Ep5gN2y3y0ih+3Hv7soM1REDti9BLkiEmbv/7HT/t+JSKcWLvdXYJB9bJy9jvsxe63zLvb+VHut+1wRWWnv+5WILBKR+7DWTfijfc0gu6U+SkSSRWSFU8yzROSla4xzL04LrIjIf4lItoh8LCK/tvc9jvUFZJeI7LL3JYjIXjuPb4hItxauo5T6ClrxK9V2QU7d/JvtfaeBeGNMDJAErHFx3Fzgt8aYaKyK96Q9RWsSMM7eXwc81ML1pwDHRCQQSAOSjDHDsGbmTBaRnsA9wFBjTBTwrPPBxpg3gWyslnm0MabS6eU3gWlO20lA5jXGeRfW1LwNnjbGjAKigB+KSJQxZg3WXOzjjTHj7el7lwB32LnMBp5s4TpKqa+gU/Yq1XaVduXnrDOw1r6nXYc193xTe4GnRaQf8LYx5lMRiQNGAgftKY2DsL5EuPJHEakEirGWbh0MfGaMKbRf3wA8BqwFqoA/iMi7gNvLABtjykTkuD1P+qf2NfbY521NnDdgTVEb47R/uoj8FOv/UF8gAshtcuwYe/8e+zrXY+VNKXWNtOJXyjMWAqXAcKyetaqmBYwxm0RkPzAJ+IuI/ARrGdINxpin3LjGQ86L+ohIL1eF7Pnhv4e18MsDwHxgQiveSyYwHfg7sNkYY8Sqhd2OEzgKpAIvA9NEZACwCBhtjCkXkTSsxWiaEmCbMWZGK+JVSn0F7epXyjN6AKfsNdQfxmrtXkFEBgLH7e7trVhd3juA+0Skj12mp4j0d/OafwdCRWSQvf0w8KF9T7yHMeZPWAPnXI2sr8BaKtiVt4G7sdaJz7T3tSpOY0wNVpf9GPs2QXfgS+C8iIQAP7pKLPuAcQ3vSUS6ioir3hOllJu04lfKM9YBM0VkH1Y3/5cuyiQBeSKSAwwBXrNH0i8B3heRXGAbVjd4i4wxVVirl70hIseAemA9ViWaZZ/vQ6zeiKbSgPUNg/uanLccyAf6G2MO2PtaHac9duAFYJEx5ihwBPgYeBXr9kGDV4A/i8guY0wZ1i8O0u3r7MPKlVLqGunqfEoppZQf0Ra/Ukop5Ue04ldKKaX8iFb8SimllB/Ril8ppZTyI1rxK6WUUn5EK36llFLKj2jFr5RSSvmR/weg7rxyhqL8uAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "auroc = get_roc(clf_reduced_tuned, y_test, X_test_pca, \n", + " \"SVM reduced features and tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Of 2020 Defaulters, the SVM reduced features and tuning RBF kernal identified 727\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Predicted01
Actual
06383346
11293727
\n", + "
" + ], + "text/plain": [ + "Predicted 0 1\n", + "Actual \n", + "0 6383 346\n", + "1 1293 727" + ] + }, + "execution_count": 87, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#confusion matrix\n", + "confusion(y_test,clf_reduced_tuned.predict(X_test_pca), \"SVM reduced features and tuning RBF kernal\")" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.95 0.89 6729\n", + " 1 0.68 0.36 0.47 2020\n", + "\n", + " accuracy 0.81 8749\n", + " macro avg 0.75 0.65 0.68 8749\n", + "weighted avg 0.80 0.81 0.79 8749\n", + "\n" + ] + } + ], + "source": [ + "print(classification_report(y_test, clf_reduced_tuned.predict(X_test_pca)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From this, we can conclude that fitting SVM model with PCA-reduced features but no parameter tuning is most accurate based on Recall value" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Thus, fitting an SVM with PCA-reduced features with default gamma = 1/13 and C = 1 with kernal = 'rbf' is the best model. However, this is only for rbf kernel." + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [ + { + "ename": "ValueError", + "evalue": "X.shape[1] = 44 should be equal to 13, the number of features at training time", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 1\u001b[0m evaluation.loc[4] = ([\"SVM\" , \n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mclassification_report\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my_test\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mclf_reduced_tuned\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_test\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moutput_dict\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"1\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"recall\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 3\u001b[0m auroc])\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36mpredict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 572\u001b[0m \u001b[0mClass\u001b[0m \u001b[0mlabels\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0msamples\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 573\u001b[0m \"\"\"\n\u001b[1;32m--> 574\u001b[1;33m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 575\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclasses_\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mintp\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 576\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36mpredict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 320\u001b[0m \u001b[0my_pred\u001b[0m \u001b[1;33m:\u001b[0m \u001b[0marray\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mshape\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mn_samples\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 321\u001b[0m \"\"\"\n\u001b[1;32m--> 322\u001b[1;33m \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_validate_for_predict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 323\u001b[0m \u001b[0mpredict\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_sparse_predict\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_sparse\u001b[0m \u001b[1;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dense_predict\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 324\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py\u001b[0m in \u001b[0;36m_validate_for_predict\u001b[1;34m(self, X)\u001b[0m\n\u001b[0;32m 472\u001b[0m raise ValueError(\"X.shape[1] = %d should be equal to %d, \"\n\u001b[0;32m 473\u001b[0m \u001b[1;34m\"the number of features at training time\"\u001b[0m \u001b[1;33m%\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 474\u001b[1;33m (n_features, self.shape_fit_[1]))\n\u001b[0m\u001b[0;32m 475\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 476\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mValueError\u001b[0m: X.shape[1] = 44 should be equal to 13, the number of features at training time" + ] + } + ], + "source": [ + "evaluation.loc[4] = ([\"SVM\" , \n", + " classification_report(y_test, clf_reduced_tuned.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Neural Networks\n", + "We will now use the train and test sets as defined above and attempt to implement a neural network model on the data\n", + "\n", + "#### Theory\n", + "A neural network is comprised of many layers of perceptrons that take in a vector as input and outputs a value. The outputs from one layer of perceptrons are passed into the next layer of perceptrons as input, until we reach the output layer. Each perceptron combines its input via an activation function. \n", + "\n", + ".\n", + "\n", + "\n", + "![image.png](https://www.researchgate.net/profile/Leslaw_Plonka/publication/260080460/figure/fig1/AS:340931325775876@1458295770470/A-simple-neural-network-diagram.png)\n", + "\n", + "\n", + "The network is at first randomly initialised with random weights on all its layers. Training samples are then passed into the network and predictions are made. The training error (difference between the actual value and the predicted value) is used to recalibrate the neural network by changing the weights. The change in weights is found via gradient descent, and then backpropogated through the neural network to update all layers.\n", + "\n", + "\n", + "This process is repeated iteratively until the model converges (i.e. it cannot be improved further).\n", + "\n", + "#### Training\n", + "Here we create an instance of our model, with 5 layers of 26 neurons each, identical to that of our training data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.neural_network import MLPClassifier" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "mlp = MLPClassifier(hidden_layer_sizes=(26,26,26,26,26), activation = \"logistic\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mlp.fit(X_train,y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "predictions = mlp.predict(X_test)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "confusion(y_test,predictions,\"Neural Network (5x26)\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "auroc = get_roc(mlp, y_test, X_test, \"Neural Network (5x26)\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(classification_report(y_test,predictions))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "evaluation.loc[5] = ([\"Neural Network\" , \n", + " classification_report(y_test, mlp.predict(X_test), output_dict = True)[\"1\"][\"recall\"],\n", + " auroc])\n", + "\n", + "evaluation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Deep Learning\n", + "\n", + "#### Theory\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from numpy import loadtxt\n", + "from keras.models import Sequential\n", + "from keras.layers import Dense\n", + "\n", + "# define the keras model\n", + "model = Sequential()\n", + "model.add(Dense(12, input_dim=26, activation='relu'))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1, activation='sigmoid'))\n", + "# compile the keras model\n", + "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n", + "# fit the keras model on the dataset\n", + "model.fit(X_train, y_train, epochs=10, batch_size=10)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# evaluate the keras model\n", + "#recall, accuracy = model.evaluate(df1, target)\n", + "#print('Accuracy: %.2f' % (accuracy*100))\n", + "#print('Recall: %.2f' % (recall*100))\n", + "\n", + "predictions = list(model.predict(X_test).ravel() > 0.5)\n", + "print(classification_report(y_test,predictions))" + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "BT2101 disrudy ", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.1" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +}