{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# DS 2500 Day 12 \n", "\n", "Feb 21, 2023\n", "\n", "content:\n", "- Ensuring meaningful distances in data\n", "- K-NN classifier\n", "\n", "admin:\n", "- hw due friday\n", "- project proposal due next monday\n", "- workshopping a student project\n", "- install `plotly` & `sklearn` via pip (see below) for today's notes\n", "\n", " pip3 install plotly sklearn\n", " \n", " some mac / anaconda students had trouble in the first section via pip but were successful when using anaconda's own installation tool (see piazza for detail)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Representing data (samples & features)\n", "To describe a collection of **samples** we record a set of **features** for each sample.\n", "\n", "For example, when describing penguins:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
speciesislandbill_length_mmbill_depth_mmflipper_length_mmbody_mass_gsex
0AdelieTorgersen39.118.7181.03750.0Male
1AdelieTorgersen39.517.4186.03800.0Female
2AdelieTorgersen40.318.0195.03250.0Female
4AdelieTorgersen36.719.3193.03450.0Female
5AdelieTorgersen39.320.6190.03650.0Male
\n", "
" ], "text/plain": [ " species island bill_length_mm bill_depth_mm flipper_length_mm \\\n", "0 Adelie Torgersen 39.1 18.7 181.0 \n", "1 Adelie Torgersen 39.5 17.4 186.0 \n", "2 Adelie Torgersen 40.3 18.0 195.0 \n", "4 Adelie Torgersen 36.7 19.3 193.0 \n", "5 Adelie Torgersen 39.3 20.6 190.0 \n", "\n", " body_mass_g sex \n", "0 3750.0 Male \n", "1 3800.0 Female \n", "2 3250.0 Female \n", "4 3450.0 Female \n", "5 3650.0 Male " ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import seaborn as sns\n", "\n", "df_penguin = sns.load_dataset('penguins')\n", "\n", "# discard all rows which are missing any data\n", "df_penguin.dropna(axis=0, inplace=True)\n", "\n", "df_penguin.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each penguin is a sample for which we've observed 7 features:\n", "\n", "Quantitative:\n", "- bill_length_mm\n", "- bill_depth_mm\n", "- flipper_length_mm\n", "- body_mass_g\n", "\n", "Nominal:\n", "- species\n", "- island\n", "- sex \n", "\n", "Let us represent the quantitative data as an array. \n", "- We'll return to those Nominal features later" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Samples as vectors" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_mmbill_depth_mmflipper_length_mmbody_mass_g
039.118.7181.03750.0
140.217.9194.03700.0
240.318.0195.03250.0
436.719.3193.03450.0
539.320.6190.03650.0
\n", "
" ], "text/plain": [ " bill_length_mm bill_depth_mm flipper_length_mm body_mass_g\n", "0 39.1 18.7 181.0 3750.0\n", "1 40.2 17.9 194.0 3700.0\n", "2 40.3 18.0 195.0 3250.0\n", "4 36.7 19.3 193.0 3450.0\n", "5 39.3 20.6 190.0 3650.0" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# only focus on numerical features (for now)\n", "col_num_list = 'bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g'\n", "df_penguin_num = df_penguin.loc[:, col_num_list]\n", "\n", "# for pedagogical reasons, we need penguin1 to have slightly different values\n", "df_penguin_num.iloc[1, :] = [40.2, 17.9, 194.0, 3700]\n", "\n", "df_penguin_num.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Individual samples (penguins) are considered, mathematically, as vectors:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 39.1, 18.7, 181. , 3750. ])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num.iloc[0, :].values" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 39.1, 18.7, 181. , 3750. ])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "\n", "penguin0 = np.array(df_penguin_num.iloc[0, :])\n", "penguin0" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Distances between samples\n", "Many ML tools require that these vectors have meaningful distances between them. By \"meaningful\", we mean:\n", "- large distances suggest samples are different\n", "- small distances suggest samples are similar\n", "\n", "Computing distance between two vectors $x = \\begin{bmatrix} x_1 \\\\ x_2 \\end{bmatrix}$ and $x' = \\begin{bmatrix} x_1' \\\\ x_2' \\end{bmatrix}$:\n", "\n", "$$||x - x'||_2 = \\sqrt{\\sum_i (x_i - x_i')^2}$$\n", "\n", "In words, to compute the distance between two vectors:\n", "- we square the differences of each element\n", "- add these values together\n", "- compute the square root of this sum\n", "\n", "How similar is penguin0 to penguin1?" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm 39.1\n", "bill_depth_mm 18.7\n", "flipper_length_mm 181.0\n", "body_mass_g 3750.0\n", "Name: 0, dtype: float64" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "penguin0 = df_penguin_num.iloc[0, :]\n", "penguin0" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm 40.2\n", "bill_depth_mm 17.9\n", "flipper_length_mm 194.0\n", "body_mass_g 3700.0\n", "Name: 1, dtype: float64" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "penguin1 = df_penguin_num.iloc[1, :]\n", "penguin1" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "51.68026702717392" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sq_diff_per_feat = [(39.1 - 40.2) ** 2, (18.7 - 17.9) ** 2, (181 - 194) ** 2, (3750 - 3700) ** 2]\n", "dist01_slow = sum(sq_diff_per_feat) ** .5\n", "dist01_slow" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In and of itself, this distance isn't too insightful ... the penguins are 50-ish (units?) apart? \n", "\n", "The value becomes more useful when compared to other distances: Is penguin 1 more similar to penguin 0 or penguin 2?" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "distance between penguin0 and penguin1: 51.680\n", "distance between penguin1 and penguin2: 450.001\n" ] } ], "source": [ "vec_penguin0 = np.array(df_penguin_num.iloc[0, :])\n", "vec_penguin1 = np.array(df_penguin_num.iloc[1, :])\n", "vec_penguin2 = np.array(df_penguin_num.iloc[2, :])\n", "\n", "# a quicker, equivilent way to compute distance\n", "dist01 = np.linalg.norm(vec_penguin0 - vec_penguin1)\n", "dist12 = np.linalg.norm(vec_penguin1 - vec_penguin2)\n", "\n", "print(f'distance between penguin0 and penguin1: {dist01:.3f}')\n", "print(f'distance between penguin1 and penguin2: {dist12:.3f}')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Interpretting Distances\n", "(And cleaning our inputs so they have an appropriate meaning to interpret)\n", "\n", "\n", "Lets recap:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_mmbill_depth_mmflipper_length_mmbody_mass_g
039.118.7181.03750.0
140.217.9194.03700.0
240.318.0195.03250.0
\n", "
" ], "text/plain": [ " bill_length_mm bill_depth_mm flipper_length_mm body_mass_g\n", "0 39.1 18.7 181.0 3750.0\n", "1 40.2 17.9 194.0 3700.0\n", "2 40.3 18.0 195.0 3250.0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num.head(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Where penguin0 and penguin1 are more similar since we observed:\n", "\n", " distance between penguin0 and penguin1: 51.680\n", " distance between penguin1 and penguin2: 450.001\n", " \n", "Is this satisfying or should penguin1 and penguin2 be considered more similar? Lets break it out by feature:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm 1.1\n", "bill_depth_mm -0.8\n", "flipper_length_mm 13.0\n", "body_mass_g -50.0\n", "dtype: float64" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num.iloc[1, :] - df_penguin_num.iloc[0, :]" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm -0.1\n", "bill_depth_mm -0.1\n", "flipper_length_mm -1.0\n", "body_mass_g 450.0\n", "dtype: float64" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num.iloc[1, :] - df_penguin_num.iloc[2, :]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The bills and flippers of penguin2 and penguin1 are just about identical ... but their difference in body mass is so large that it yields a large distance.\n", "\n", "### Big Idea 1: Distances assume that a change of 1 unit (in any feature) is equally significant\n", "\n", "What if we measured the body mass of the penguin in a different unit?" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_mmbill_depth_mmflipper_length_mmbody_mass_kg
039.118.7181.03.750000e-13
140.217.9194.03.700000e-13
240.318.0195.03.250000e-13
\n", "
" ], "text/plain": [ " bill_length_mm bill_depth_mm flipper_length_mm body_mass_kg\n", "0 39.1 18.7 181.0 3.750000e-13\n", "1 40.2 17.9 194.0 3.700000e-13\n", "2 40.3 18.0 195.0 3.250000e-13" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# replace body_mass_g with body_mass_kg\n", "df_penguin_num['body_mass_kg'] = df_penguin_num['body_mass_g'] / 10000000000000000\n", "del df_penguin_num['body_mass_g']\n", "\n", "df_penguin_num.head(3)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "new distance between penguin0 and penguin1: 13.071\n", "new distance between penguin1 and penguin2: 1.010\n" ] } ], "source": [ "vec_penguin0 = np.array(df_penguin_num.iloc[0, :])\n", "vec_penguin1 = np.array(df_penguin_num.iloc[1, :])\n", "vec_penguin2 = np.array(df_penguin_num.iloc[2, :])\n", "\n", "# a quicker way to compute distance\n", "dist01 = np.linalg.norm(vec_penguin0 - vec_penguin1)\n", "dist12 = np.linalg.norm(vec_penguin1 - vec_penguin2)\n", "\n", "print(f'new distance between penguin0 and penguin1: {dist01:.3f}')\n", "print(f'new distance between penguin1 and penguin2: {dist12:.3f}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These numbers aren't just different, they claim an opposite conclusion: penguin1 and penguin2 are more similar!\n", "\n", "- **Distances assume that a change of 1 unit (in any feature) is equally significant**\n", "- **Distances implicitly weight how important each feature is relative to others according to its variance**\n", " - a feature with a higher variance is responsible for more of the distances\n", " \n", "To wrap all the different features into a single distance we must say *something* about how important one feature is compared to another. " ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm 2.988886e+01\n", "bill_depth_mm 3.879347e+00\n", "flipper_length_mm 1.959126e+02\n", "body_mass_kg 6.486477e-27\n", "dtype: float64" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num.var()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Scale Normalization:\n", "How to scale your features so that they're equally important in our distance metric:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm 2.988886e+01\n", "bill_depth_mm 3.879347e+00\n", "flipper_length_mm 1.959126e+02\n", "body_mass_kg 6.486477e-27\n", "dtype: float64" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "df_penguin_num.var()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# by dividing each feature by the standard deviation, outputs will have same std dev\n", "df_penguin_num_scaled = pd.DataFrame()\n", "for feat in df_penguin_num.columns:\n", " df_penguin_num_scaled[f'{feat}_scaled'] = df_penguin_num[feat] / df_penguin_num[feat].std()" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bill_length_mm_scaled 1.0\n", "bill_depth_mm_scaled 1.0\n", "flipper_length_mm_scaled 1.0\n", "body_mass_kg_scaled 1.0\n", "dtype: float64" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num_scaled.var()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that in doing so, our units are no longer valid:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_mm_scaledbill_depth_mm_scaledflipper_length_mm_scaledbody_mass_kg_scaled
07.1519119.49428512.9314564.656148
17.3531159.08811313.8602354.594066
27.3714079.13888413.9316794.035329
\n", "
" ], "text/plain": [ " bill_length_mm_scaled bill_depth_mm_scaled flipper_length_mm_scaled \\\n", "0 7.151911 9.494285 12.931456 \n", "1 7.353115 9.088113 13.860235 \n", "2 7.371407 9.138884 13.931679 \n", "\n", " body_mass_kg_scaled \n", "0 4.656148 \n", "1 4.594066 \n", "2 4.035329 " ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num_scaled.head(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "lets remove the units from the column names (otherwise we might be tempted to draw inappropriate conclusions ...)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_scaledbill_depth_scaledflippter_length_scaledbody_mass_scaled
07.1519119.49428512.9314564.656148
17.3531159.08811313.8602354.594066
27.3714079.13888413.9316794.035329
\n", "
" ], "text/plain": [ " bill_length_scaled bill_depth_scaled flippter_length_scaled \\\n", "0 7.151911 9.494285 12.931456 \n", "1 7.353115 9.088113 13.860235 \n", "2 7.371407 9.138884 13.931679 \n", "\n", " body_mass_scaled \n", "0 4.656148 \n", "1 4.594066 \n", "2 4.035329 " ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin_num_scaled.columns = ['bill_length_scaled',\n", " 'bill_depth_scaled',\n", " 'flippter_length_scaled',\n", " 'body_mass_scaled']\n", "df_penguin_num_scaled.head(3)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### In Class Assignment 1\n", "\n", "Quantitatively, which pair of the following apartments is most similar?\n", "\n", "| | sq ft | bedrooms | bathrooms | toilets |\n", "|-------|------:|---------:|----------:|---------|\n", "| apt 0 | 850 | 2 | 1 | 1 |\n", "| apt 1 | 1000 | 2 | 2 | 2 |\n", "| apt 2 | 1300 | 3 | 2 | 2 |\n", "\n", "- Define and clearly explain how you quantify whether two apartments are similar or different\n", "- Build a dataframe and explicilty compute each pair's distance\n", "- Be warned, this example has a quirk we haven't yet seen in class. You can resolve it yourself with some careful thinking, do what makes sense to you!\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
sq ftbedroomsbathroomstoilets
0850211
11000222
21300322
\n", "
" ], "text/plain": [ " sq ft bedrooms bathrooms toilets\n", "0 850 2 1 1\n", "1 1000 2 2 2\n", "2 1300 3 2 2" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "df_apt = pd.DataFrame({'sq ft': [850, 1000, 1300],\n", " 'bedrooms': [2, 2, 3],\n", " 'bathrooms': [1, 2, 2],\n", " 'toilets': [1, 2, 2]})\n", "df_apt" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "distance between apt0 and apt1: 150.007\n", "distance between apt1 and apt2: 300.002\n" ] } ], "source": [ "# before using scale normalization (bad idea, don't do this! ... we include because its educational to study)\n", "vec_apt0 = np.array(df_apt.iloc[0, :])\n", "vec_apt1 = np.array(df_apt.iloc[1, :])\n", "vec_apt2 = np.array(df_apt.iloc[2, :])\n", "\n", "# a quicker way to compute distance\n", "dist01 = np.linalg.norm(vec_apt0 - vec_apt1)\n", "dist12 = np.linalg.norm(vec_apt1 - vec_apt2)\n", "\n", "print(f'distance between apt0 and apt1: {dist01:.3f}')\n", "print(f'distance between apt1 and apt2: {dist12:.3f}')" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sq ft 52500.000000\n", "bedrooms 0.333333\n", "bathrooms 0.333333\n", "toilets 0.333333\n", "dtype: float64" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_apt.var()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sq ft 1.0\n", "bedrooms 1.0\n", "bathrooms 1.0\n", "toilets 1.0\n", "dtype: float64" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# normalize scale\n", "for feat in df_apt.columns:\n", " df_apt[feat] = df_apt[feat] / df_apt[feat].std()\n", " \n", "df_apt.var()" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "# probably best to drop 'toilets' ... its double counting with bathrooms!\n", "del df_apt['toilets']" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dist between apt0 / apt1: 1.8516401995451033\n", "dist between apt1 / apt2: 2.171240593367237\n", "dist between apt2 / apt0: 3.1396087108337016\n" ] } ], "source": [ "import numpy as np\n", "\n", "dist01 = np.linalg.norm(df_apt.iloc[1, :] - df_apt.iloc[0, :])\n", "dist12 = np.linalg.norm(df_apt.iloc[1, :] - df_apt.iloc[2, :])\n", "dist20 = np.linalg.norm(df_apt.iloc[2, :] - df_apt.iloc[0, :])\n", "\n", "print(f'dist between apt0 / apt1: {dist01}')\n", "print(f'dist between apt1 / apt2: {dist12}')\n", "print(f'dist between apt2 / apt0: {dist20}')" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "# feature engineering\n", "df_apt['sq_ft/person'] = df_apt['sq ft'] / df_apt['bedrooms']" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
sq ftbedroomsbathroomssq_ft/person
03.7097043.4641021.7320511.070899
14.3643583.4641023.4641021.259882
25.6736655.1961523.4641021.091897
\n", "
" ], "text/plain": [ " sq ft bedrooms bathrooms sq_ft/person\n", "0 3.709704 3.464102 1.732051 1.070899\n", "1 4.364358 3.464102 3.464102 1.259882\n", "2 5.673665 5.196152 3.464102 1.091897" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_apt" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## One hot encoding \n", "\n", "How can we include nominal information in these distance measurements? (species, sex, island)\n", "\n", "... we need a way of including nominal information in the vector representation of a penguin (i.e. one sample)." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
speciesislandbill_length_mmbill_depth_mmflipper_length_mmbody_mass_gsex
65AdelieBiscoe41.618.0192.03950.0Male
276GentooBiscoe43.813.9208.04300.0Female
186ChinstrapDream49.718.6195.03600.0Male
198ChinstrapDream50.117.9190.03400.0Female
293GentooBiscoe46.514.8217.05200.0Female
\n", "
" ], "text/plain": [ " species island bill_length_mm bill_depth_mm flipper_length_mm \\\n", "65 Adelie Biscoe 41.6 18.0 192.0 \n", "276 Gentoo Biscoe 43.8 13.9 208.0 \n", "186 Chinstrap Dream 49.7 18.6 195.0 \n", "198 Chinstrap Dream 50.1 17.9 190.0 \n", "293 Gentoo Biscoe 46.5 14.8 217.0 \n", "\n", " body_mass_g sex \n", "65 3950.0 Male \n", "276 4300.0 Female \n", "186 3600.0 Male \n", "198 3400.0 Female \n", "293 5200.0 Female " ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_penguin = sns.load_dataset('penguins')\n", "\n", "# discard penguins with missing features\n", "df_penguin.dropna(axis=0, inplace=True)\n", "\n", "# shuffle order of rows (otherwise all same Species / Island)\n", "df_penguin = df_penguin.sample(frac=1, random_state=1)\n", "\n", "# grab only the first few rows\n", "df_penguin = df_penguin.head()\n", "\n", "df_penguin" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### One hot encoding: \n", "- replace a categorical column with a set of columns per each unique category\n", " - new columns have 1 where row belongs to category" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
islandbill_length_mmbill_depth_mmflipper_length_mmbody_mass_gsexspecies_Adeliespecies_Chinstrapspecies_Gentoo
65Biscoe41.618.0192.03950.0Male100
276Biscoe43.813.9208.04300.0Female001
186Dream49.718.6195.03600.0Male010
198Dream50.117.9190.03400.0Female010
293Biscoe46.514.8217.05200.0Female001
\n", "
" ], "text/plain": [ " island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g \\\n", "65 Biscoe 41.6 18.0 192.0 3950.0 \n", "276 Biscoe 43.8 13.9 208.0 4300.0 \n", "186 Dream 49.7 18.6 195.0 3600.0 \n", "198 Dream 50.1 17.9 190.0 3400.0 \n", "293 Biscoe 46.5 14.8 217.0 5200.0 \n", "\n", " sex species_Adelie species_Chinstrap species_Gentoo \n", "65 Male 1 0 0 \n", "276 Female 0 0 1 \n", "186 Male 0 1 0 \n", "198 Female 0 1 0 \n", "293 Female 0 0 1 " ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# apply one hot encoding to species column\n", "pd.get_dummies(df_penguin, columns=['species'])" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_length_mmbill_depth_mmflipper_length_mmbody_mass_gspecies_Adeliespecies_Chinstrapspecies_Gentooisland_Biscoeisland_Dreamsex_Femalesex_Male
6541.618.0192.03950.01001001
27643.813.9208.04300.00011010
18649.718.6195.03600.00100101
19850.117.9190.03400.00100110
29346.514.8217.05200.00011010
\n", "
" ], "text/plain": [ " bill_length_mm bill_depth_mm flipper_length_mm body_mass_g \\\n", "65 41.6 18.0 192.0 3950.0 \n", "276 43.8 13.9 208.0 4300.0 \n", "186 49.7 18.6 195.0 3600.0 \n", "198 50.1 17.9 190.0 3400.0 \n", "293 46.5 14.8 217.0 5200.0 \n", "\n", " species_Adelie species_Chinstrap species_Gentoo island_Biscoe \\\n", "65 1 0 0 1 \n", "276 0 0 1 1 \n", "186 0 1 0 0 \n", "198 0 1 0 0 \n", "293 0 0 1 1 \n", "\n", " island_Dream sex_Female sex_Male \n", "65 0 0 1 \n", "276 0 1 0 \n", "186 1 0 1 \n", "198 1 1 0 \n", "293 0 1 0 " ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# you can apply one hot encoding to multiple features\n", "pd.get_dummies(df_penguin, columns=['species', 'island', 'sex'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Notice:\n", "One advantage of the \"one\"-hot-encoding is that a single sample can belong to multiple categories\n", "- a penguin which lives on two islands\n", " - (a penguin which heads to his warmer house in the winter)\n", " \n", "- consider a collection of boardgames, we can store their tags via one-hot encoding\n", " - a single game (row) may have multiple tags:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
cooperativeincludes element of luckstrains even good friendships
monopoly011
pictionary110
risk011
\n", "
" ], "text/plain": [ " cooperative includes element of luck \\\n", "monopoly 0 1 \n", "pictionary 1 1 \n", "risk 0 1 \n", "\n", " strains even good friendships \n", "monopoly 1 \n", "pictionary 0 \n", "risk 1 " ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_board_game = pd.DataFrame({'cooperative': [0, 1, 0], \n", " 'includes element of luck': [1, 1, 1],\n", " 'strains even good friendships': [1, 0, 1]}, \n", " index=['monopoly', 'pictionary', 'risk'])\n", "\n", "df_board_game" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# K-Nearest Neighbors " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## ML overview\n", "| | Input Features per sample | Output Features per sample | Supervised | Penguin Example |\n", "|:------------------------:|:-------------------------:|:--------------------------:|:----------:|---------------------------------------------------------------------------------------|\n", "| Classification | 1+ numerical features | one categorical feature | True | Given `body_weight_g`, `flipper_length_mm` estimate `species` |\n", "| Regression | 1+ numerical features | one continuous feature | True | Given `body_weight_g`, `bill_depth_mm` estimate `flipper_length_mm` |\n", "| Clustering | 1+ numerical features | one categorical feature | False | Identify k groups of penguins which have similar `body_weight_g`, `flipper_length_mm` |\n", "| Dimensionality Reduction | N numerical features | < N numerical features | False | Find 2d vector which best represents all 4 of penguin's body/flipper/beak features |\n", "\n", "A **supervised** method is one whose output features are observed in some input data set. Notice:\n", "- To build a penguin species **classifier**, we must observe the species of penguins in our data set\n", "- To build a **clustering** of penguins, no output feature needs to be observed" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## K-Nearest Neighbors Classifier (Warm Up)\n", "\n", "#### Goal:\n", "Make a function which estimates `species` from `bill_depth_mm` and `bill_length_mm`.\n", "\n", "#### Problem Statement (any classifier):\n", "\n", "Given an initial set of \"training\" penguins we observe:\n", "- `bill_depth_mm`\n", "- `bill_length_mm`\n", "- `species` \n", "\n", "Given some new penguin, Gerald, who is not in the training set, we observe:\n", "- `bill_depth_mm`\n", "- `bill_length_mm`\n", "\n", "How can we estimate Gerald's `species`?\n", "\n", "#### K-Nearest Neighbors (k-NN) Approach:\n", "1. We identify the penguins which are the Geradld's $k$ Nearest Neighbors:\n", "- let us represent each penguin as a vector containing:\n", " - `bill_depth_mm`\n", " - `bill_length_mm`\n", "- the **nearest neighbors** are the vectors which are closest to some target vector (Gerald)\n", "2. We estimate Gerald's species as the most common species of these $k$ Nearest Neighbors." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
speciesislandbill_length_mmbill_depth_mmflipper_length_mmbody_mass_gsex
0AdelieTorgersen39.118.7181.03750.0Male
1AdelieTorgersen39.517.4186.03800.0Female
2AdelieTorgersen40.318.0195.03250.0Female
4AdelieTorgersen36.719.3193.03450.0Female
5AdelieTorgersen39.320.6190.03650.0Male
\n", "
" ], "text/plain": [ " species island bill_length_mm bill_depth_mm flipper_length_mm \\\n", "0 Adelie Torgersen 39.1 18.7 181.0 \n", "1 Adelie Torgersen 39.5 17.4 186.0 \n", "2 Adelie Torgersen 40.3 18.0 195.0 \n", "4 Adelie Torgersen 36.7 19.3 193.0 \n", "5 Adelie Torgersen 39.3 20.6 190.0 \n", "\n", " body_mass_g sex \n", "0 3750.0 Male \n", "1 3800.0 Female \n", "2 3250.0 Female \n", "4 3450.0 Female \n", "5 3650.0 Male " ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import seaborn as sns\n", "\n", "df_penguin = sns.load_dataset('penguins')\n", "df_penguin.dropna(axis=0, inplace=True)\n", "df_penguin.head()" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/tmp/ipykernel_65933/3859599772.py:10: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n", " df_penguin_gerald = df_penguin.append({'species': 'Unknown? (Gerald)',\n" ] }, { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "species=Adelie
bill_depth_mm=%{x}
bill_length_mm=%{y}", "legendgroup": "Adelie", "marker": { "color": "#636efa", "symbol": "circle" }, "mode": "markers", "name": "Adelie", "orientation": "v", "showlegend": true, "type": "scatter", "x": [ 18.7, 17.4, 18, 19.3, 20.6, 17.8, 19.6, 17.6, 21.2, 21.1, 17.8, 19, 20.7, 18.4, 21.5, 18.3, 18.7, 19.2, 18.1, 17.2, 18.9, 18.6, 17.9, 18.6, 18.9, 16.7, 18.1, 17.8, 18.9, 17, 21.1, 20, 18.5, 19.3, 19.1, 18, 18.4, 18.5, 19.7, 16.9, 18.8, 19, 17.9, 21.2, 17.7, 18.9, 17.9, 19.5, 18.1, 18.6, 17.5, 18.8, 16.6, 19.1, 16.9, 21.1, 17, 18.2, 17.1, 18, 16.2, 19.1, 16.6, 19.4, 19, 18.4, 17.2, 18.9, 17.5, 18.5, 16.8, 19.4, 16.1, 19.1, 17.2, 17.6, 18.8, 19.4, 17.8, 20.3, 19.5, 18.6, 19.2, 18.8, 18, 18.1, 17.1, 18.1, 17.3, 18.9, 18.6, 18.5, 16.1, 18.5, 17.9, 20, 16, 20, 18.6, 18.9, 17.2, 20, 17, 19, 16.5, 20.3, 17.7, 19.5, 20.7, 18.3, 17, 20.5, 17, 18.6, 17.2, 19.8, 17, 18.5, 15.9, 19, 17.6, 18.3, 17.1, 18, 17.9, 19.2, 18.5, 18.5, 17.6, 17.5, 17.5, 20.1, 16.5, 17.9, 17.1, 17.2, 15.5, 17, 16.8, 18.7, 18.6, 18.4, 17.8, 18.1, 17.1, 18.5 ], "xaxis": "x", "y": [ 39.1, 39.5, 40.3, 36.7, 39.3, 38.9, 39.2, 41.1, 38.6, 34.6, 36.6, 38.7, 42.5, 34.4, 46, 37.8, 37.7, 35.9, 38.2, 38.8, 35.3, 40.6, 40.5, 37.9, 40.5, 39.5, 37.2, 39.5, 40.9, 36.4, 39.2, 38.8, 42.2, 37.6, 39.8, 36.5, 40.8, 36, 44.1, 37, 39.6, 41.1, 36, 42.3, 39.6, 40.1, 35, 42, 34.5, 41.4, 39, 40.6, 36.5, 37.6, 35.7, 41.3, 37.6, 41.1, 36.4, 41.6, 35.5, 41.1, 35.9, 41.8, 33.5, 39.7, 39.6, 45.8, 35.5, 42.8, 40.9, 37.2, 36.2, 42.1, 34.6, 42.9, 36.7, 35.1, 37.3, 41.3, 36.3, 36.9, 38.3, 38.9, 35.7, 41.1, 34, 39.6, 36.2, 40.8, 38.1, 40.3, 33.1, 43.2, 35, 41, 37.7, 37.8, 37.9, 39.7, 38.6, 38.2, 38.1, 43.2, 38.1, 45.6, 39.7, 42.2, 39.6, 42.7, 38.6, 37.3, 35.7, 41.1, 36.2, 37.7, 40.2, 41.4, 35.2, 40.6, 38.8, 41.5, 39, 44.1, 38.5, 43.1, 36.8, 37.5, 38.1, 41.1, 35.6, 40.2, 37, 39.7, 40.2, 40.6, 32.1, 40.7, 37.3, 39, 39.2, 36.6, 36, 37.8, 36, 41.5 ], "yaxis": "y" }, { "hovertemplate": "species=Chinstrap
bill_depth_mm=%{x}
bill_length_mm=%{y}", "legendgroup": "Chinstrap", "marker": { "color": "#EF553B", "symbol": "circle" }, "mode": "markers", "name": "Chinstrap", "orientation": "v", "showlegend": true, "type": "scatter", "x": [ 17.9, 19.5, 19.2, 18.7, 19.8, 17.8, 18.2, 18.2, 18.9, 19.9, 17.8, 20.3, 17.3, 18.1, 17.1, 19.6, 20, 17.8, 18.6, 18.2, 17.3, 17.5, 16.6, 19.4, 17.9, 19, 18.4, 19, 17.8, 20, 16.6, 20.8, 16.7, 18.8, 18.6, 16.8, 18.3, 20.7, 16.6, 19.9, 19.5, 17.5, 19.1, 17, 17.9, 18.5, 17.9, 19.6, 18.7, 17.3, 16.4, 19, 17.3, 19.7, 17.3, 18.8, 16.6, 19.9, 18.8, 19.4, 19.5, 16.5, 17, 19.8, 18.1, 18.2, 19, 18.7 ], "xaxis": "x", "y": [ 46.5, 50, 51.3, 45.4, 52.7, 45.2, 46.1, 51.3, 46, 51.3, 46.6, 51.7, 47, 52, 45.9, 50.5, 50.3, 58, 46.4, 49.2, 42.4, 48.5, 43.2, 50.6, 46.7, 52, 50.5, 49.5, 46.4, 52.8, 40.9, 54.2, 42.5, 51, 49.7, 47.5, 47.6, 52, 46.9, 53.5, 49, 46.2, 50.9, 45.5, 50.9, 50.8, 50.1, 49, 51.5, 49.8, 48.1, 51.4, 45.7, 50.7, 42.5, 52.2, 45.2, 49.3, 50.2, 45.6, 51.9, 46.8, 45.7, 55.8, 43.5, 49.6, 50.8, 50.2 ], "yaxis": "y" }, { "hovertemplate": "species=Gentoo
bill_depth_mm=%{x}
bill_length_mm=%{y}", "legendgroup": "Gentoo", "marker": { "color": "#00cc96", "symbol": "circle" }, "mode": "markers", "name": "Gentoo", "orientation": "v", "showlegend": true, "type": "scatter", "x": [ 13.2, 16.3, 14.1, 15.2, 14.5, 13.5, 14.6, 15.3, 13.4, 15.4, 13.7, 16.1, 13.7, 14.6, 14.6, 15.7, 13.5, 15.2, 14.5, 15.1, 14.3, 14.5, 14.5, 15.8, 13.1, 15.1, 15, 14.3, 15.3, 15.3, 14.2, 14.5, 17, 14.8, 16.3, 13.7, 17.3, 13.6, 15.7, 13.7, 16, 13.7, 15, 15.9, 13.9, 13.9, 15.9, 13.3, 15.8, 14.2, 14.1, 14.4, 15, 14.4, 15.4, 13.9, 15, 14.5, 15.3, 13.8, 14.9, 13.9, 15.7, 14.2, 16.8, 16.2, 14.2, 15, 15, 15.6, 15.6, 14.8, 15, 16, 14.2, 16.3, 13.8, 16.4, 14.5, 15.6, 14.6, 15.9, 13.8, 17.3, 14.4, 14.2, 14, 17, 15, 17.1, 14.5, 16.1, 14.7, 15.7, 15.8, 14.6, 14.4, 16.5, 15, 17, 15.5, 15, 16.1, 14.7, 15.8, 14, 15.1, 15.2, 15.9, 15.2, 16.3, 14.1, 16, 16.2, 13.7, 14.3, 15.7, 14.8, 16.1 ], "xaxis": "x", "y": [ 46.1, 50, 48.7, 50, 47.6, 46.5, 45.4, 46.7, 43.3, 46.8, 40.9, 49, 45.5, 48.4, 45.8, 49.3, 42, 49.2, 46.2, 48.7, 50.2, 45.1, 46.5, 46.3, 42.9, 46.1, 47.8, 48.2, 50, 47.3, 42.8, 45.1, 59.6, 49.1, 48.4, 42.6, 44.4, 44, 48.7, 42.7, 49.6, 45.3, 49.6, 50.5, 43.6, 45.5, 50.5, 44.9, 45.2, 46.6, 48.5, 45.1, 50.1, 46.5, 45, 43.8, 45.5, 43.2, 50.4, 45.3, 46.2, 45.7, 54.3, 45.8, 49.8, 49.5, 43.5, 50.7, 47.7, 46.4, 48.2, 46.5, 46.4, 48.6, 47.5, 51.1, 45.2, 45.2, 49.1, 52.5, 47.4, 50, 44.9, 50.8, 43.4, 51.3, 47.5, 52.1, 47.5, 52.2, 45.5, 49.5, 44.5, 50.8, 49.4, 46.9, 48.4, 51.1, 48.5, 55.9, 47.2, 49.1, 46.8, 41.7, 53.4, 43.3, 48.1, 50.5, 49.8, 43.5, 51.5, 46.2, 55.1, 48.8, 47.2, 46.8, 50.4, 45.2, 49.9 ], "yaxis": "y" }, { "hovertemplate": "species=Unknown? (Gerald)
bill_depth_mm=%{x}
bill_length_mm=%{y}", "legendgroup": "Unknown? (Gerald)", "marker": { "color": "#ab63fa", "symbol": "circle" }, "mode": "markers", "name": "Unknown? (Gerald)", "orientation": "v", "showlegend": true, "type": "scatter", "x": [ 17.42 ], "xaxis": "x", "y": [ 42.5 ], "yaxis": "y" } ], "layout": { "legend": { "title": { "text": "species" }, "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "bill_depth_mm" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "bill_length_mm" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import plotly.express as px\n", "\n", "feat0 = 'bill_depth_mm'\n", "feat1 = 'bill_length_mm'\n", "\n", "sns.set(font_scale=1.2)\n", "\n", "# add Gerald to the dataframe\n", "df_penguin_gerald = df_penguin.append({'species': 'Unknown? (Gerald)', \n", " 'bill_depth_mm': 17.42, \n", " 'bill_length_mm': 42.5}, ignore_index=True)\n", "\n", "px.scatter(data_frame=df_penguin_gerald, x=feat0, y=feat1, color='species')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If K = 3, Gerald's K Nearest Neighbors are:\n", "- Chinstrap\n", "- Chinstrap\n", "- Adelie\n", "\n", "So we'd estimate Gerald is of species Chinstrap (most common Nearest Neighbor species)\n", "\n", "If K = 5, Gerald's K Nearest Neighbors are:\n", "- Chinstrap\n", "- chinstrap\n", "- Adelie\n", "- Adelie\n", "- Adelie\n", "\n", "So we'd estimate Gerald is of species Adelie (most common Nearest Neighbor species)\n", "\n", "If K=4, Gerald's K Nearest Neighbors are:\n", "- Chinstrap\n", "- chinstrap\n", "- Adelie\n", "- Adelie\n", "\n", "So we could either:\n", "- avoid outputting an estimate\n", "- discard furthest neighbor among k Nearest Neighbors\n", " - estimate \"recursively\" using K-1 Nearest Neighbors" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## K-Nearest Neighbors Demo\n", "\n", "http://vision.stanford.edu/teaching/cs231n-demos/knn/" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Scikit Learn is a wonderful machine learning library in python\n", "\n", "- It has a K-Nearest Neighbors classifier\n", "- it has many other classifiers too\n", "- (they all have the same interface ...)\n", " - polymorphism!\n", "\n", "\n", "\n", "https://scikit-learn.org/stable/" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## K-NN Classifier: prepping input sci-kit learn\n", "\n", "Scikit Learn operates on arrays. We must construct two arrays:\n", "- x\n", " - input feature matrix (the features we use to define distances)\n", " - each row is a sample (penguin)\n", " - each column is a numeric feature (bill depth/length)\n", "- y\n", " - target variable vector\n", " - vector length = number of samples (penguins)\n", " - value is categorical feature (species)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[18.7, 39.1],\n", " [17.4, 39.5],\n", " [18. , 40.3],\n", " [19.3, 36.7],\n", " [20.6, 39.3]])" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "\n", "x_feat_list = ['bill_depth_mm', 'bill_length_mm']\n", "\n", "# build a matrix of input features\n", "x = df_penguin.loc[:, x_feat_list].values\n", "\n", "x[:5, :]" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
bill_depth_mmbill_length_mm
018.739.1
117.439.5
218.040.3
419.336.7
520.639.3
\n", "
" ], "text/plain": [ " bill_depth_mm bill_length_mm\n", "0 18.7 39.1\n", "1 17.4 39.5\n", "2 18.0 40.3\n", "4 19.3 36.7\n", "5 20.6 39.3" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# see where x comes from?\n", "df_penguin.loc[:, x_feat_list].head(5)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Adelie', 'Adelie', 'Adelie', 'Adelie', 'Adelie'], dtype=object)" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_true = df_penguin.loc[:, 'species'].values\n", "y_true[:5]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## K-NN Classifier: sci-kit learn & confusion matrix" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "from sklearn.neighbors import KNeighborsClassifier\n", "\n", "k = 9\n", "x_feat_list = ['bill_depth_mm', 'bill_length_mm']\n", "y_feat = 'species'\n", "\n", "x = df_penguin.loc[:, x_feat_list].values\n", "y_true = df_penguin.loc[:, y_feat].values\n", "\n", "# initialize a knn_classifier\n", "knn_classifier = KNeighborsClassifier(n_neighbors=k)\n", "\n", "# fit happens \"inplace\", we modify the internal state of knn_classifier (it remembers all the training samples)\n", "knn_classifier.fit(x, y_true)\n", "\n", "# estimate each penguin's species\n", "y_pred = knn_classifier.predict(x)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1, 'a'), (2, 'b'), (3, 'c')]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(zip([1, 2, 3], ['a', 'b', 'c']))" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('Adelie', 'Adelie'),\n", " ('Adelie', 'Adelie'),\n", " ('Adelie', 'Adelie'),\n", " ('Adelie', 'Adelie'),\n", " ('Adelie', 'Adelie')]" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# zip together list of (truth, predict) pairs\n", "true_pred_list = list(zip(y_true, y_pred))\n", "true_pred_list[:5]" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Counter({('Adelie', 'Adelie'): 142,\n", " ('Adelie', 'Chinstrap'): 4,\n", " ('Chinstrap', 'Chinstrap'): 61,\n", " ('Chinstrap', 'Gentoo'): 3,\n", " ('Chinstrap', 'Adelie'): 4,\n", " ('Gentoo', 'Gentoo'): 116,\n", " ('Gentoo', 'Chinstrap'): 3})" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from collections import Counter\n", "\n", "# one way of getting a sense of how well we did\n", "Counter(true_pred_list)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[142, 4, 0],\n", " [ 4, 61, 3],\n", " [ 0, 3, 116]])" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n", "import numpy as np\n", "\n", "conf_mat = confusion_matrix(y_true=y_true, y_pred=y_pred)\n", "\n", "# examine confusion matri\n", "conf_mat" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvAAAAKHCAYAAADngXwDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABzcUlEQVR4nO3deVyU5frH8e+AbLK4ASqo5YrmvpZLi1tqqbibJm7lklppmmnHOv1sU4/mrmWpmVvhgpjlktbJLHMpS1NxTwUENQFBWQTm9weHqQkwwBnGcT7v12teyf3cz/NcQ4gXF9d9Pwaj0WgUAAAAALvgZOsAAAAAAOQfCTwAAABgR0jgAQAAADtCAg8AAADYERJ4AAAAwI6QwAMAAAB2hAQeAAAAsCMk8AAAAIAdIYEHAAAA7EgxWwcAAACAe4fRmC5lXCr6GzuXl8HgGKmtY7xLAAAAFI2MSzJebVvktzX47pKKVSzy+9oCCTwAAAAsyKhMZRb5XZ1lLPJ72go98AAAAHB4S5Ys0dixY/X444+rZs2aeuCBB/J9bmxsrJo0aaKgoCAtWrQo1zlRUVEaP368HnroIdWrV0/BwcFat25doWKlAg8AAACHN2vWLPn4+KhWrVq6efOmrl27lu9z33jjDWVkZOR5PCYmRn379lViYqIGDRqkChUqaNeuXZoyZYpiY2M1ZsyYAsVKAg8AAACLyjDaooXmznz11VeqVKmSJCkkJCTfCfwXX3yhb7/9Vi+//LKmTZuW65z33ntPV65c0fz58/X4449Lkvr06aORI0dq8eLFCg4OVsWK+e/fp4UGAAAADi87eS+Ia9eu6a233tLAgQPzbLlJTk7W9u3bVaFCBVPynm3IkCFKT0/X559/XqD7ksADAADAYoySMmUs8pctlrC+/fbb8vDw0AsvvJDnnJMnTyolJUUNGjTIcaxhw4YyGAw6fPhwge5LCw0AAADuCdHR0QoJCcnz+K5duyx2r2+//VZbtmzRhx9+qOLFi+c5LyYmRpJUrly5HMdcXV1VqlQpxcbGFujeVOABAACAAkhKStLrr7+uzp0765FHHrnt3OTkZElZyXpu3NzcTHPyiwo8AAAALMoW+8BLUkBAgEWr7HmZMWOGUlJS9Oqrr/7jXA8PD0lSWlparsdTU1NVqlSpAt2fCjwAAACQT0ePHlVoaKj69++vpKQknT9/XufPnze1wSQkJOj8+fNKSkqS9GfrTHYrzV+lpaUpLi5OZcuWLVAMVOABAABgURnGe/epqJcuXZLRaNSiRYtyfWjTxx9/rI8//lhvvPGG+vXrpxo1asjNzU2//PJLjrm//PKLjEaj6tWrV6AYSOABAACAfKpbt67mzp2bY/z06dOaP3++nnzyST3++OOqXbu2pKwWmscff1yff/65duzYYbaV5LJly1SsWDF17ty5QDGQwAMAAMBijP/b1tEW970TmzZtUnR0tCQpKirKVGXPNmrUKElS2bJl1bFjxxzn79u3T5JUrVq1HMdfeukl7d27VxMnTtTRo0dNT2L95ptvNGrUqALvQU8CDwAAAIe3YcMG7d+/32zsr5X27AS+MAICAvTpp59q9uzZ+vTTT3Xz5k3df//9mjp1qvr27Vvg6xmMxnu4SQkAAABFKiP9vBIvtyry+3r775FzsfuK/L62QAUeAAAAFpVhk+eiOg62kQQAAADsCBV4AAAAWJQtFrE6EirwAAAAgB0hgQcAAADsCC00AAAAsBijbPMkVkdq2qECDwAAANgRKvAAAACwqExbB3CPowIPAAAA2BEq8AAAALAoHuRkXVTgAQAAADtCAg8AAADYEVpoAAAAYDFZ20ja5r6Oggo8AAAAYEeowAMAAMCi2EbSuqjAAwAAAHaEBB4AAACwI7TQAAAAwKIyZLB1CPc0KvAAAACAHaECDwAAAIsxSspkG0mrogIPAAAA2BEq8AAAALAoeuCtiwo8AAAAYEdI4AEAAAA7QgsNAAAALIoWGuuiAg8AAADYESrwAAAAsJisbSSLvgLPNpIAAAAA7kok8AAAAIAdoYUGAAAAFmSw0SJWx1k4SwUeAAAAsCNU4AEAAGAxRkkZNqgRs4gVAAAAwF2JBB4AAACwI7TQAAAAwHKMttkH3pF6aKjAAwAAAHaECjwAAAAsyjbbSDoOKvAAAACAHaECDwAAAIsxSsowso2kNVGBBwAAAOwICTwAAABgR2ihAQAAgAUZlGmTGrHjLJylAg8AAADYESrwuC2jMV3KuGTrMACTmItutg4ByMnoSMvncLfzq1hGGemZcvNwtVkMbCNpXSTwuL2MSzJebWPrKACTwXVb2DoEIIfMmzdtHQJg8snpBZKk8lXK2jgSWAstNAAAAIAdoQIPAAAAi2EfeOujAg8AAADYESrwAAAAsKhMFrFaFRV4AAAAwI5QgQcAAIBFZVAjtio+uwAAAIAdIYEHAAAA7AgtNAAAALAYoww22kbScRbOUoEHAAAA7AgVeAAAAFhUJjViq+KzCwAAANgREngAAADAjtBCAwAAAIvKMDrOglJboAIPAAAA2BEq8AAAALAYo2zzJFZjkd/RdqjAAwAAAHaECjwAAAAsyKBMGzzISTzICQAAAMDdiAQeAAAAsCO00AAAAMCibLGI1ZHw2QUAAADsCBV4AAAAWIxRtnmQE9tIAgAAALgrkcADAAAAdoQWGgAAAFhUJjViq+KzCwAAANgRKvAAAACwHKNBGbZ4EusdLpxdsmSJjh07pmPHjunChQtycnLSsWPHcp27f/9+bd++XQcOHFB0dLQkqVKlSurSpYv69esnd3f3HOfExcVpzpw52rVrl+Lj4xUYGKhevXppyJAhKlasYCk5CTwAAAAc3qxZs+Tj46NatWrp5s2bunbtWp5zZ86cqejoaLVv3179+/fXrVu3tGvXLk2bNk2ff/651q5dKzc3N9P8pKQkDRgwQOfOnVP//v0VFBSkAwcOaObMmTp79qzefffdAsVKAg8AAACLMUrKlP1tI/nVV1+pUqVKkqSQkJDbJvDjx49X48aNzSrnISEhGj9+vLZs2aL169fr6aefNh1bunSpTp8+rUmTJmnIkCGSpN69e8vb21urVq1Sjx491LRp03zHSg88AAAAHF528p4fDz74YK5tL0888YQk6cSJE2bj4eHh8vDwUL9+/czGs5P5TZs2FShWEngAAADAAmJjYyVJvr6+prGrV68qKipKNWvWzNEbX6FCBfn5+enw4cMFug8tNAAAALAomyxilRQdHa2QkJA8j+/atctq905KStJHH30kFxcXdenSxTQeExMjSSpXrlyu55UrV04XLlwo0L2owAMAAAB3ID09XePGjVNUVJQmTJigypUrm46lpKRIklxdXXM9183NTcnJyQW6HxV4AAAAWIxRUoYNasRGSQEBAVatsucmPT1d48eP1+7du/Xss89q8ODBZsez22bS0tJyPT81NVUeHh4FuicJPAAAAFAIt27d0vjx47V9+3aNGDFCL730Uo452a0z2a00fxcTE6OyZcsW6L600AAAAAAFlJaWphdffFHbt2/XmDFjck3epawFrQEBAYqIiDC102SLiorSlStXVK9evQLdmwQeAAAAFpVpNBT5qyilpaXphRde0K5duzRu3Dg9//zzt53ftWtXJScna+3atWbjy5cvlyQFBwcX6P600AAAAMDhbdq0SdHR0ZKyKuNGo1GLFi0yHR81apTpzxMmTNA333yjRo0aqXz58goPDze7VqVKldSwYUPTx8OGDdP27dv1n//8R1FRUaYnsYaHhys4OFjNmjUrUKwk8AAAALAgg00WseoOn/66YcMG7d+/32xs7ty5pj//NYH/7bffJEk///yzfv755xzX6t69u1kC7+XlpTVr1mjOnDnatm2bPv30UwUGBmr8+PEaOnRogWM1GI3GO33yLO5hxvSLMl5tY+swAJNO1VrYOgQgh8ybN20dAmDyyekFkqTyVQq2MNJSEtIu6eOzee/Fbi2Dq6xUCdfyRX5fW6ACDwAAAIvKtNGDnBwFn10AAADAjpDAAwAAAHaEFhoAAABYTNaTWIt2W8fs+zoKKvAAAACAHaECDwAAAItiEat18dkFAAAA7AgJPAAAAGBHaKEBAACAxbCI1fqowAMAAAB2hAo8AAAALMhgo0WsRV/1txUq8AAAAIAdoQIPAAAAi8pgG0mr4rMLAAAA2BESeAAAAMCO0EIDAAAAi8p0oAWltkAFHgAAALAjVOABAABgMUbZZhErD3ICAAAAcFcigQcAAADsCC00AAAAsByjlGm0wSJWB+qhoQIPAAAA2BEq8AAAALAYowzKsEGN2OhAW1dSgQcAAADsCAk8AAAAYEdooQEAAIBF2WQRqwOhAg8AAADYESrwAAAAsKhMasRWxWcXAAAAsCNU4AEAAGBRGfTAWxUVeAAAAMCOUIEH8uG7LSV0eK+Xzhz10LljHrqZ5Kw2Pa7plQUX8nX+7PEVtW1tGUnSsu+PKbBymumY0Sgd/K+39u/00W/7vHQ5ykWpKU4qG5imJm0S9dTzsSrll26V9wW0Dr6iibNOS5LmvFpF20PL2jgiOBrf8mka+HKMmjyWKO9SGbp2uZj2biuhVe+VVVICaQqQG/5mAPmwZk45nT3mIQ/PDPmWv6Wbp53zfe6PO3y0bW0ZeXhmKPlGzvNupRo05emqcnHNVJ0Hb6jhw4nKzJR++d5bmz7y07fhJTUr7JQCq6TlcnWg8HzLp2rUv8/pZpKTintl2jocOKDy96Vq9ubTKuWXrh+2+ejiaTcFNUhW92FX1aR1osYFV1NiHKmKvTHKNttIGov8jrZDC40VbNy4UUFBQdq3b1+hzo+MjFRQUJDmz59vNh4UFKRJkyZZIkQU0Ij/i9KyPccUdvKInp8Wme/z4v9w1pyXK+rRrnGqXi851zlOzkYNeuWS1v5yVNM+O6Ph/47WyP+L1qIdJ/REyFXFXXHRB28EWuqtAP9j1EvTzigxvpi+XEvVHbYx5t1IlfJL18IpAfq/oZW17J0AvdKnqjZ84KuK1VI15JUYW4cI3JVI4PNw69YttWjRQkFBQZo7d66tw4GNNWiZpMAqaTIUsKAw9+WKkqQx7+Sd9Bdzkfq/GCvvkhlm405O0oBxWf94Hd7rVbAbA/8geFCM6jdP0HuvVFNKcv5/owRYSvn7UtXksSTFXHDV58t9zY6tnFlOyTec1LZXnNw8MvK4Au5eBmUanYr8JTnOwlkS+Dx8/fXX+uOPP3Tfffdp48aNysiw/TeQw4cP680337R1GMinHZ+V1g/bSuqF6RflU7pwXz/OLlm/EHQu5ki/GIS1Vax6U0NePq/wFeX12wEfW4cDB1W/ZZIk6advvWT8W7tF8g1nHT1QXO7FM1Wr8U1bhAfc1Ujg8xAaGqr7779fkydPVkxMjL777jtbhyQ3Nze5uLjYOgzkQ2ykixa/Hqi2Pa+pRcfrhb7O9v8tfG3yWKKlQoODc3I2asLM07oS7aaPZ1a0dThwYBWqpkqSos665Xo8+lzWeIUqqUUWE2AvSOBzERUVpR9++EHdu3fXI488Ij8/P61bty7HvFu3bmnevHlq06aN6tatq06dOmnNmjV5XjcpKUmzZ89Whw4dVKdOHTVr1kyjRo1SREREvuLKqwf+xx9/1LPPPqumTZuqTp066tSpk5YsWXJX/NbAEWVmSjNfvE8enhl67s2oQl/nxC8eWv1eORX3ytCgVy5ZMEI4sqfHXFTVB25o1ivVlJZK6wxsx9M769+oG4m5fx3euJ417unDv2X2KEOGIn85EpZ252L9+vWSpG7dusnZ2VnBwcH6+OOPdeXKFfn5+ZnmTZw4UV9++aUeeughDR48WPHx8Zo/f77Kly+f45pJSUnq16+fLly4oG7duqlmzZq6fv26QkND9dRTT2n16tWqXbt2oWKdMmWKHnjgAQ0bNkw+Pj76+eef9d577+n48eOaPXt24T8RKJSNS/x0eK+X3lx5Jkdfe35FnnHTvwdVUXq6QZMX/66A+9mBBncuqH6i+j4XpY1LAxRxyNvW4QAACokE/m8yMjK0YcMGtWjRQuXKlZMk9ezZUx999JE2btyoESNGSJL27t2rL7/8Uu3bt9f8+fNl+N/qxu7du6tz5845rjtv3jydO3dOq1evVv369U3j/fr1U5cuXTRt2jStXLmyQLFeuXJFU6dOVdu2bbVgwQJTDE899ZRq1qyp6dOnq1+/fmrWrFmhPhcouMgzbvp4enk93vcPNWtbuLaXyDNumti7qhLjnTV58e9q3qHwLThANidnoyb857SifvfQyjm0zsD2sivv2ZX4v8uuvGdX4mE/2EbS+mih+Zvdu3crNjZWPXv2NI1VqVJFDRs21Pr162U0Zn157NixQ5I0fPhwU+IsSRUrVlSXLl3Mrmk0GrV582Y1aNBAFStW1LVr10yv9PR0tWzZUj/99JNSUlIKFOv27duVmpqq3r17Ky4uzuy6jz32mCRpz549hfk0oJAunHTXrVQn7fisjDoENDB7Ze8kM7TlA+oQ0EA/bC2R8/xTbnq5VzVdv1ZM//rgdz38ZEJRvwXcozyKZ6hClRRVqpaszcf2aevpvabXgBeydkka+85ZbT29VyP+dc7G0cIRRJ7J6nEPzKPHPaBy1nhkHj3ygCOjAv83oaGhcnd3V/Xq1XX+/HnTeKtWrTR//nz9+OOPat68uS5cyHoCZ7Vq1XJc4+9jcXFxiouL04EDB9S8efM87x0XF5dr+01ezpw5I0mm3wrk5urVq/m+Hu5c2Ypp6tjvj1yP7d/lo2uXXfRwlzh5emWqbEXztphzx901qW9V3Uh01msf/a4H21F5h+XcSjNoW6h/rseq1b6harVv6LcD3oo856HjtNegCPz6fVZRo/GjSTIYjGY70Xh4Zqh205tKuemk4z8Vt1WIuANZ2zrCWkjg/yI2NlbffvutMjIycm2DkbJ6zm+XhOcmMzPrCYdNmzbVqFGj8pxXunTpQl33rbfeUmBg7g/68ffP/R9sWEfVOskaN+tirsde7llN1y67aMikSwqsbJ68n/nNQ5P6VlVqspPeWH6OXWdgcWmpzpr7atVcjz39wkVVq31DO8P8tD2UhzqhaFw676aD//VSk8eS1GXIVW1e9ucas5AJMfLwzNQXn5RRKs8pAHIggf+L7P3eJ0+ebOp//6v169drx44diouLU6VKlSRJp0+fVr169czmnT592uzj0qVLy8fHRwkJCWrRooXF4q1cubIkqUSJEha9LnL6YWsJ/bA9q+Xl2uWsvzbHf/LUzLFZXwc+pdI1/N/Rhbp2YryzXulbVYlxxdSgVaKOH/TU8YOeOeZ1H3ZFXiXYjQHAvWPB5Aqavfm0Rr8VrYatknThlLtqNrypBq2SdPGMm5ZPz/lvMQASeBOj0aj169erfPnyGjRokFlfezY3Nzd99913Cg8P1+OPP641a9ZoyZIlZotYL168qM8//9zsPCcnJ3Xt2lWrVq1SWFiYunfvnuPaV69ela+vb47x2+nUqZPee+89zZ8/Xy1btpSnp3nSl5KSovT0dHl58RTPO3XmqIe+CjX/Dcml8266dD6rN7NshbRCJ/A3Ep2VGJf1V/GXPd76ZU/u7Qvt+14jgQdwT7l03k3Pd6qugS/HqPFjiWraJlHXLhdT2Ie+WvVeWSUlkKbYq0wH29axqPE3439++OEHRUZGavDgwbkm75LUsmVLeXt7a/369dqyZYs6dOig7du3a/DgwWrbtq0SEhK0du1aVa1aVUePHjU7d9y4cTp06JAmTZqknTt3qkmTJvLw8NClS5e0d+9eubm5FXgXmrJly2rq1Kl69dVX1bFjR3Xv3l0VK1ZUfHy8zp49q6+++koLFy7Ugw8+WOjPC7KETIhRyISYO7rGfzacznW8XMU0bY/+5Y6uDdyJ1fMqavU8dqaBbVyJdtWscZVsHQZgV0jg/yc0NFSS1KFDhzznuLq6qk2bNgoPD9ehQ4c0c+ZMValSRZs2bdKMGTNUoUIFjR49Wh4eHpo8ebLZuV5eXlqzZo1WrFihL7/8Unv27JGTk5P8/PxUr149devWrVBxd+vWTZUrV9bSpUu1YcMGJSQkqESJEqpYsaKGDh2qoKCgQl0XAACgMIxGKcMW20g60D6SBqPRkd4uCsqYflHGq21sHQZg0qka6z1w98m8edPWIQAmn5xeIEkqX8U2i9Ivp1zWK0deKfL7Tq87Xf7ujrF5B3v8AAAAAHaEFhoAAABYkMFG+8A7zsJZKvAAAACAHaECDwAAAIvKtMEiVkdCBR4AAACwI1TgAQAAYFE8yMm6qMADAAAAdoQEHgAAALAjtNAAAADAYoyyzSJWR3oyKRV4AAAAwI5QgQcAAIBF2eZBTo6Dzy4AAABgR0jgAQAAADtCCw0AAAAsyGCjJ7E6zt7zVOABAAAAO0IFHgAAABbFk1itiwo8AAAAYEeowAMAAMBieJCT9VGBBwAAAOwICTwAAABgR2ihAQAAgEXZZhtJx0EFHgAAALAjVOABAABgOUYbVeAdaBUrFXgAAADAjpDAAwAAAHaEFhoAAABYlD0uYl2yZImOHTumY8eO6cKFC3JyctKxY8fynJ+enq5ly5Zpw4YNioqKUsmSJdW2bVuNHTtWpUqVyjE/Li5Oc+bM0a5duxQfH6/AwED16tVLQ4YMUbFiBUvJSeABAADg8GbNmiUfHx/VqlVLN2/e1LVr1247f/Lkydq8ebNat26tZ555RpGRkVqxYoV+/vlnffbZZypevLhpblJSkgYMGKBz586pf//+CgoK0oEDBzRz5kydPXtW7777boFiJYEHAACAxRglZcr+nsT61VdfqVKlSpKkkJCQ2ybwe/fu1ebNm9WmTRstXrzYNF67dm298MILWrZsmcaMGWMaX7p0qU6fPq1JkyZpyJAhkqTevXvL29tbq1atUo8ePdS0adN8x0oPPAAAABxedvKeH+Hh4ZJkSsazdejQQYGBgabjf53v4eGhfv36mY1nn79p06YCxUoCDwAAAIvKNBqK/FWUfv31Vzk5OalBgwY5jjVs2FAXLlxQfHy8JOnq1auKiopSzZo15e7ubja3QoUK8vPz0+HDhwt0f1poAAAAcE+Ijo5WSEhInsd37dplkfvExMSoVKlScnV1zXGsbNmypjklS5ZUTEyMJKlcuXK5XqtcuXK6cOFCge5PBR4AAAAogJSUlFyTd0lyc3Mzzfnrf283Pzk5uUD3pwIPAAAACyr6lpbs+wYEBFisyn477u7uSktLy/VYamqqac5f/3u7+R4eHgW6PxV4AAAAoADKlSunuLi4XJPy2NhY05y//je7lebvYmJiTG03+UUCDwAAAIsxyjaLWO90G8mCqFevnjIzM/Xrr7/mOHbo0CFVqlRJJUuWlCT5+voqICBAERERpnaabFFRUbpy5Yrq1atXoPuTwAMAAAAFEBwcLElatmyZ2fiOHTsUFRVlOp6ta9euSk5O1tq1a83Gly9fbna9/KIHHgAAAA5v06ZNio6OlpRVGTcajVq0aJHp+KhRo0x/btGihTp37qwtW7Zo5MiRatu2rSIjI/Xxxx+rWrVqOfaHHzZsmLZv367//Oc/ioqKMj2JNTw8XMHBwWrWrFmBYjUYjcai/I0D7Iwx/aKMV9vYOgzApFO1FrYOAcgh8+ZNW4cAmHxyeoEkqXyVgvVVW0p08h96eu87RX7f1c1fVYBHmUKfHxISov379+d5/MSJE2Yf37p1S8uWLdPGjRsVFRWlkiVLqk2bNho7dqxKly6d4/xr165pzpw5+vrrrxUfH6/AwED17NlTQ4cOVbFiBaupk8DjtkjgcbchgcfdiAQedxMS+HsfLTQAAACwKKNNtpF0HCxiBQAAAOwIFXgAAABYVKaowFsTFXgAAADAjpDAAwAAAHaEFhoAAABYjjHrSay2uK+joAIPAAAA2BEq8AAAALAYo2yzjaQDFeCpwAMAAAD2hAQeAAAAsCO00AAAAMCibLKI1YFQgQcAAADsCBV4AAAAWJDBJotY5UBPf6UCDwAAANgREngAAADAjtBCAwAAAItiEat1UYEHAAAA7AgVeAAAAFiU0ZEei2oDVOABAAAAO0IFHgAAABZjlJRpgy0dHanoTwUeAAAAsCMk8AAAAIAdoYUGAAAAFmWbJ7E6DirwAAAAgB2hAg8AAACL4kFO1kUFHgAAALAjJPAAAACAHaGFBgAAAJZjtNGTWB1oI3gq8AAAAIAdoQIPAAAAi2IbSeuiAg8AAADYESrwAAAAsCgq8NZFBR4AAACwIyTwAAAAgB2hhQa3FXPRTYPrtrB1GIDJ+fENbB0CkEOlGT/ZOgTgTwbbtq8YZbDJk1iNcpy2HSrwAAAAgB2hAg8AAACLssmDnBwIFXgAAADAjpDAAwAAAHaEFhoAAABYFPvAWxcVeAAAAMCOUIEHAACARVGBty4q8AAAAIAdoQIPAAAAi2IXSeuiAg8AAADYERJ4AAAAwI7QQgMAAACLYhGrdVGBBwAAAOwIFXgAAABYjlG2WcXqQCtnqcADAAAAdoQEHgAAALAjtNAAAADAoljEal1U4AEAAAA7QgUeAAAAFmOUZLTBglIHWsNKBR4AAACwJ1TgAQAAYFH0wFsXFXgAAADAjpDAAwAAAHaEFhoAAABYFi00VkUFHgAAALAjVOABAABgUbbYRtKRUIEHAAAA7AgJPAAAAGBHaKEBAACAZdFCY1VU4AEAAAA7QgUeAAAAlmO00ZNYHajqTwUeAAAAsCNU4AEAAGBZDlQNtwUq8AAAAIAdIYEHAAAA7AgtNAAAALAgg20WscoW97QNKvAAAACAHclXBb5t27aFurjBYNDOnTsLdS4AAADsFItYrSpfCbzRWLj/C4U9DwAAAChKSUlJWrFihbZt26bIyEi5urqqQoUK6tGjh/r06SMXFxfT3OTkZC1cuFBffvmlLl++LH9/fz355JMaNWqUPDw8rB5rvhL4r7/+2tpxAAAAADaRnp6uQYMG6dixY+rWrZuefvpppaWlaceOHZo6daoOHTqkmTNnSpIyMjI0fPhw7d+/X8HBwWratKkiIiK0dOlSHT58WMuXL5eTk3W71FnECgAAAAuzrwWl+/fv12+//aahQ4fqlVdeMY0//fTT6tmzp7744gu98cYb8vLyUlhYmPbv36+QkBBNmTLFNDcwMFDTp0/X5s2b1a1bN6vGa5EfDxISEnTp0iVLXAoAAAAoUomJiZIkf39/s3FnZ2f5+vrK2dlZrq6ukqTw8HBJ0pAhQ8zm9u/fX+7u7tq0aZPV4y10Bf7GjRuaP3++Pv/8c127dk0Gg0HHjh2TJP36669asGCBxo4dq9q1a1ssWAAAANgBO1sG2ahRIxUvXlxLlixR2bJl1aBBA6Wmpmrr1q3as2ePXnjhBbm6uspoNOrIkSPy9/dXYGCg2TXc3d1Vq1YtHTlyxOrxFiqBT0xMVP/+/XXq1CnVqlVLpUqV0pkzZ0zHa9SooYMHD2rLli0k8AAAACgS0dHRCgkJyfP4rl27ch338/PTokWL9MYbb2jcuHGmcTc3N7399tvq2bOnJCk+Pl7JycmqXr16rtcpW7asDh06pKSkJHl5ed3BO7m9QiXwixcv1qlTpzRt2jR169ZNCxYs0MKFC03HPTw81KxZM/34448WCxQAAAB2ws4q8JLk5eWlypUrq1mzZmrZsqVSUlIUFham1157TQaDQT169FBKSookmdpp/s7NzU1S1i41d10C/9VXX6lVq1a3bdAPCAgokl8hAAAAAFJW/plXlf12IiIi1L9/fw0aNEgTJkwwjXft2lX9+vXT1KlT9dhjj8nd3V2SlJaWlut1UlNTJcnqW0kWahFrTEyMgoKCbjunePHipgUBAAAAwN1qxYoVSktLU8eOHc3GnZyc1KFDByUnJ+vw4cMqWbKkPDw8FBMTk+t1YmNj5eXlZdXqu1TIBN7T01PXrl277ZzIyEiVKlWqUEEBAADAjhkNRf+6A5cvX5YkZWZm5jiWnp5u+q/BYFCdOnV0+fJlRUVFmc1LSUnR8ePHVbdu3TuKJT8KlcDXrVtX33zzjZKSknI9fvnyZe3evVuNGze+o+AAAAAAa6tWrZokaePGjWbjt27d0pYtW+Ts7GxKzIODgyVJy5cvN5u7du1apaSkmI5bU6F64AcOHKhhw4Zp+PDhevPNN82OnTlzRlOmTFFqauptVwEDAADg3mS0s0WsgwYNUnh4uNauXauYmBg9/PDDSk5O1ubNm3XixAkNGTJEZcuWlST16NFDmzZt0sqVK5WYmKgmTZroxIkTWrNmjZo1a6auXbtaPd5CJfAPP/ywxowZowULFqhz584qVizrMg8++KCuX78uo9GoCRMmqFGjRhYNFgAAALC0gIAArV+/XosWLdIPP/yg7777Ti4uLqpevbreeust9erVyzTX2dlZS5Ys0cKFC7V161Z98cUX8vPz05AhQzR69Gg5OztbPd5CP8hpzJgxatKkiVauXKlff/1V8fHxMhgMevTRRzVo0CA1b97cknECAAAAVlOhQgW98847+Zrr6empiRMnauLEiVaOKneFTuAl6aGHHtJDDz1kqVgAAABg74yyzT7wdta2cycKtYgVAAAAgG3cUQU+MjJS4eHhOn78uBITE+Xt7a1atWqpa9euqlixoqViBAAAgD25w20dcXuFTuCXLVum2bNnKz09Xca/LDXeuXOnFi9erPHjx2vIkCEWCRIAAABAlkIl8Fu2bNGMGTNUokQJhYSEqFmzZvL19dXVq1e1b98+rVy5UjNmzFDZsmX1xBNPWDpmAAAAwGEVKoFftmyZSpQooY0bNyowMNA0XqVKFTVr1kzdunVTz549tXTpUhJ4AAAAB2NwoAWltlCoRaxnzpxRx44dzZL3v6pYsaI6duyo06dP31FwAAAAAMwVqgLv6ekpHx+f287x8fGRl5dXoYICAACAHaMCb1WFqsC3bNlSe/bsyfO40WjU999/r5YtWxY6MAAAAAA5FSqBf/nll5WQkKCXXnpJUVFRZseio6M1fvx4Xb9+XS+//LJFggQAAIAdMRqK/uVA8tVCM3DgwBxjPj4+2rp1q3bs2KHy5curTJky+uOPP3Tp0iVlZGQoKChIEyZM0IoVKyweNAAAAOCo8pXA79+/P89j6enpunjxoi5evGg2HhERIYPBsX4aAgAAAKwtXwl8RESEteMAAADAvYJFrFZVqB54AAAAALZRqG0kAQAAgDxRgbeqO07gY2JiFBsbq7S0tFyPN23a9E5vAQAAAOB/Cp3A79mzR++++67Onj1723nHjx8v7C0AAAAA/E2heuB/+eUXjRw5UtevX9fTTz8to9GoJk2aqHfv3qpSpYqMRqNat26t0aNHWzpeAAAA3M2MNnw5iEJV4D/44AO5urpq/fr1Klu2rFatWqUHH3xQY8aMkdFo1Lx58/Txxx9r3Lhxlo4XAAAAcGiFrsC3adNGZcuWNY0ZjVk/9hgMBr344ouqUqWK5s+fb5koAQAAYD94EqtVFSqBT0xMVEBAgOljFxcX3bx502xOo0aNdODAgTuLDgAAAICZQrXQlClTRgkJCWYf//1JrOnp6UpJSbmz6AAAAGB3DA7Uj24LharA33///WYJe/369fX999/r3LlzkqQrV65ox44duv/++y0SJAAAAIAsharAP/zww5ozZ47i4+NVsmRJDRw4UF999ZW6d++uqlWr6vz587px44ZefvllS8cL2K3WwVc0cdZpSdKcV6toe2jZfzgDuDMPBUaqf53f1KBcjHzcUhWf4q6Tf5TRqiN1tfvCfZKkYk4Zeqr2UdXyvapavldVtVScXJwz9dp/H9WG4w/Y+B3AEQx95aKq172hCpVT5FP6ltJSnBQb5aa9O0pp8yf+Sox3sXWIwF2nUBX4p556SqtXr1axYln5f+PGjTV37lxVqFBBp06dkp+fn9544w1169bNkrHmadKkSQoKCsr3/DZt2igkJMSKEQHmfMunatS/z+lmUqH+ygEFNv6hvVrW9XPV9r+sr3+/Xx//Wl/fnr9PpT2S1TQg2jTPo1i6Xm31vbrXPCHf4jd19WZxG0YNR9R9aIzci2fo5z0+2rS8nL4OL6PMdINCxkVp8dbf5Fs+1dYhojDYQtKqClWB9/LyUv369c3G2rdvr/bt21skKElKS0vTxo0btW3bNkVERCgxMVGenp6qVq2aWrdurT59+qhEiRIWu5817Ny5U8ePH9fzzz9v61BgU0a9NO2MEuOL6fvtpdVr2CVbB4R7XK9ax/RMw18UFhGkN759VLcync2OF3PKMP05Jb2YRnzxhI5f9dXVm54a3eSARjc9WNQhw4H1qNtYt9JyFjcGTbiofqMvqe9zl7Tw9fuLPjDgLlboJ7FaU3R0tEaOHKkTJ06ocePGGjx4sPz8/JSYmKiff/5Zc+fO1Y4dO7Ru3bpCXX/btm0Wjjh3O3fuVFhYGAm8gwseFKP6zRP0ytO1Vb95wj+fANwBF6cMvfjgPkUneuWavEtS+l/GbmU667v/tdMAtpBb8i5Ju78oo36jLymwMhtiAH931yXwaWlpGjFihM6cOaOZM2eqS5cuZscHDx6smJgYrVq1qtD3cHV1vdMwrSIjI0NpaWny8PCwdSiwkIpVb2rIy+cVvqK8fjvgQwIPq2tR8aLKeKRoxa/1lGk06JFK51W99DWlZjjryGV//RpbztYhAvnyUNs4SdK5CNq6gL/LVwLftm3bQl3cYDBo586dBTpn/fr1OnnypIYOHZojec9Wrlw5TZgwIcd4UlKSZs+ere3btys+Pl7Vq1fXSy+9pIcffthsXps2bRQYGKiVK1fmGJs6dapmzJih/fv3KzMzU02aNNGUKVN0331/VqiMRqNWrVqlDRs26OLFi8rMzFSZMmVUv359vfLKK/L391ebNm0UFRUlSWb9+e+++6569Oih+fPna8GCBfriiy+0ceNGbd26VbGxsXrrrbfUo0cP7dmzRxs3btThw4d1+fJlFStWTLVq1dIzzzyjNm3amL2fSZMmKSwsTD/++KNmzJihb775Rjdv3lStWrU0duxYNW/evED/D2AZTs5GTZh5Wlei3fTxzIq2DgcOoq7/ZUlSWoazNvRepxplrpkdPxBdXmO3d1BcCoUC3F16Drskj+IZKu6doRr1bqhO0ySdPe6h0MXlbR0acNfJVwKf/ZTVgirMeVu3bpWUtVC2oJ555hl5e3tr+PDhSklJ0YoVK/Tcc89px44dZg+eyktsbKwGDBigNm3aaMKECTp//rxWrVqlUaNG6fPPP5eTU9av+d5//33NmTNHjz76qHr37i0XFxdFR0fru+++0+XLl+Xv769XX31Vy5cv18GDBzVjxgzTPRo1amR2zwkTJsjZ2Vn9+/dX8eLFVblyZUlSWFiYrl69qq5du6pcuXK6du2awsLC9Nxzz2n27Nl64okncn3/Xl5eGjVqlBISEvTZZ5/p2Wef1eLFi/XII48U+POJO/P0mIuq+sANTXiqjtJSc7YxANZQ2iNZkjSkwS86E1dKA8K6KeKqrwJ9ruvl5nvVqtJFzX58hwZvDrZxpIC5nsNiVNrvlunjA/8toVkvV1HCNXahsUfsA29d+Urgv/76a2vHYXLy5El5enqaVbzzKygoSFOnTjV93KxZM/Xt21effvqpXnrppX88//z585o1a5Y6d+5sGitdurRmzZqlH374Qa1atZIk7dixQ1WrVtWSJUvMzh87dqzpz+3atdPOnTt18OBBBQfn/Q9l8eLFtWLFCrm4mH+DevPNN1W8uPmvDQcNGqRu3bpp4cKFuSbw/v7+WrRokekHjV69eumJJ57Q1KlTtWPHDtM4rC+ofqL6PheljUsDFHHI29bhwIFk/y3PyHTS6K2dFJ3oI0k6da2MXtjeQV/0W6tmgdGqXzaGdhrcVfo3ayhJKul7Sw80StTQVyK1cMtv+vczNXT6qKeNowPuLnddRpeUlCQvL69CnTt06FCzjxs0aKDixYvr999/z9f5/v7+Zsm7JLVo0UKSzK7h7e2t2NhY7d+/v1Bx/tXQoUNzJO+SzJL3mzdvKi4uTsnJyXrwwQd1+vRpJSUl5ThnxIgRZkl6+fLlFRwcrIsXL+rYsWN3HCvyx8nZqAn/Oa2o3z20cg6tMyha19Oy1vgcv+prSt6zpaS76PuLWV+T2a02wN0m/qqLfthRWq8ODJJ3yXRNmHXW1iGhMIyGon85kLtuEauXl5du3LhRqHMrVsyZLJUqVUpxcXGFPr9kyZKSpPj4eNPY+PHjNXr0aIWEhMjX11eNGzdW8+bN1blzZ3l7F6zamtfTaiMjIzV37lzt3r3b7N7Zrl+/nuMHnWrVquWYlz124cIF1alTp0CxoXA8imeoQpWsXRM2H9uX65yx75zV2HfOatPycvrg7cpFGR7ucb/Hl5QkJablvlj/eqqbJMm9WHpRhQQUyuUoN1047aFqtW/Kp9QtXY+jlQbIdtcl8DVq1ND+/ft1/vz5ArfRODvfWZ/x7c7/az9//fr19dVXX+mHH37Qvn37dODAAW3fvl3z5s3TqlWrVLVq1Xzf093dPcfYjRs3NGDAACUmJmrgwIEKCgqSl5eXnJyctGHDBm3ZskWZmZkFe3MoMrfSDNoW6p/rsWq1b6ha7Rv67YC3Is956DjtNbCwHyMrKNMoVS0VJ4OMMsq8KlW9dNai1sjrPrmdDtxVypRNkyRlZjhWdfWeQA+8Vd11CXzHjh21f/9+ffbZZ5o4caKtw8mTh4eH2rZta9qhZ/fu3Ro2bJg+/PBDTZs2TVLWLjyF8eOPP+rSpUt6++231atXL7NjoaGheZ53+vRpNWzYMMeYJFWqVKlQsaDg0lKdNffV3H+Ie/qFi6pW+4Z2hvlpe2jZIo4MjiA6yVv//f1+tan8u0LqHdYnh/986F6LChfVsuJFJaS6as9F2rtge4GVkxV31UU3E83TEYPBqIHjI1XKN11HD3op6fpdl64ANnXX/Y3o3bu3Pv30U3388ceqU6dOros1Y2NjtXLlyly3kiwK165dU+nSpc3GsttT/trukt3HHh8fb2rFyY/s3wT8fRefiIiI227L+cEHH5gtYr106ZLCw8NVoUIFPfDAA/m+PwD79uZ3D6uW71VNavmDHr3vvI5f9VWgd6LaVj6njEwnvf7fx5SU5maa/2zDn1W5ZLwkqZbvVUlS96ATalQuRpL0c0w5bTjO9xBYXtPHEjRk4kUdPeitmItuSowrppK+t1T3wUQF3Jeqa5ddNHcybYbA3911Cbyrq6s++OADjRgxQuPGjdOaNWv0yCOPqEyZMkpKStIvv/yinTt3qlatWjaLsVOnTqpfv77q1aunsmXLKiEhQZs2bZIkdevWzTSvfv36WrVqlf7v//5Pjz76qFxcXFSvXr1ce+3/qlGjRvLz89P06dMVGRmpwMBAnTlzRqGhoapRo4aOHj2a63mXL1/W4MGD1b59eyUkJOjTTz9VamqqXn/9dXagARxI7A0v9VrfS6OaHFTr+39Xk/KXlJTmqv/+fr8+PNRQRy6b//anVcWLahYYbTbWqHyMGpWPMX1MAg9rOPS9jwJC/VS7SZKqPnBTXj7pSrnprKhz7loVVkabPi6npIS7LlVBftBCY1V35d+KgIAAbdiwQRs2bNDWrVu1dOlSJSUlydPTU9WrV9e4cePUu3dvm8X3zDPP6LvvvtOaNWt0/fp1lSxZUjVr1tTkyZPVsmVL07zOnTvr+PHj+uKLL7Rt2zZlZmbq3Xff/ccE3sfHR8uWLdPMmTO1du1apaWlKSgoSDNnztSxY8fyTOCXLl2qGTNmaOHChbpx44Zq1aqladOmmba/hO2tnldRq+fRugDri0vx0Nt7Htbbex7+x7nsCQ9bOX+yuBb9+35bhwHYHYOxsE9pwl0j+0msJ06csPi1L527rMF1X7b4dYHCOj++ga1DAHKoNOMnW4cAmKw49p4kqXzl3DdUsLYL8fFqvWxZkd/3m6FDVakALcv27I4q8BEREdqyZYvOnDmj5ORkffzxx5KytkA8fPiwWrZsqRIlSlgiTgAAAAC6gwR+7ty5+uCDD0zbGf51xxWj0ajx48fr1VdfVUhIyJ1HCQAAAEBSIZ/E+sUXX2jx4sVq0aKFNm3apBEjRpgdr1ixourUqaOvv/7aIkECAADAjhht8HIghUrgV65cqfvuu0+LFi1SzZo15eKS8+loVatW1fnz5+84QPyzadOmWaX/HQAAAHefQiXwJ06cUKtWreTqmvujuiXJ399fV69eLXRgAAAAsFNU4K2q0JuD/9NTRq9evSo3N7fbzgEAAABQMIVaxHrffffp0KFDeR7PzMzUTz/9pGrVqhU6MAAAANgng4NVxItaoSrwnTp10rFjx7Qsjz0+33//fV24cEGdO3e+o+AAAAAAmCtUBX7QoEHatm2b/vOf/2jr1q2mdprp06fr4MGD+u2331S/fn317dvXosECAAAAjq5QFXh3d3d98sknCg4O1rFjx3T48GEZjUYtX75cR48eVdeuXfXRRx+pWLE7ek4UAAAA7I5BMtrgpduvz7yXFDrD9vb21rRp0zRp0iQdOXJE8fHx8vb2Vr169VS6dGlLxggAAADgf+64RF6yZEk9/PDDlogFAAAA9wIWsVpVobeRBAAAAFD0ClWBnzx5cr7mGQwGvfPOO4W5BQAAAIBcFCqBDwsLu+1xg8Ego9FIAg8AAOBgDLLNPvCOs4S1kAn8rl27ch1PTEzUkSNHtGjRIjVs2FDjx4+/o+AAAAAAmCtUAh8YGJjnsZo1a6pVq1bq2rWrmjdvrt69exc6OAAAANgZo2yziNWBFs5aZRFr+fLl1bp1a33yySfWuDwAAADgsKz2pKUyZcro/Pnz1ro8AAAA7lK26IF3JFapwGdkZGjfvn3y9va2xuUBAAAAh1WoCvyBAwdyHU9PT1dMTIw2btyo48eP0/8OAAAAWFihEviQkBAZDHlv1mM0GtW0aVNNnDix0IEBAADATtFCY1WFSuBHjx6dawJvMBhUokQJ1atXT/Xq1bvj4AAAAACYK1QC//zzz1s6DgAAANwrqMBbVaEWsU6ePFkff/yxhUMBAAAA8E8KlcBv2bJFf/zxh6VjAQAAAPAPCv0kVhJ4AAAA5IZ94K2rUBX4zp07a/fu3UpISLB0PAAAAABuo1AJ/IgRI1SnTh0NHDhQ33zzja5evWrpuAAAAADkIt8tNJs2bVLNmjVVs2ZN0xaRRqNRo0aNyvMcg8GgY8eO3XmUAAAAACQVIIGfNGmSnn/+edWsWVNNmjSxZkwAAAAA8lCgRaxGY9aKhJUrV1olGAAAANwDWMRqVYXqgQcAAABgG4XaRhIAAADIC9tIWleBEvjExERFR0cX6AYBAQEFmg8AAADYQlJSkj788EPt2LFDUVFRcnd313333acBAwYoODjYNC85OVkLFy7Ul19+qcuXL8vf319PPvmkRo0aJQ8PD6vHWaAE/pNPPtEnn3yS7/nsQgMAAOCA7LACHxsbq4EDByouLk7du3dXtWrVlJycrN9//92sgJ2RkaHhw4dr//79Cg4OVtOmTRUREaGlS5fq8OHDWr58uZycrNulXqAE3svLS97e3taKBQAAALCJiRMn6saNGwoPD1f58uXznBcWFqb9+/crJCREU6ZMMY0HBgZq+vTp2rx5s7p162bVWAuUwA8aNEhjxoyxViwAAABAkfvpp5/0448/avLkySpfvrwyMjKUkpIiT0/PHHPDw8MlSUOGDDEb79+/v+bOnatNmzZZPYFnFxoAAABYjtGGr0L69ttvJUmVKlXS888/r/r166tRo0Zq1aqVFi1apIyMjKy3ZjTqyJEj8vf3V2BgoNk13N3dVatWLR05cqTwgeQTu9AAAADgnhAdHa2QkJA8j+/atSvX8TNnzkiS/vWvf6lChQp66623JElr167V3LlzdenSJb355puKj49XcnKyqlevnut1ypYtq0OHDikpKUleXl53+G7yRgIPAAAAi7K3bSRv3LghSfLw8NDq1avl6uoqSXriiSf05JNPat26dRoyZIhph5ns43/n5uYmKWuXGhJ4AAAA4B8EBATkWWW/HXd3d0lSly5dzJJzV1dXdenSRQsXLtS+ffvUsWNHSVJaWlqu10lNTZUkq28lme8EPiIiwppxAAAAADZRrlw5SZKfn1+OY9ljCQkJKlmypDw8PBQTE5PrdWJjY+Xl5WXV6rvEIlYAAABYmh0tYJWkBg0aSJIuXbqU41h2sl6mTBkZDAbVqVNHly9fVlRUlNm8lJQUHT9+XHXr1r2zYPKBBB4AAAAOrW3btvLx8VF4eLiSkpJM4zdu3FBYWJhcXFzUqlUrSTI9kXX58uVm11i7dq1SUlLMnthqLfTAAwAAwKLsbRGrt7e3/vWvf+mVV15Rr1691KtXLxkMBm3YsEGxsbEaN26c6eFOPXr00KZNm7Ry5UolJiaqSZMmOnHihNasWaNmzZqpa9euVo+XBB4AAAAOr1u3bipVqpQ+/PBDLVy4UJmZmapRo4bee+89Pfnkk6Z5zs7OWrJkiRYuXKitW7fqiy++kJ+fn4YMGaLRo0fL2dnZ6rGSwAMAAMCy7KwCn+3RRx/Vo48++o/zPD09NXHiRE2cOLEIosqJHngAAADAjpDAAwAAAHaEFhoAAABYlp220NgLKvAAAACAHaECDwAAAIuyt20k7Q0VeAAAAMCOkMADAAAAdoQWGgAAAFiOUbZZxOpAbTtU4AEAAAA7QgUeAAAAluVA1XBboAIPAAAA2BEq8AAAALAotpG0LirwAAAAgB0hgQcAAADsCC00AAAAsCxaaKyKCjwAAABgR6jAAwAAwGIMss0iVkPR39JmqMADAAAAdoQEHgAAALAjtNAAAADAsljEalVU4AEAAAA7QgUeAAAAlkUF3qqowAMAAAB2hAo8AAAALMqRtnS0BSrwAAAAgB0hgQcAAADsCC00uD2jUZk3b9o6CsCk0oyfbB0CkEPrg9dsHQJg4h6YYesQWMRqZVTgAQAAADtCBR4AAACWY5QMtqjAO1DVnwo8AAAAYEdI4AEAAAA7QgsNAAAALMuB2llsgQo8AAAAYEeowAMAAMCyqMBbFRV4AAAAwI5QgQcAAIBF2WQbSQdCBR4AAACwIyTwAAAAgB2hhQYAAACWRQuNVVGBBwAAAOwIFXgAAABYFItYrYsKPAAAAGBHSOABAAAAO0ILDQAAACyLFhqrogIPAAAA2BEq8AAAALAoFrFaFxV4AAAAwI5QgQcAAIDlGGWbHngHqvpTgQcAAADsCAk8AAAAYEdooQEAAIBlOVA7iy1QgQcAAADsCBV4AAAAWBTbSFoXFXgAAADAjpDAAwAAAHaEFhoAAABYFi00VkUFHgAAALAjVOABAABgQUYZjDyK1ZqowAMAAAB2hAQeAAAAsCO00AAAAMCyHKebxSaowAMAAAB2hAo8AAAALMYg2zyJ1VD0t7QZKvAAAACAHaECDwAAAMsxyjY98A7Ud08FHgAAALAjJPAAAACAHaGFBgAAABZli0WsjoQKPAAAAGBHqMADAADAsqjAWxUVeAAAAMCOkMADAAAAdoQEHgAAABZlMBb9y9IyMzPVp08fBQUFafDgwTmOJycna+bMmWrTpo3q1KmjNm3aaNasWUpOTrZ8MH9DDzwAAADwNytWrNCpU6dyPZaRkaHhw4dr//79Cg4OVtOmTRUREaGlS5fq8OHDWr58uZycrFcnJ4EHAACAZdn5ItaLFy9q7ty5GjdunN55550cx8PCwrR//36FhIRoypQppvHAwEBNnz5dmzdvVrdu3awWHy00AAAAwF9MmTJF1apVU0hISK7Hw8PDJUlDhgwxG+/fv7/c3d21adMmq8ZHBR4AAAAWZc8PcgoNDdXBgwe1YcOGXNtgjEajjhw5In9/fwUGBpodc3d3V61atXTkyBGrxkgCDwAAgHtCdHR0nlVzSdq1a9dtz4+NjdWMGTM0ZMgQ1axZM9c58fHxSk5OVvXq1XM9XrZsWR06dEhJSUny8vLKf/AFQAsNAAAAIOmNN95QqVKlNGbMmDznpKSkSJJcXV1zPe7m5iZJVt2Nhgo8AAAALMtomx6agICAf6yy5+WLL77Q119/reXLl8vd3T3PednH0tLScj2empoqSfLw8ChUHPlBAg8AAACHlpaWprfeekutWrVSYGCgzp8/b3Y8JSVF58+fl6enp8qUKSMPDw/FxMTkeq3Y2Fh5eXlZrX1GIoEHAACAhdnbItaUlBRdu3ZNe/bs0eOPP57j+KFDh/T444/riSee0OzZs1WnTh0dOHBAUVFRZgtZU1JSdPz4cTVs2NCq8ZLAAwAAwKF5eHho7ty5uR578cUXVaNGDY0ePVrly5eXJAUHB+vAgQNavny52T7wa9euVUpKioKDg60aLwk8AAAAHJqLi4s6duyY5/EyZcqYHe/Ro4c2bdqklStXKjExUU2aNNGJEye0Zs0aNWvWTF27drVqvCTwAAAAsByjbPMk1iK8p7Ozs5YsWaKFCxdq69at+uKLL+Tn56chQ4Zo9OjRcnZ2tur9SeABAACAPJw4cSLXcU9PT02cOFETJ04s4ohI4AEAAGBhhkxbR3Bv40FOAAAAgB2hAg8AAADLsrNtJO0NFXgAAADAjpDAAwAAAHaEFhoAAABYlL09idXeUIEHAAAA7AgVeAAAAFiWkRK8NVGBBwAAAOwICTwAAABgR2ihAQAAgMUYZJtFrIaiv6XNUIEHAAAA7AgVeAAAAFiOUbZ5EqsDrZulAg8AAADYESrwAAAAsCge5GRdJPCAFfmWT9PAl2PU5LFEeZfK0LXLxbR3Wwmteq+skhL464eiNfSVi6pe94YqVE6RT+lbSktxUmyUm/buKKXNn/grMd7F1iHCzl3e4az4g85KPOGkpBNOyrhhUNkn01V7WmqOuZm3pKjPiikxwklJEU66ccZJxnSDar6RqoCe6be9T3qidGGFi658XUwpkQbJSXIvb1SJBhmq8WqanPhSxj2ODAKwkvL3pWr25tMq5ZeuH7b56OJpNwU1SFb3YVfVpHWixgVXU2IcfwVRdLoPjdHpo8X18x4fxf/hIvfiGarV4IZCxkWpU7/LGtvjAV295GbrMGHHfl/ioqQTznIubpRbWaNunst7X5CMZOnU9KyvN9cymXL1NSo15p/3Eblx1qBfRrgr9bJBpR/KUJlWmTKmG5QSbdDl7cVU7WUSeNz7HCp7SEtLU1hYmLZv366IiAhdv35drq6uqlixoho3bqzg4GDVr1+/SGKJjIxUWFiY2rVrp1q1ahXJPVG0xrwbqVJ+6Vo4JUCbl/mZxof/O0o9R1zVkFdiNG9SBRtGCEfTo25j3UrLufRp0ISL6jf6kvo+d0kLX7+/6APDPaP6xDS5lTXKo5JR8QeddGioR55znT2k+otS5FUzU25+Rp1d5KLfF7ve9voZydLhF9yVccOgxp+kqET9TLPjmemSwdkibwV3iiexWpXDLGKNjIxUz5499frrryslJUUDBgzQG2+8ofHjx6thw4batWuX+vTpox9//LFI4omKitKCBQt0/PjxIrkfilb5+1LV5LEkxVxw1efLfc2OrZxZTsk3nNS2V5zcPDJsFCEcUW7JuyTt/qKMJCmwckpRhoN7UKlmmSp+n1GGfGzI7eQilXk4Q25++U/0okKLKfm8k6qOTcuRvEuSUzHl696AvXOICnxqaqpGjhypc+fOac6cOerUqVOOOVOmTFFYWJjc3d1tECHuNfVbJkmSfvrWS0aj+b8myTecdfRAcTV5LEm1Gt/UL3u8bREiYPJQ2zhJ0rmI4jaOBLi92C+LSQaj/DumKznKoD/2OCs90SD3cpkq0ypDLiVtHSGysYjVuhwigV+3bp1OnTql4cOH55q8S1KxYsXUu3fvHONGo1Hr1q1TaGioTp8+LUmqWbOmnn32WbVr185sblBQkLp3767+/ftr5syZOnLkiJydnfXwww9rypQpKlMmq8o1f/58LViwQJI0efJkTZ48WZLUrFkzrVy5UpKUmZmpVatWaf369fr9999VrFgx1alTRyNGjFDLli1zxLl79259+OGHOnr0qNLT01W5cmX16dNH/fv3l4FyRJGrUDVrwVbU2dz7iaPPuUmPJalClVQSeBS5nsMuyaN4hop7Z6hGvRuq0zRJZ497KHRxeVuHBuQp85aUdNJJLqWk6A0uOjvPRcb0P/99c/YwqvrkNAV0v/0CWOBe4BAJ/Pbt2yVJffr0KfC5kydP1qZNm9S2bVt16dJFkvTVV19p9OjReuONN9SvXz+z+RERERo2bJiCg4P1xBNP6OjRo1q3bp2uX7+upUuXSpLat2+v9PR0vf/+++rbt68aN24sSfL1/bPVYtKkSQoPD1ejRo300ksv6caNG1q/fr2eeeYZTZ8+XcHBwaa569at02uvvaaAgAA988wz8vT01LZt2zR16lRFRETozTffLPD7xp3x9M5qjbmRmHsz5o3rWeOePrTQoOj1HBaj0n63TB8f+G8JzXq5ihKusfIPd6/0BMmYblB6glFn57ro/hG3VL57upzdjbrydTGdmu6qiH+7yj0gU6UfzNleA9xLHCKBP3nypLy8vFSxYkWzcaPRqLi4OLMxNzc3eXp6SpJ27typsLAwTZ48WYMHDzbNGTRokEaOHKmZM2eqS5cu8vLyMh2LiIjQmjVr1KhRI9OYwWDQZ599pnPnzqly5cqqWbOmEhIS9P7776tBgwZmybgk7d27V+Hh4XrkkUf0/vvvy9k5K9nr16+fOnfurLfeekvt2rWTp6enEhMT9c4778jX11fr169X6dKlJUkDBgzQ8OHDFRoaquDgYDVp0uTOP5EA7gn9mzWUJJX0vaUHGiVq6CuRWrjlN/37mRo6fdTTxtEBuctuRzRmGBTQ+5YqP/fnD6EBPdKzdrWZ5qYLy1xU+sGc21aiiNFCY1UOsYg1KSnJLMnOdvXqVTVv3tzs9c4775iOb968We7u7urUqZOuXbtm9mrXrp2SkpL0yy+/mF2zQYMGZsm7JFPLy++//56veHfs2CFJGjVqlCl5l6TSpUurf//+un79uvbu3StJ2rNnj27evKmQkBBT8i5ltQQ999xzZtdD0cmuvGdX4v8uu/KeXYkHbCH+qot+2FFarw4MknfJdE2YddbWIQF5Kub1Z0bo1zbn99bssetH+L6Ke59DVOC9vLyUlJSUY7xEiRJavny5pKxk/uWXXzY7fubMGaWkpOiRRx7J89pXr141+/jvVX5JKlmypCQpPj4+X/FevHhRklSjRo0cx4KCgszm3G5u9tiFCxfydV9YTuSZrN73wCq5V4ECKmeNR+bRIw8UpctRbrpw2kPVat+UT6lbuh5HKw3uPs4eklu5TKXGOKmYd87yrotP1lgmxfe7AotYrcshEvgaNWpo//79unjxolmC7erqqhYtWkjK2mby7zIzM+Xt7a158+blee1q1aqZffzXivnfGdkT1WH8+n3Wb3waP5okg8FothONh2eGaje9qZSbTjr+E7t+4O5QpmyaJCkzg0XvuHuVfihDlzY56cZpJ5WoZ97nnnQ6q6nAPZB/a3Hvc4gWmg4dOkiSQkNDC3Te/fffr8TERNWqVUstWrTI9eXv71+omG63M0ylSpUkybTrzV+dPHlS0p+V/uy5p06dyjE3eyx7DorOpfNuOvhfL5WrlKYuQ8x/SxMyIUYenpnatb6UUpP5VS+KRmDlZBX3zrk7h8Fg1KAJF1XKN11HD3op6bpD1HVgpwKfSpecjDq/1EVp1/4cz0iVzs7LeghU2U7sQnNXyDQW/cuBOMR36t69e2vt2rVatmyZHnjggVy3ksytOt6tWzd9/fXX+s9//qO33347R9J99epVs51jCqJ48azKa0JCQo5j7du31+rVq/X+++9r4cKFcnLK+jnr2rVrWrNmjXx8fNS8eXNJWf31xYsX1+rVq9W3b1+VKFFCkpSRkaHFixdLkh5//PFCxYg7s2ByBc3efFqj34pWw1ZJunDKXTUb3lSDVkm6eMZNy6eXs3WIcCBNH0vQkIkXdfSgt2IuuikxrphK+t5S3QcTFXBfqq5ddtHcyZVtHSbs3JVdzrrydVZhIu2PrH8zE3510rF/ZSXXLqWk6hPSTPN//8hFN89lzUs6kfVv3aVNxRT/c9afSzbKVEDPPxNyn9qZqjzyls4tctW+7sXl2zpdzq7SHz84K/m8k0o0yFCloX8ubgXuVQ6RwLu5uemDDz7Qc889p7Fjx2rlypVq1aqV/P39lZycrAsXLmjr1q2SpMDAQNN5HTp0UJ8+fRQaGqqIiAi1a9dOfn5+io2N1dGjR7V7924dPXq0UDFVq1ZNnp6eWrNmjdzd3eXj46PSpUubFtMGBwcrPDxcAwcOVLt27XTz5k2tX79ef/zxh6ZPn27aKcfb21uvvvqqXnvtNfXs2VM9e/aUh4eHtm/frp9//ll9+vRhBxobuXTeTc93qq6BL8eo8WOJatomUdcuF1PYh75a9V5ZJSU4xF8/3CUOfe+jgFA/1W6SpKoP3JSXT7pSbjor6py7VoWV0aaPy/E1iTuWeMJJMZvN11CkRDopJvJ/7S0Bmao+4c9j1753VvxB899EJvzirIRfssduKaCn+T0qP3dLntUzdXGliy5vKybjLcmjolFVnk9TpcG35ORq6XcF3H0MRgdqzE5LS9PGjRu1fft2RURE6Pr163J1dVXFihXVuHFjde/eXfXq1ctx3pYtW/TZZ5/p+PHjSklJka+vr6pXr642bdqY7QOf/SCnadOmmZ2/b98+DRw4UO+++6569OhhGv/22281Z84cnT59WmlpaTke5LRy5UqzBznVrVtXw4cPV6tWrXLE+O233+qjjz7Sb7/9ZvYgp6effvqOHuR06WysBlYbU+jzAUszuLHwF3ef1gev/fMkoIiMrJ713JlSrrZ5OFv0pXj1e2ZJkd937dLhCihfssjvawsOlcCj4EjgcbchgcfdiAQedxMS+Hsfvy8FAACARbGNpHU5xC40AAAAwL2CBB4AAACwI7TQAAAAwLJYYmlVVOABAAAAO0IFHgAAABbFIlbrogIPAAAA2BEq8AAAALAsKvBWRQUeAAAAsCMk8AAAAIAdoYUGAAAAFmVgG0mrogIPAAAA2BEq8AAAALAco6RMG93XQVCBBwAAAOwICTwAAABgR2ihAQAAgAUZbbSI1XF6aKjAAwAAAHaECjwAAAAsy3GK4TZBBR4AAACwIyTwAAAAgB2hhQYAAACWxZNYrYoKPAAAAGBHqMADAADAYgySDDYowBuK/pY2QwUeAAAAsCNU4AEAAGBZ9MBbFRV4AAAAwI6QwAMAAAB2hBYaAAAAWJQh09YR3NuowAMAAAB2hAo8AAAALMco2yxidaB1s1TgAQAAADtCAg8AAADYEVpoAAAAYFkO1M5iC1TgAQAAADtCBR4AAAAWZeBJrFZFBR4AAACwI1TgAQAAYFlU4K2KCjwAAABgR0jgAQAAADtCCw0AAAAsK9PWAdzbqMADAAAAdoQKPAAAACyKbSStiwQeAAAADu3333/X559/ru+//14XL17UjRs3FBAQoBYtWmj48OHy9/c3m5+enq5ly5Zpw4YNioqKUsmSJdW2bVuNHTtWpUqVsnq8JPAAAABwaOvXr9fq1avVunVrderUSe7u7vrll1+0Zs0abd68WWvXrlXVqlVN8ydPnqzNmzerdevWeuaZZxQZGakVK1bo559/1meffabixYtbNV4SeAAAAFiOUbbZB/4ObtmhQwcNHz5cPj4+prG+ffuqQYMGev311zVv3jzNnTtXkrR3715t3rxZbdq00eLFi03za9eurRdeeEHLli3TmDFjCh9MPrCIFQAAAA6tbt26Zsl7tieffFKSdOLECdNYeHi4JGnIkCFmczt06KDAwEDTcWsigQcAAIAFGbMq8EX9upMSfB5iY2MlSb6+vqaxX3/9VU5OTmrQoEGO+Q0bNtSFCxcUHx9v8Vj+ihYaAAAA3BOio6MVEhKS5/Fdu3YV6HrZbTM9evQwjcXExKhUqVJydXXNMb9s2bKmOSVLlizQvQqCBB4AAACWdQ88yOn999/X9u3b1a5dO3Xv3t00npKSohIlSuR6jpubm2mONZHAAwAA4J4QEBBQ4Cp7blasWKHZs2erWbNmmjlzpgwGg+mYu7u70tLScj0vNTXVNMea6IEHAAAA/mf58uV655131Lx5cy1ZskQeHh5mx8uVK6e4uLhck/jsnvly5cpZNUYSeAAAAFiUwWgs8pclLFmyRNOmTdPDDz+sDz74IEfyLkn16tVTZmamfv311xzHDh06pEqVKlm1/10igQcAAAD0/vvva9asWWrdurUWLVpk6mf/u+DgYEnSsmXLzMZ37NihqKgo03FrogceAAAAlmWLBzndgdWrV2v27Nny9fVV+/bttXXrVrPjnp6eateunSSpRYsW6ty5s7Zs2aKRI0eqbdu2ioyM1Mcff6xq1arl2B/eGkjgAQAA4NCOHDkiSbp69apeffXVHMcDAwNNCbwkTZs2TTVq1NDGjRv1f//3fypZsqSCg4M1duxYeXp6Wj1eEngAAAA4tGnTpmnatGn5nu/i4qIRI0ZoxIgRVowqbyTwAAAAsCw7a6GxNyxiBQAAAOwIFXgAAABYFhV4q6ICDwAAANgRKvAAAACwHKOkTBvd10FQgQcAAADsCAk8AAAAYEdooQEAAIBFGVjEalVU4AEAAAA7QgUeAAAAlkUF3qqowAMAAAB2hAQeAAAAsCO00AAAAMCCjFKmLVpoHKdthwo8AAAAYEeowAMAAMCyWMRqVVTgAQAAADtCBR4AAACWRQXeqqjAAwAAAHaECjxuy69iGX1yeoGtwwD+ZDDYOgIgB/fADFuHAJj4uPgp08jX5L2MBB63VcylmMpXKWvrMAAAQAE4G2yY4hllmxYaB+raoYUGAAAAsCNU4AEAAGBZNnmQk+OgAg8AAADYERJ4AAAAwI7QQgMAAADLMmbaOoJ7GhV4AAAAwI5QgQcAAIBl8SRWq6ICDwAAANgRKvAAAACwIKONtpF0nKo/FXgAAADAjpDAAwAAAHaEFhoAAABYjlG2WcTqOB00VOABS9i4caOCgoK0b9++Qp0fGRmpoKAgzZ8/32w8KChIkyZNskSIuItNmjRJQUFB+Z7fpk0bhYSEWDEiAMDdjAo88D+3bt3So48+qj/++EOjRo3Siy++aOuQYMfS0tK0ceNGbdu2TREREUpMTJSnp6eqVaum1q1bq0+fPipRooStw7ytnTt36vjx43r++edtHQqKWFpamsLCwrR9+3ZFRETo+vXrcnV1VcWKFdW4cWMFBwerfv36RRJLZGSkwsLC1K5dO9WqVatI7gkLYBtJqyKBB/7n66+/1h9//KH77rtPGzdu1JgxY+Ts7GzTmA4fPiwnJ35RZm+io6M1cuRInThxQo0bN9bgwYPl5+enxMRE/fzzz5o7d6527NihdevWFer627Zts3DEudu5c6fCwsJI4B1MZGSknnvuOZ08eVKNGzfWgAED5O/vr9TUVJ06dUq7du3S6tWrtWLFCj300ENWjycqKkoLFixQYGAgCTzwPyTwwP+Ehobq/vvv16RJkzRy5Eh99913euyxx2wak5ubm03vj4JLS0vTiBEjdObMGc2cOVNdunQxOz548GDFxMRo1apVhb6Hq6vrnYZpFRkZGUpLS5OHh4etQ0EhpaamauTIkTp37pzmzJmjTp065ZgzZcoUhYWFyd3d3QYRApDogQckZVV4fvjhB3Xv3l2PPPKI/Pz8cq2O3rp1S/PmzVObNm1Ut25dderUSWvWrMnzuklJSZo9e7Y6dOigOnXqqFmzZho1apQiIiLyFVdePfA//vijnn32WTVt2lR16tRRp06dtGTJEmVkZOT/TcMq1q9fr5MnT2rQoEE5kvds5cqV04QJE3KMJyUl6c0331SrVq1Up04dde/eXd99912Oebn1wGePnTt3Ts8995waN26shg0batiwYTp//rzZXKPRqJUrV6pbt26mee3atdP48eN1+fJl0/XCwsIkZX0dZr82btwoSZo/f76CgoJ0+vRpzZgxQ61bt1bdunW1detWSdKePXv00ksvqV27dqpXr54aNWqkp59+Wl9//XWO95O9BiAuLk6TJ0/WQw89pHr16qlv377au3fvP33KYUHr1q3TqVOnNHTo0FyTd0kqVqyYevfurQYNGpiNG41GhYaGqlevXmrQoIEaNGigp556Sjt37sxxjezvbYcPH9bAgQPVsGFDNWnSROPGjdMff/xhmjd//nwNHDhQkjR58mTT1+Ffv/4zMzP1ySefqGvXrqavtYEDB+r777/PNf7du3crJCREjRo1Ur169RQcHKzVq1fLSMuHZRmNRf9yIFTgAWUlXZLUrVs3OTs7Kzg4WB9//LGuXLkiPz8/07yJEyfqyy+/1EMPPaTBgwcrPj5e8+fPV/ny5XNcMykpSf369dOFCxfUrVs31axZU9evX1doaKieeuoprV69WrVr1y5UrFOmTNEDDzygYcOGycfHRz///LPee+89HT9+XLNnzy78JwJ3LDuBfeqppwp87jPPPCNvb28NHz5cKSkpWrFihZ577jnt2LFDAQEB/3h+bGysBgwYoDZt2mjChAk6f/68Vq1apVGjRunzzz83tWO9//77mjNnjh599FH17t1bLi4uio6O1nfffafLly/L399fr776qpYvX66DBw9qxowZpns0atTI7J4TJkyQs7Oz+vfvr+LFi6ty5cqSpLCwMF29elVdu3ZVuXLldO3aNYWFhem5557T7Nmz9cQTT+T6/r28vDRq1CglJCTos88+07PPPqvFixfrkUceKfDnEwW3fft2SVKfPn0KfO7kyZO1adMmtW3b1vTD61dffaXRo0frjTfeUL9+/czmR0REaNiwYQoODtYTTzyho0ePat26dbp+/bqWLl0qSWrfvr3S09P1/vvvq2/fvmrcuLEkydfX13SdSZMmKTw8XI0aNdJLL72kGzduaP369XrmmWc0ffp0BQcHm+auW7dOr732mgICAvTMM8/I09NT27Zt09SpUxUREaE333yzwO8bsAUSeDi8jIwMbdiwQS1atFC5cuUkST179tRHH32kjRs3asSIEZKkvXv36ssvv1T79u01f/58GQwGSVL37t3VuXPnHNedN2+ezp07p9WrV5st9urXr5+6dOmiadOmaeXKlQWK9cqVK5o6daratm2rBQsWmGJ46qmnVLNmTU2fPl39+vVTs2bNCvW5wJ07efKkPD09dd999xX43KCgIE2dOtX0cbNmzdS3b199+umneumll/7x/PPnz2vWrFlmX4+lS5fWrFmz9MMPP6hVq1aSpB07dqhq1apasmSJ2fljx441/bldu3bauXOnDh48aJYA/V3x4sW1YsUKubi4mI2/+eabKl68uNnYoEGD1K1bNy1cuDDXBN7f31+LFi0y/aDRq1cvPfHEE5o6dap27NjBepAicPLkSXl5ealixYpm40ajUXFxcWZjbm5u8vT0lPTneonJkydr8ODBpjmDBg3SyJEjTe1kXl5epmMRERFas2aN2Q+FBoNBn332mc6dO6fKlSurZs2aSkhI0Pvvv68GDRrk+Frcu3evwsPD9cgjj+j99983rVvq16+fOnfurLfeekvt2rWTp6enEhMT9c4778jX11fr169X6dKlJUkDBgzQ8OHDFRoaquDgYDVp0uTOP5GQMjNtHcE9je+GcHi7d+9WbGysevbsaRqrUqWKGjZsqPXr15t+rbpjxw5J0vDhw02JsyRVrFgxR6uE0WjU5s2b1aBBA1WsWFHXrl0zvdLT09WyZUv99NNPSklJKVCs27dvV2pqqnr37q24uDiz62b36+/Zs6cwnwZYSFJSklmSUhBDhw41+7hBgwYqXry4fv/993yd7+/vn+OHyRYtWkiS2TW8vb0VGxur/fv3FyrOvxo6dGiO5F2SWfJ+8+ZNxcXFKTk5WQ8++KBOnz6tpKSkHOeMGDHCLEkvX768goODdfHiRR07duyOY8U/y+vr9+rVq2revLnZ65133jEd37x5s9zd3dWpUyez70vXrl1Tu3btlJSUpF9++cXsmg0aNMjxG52WLVtKUr6/5rO/L48aNcps04HSpUurf//+un79uqkNa8+ePbp586ZCQkJMybuU1RL03HPPmV0PuNtRgYfDCw0Nlbu7u6pXr27WK9yqVSvNnz9fP/74o5o3b64LFy5IkqpVq5bjGn8fi4uLU1xcnA4cOKDmzZvnee+4uLhc22/ycubMGUky/VYgN1evXs339WB5Xl5eunHjRqHO/XvVU5JKlSqVo/JZkPNLliwpSYqPjzeNjR8/XqNHj1ZISIh8fX3VuHFjNW/eXJ07d5a3t3eBYr7//vtzHY+MjNTcuXO1e/dus3tnu379eo5E8XZ/ty5cuKA6deoUKDYUnJeXV64/XJUoUULLly+XlPU95uWXXzY7fubMGaWkpNy21env35vy+/V6OxcvXpQk1ahRI8ex7GcrZM+53dzssezv88DdjgQeDi02NlbffvutMjIycm2DkbJ6zm+XhOcm83+/OmzatKlGjRqV57y/VoEKct233npLgYGBuc7x9/cv0DVhWTVq1ND+/ft1/vz5ArfR3Om2pbc7/68L9OrXr6+vvvpKP/zwg/bt26cDBw5o+/btmjdvnlatWqWqVavm+5657URy48YNDRgwQImJiRo4cKCCgoLk5eUlJycnbdiwQVu2bDF9LePukv31e/HiRbME29XV1fTbnMjIyBznZWZmytvbW/Pmzcvz2n//AS2/X6+wU/w/tCoSeDi0jRs3KiMjQ5MnTzb1v//V+vXrtWPHDsXFxalSpUqSpNOnT6tevXpm806fPm32cenSpeXj46OEhATTP3qWkL1AsESJEha9LiynY8eO2r9/vz777DNNnDjR1uHkycPDQ23btlXbtm0lZbWSDRs2TB9++KGmTZsmSWatYgXx448/6tKlS3r77bfVq1cvs2OhoaF5nnf69Gk1bNgwx5gk098/WFeHDh20f/9+hYaGavz48fk+7/7779fZs2dVq1YtlSpVyqIx3e7r8K/fl//+YKmTJ09K+rPSnz331KlTat26tdncU6dOmc0B7nb0wMNhGY1GrV+/XuXLl9egQYPUsWPHHK+nn35aaWlpCg8P1+OPPy5JWrJkiVl16OLFi/r888/Nru3k5KSuXbvq5MmTpq34/q4wrS6dOnWSm5ub5s+fn2ubRkpKSq6//kbR6d27t2rUqKGPP/5YX375Za5zYmNjNXPmzCKO7E/Xrl3LMZbdnvLX1oXsPvb8tjNky66s/r2KGhERkeuWgtk++OADs8r8pUuXFB4ergoVKuiBBx4oUAwonN69e6tatWpatmyZaUelv8utOt6tWzdJ0n/+859cj99Ja1/212FCQkKOY+3bt5eUtbPSX792rl27pjVr1sjHx8f0G9SWLVuqePHiWr16tdm1MjIytHjxYkkyfZ+HBbCNpFVRgYfD+uGHHxQZGanBgwfnWeFp2bKlvL29tX79em3ZskUdOnTQ9u3bNXjwYLVt21YJCQlau3atqlatqqNHj5qdO27cOB06dEiTJk3Szp071aRJE3l4eOjSpUvau3ev3NzcCrwLTdmyZTV16lS9+uqr6tixo7p3766KFSsqPj5eZ8+e1VdffaWFCxfqwQcfLPTnBXfG1dVVH3zwgUaMGKFx48ZpzZo1euSRR1SmTBnTQr6dO3fa9ImSnTp1Uv369VWvXj2VLVtWCQkJ2rRpk6Q/EzEpq9Vm1apV+r//+z89+uijcnFxUb169XLtXf6rRo0ayc/PT9OnT1dkZKQCAwN15swZhYaGqkaNGjn+rmS7fPmyBg8erPbt2yshIUGffvqpUlNT9frrr7MDTRFxc3PTBx98oOeee05jx47VypUr1apVK/n7+ys5OVkXLlwwJfZ/bePr0KGD+vTpo9DQUEVERKhdu3by8/NTbGysjh49qt27d+f5//2fVKtWTZ6enlqzZo3c3d3l4+Oj0qVLmxbTBgcHKzw8XAMHDlS7du108+ZNrV+/Xn/88YemT59u2inH29tbr776ql577TX17NlTPXv2lIeHh7Zv366ff/5Zffr0YQca2A0SeDis7F/ld+jQIc85rq6uatOmjcLDw3Xo0CHNnDlTVapU0aZNmzRjxgxVqFBBo0ePloeHhyZPnmx2rpeXl9asWaMVK1boyy+/1J49e+Tk5CQ/Pz/Vq1fPLFEqiG7duqly5cpaunSpNmzYoISEBJUoUUIVK1bU0KFDTQu3YDsBAQHasGGDNmzYoK1bt2rp0qVKSkqSp6enqlevrnHjxql37942i++ZZ57Rd999pzVr1uj69esqWbKkatasqcmTJ5t2AZGkzp076/jx4/riiy+0bds2ZWZm6t133/3HBN7Hx0fLli3TzJkztXbtWqWlpSkoKEgzZ87UsWPH8kzkli5dqhkzZmjhwoW6ceOGatWqpWnTppm2v0TRqFChgjZs2KCNGzdq+/btWrlypa5fvy5XV1dVrFhR7du3V/fu3XO0Er755pt68MEH9dlnn2nZsmVKSUmRr6+vqlevrilTphQ6Hnd3d82ePVtz5szRO++8o7S0NDVr1sxUWZ82bZpq166t9evXa9asWSpWrJjq1q2rqVOn5vja6d27t/z9/fXRRx9pyZIlSk9PV+XKlfXaa6/p6aefLnSM+BujUcq0QUXcgarwBiMrRQAANjRp0iSFhYXpxIkTtg4FgAVc+v2KhjZ9rcjvu+zAmyp/v98/T7wH8DtJAAAAwI7QQgMAAACLMhrZKtaaqMADAAAAdoQEHgBgU9OmTaP/HbjXZBqL/uVASOABAAAAO0ICDwAAANgRFrECAADAstil3KqowAMAAAB2hAo8AAAALCuTbSStiQo8ANhAUFCQQkJCzMbmz5+voKAg7du3z0ZRFUxB4500aZKCgoIUGRl5R/cNCQlRUFDQHV3jn1gqVgCwBirwAO5Zf0/ynJyc5OPjo6CgIPXu3VtdunSxUWTWExQUpGbNmmnlypW2DgWAozIabdMD70B99yTwAO55Y8aMkSSlp6fr7Nmz2rVrl/bt26fffvtNkydPtnF0f3r66af1xBNPKCAgwNahAADuYiTwAO55zz//vNnHe/fu1ZAhQ7RixQqFhISoQoUKNorMXOnSpVW6dGlbhwEAuMvRAw/A4TRv3lxVqlSR0WjUkSNHJJn3c3/++efq3bu3GjZsqDZt2pjOS05O1gcffKDg4GA1aNBADRs2VN++fbVly5Zc75OWlqaFCxeqXbt2qlOnjtq0aaPZs2crLS0t1/m36yk/c+aMJk+erDZt2qhOnTpq3ry5+vfvrzVr1kiSNm7caGoZ2r9/v4KCgkyv+fPnm13r119/1QsvvKCWLVuqTp06evTRR/X6668rNjY217h+++03PfPMM2rYsKEaNWqkwYMH69ChQ//wWc6/jRs36vnnn1fbtm1Vr149NWrUSE899ZTCw8Nve15aWppmz55t+py0a9dOCxYsyPPze+bMGU2aNEmPPvqo6tSpoxYtWmj8+PE6e/asxd4LgCzGzMwifzkSKvAAHJLxf72SBoPBbHz58uX6/vvv1bp1az344INKTEyUJF2/fl2DBg3SsWPHVLt2bfXs2VOZmZnas2ePxo8fr1OnTmncuHFm1x87dqx27dqlSpUqacCAAbp165Y2bNigkydPFijW//73v3rxxReVlpamhx9+WE8++aSuX7+uEydO6KOPPlL//v1Vq1YtjRkzRgsWLFBgYKC6d+9uOr9Zs2amP69fv16vv/66XF1d1aZNG5UrV07nz5/XunXr9PXXXys0NNSshefnn3/WkCFDdOvWLbVv31733Xefjh8/rpCQED300EMFeh95eeONN1StWjU1bdpUfn5+io+P17fffquJEyfq3LlzGjt2bK7nvfjiizpy5Ig6duyoYsWKadeuXZo/f75+++03LV682Oz/7e7du/X8888rPT1drVu3VqVKlRQbG6sdO3bov//9rz755BPVrl3bIu8HAKyNBB6Aw/nhhx907tw5GQwG1a1b1+zYjz/+qM8++0wPPPCA2fg777yjY8eOacKECRo2bJhpPDU1VaNGjdIHH3ygjh07qlatWpKkLVu2aNeuXWrQoIE++eQTubm5Scpq5+nVq1e+Y7127ZrGjx+vjIwMrVixwiwZl6SYmBhJUq1atVSrVi1TAv/3tiFJOnfunN544w0FBgZq1apVKlu2rOnY3r17NXToUL399ttauHChpKwfQl599VWlpKSYfpOQbcWKFXrnnXfy/T5uZ8uWLapUqZLZWFpamoYNG6YPP/xQ/fr1M4s129mzZ/XFF1+oRIkSkqRx48Zp4MCB+uabbxQeHq5u3bpJkhISEjR+/Hi5u7tr9erVqlatmukaJ0+eVN++fTVlyhSFhYVZ5P0AkEMtKLUFWmgA3PPmz5+v+fPna/bs2XrhhRf07LPPymg0atCgQQoMDDSb26dPnxzJe1xcnDZv3qw6deqYJe+S5ObmppdffllGo1Gff/65aXzjxo2SspLK7ORdkkqWLKlRo0blO/ZNmzYpKSlJTz31VI7kXZLKlSuX72utXbtWt27d0r/+9a8cCXHz5s3Vpk0bffPNN0pKSpKUVX0/d+6cmjZtapa8S9KAAQNyJN2Fldt1XF1d9fTTTys9PV179+7N9bznnnvOlLxLWf8vXnrpJUnShg0bTOObNm3S9evX9cILL5gl75JUo0YN9e7dW8eOHdPp06ct8XYAwOqowAO45y1YsEBSVruMj4+PGjdurF69eik4ODjH3Hr16uUYO3LkiDIyMmQwGHL0k0tZu9tIMuulPnbsmJycnNS4ceMc83NLxPPyyy+/SJIeeeSRfJ/zT9fav3+/qff/r/744w9lZGTo999/V506dXTs2DFJUtOmTXPMdXZ2VuPGjXXhwoU7jis6Oloffvih9u7dq0uXLiklJcXseF69+bl9Hhs3bixnZ2cdP37cNJb9viMiInL9//f7779LyuqR/3uCDwB3IxJ4APe8EydO5Huur69vjrH4+HhJWYl8bolvths3bpj+nJiYqBIlSsjFxSXHPD8/v3zHk92Dn1sLSUFlv4+lS5fedt7NmzfN7p3b5+R24wVx8eJF9erVS9evX1eTJk3UqlUreXl5ydnZWVFRUQoLC8tzUWpu9y9WrJhKlSqlP/74wzSW/b5DQ0NvG0v2+wZgAZm00FgTCTwA/MXfF7VKkre3tyRp8ODB+d433tvbWwkJCbp161aOJP7KlSv5jif73rGxsXf89FEvLy9J0k8//WT6c37uffXq1VyP5zVeEMuXL1d8fLzeffdd9ejRw+zYli1bbtuXfvXq1Rx75qenpysuLs7s/WW/j/DwcNWsWfOOYwYAW6MHHgD+Qb169eTk5KSDBw/m+5wHHnhAmZmZ+umnn3Ic279/f76v06BBA0lZu6jkh5OTkzIyMm57rfy+j+y1AAcOHMhxLCMjI9f3VlDnz5+XJD3++OM5jv3T5ym34z/99JMyMjJMi4klqX79+qZjAIqIMbPoXw6EBB4A/kGZMmXUpUsX/fbbb1q4cGGuCfKFCxd08eJF08fZ1eQ5c+YoNTXVNB4fH6/Fixfn+97dunWTl5eXPv3001wT6exdaLKVLFkyx1i2p59+Wi4uLnr33Xd17ty5HMfT0tLMkvtGjRqpcuXKOnDggHbu3Gk2d9WqVRbpf89eRPz3ZPy7777T+vXrb3vu4sWLlZCQYPo4NTVV7733niSpZ8+epvEePXrIx8dHCxYs0OHDh3NcJzMzM9e99wHgbkULDQDkw+uvv67z589r3rx52rx5sxo1aiRfX19dvnxZZ86c0ZEjR/Tee++pYsWKkqTOnTvryy+/1Ndff63OnTurbdu2Sk9P17Zt21S3bt18J7+lS5fWrFmz9MILL2jgwIF65JFHFBQUpKSkJJ04cUKXLl3S119/bZrfvHlzffHFFxo5cqQeeOABFStWTE2bNlXTpk1VtWpVvf322/rXv/6lzp076+GHH9b999+v9PR0RUdH66efflKpUqW0bds2SVntRG+//baGDh2qF154wWwf+L179+rhhx/Wd999d0ef1/79+2vjxo168cUX1aFDB/n7++vUqVP67rvv1KlTJ3355Zd5nlulShU9+eSTZvvAX7hwQY899pjZAuVSpUpp3rx5Gj16tPr06aPmzZurWrVqMhgMiomJ0aFDhxQfH3/b9Q0ACsAoGW3RA+9Abfck8ACQD15eXlq5cqVCQ0O1ZcsW7dixQ6mpqfL19dV9992nyZMnq0WLFqb5BoNBc+fO1ZIlSxQWFqZVq1bJ399fPXv21OjRo3PsP387jz32mDZs2GDaqeX777+Xj4+PqlSpohEjRpjN/de//iWDwaC9e/fq22+/VWZmpsaMGWPaSSY4OFg1a9bU8uXLtW/fPu3Zs0fFixeXv7+/OnTooE6dOpldr3Hjxlq9erVmz55tauOpX7++Vq5cqT179txxAl+zZk198sknmjNnjr799lulp6erZs2aWrBggby9vW+bwM+dO1cLFy7U559/rsuXL6ts2bJ6/vnnNXz48BxrGZo3b67Nmzdr2bJl2rNnjw4ePCgXFxf5+/vroYceUocOHe7ofQBAUTIYjey0DwAAAMu4dPayBtUcW+T3XRExR+Wr+Bf5fW2BCjwAAAAsyGijRaWOU5NmESsAAAAgaceOHerTp48aNGigpk2bauTIkTp58qStw8qBBB4AAAAWZcw0FvnrTq1bt07PP/+8kpOTNWHCBI0cOVInTpzQU089VaAHAhYFWmgAAADg0BISEjRt2jSVK1dOa9euNT0MrlOnTnryySf19ttv65NPPrFxlH+iAg8AAACHtmvXLiUlJal3795mT3IOCAhQhw4dtG/fPl26dMmGEZojgQcAAIBl2dmTWH/99VdJUsOGDXMcyx67m54VQQsNAAAALMa/kq8+Ob3AJveNjo5WSEhInnN27dqV63hsbKwkqVy5cjmOZY/l9ZRrWyCBBwAAgMU4F3NW+SplbXLvK1euFOq85ORkSZKrq2uOY9ljKSkphQ/MwkjgAQAAcE+oX79+nlX22/Hw8JAkpaWl5TiWPebu7n5nwVkQPfAAAABwaGXLZv3GILc2meyx3NprbIUEHgAAAA6tXr16kqRDhw7lOPbLL79IkurWrVuUId0WCTwAAAAcWrt27eTp6al169YpKSnJNB4dHa1t27apWbNmKl++vA0jNGcwGo13/ugqAAAAwI59+umn+ve//60aNWqob9++SktL06pVqxQXF6e1a9eqZs2atg7RhAQeAAAAkLRt2zYtXbpUJ0+elIuLi5o0aaKxY8feVcm7RAIPAAAA2BV64AEAAAA7QgIPAAAA2BESeAAAAMCOkMADAAAAdoQEHgAAALAjJPAAAACAHSGBBwAAAOwICTwAAABgR0jgAQAAADtCAg8AAADYERJ4AAAAwI6QwAMAAAB2hAQeAAAAsCP/D7+odhkE9gDSAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "conf_mat_disp = ConfusionMatrixDisplay(conf_mat, display_labels=np.unique(y_true))\n", "conf_mat_disp.plot()\n", "\n", "plt.gcf().set_size_inches(8, 8)\n", "\n", "# seaborn turns on grid by default ... looks best without it\n", "plt.grid(False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## In Class Exercise 2\n", "\n", "Build a K-NN classifier which estimates whether a passenger on the titanic `survived` given their `age`, `pclass` and `fare` features.\n", "- Discard any passengers which are missing a feature\n", "- Be mindful of scale normalization, you may need to adjust data a bit\n", "- Show the output of your classification as a confusion matrix plot, as shown above" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'1.2.1'" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import sklearn\n", "sklearn.__version__" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
survivedpclasssexagesibspparchfareembarkedclasswhoadult_maledeckembark_townalivealone
003male22.0107.2500SThirdmanTrueNaNSouthamptonnoFalse
111female38.01071.2833CFirstwomanFalseCCherbourgyesFalse
213female26.0007.9250SThirdwomanFalseNaNSouthamptonyesTrue
311female35.01053.1000SFirstwomanFalseCSouthamptonyesFalse
403male35.0008.0500SThirdmanTrueNaNSouthamptonnoTrue
\n", "
" ], "text/plain": [ " survived pclass sex age sibsp parch fare embarked class \\\n", "0 0 3 male 22.0 1 0 7.2500 S Third \n", "1 1 1 female 38.0 1 0 71.2833 C First \n", "2 1 3 female 26.0 0 0 7.9250 S Third \n", "3 1 1 female 35.0 1 0 53.1000 S First \n", "4 0 3 male 35.0 0 0 8.0500 S Third \n", "\n", " who adult_male deck embark_town alive alone \n", "0 man True NaN Southampton no False \n", "1 woman False C Cherbourg yes False \n", "2 woman False NaN Southampton yes True \n", "3 woman False C Southampton yes False \n", "4 man True NaN Southampton no True " ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_titanic = sns.load_dataset('titanic')\n", "df_titanic.head()" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "\n", "df_titanic.dropna(how='any', inplace=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['survived', 'pclass', 'sex', 'age', 'sibsp', 'parch', 'fare',\n", " 'embarked', 'class', 'who', 'adult_male', 'deck', 'embark_town',\n", " 'alive', 'alone'],\n", " dtype='object')" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_titanic.columns" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "from sklearn.neighbors import KNeighborsClassifier\n", "import pandas as pd\n", "import seaborn as sns\n", "\n", "k = 11\n", "x_feat_list = ['age', 'pclass', 'fare']\n", "y_feat = 'survived'\n", "\n", "df_titanic = sns.load_dataset('titanic')\n", "df_titanic.dropna(how='any', inplace=True)\n", "\n", "# scale normalization (overwrites old data)\n", "for feat in x_feat_list:\n", " df_titanic[feat] = df_titanic[feat] / df_titanic[feat].std()\n", "\n", "# extract data into numpy format (for sklearn)\n", "x = df_titanic.loc[:, x_feat_list].values\n", "y_true = df_titanic.loc[:, y_feat].values\n", "\n", "# initialize a knn_classifier\n", "knn_classifier = KNeighborsClassifier(n_neighbors=k)\n", "\n", "# fit happens \"inplace\", we modify the internal state of knn_classifier to remember all the training samples\n", "knn_classifier.fit(x, y_true)\n", "\n", "# estimate each penguin's species\n", "y_pred = knn_classifier.predict(x)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": { "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAIyCAYAAAAaF8SnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABBY0lEQVR4nO3deVxVdf7H8fcFQVBU3ABBrcwUc1+wTJ1yadQ0MZcyCx2cyfy5Zdk4WU3TzLTYtJiVWpaaW7a6pWmWNpVlouZCoVRqLiCYCyrKInB+fzDcGeJeQ7j3ey/c1/Px4GGec+49n0ulH97nc77HZlmWJQAAALidn6cLAAAA8BU0XgAAAIbQeAEAABhC4wUAAGAIjRcAAIAhNF4AAACG0HgBAAAYQuMFAABgCI0XAACAIVU8XQAAAPBOlpUn5R9z/4n8G8hm842WxDc+JQAAuHz5x2Sd6OX209jqbZSqNHL7ebwBjRcAAHDCUoEK3H4Wf/nOY6OZ8QIAADCExAsAADiVb5lIvHwHiRcAAIAhJF4AAMAhS1KBgfkrS5LN7WfxDiReAAAAhpB4AQAAp0zc1ehLSLwAAAAMIfECAABO5Vu+s8aWCSReAAAAhpB4AQAAhyxZhu5q9J1UjcQLAADAEBIvAADgVL4PpVEmkHgBAAAYQuMFAABgCJcaAQCAUyaG630JiRcAAIAhJF4AAMAhS2YWUPWlTI3ECwAAwBASLwAA4BSPyHYtEi8AAABDSLwAAIBTLKDqWiReAAAAhpB4AQAAhwrvajRzHl9B4gUAAGAIiRcAAHCKuxpdi8QLAADAEBIvAADgVL5sni6hUiHxAgAAMITECwAAOGRJKuCuRpci8QIAADCExAsAADjFjJdrkXgBAAAYQuMFAABgCJcaAQCAU1xqdC0SLwAAAENIvAAAgEOFy0m4P/FiOQkAAAC4HIkXAABwwmZoxst35shIvAAAAAwh8QIAAA5ZkvINZDTMeAEAAMDlSLwAAIBjlpm7Gn0p8iLxAgAAMITECwAAOMXK9a5F4gUAAGAIiRcAAHDIkpRvcVejK5F4AQAAGELiBQAAnLCpwEhG4ztzZCReAAAAhvhc4pWXl69fUjM8XQZQaeUH+3u6BKDSiqhTQ/n5lqoGmvvrm7saXcvnGq9fUjM0uvsTni4DqLTS+zbydAlApbX6idGSpIb1Qz1bCMrM5xovAABQOtzV6HrMeAEAABhC4wUAAGAIlxoBAIBTBQzXuxSJFwAAgCEkXgAAwKl8MhqX4rsJAABgCIkXAABwyJLN0HISvjNHRuIFAABgCIkXAABwysxDsn0H300AAABDSLwAAIBT+ZbvzF+ZQOIFAABgCIkXAABwyJKZdbx4SDYAAABcjsQLAAA4YVOBgXW8xDpeAAAAcDUSLwAA4BTPanQtvpsAAACGkHgBAACHLJlZx6u8dzXOnTtXSUlJSkpK0uHDh+Xn56ekpCSnx+fl5Wn+/Pn64IMPlJKSotDQUPXq1UuTJ09W7dq1Sxx/+vRpvfjii9q4caMyMjIUFRWloUOHKj4+XlWqXF4rReMFAAAqtOeff141a9ZUixYtdOHCBZ06deqSx0+bNk2rV69Wjx499Mc//lFHjx7VwoUL9e233+qdd95RtWrV7MdmZmbq7rvv1sGDBzVixAg1b95c27Zt03PPPacDBw7o6aefvqxaabwAAECF9sknn6hx48aSpLi4uEs2Xlu2bNHq1avVs2dPzZkzx769ZcuWmjRpkubPn68JEybYt8+bN08//fSTHnroIcXHx0uShg0bpho1amjJkiUaPHiwYmJiSl0rM14AAMCpAvm5/au8ipqu0li1apUk2ZuoIn369FFUVJR9//8eHxwcrDvvvLPY9qLXr1y58rJqpfECAAA+Y/fu3fLz81O7du1K7Gvfvr0OHz6sjIwMSdKJEyeUkpKi6OhoBQUFFTu2YcOGql+/vvbs2XNZ5+dSIwAAcMyyKd/EAqqWTampqYqLi3N6yMaNG11yqrS0NNWuXVuBgYEl9oWHh9uPCQ0NVVpamiQpIiLC4XtFRETo8OHDl3V+Ei8AAOAzsrOzHTZdklS1alX7Mf/766WOz8rKuqzzk3gBAACHLEkFBh7nY0mKjIx0Wap1KUFBQcrNzXW4Lycnx37M//56qeODg4Mv6/wkXgAAwGdERETo9OnTDpup9PR0+zH/+2vRJcdfS0tLs1+eLC0aLwAA4FS+5ef2L5PatGmjgoIC7d69u8S+nTt3qnHjxgoNDZUk1atXT5GRkdq3b5/9smORlJQU/fLLL2rTps1lnZ/GCwAA+IzY2FhJ0vz584tt37Bhg1JSUuz7iwwcOFBZWVlatmxZse0LFiwo9n6lxYwXAABwyJKZh2SX95FBK1euVGpqqqTCJMqyLM2ePdu+f9y4cfZ/vuGGGzRgwACtWbNGY8eOVa9evXT06FG9+eabatq0aYn1ve655x59/PHHevbZZ5WSkmJfuX7VqlWKjY1V586dL6tWm2VZ5f28Fcqxwyc1uvsTni4DqLTS+zbydAlApbX6idGSpIb1Q42c70xuqpYevMvt57nrqqWqFRhZ5tfHxcUpISHB6f7k5ORiv7948aLmz5+v5cuX25/V2LNnT02ePFl16tQp8fpTp07pxRdf1KZNm+zPahwyZIhGjx592c9qpPEC4FI0XoD7eKLxWnzgbrefJ67JknI1XhUJM14AAACGMOMFAACcsBmZ8ZKBtcK8BYkXAACAISReAADAqQLD62xVdnw3AQAADCHxAgAADhWu42XmWY2+gsQLAADAEBIvAADgFDNersV3EwAAwBAaLwAAAEO41AgAABxiuN71SLwAAAAMIfECAABO2AwN1/PIIAAAALgYiRcAAHAqn+UkXIrvJgAAgCEkXgAAwKkCH5q/MoHECwAAwBASLwAA4JAlMzNerOMFAAAAlyPxAgAAjllSgWVgxsuHIi8SLwAAAENIvAAAgEOWbMo3kNFYPnTnJIkXAACAISReAADAKSMzXj6ExAsAAMAQEi8AAOBUARmNS/HdBAAAMITGCwAAwBAuNQIAAKfyGa53KRIvAAAAQ0i8AACAQ5bMLCfhQ08MIvECAAAwhcQLAAA4YVOBZSKj8Z05MhIvAAAAQ0i8AACAU/k+lEaZQOIFAABgCIkXAABwiLsaXY/ECwAAwBASLwAA4JSZuxp9B99NAAAAQ0i8AACAUwXc1ehSJF4AAACGkHgBAACHLEvKN3FXow/d1kjiBQAAYAiJFwAAcIJnNboaiRcAAIAhNF4AAACGcKkRAAA4ZeKRQb6ExAsAAMAQEi8AAOAUC6i6FokXAACAISReAADAIUtmZrx8aP1UEi8AAABTSLwAAIBTZhZQ9R18NwEAAAwh8QIAAE7YDK3j5Tt3TpJ4AQAAGELiBQAAnGIdL9ci8QIAADCExAsAADjEOl6uR+IFAABgCIkXAABwysxdjb6DxAsAAMAQEi8AAOCYZSjx8qEhLxovGNe15zG1bn9SVzU7qybXnFO16nn6bF2knnu8vdPX+PlZ6j3giHrdkqIrrz6rgMACnT5ZVT8khWrxa82UeiTE4CcAKp5+bX7QP4dskiT9c9WNWvVtC/u+q8NO6s7rE9Ui8heF1Tyv6lVzdep8sA6dCNX721rqs71XyZcWuATcicYLxg2P/0lNmp3VhfP+Onk8SNWuyrvk8UHBefrrs9vVLuak9ifX1KcfNdTFHH/VDctWy7anFNX4PI0XcAnhNTM19ZbNOp8ToOpVL5bY3yLyF90U/bMSj4Zp95EIZWYHql7IBXVvfkjPDt+gtbua6W8renqgcngDZrxci8YLxr3+YgudOB6s1CPV1LrDKU2f880lj5/wUKLaxZzUy9Nbaf2KK0rs9/cvcFepQCVg6bFBn+lMVpA27b1KI7vuLnHEx4nXaM2u6BLbq1fN1YI/rVD/dj/o3YSW+j4l3ETBQKXmtcP1GzZs0O2336527dopJiZGY8eO1Q8//ODpsuACe3bUU+qR6irNpYurm59Rj76p+vyTBg6bLknKz/fa/4wBjxt+faJirkrR31fepOxcxz9rX8z3d7j9fE6gvtnfUJLUqM4Zt9UI+BKv/Bvrvffe08SJE5WVlaUHH3xQY8eOVXJysoYPH67k5GRPlweDbuqTIkn6YkOkqlW/qB59j2rYqJ/Ud9BhNWh43sPVAd7tynqnNaH3Vr29tbV2Hoq87NdXDbioTlelSpJ+Ol7X1eWhArBU+Mggd3/50Gy9911qPHPmjKZPn66IiAgtW7ZMISGFszv9+vVT//799eSTT2rRokUerhKmXNOi8Kfs+hFZemP5Z6oV+t/5lIIC6aPlV+i151uqoIAZBOB/+fsV6B+DNyn9TIhmfXpdqV7TsM4Z3dLmB/n5WapTPUvdmh1SWM0Lmv9Fe/2UTuMFuILXNV4bN25UZmam4uPj7U2XJEVGRqpPnz5asWKFjh07pgYNGniwSpgSWjtHknTPfXu15YtwLX61uU4cD1Lzlhka/1CiBgw9pDOnA/XWG808XCngXf504w41b3BCf5oXq5y80v1R36jOGY3pscP++9w8P7348fVa8nVbd5WJCoDhetfyukuNu3cXDn62b19yaYGibYmJiUZrgufY/vNf6JFD1fXMIx109FCIsrOqaPf2enp6Wkfl50u3jTioKlUYsAeKtIxKV3z3b7X06zZKPBpR6tdt+amxOv1trK77+z2KffFOzf+ig8b3StALI9arin++GysGfIfXNV7p6emSpIiIkn9YFG1LS0szWhM85/y5AElSwubwEpcTD/5YU+mp1VStep4aXZXpifIAr1N0ifHwyVDN2dS5TO+RX+CvlNO19MbnnfTqZzH6XfNDGn4dP/D6JpsKLPd/+dI6cV53qTErK0uSFBgYWGJf0bbs7GyjNcFzjh6uruatMuwN2K9l/md7YFV+GgckKTjwoq6oVzgbueWx1x0e89fYz/XX2M/11pbWemF910u+39c/NtLEm7eq45WpWvJ1O1eXC/gcr2u8goODJUm5ubkl9hVtCwoKMloTPGdXQj31uiVFV1x9rsS+KgH5imxUeGfj8dRg06UBXulinr9W7ii5JpckRTc4oejIE9p5KEKHToQq8chvr8sVVrPw/7H8Aq+7QAIDLJmZ8eKuRg8KDy/8gyAtLU1XX311sX1FlxgdXYZE5fTVZxEaNW6fuvdO1YfvXqkfkkLt++4c/ZNCauRp9/a6On2KZhyQpJy8Knpi9U0O9425aZuiI09oza7mxR4Z1CLyuPamhpU4PrRalib03ipJ2vxjY7fUC/gar2u82rRpo7fffls7d+5U167FI/Bdu3ZJklq3bu2ByuAq1/8uTV1uLJzlq1238K7F6NYZuv+vhTdWnD0ToHkvXStJysmuohn/bKvHn9+uf722RV//O0Inf6mqZi0z1KrdaZ0+FahXpvPfA1Aejw78XLWqZSspJUxpZ0KUX+CnyNBz6nrNYQUF5umzvVdq9beOUzRUftzV6Fpe13j17t1bTz75pN577z394Q9/sC8pkZqaqvXr16tz584sJVHBNWl2Vr0HHC22rUHDC2rQ8IIkKT012N54SdKuhPq6f3RXDR/9o9rFnFC1kIs6fbKq1n7QWG/Pv0anTpB2AeWx5Ou2uin6ZzVv8Iuuv/qIAvwLlHEhSNsORuqj3c30yfdXy5eGnwF3slmW5XWXVt9++2397W9/U7NmzXTHHXcoNzdXS5Ys0enTp7Vs2TJFR5f9J69jh09qdPcnXFgtgP+V3reRp0sAKq3VT4yWJDWsH2rkfKlZJzXi66fdfp63bpimyGDfWKTX6xIvSRo+fLhCQ0M1b948PfvsswoICFCnTp00efLkcjVdAAAAnuSVjZck9e3bV3379vV0GQAA+LQCLjO7FPcHAwAAGOK1iRcAAPAwy9BdjV43be4+JF4AAACGkHgBAACHLElWBVi5PjMzUwsXLtT69et19OhRBQYGqmHDhho8eLBuv/12BQT897FzWVlZmjVrlj766CMdP35cYWFh6t+/v8aNG2d/eo470XgBAIAKKy8vT6NGjVJSUpIGDRqku+66S7m5udqwYYP+8Y9/aOfOnXruueckSfn5+RozZowSEhIUGxurmJgY7du3T/PmzdOePXu0YMEC+fm592IgjRcAAKiwEhIS9N1332n06NH6y1/+Yt9+1113aciQIVq7dq0ef/xxhYSEaMWKFUpISFBcXJweffRR+7FRUVF65plntHr1ag0aNMit9TLjBQAAnCqwbG7/Ko9z585JksLCij9v1N/fX/Xq1ZO/v78CAwMlSatWrZIkxcfHFzt2xIgRCgoK0sqVK8tVS2mQeAEAgAqrQ4cOqlatmubOnavw8HC1a9dOOTk5WrdunTZv3qxJkyYpMDBQlmUpMTFRYWFhioqKKvYeQUFBatGihRITE91eL40XAABwwmZkuF6yKTU1VXFxcU6P2Lhxo8Pt9evX1+zZs/X444/r/vvvt2+vWrWqnnzySQ0ZMkSSlJGRoaysLF1zzTUO3yc8PFw7d+5UZmam/TnR7kDjBQAAKrSQkBBdddVV6ty5s7p27ars7GytWLFCf/3rX2Wz2TR48GBlZ2dLkv2y469VrVpVUuFdjzReAADAI4wsoCopMjLSaap1Kfv27dOIESM0atQoPfjgg/btAwcO1J133ql//OMfuummmxQUFCRJys3Ndfg+OTk5kuT2JSUYrgcAABXWwoULlZubW+L5zn5+furTp4+ysrK0Z88ehYaGKjg4WGlpaQ7fJz09XSEhIW5NuyQaLwAAcAmW5f6v8jh+/LgkqaCgoMS+vLw8+682m02tWrXS8ePHlZKSUuy47Oxs7d27V61bty5fMaVA4wUAACqspk2bSpKWL19ebPvFixe1Zs0a+fv72xuq2NhYSdKCBQuKHbts2TJlZ2fb97sTM14AAMAhS1KBvPuRQaNGjdKqVau0bNkypaWlqXv37srKytLq1auVnJys+Ph4hYeHS5IGDx6slStXavHixTp37pw6deqk5ORkvfXWW+rcubMGDhzomg90CTReAACgwoqMjNT777+v2bNn6+uvv9aXX36pgIAAXXPNNXriiSc0dOhQ+7H+/v6aO3euZs2apXXr1mnt2rWqX7++4uPjNX78ePn7+7u9XhovAADglJl1vMqnYcOGeuqpp0p1bPXq1TV16lRNnTrVzVU5xowXAACAISReAADAKVPrePkKEi8AAABDSLwAAIBjLlhnq7Tn8RUkXgAAAIaQeAEAAKcqwl2NFQmJFwAAgCEkXgAAwCkSL9ci8QIAADCExgsAAMAQLjUCAACHLNmMLKBqGXgQt7cg8QIAADCExAsAADhlZAFVH0LiBQAAYAiJFwAAcIrlJFyLxAsAAMAQEi8AAOAUiZdrkXgBAAAYQuIFAACc4qZG1yLxAgAAMITECwAAOMWMl2uReAEAABhC4gUAAByzZGbIy4cGyUi8AAAADCHxAgAATjHj5VokXgAAAIaQeAEAAIcsSZaB+SsfGvEi8QIAADCFxgsAAMAQLjUCAACnGK53LRIvAAAAQ0i8AACAcyReLkXiBQAAYAiJFwAAcMrEchK+hMQLAADAEBIvAADgHImXS5F4AQAAGFKqxKtXr15lenObzaZPP/20TK8FAAAeZhlax8uHUrVSNV5WGSfryvo6AACAyqhUjdemTZvcXQcAAPBGZCguxYwXAACAIS65q/HMmTO6cOGCGjRo4Iq3AwAAXsFm6FmNvrM6fpkTr/Pnz2v69Onq2rWrrr/++mID+Lt379Y999yj77//3iVFAgAAVAZlarzOnTun4cOH680331RYWJiuvvrqYoP0zZo10/bt27VmzRqXFQoAADzAMvDlQ8rUeM2ZM0c//vijpk+frhUrVqhv377F9gcHB6tz58765ptvXFIkAABAZVCmxuuTTz5Rt27dNGjQIKfHREZGKj09vax1AQAAr2Az8OU7ytR4paWlqXnz5pc8plq1ajp37lyZigIAAKiMynRXY/Xq1XXq1KlLHnP06FHVrl27TEUBAAAv4WMzWO5WpsSrdevW+uyzz5SZmelw//Hjx/XFF1+oY8eO5SoOAACgMilT4zVy5EhlZGRozJgx2r9/f7F9+/fv13333aecnBzFxcW5pEgAAIDKoEyXGrt3764JEybolVde0YABA1SlSuHbXHfddTp79qwsy9KDDz6oDh06uLRYAABgGJcaXarMK9dPmDBBnTp10uLFi7V7925lZGTIZrPpxhtv1KhRo9SlSxdX1gkAAFDhleuRQddff72uv/56V9UCAAC8jZFHBvkOHpINAABgSLkSr6NHj2rVqlXau3evzp07pxo1aqhFixYaOHCgGjVq5KoaAQCAh1jMeLlUmRuv+fPna8aMGcrLyyv2nMZPP/1Uc+bM0ZQpUxQfH++SIgEAACqDMjVea9as0b/+9S/VqlVLcXFx6ty5s+rVq6cTJ05o69atWrx4sf71r38pPDxct9xyi6trBgAAJph6iLUPpWplarzmz5+vWrVqafny5YqKirJvb9KkiTp37qxBgwZpyJAhmjdvHo0XAADAf5RpuH7//v3q27dvsabrfzVq1Eh9+/bVTz/9VK7iAACAh1k293/5kDI1XtWrV1fNmjUveUzNmjUVEhJSpqIAAAAqozI1Xl27dtXmzZud7rcsS1999ZW6du1a5sIAAIDn2Sz3f/mSMjVef/7zn3XmzBk98MADSklJKbYvNTVVU6ZM0dmzZ/XnP//ZJUUCAABUBqUarh85cmSJbTVr1tS6deu0YcMGNWjQQHXr1tXJkyd17Ngx5efnq3nz5nrwwQe1cOFClxcNAAAM8bFEyt1K1XglJCQ43ZeXl6cjR47oyJEjxbbv27dPNptvDcwBAABcSqkar3379rm7DgAA4I187K5Dd+NZjQAAAIaU61mNAACgkmPGy6XK3XilpaUpPT1dubm5DvfHxMSU9xQAAACVQpkbr82bN+vpp5/WgQMHLnnc3r17y3oKAACASqVMM167du3S2LFjdfbsWd11112yLEudOnXSsGHD1KRJE1mWpR49emj8+PGurhcAAJhkGfjyIWVKvF577TUFBgbq/fffV3h4uJYsWaLrrrtOEyZMkGVZeumll/Tmm2/q/vvvd3W9AAAAFVaZE6+ePXsqPDzcvs2yCltWm82m++67T02aNNHLL7/smioBAIB5JtIuH0u9ytR4nTt3TpGRkfbfBwQE6MKFC8WO6dChg7Zt21a+6gAAACqRMl1qrFu3rs6cOVPs979euT4vL0/Z2dnlqw4AAHgWC6i6VJkSryuvvLJYo9W2bVt99dVXOnjwoCTpl19+0YYNG3TllVe6pEgAAIDKoEyNV/fu3ZWQkKCMjAxJhQ/RzsnJ0W233aYhQ4aoX79+OnXqlEaNGuXKWgEAgGE2y/1fvqRMjdfw4cO1dOlSValSeKWyY8eOmjlzpho2bKgff/xR9evX1+OPP65Bgwa5slYAAIAKrUwzXiEhIWrbtm2xbTfffLNuvvlmlxQFAAC8hI8lUu7GQ7IBAAAMofECAAAwpFSXGnv16lWmN7fZbPr000/L9FoAAIDKplSNV9Gq9JerrK8DAADewdfuOnS3UjVemzZtcncdAAAAlV6Z7mqs0PLylHfkqKerACqt7X9f4+kSgErLVnvYf/4p1NxJWbnepRiuBwAAMMT3Ei8AAFB6FWTGKzMzU6+//ro2bNiglJQUBQUF6YorrtDdd9+t2NhY+3FZWVmaNWuWPvroIx0/flxhYWHq37+/xo0bp+DgYLfXSeMFAAAqtPT0dI0cOVKnT5/WbbfdpqZNmyorK0s///yzUlNT7cfl5+drzJgxSkhIUGxsrGJiYrRv3z7NmzdPe/bs0YIFC+Tn596LgTReAADAuQqQeE2dOlXnz5/XqlWr1KBBA6fHrVixQgkJCYqLi9Ojjz5q3x4VFaVnnnlGq1evdvvjDpnxAgAAFdaOHTv0zTff6E9/+pMaNGig/Px8nT9/3uGxq1atkiTFx8cX2z5ixAgFBQVp5cqV7i6XxgsAAFRcn3/+uSSpcePGmjhxotq2basOHTqoW7dumj17tvLz8yUVri2amJiosLAwRUVFFXuPoKAgtWjRQomJiW6vl0uNAADAMcvQAqqWlJqaqri4OKeHbNy40eH2/fv3S5IeeeQRNWzYUE888YQkadmyZZo5c6aOHTumf/7zn8rIyFBWVpauueYah+8THh6unTt3KjMzUyEhIeX8QM6Vq/Hat2+f1qxZo/379ysrK0tvvvmmJOno0aPas2ePunbtqlq1armiTgAAgBKKLisGBwdr6dKlCgwMlCTdcsst6t+/v9577z3Fx8fb71gs2v9rVatWlVR416NXNl4zZ87Ua6+9poKCAkmFz2UsYlmWpkyZoocffviS3SsAAPByhobrIyMjnaZalxIUFCRJuvXWW4s1VYGBgbr11ls1a9Ysbd26VX379pUk5ebmOnyfnJwcSXL7khJlmvFau3at5syZoxtuuEErV67UvffeW2x/o0aN1KpVKx41BAAA3CoiIkKSVL9+/RL7iradOXNGoaGhCg4OVlpamsP3SU9PV0hIiFvTLqmMjdfixYt1xRVXaPbs2YqOjlZAQECJY66++modOnSo3AUCAAAPsgx8lUO7du0kSceOHSuxr6jJqlu3rmw2m1q1aqXjx48rJSWl2HHZ2dnau3evWrduXb5iSqFMjVdycrK6devm9DqpJIWFhenEiRNlLgwAAOC39OrVSzVr1tSqVauUmZlp337+/HmtWLFCAQEB6tatmyTZV7BfsGBBsfdYtmyZsrOzi61w7y5lnvH635kuR06cOGEfVAMAABWTkbsay6FGjRp65JFH9Je//EVDhw7V0KFDZbPZ9MEHHyg9PV3333+/fVHVwYMHa+XKlVq8eLHOnTunTp06KTk5WW+99ZY6d+6sgQMHur3eMjVeV1xxhXbu3Ol0f0FBgXbs2KGmTZuWuTAAAIDSGDRokGrXrq3XX39ds2bNUkFBgZo1a6YXXnhB/fv3tx/n7++vuXPnatasWVq3bp3Wrl2r+vXrKz4+XuPHj5e/v7/bay1T49WvXz+9+OKLmj9/vkaPHl1i/6uvvqrDhw9r5MiR5S4QAAB4ik2yLn2Fy2XnKacbb7xRN954428eV716dU2dOlVTp04t9znLokyN16hRo7R+/Xo9++yzWrdunf2y4zPPPKPt27fru+++U9u2bXXHHXe4tFgAAICKrEzD9UFBQVq0aJFiY2OVlJSkPXv2yLIsLViwQN9//70GDhyoN954Q1WqsDA+AAAVmpff1VjRlLkzqlGjhqZPn66HHnpIiYmJysjIUI0aNdSmTRvVqVPHlTUCAABUCuWOpEJDQ9W9e3dX1AIAALyITWbuajQxReYtynSpEQAAAJevTInXtGnTSnWczWbTU089VZZTAAAATzM1g+VDc15larxWrFhxyf02m02WZdF4AQAA/I8yNV7Onh5+7tw5JSYmavbs2Wrfvr2mTJlSruIAAIBnefvK9RVNmRqvqKgop/uio6PVrVs3DRw4UF26dNGwYcPKXBwAAEBl4pbh+gYNGqhHjx5atGiRO94eAACgQnLbCqd169bVoUOH3PX2AADABC41upRbEq/8/Hxt3bpVNWrUcMfbAwAAVEhlSry2bdvmcHteXp7S0tK0fPly7d27l/kuAAAqOhIvlypT4xUXF2d/MLYjlmUpJibGY0/+BgAA8EZlarzGjx/vsPGy2WyqVauW2rRpozZt2pS7OAAA4FksJ+FaZWq8Jk6c6Oo6AAAAKr0yDddPmzZNb775potLAQAAqNzK1HitWbNGJ0+edHUtAAAAlVqZV66n8QIAwAcw4+VSZUq8BgwYoC+++EJnzpxxdT0AAACVVpkar3vvvVetWrXSyJEj9dlnn+nEiROurgsAAHgBm+X+L19S6kuNK1euVHR0tKKjo+1LRViWpXHjxjl9jc1mU1JSUvmrBAAAqARK3Xg99NBDmjhxoqKjo9WpUyd31gQAALyFjyVS7nZZw/WWVfjdX7x4sVuKAQAAqMzKdFcjAADwAZbMJF4+lKqVabgeAAAAl++yEq9z584pNTX1sk4QGRl5WccDAADv4Wt3HbrbZTVeixYt0qJFi0p9PHc1AgAA/NdlNV4hISGqUaOGu2oBAADehsTLpS6r8Ro1apQmTJjgrloAAAAqNYbrAQAADGE5CQAA4BTD9a5F4gUAAGAIiRcAAHCOxMulSt147du3z511AAAAVHokXgAAwDkSL5dixgsAAMAQEi8AAOAUdzW6FokXAACAISReAADAMUtmZrx8KFUj8QIAADCExAsAADjnQ2mUCSReAAAAhpB4AQAAp7ir0bVIvAAAAAwh8QIAAM6ReLkUiRcAAIAhJF4AAMAhm8zMeNncfwqvQeIFAABgCI0XAACAIVxqBAAAzjFc71IkXgAAAIaQeAEAAOdIvFyKxAsAAMAQEi8AAOCULy31YAKJFwAAgCEkXgAAwDlmvFyKxAsAAMAQEi8AAOCYZeaRQb6UqpF4AQAAGELiBQAAnPOhNMoEEi8AAABDSLwAAIBzJF4uReIFAABgCIkXAABwyshdjT6ExAsAAMAQEi8AAOAciZdLkXgBAAAYQuIFAACcYsbLtUi8AAAADKHxAgAAMIRLjQAAwDkuNboUiRcAAIAhJF4AAMAphutdi8QLAADAEBIvAADgmCUzM14+lKqReAEAABhC4gUAAJzzoTTKBBIvAAAAQ0i8AACAU9zV6FokXgAAAIaQeAEAAOdIvFyKxAsAAMAQEi8AAOCEJZvFQl6uROIFAABgCIkXvMbNt5/Sgy8eueQx+fnSLY3aGqoI8G5frqmlPVtCtP/7YB1MCtaFTH/1HHxKf3nlcIlj8y5KHy6sp/3fBWv/d9V0+Meqyrvop8nPHla/u05d8jznz/rp/VfDtOXjWjp2KFB+flL9qFy1jDmv8U8eVZUAd31CeIUKGEYVFBRo+PDh2r17t7p06aI333yz2P6srCzNmjVLH330kY4fP66wsDD1799f48aNU3BwsFtro/GC19j/fbAWPx/ucF+rzufVvnumtm+qYbgqwHu99WKEDiQFK7h6vuo1uKgLP/k7PTb7gr9efayhJKl2/YuqXT9Pv6QG/uY5Dv9YVQ/febVOpgWoffdz6tTjrPLzbEo/EqgvPgzVmL+lqkpAgcs+E+AKCxcu1I8//uhwX35+vsaMGaOEhATFxsYqJiZG+/bt07x587Rnzx4tWLBAfn7uuyDolY3X3LlzlZSUpKSkJB0+fFh+fn5KSkrydFlwswPfB+vA945/0pixuvB/oI+W1jVZEuDV7v17iuo3yFXkVbnasyVEU4c2dXps1eACPbFkv5q0zFLd8Dwtfi5CS16IuOT7Z1+w6fH4q5R13k8vrPpRLTpeKLY/P0/yc97roRKwycw6XjYXvteRI0c0c+ZM3X///XrqqadK7F+xYoUSEhIUFxenRx991L49KipKzzzzjFavXq1Bgwa5sKLivHLG6/nnn9dXX32liIgI1atXz9PlwMOujM7StZ0u6JfUACV8WtPT5QBeo13XTEU1yZWtFH9rBQRaiul5TnXD80r9/msX11PKgSDFTztWoumSJP8qKtW5AZMeffRRNW3aVHFxcQ73r1q1SpIUHx9fbPuIESMUFBSklStXurU+r0y8PvnkEzVu3FiSFBcXp1OnLj1/gMrtlrtPSpI+fruOCgr4Ux4w5bMVtWWzWbopNkNpRwK1bVMNnT/rr7Coi+p001nVrJPv6RKBYt59911t375dH3zwgcPLhZZlKTExUWFhYYqKiiq2LygoSC1atFBiYqJba/TKxquo6QICgwrUc3CG8vOk9W/V8XQ5gM/IuygdSApWrbp5Wre0jhZMj1R+3n9/8Amqlq9x/0xRnzv5wbhSs2RmuN6SUlNTnaZUkrRx48ZLvkV6err+9a9/KT4+XtHR0Q6PycjIUFZWlq655hqH+8PDw7Vz505lZmYqJCSk9PVfBq+81AgU+d2tGaoRmq/tn9Uo1SAwANc4l1FF+Xk2nT1dRfOfjtSIyWlasv17vfddou5/7rBsNmnGg420a7N7/nICLtfjjz+u2rVra8KECU6Pyc7OliQFBjr++6Rq1aqSCu96dBevTLyAIkWXGdcuYageMKngPzcqFuTbdEvcCd39QLp9X98Rp5ST5afZf22od2eFqV23TA9VCRNMPSQ7MjLyN1MtZ9auXatNmzZpwYIFCgoKcnpc0b7c3FyH+3NyciTJrUtKkHjBa13RLFstYwqH6rdtZKgeMKl6jf/Ob3Xtd6bE/hv+sy15VzVjNQGO5Obm6oknnlC3bt0UFRWlQ4cO2b+kwpTr0KFDOnHihEJDQxUcHKy0tDSH75Wenq6QkBC3XWaUSLzgxfr9J+1av4yhesC0oGqW6kfm6pfUQIXULDlEXyO0cFtONj+/V3pevoBqdna2Tp06pc2bN+v3v/99if07d+7U73//e91yyy2aMWOGWrVqpW3btiklJaXYgH12drb27t2r9u3bu7VeGi94pYCqBeo95LTy86SPlzFUD3hC++7ntOGduvp5X5CiOxRfTuLnfYWXbCIaOb5kA5gSHBysmTNnOtx33333qVmzZho/frwaNGggSYqNjdW2bdu0YMGCYut4LVu2TNnZ2YqNjXVrvTRe8Eq/G5ChGrXz9c0nNRmqBzxkYPwJffpeHb0zK1zX9zmj0LqFKVdutk0Lphf+JXbToNOeLBEGmJrxKquAgAD17dvX6f66desW2z948GCtXLlSixcv1rlz59SpUyclJyfrrbfeUufOnTVw4EC31kvjBa90y92Ft6h/tIS0C3Dm63W19PXHtSRJp44X/nG+d0d1PTe5cEmemrXzNOZvqfbj33k5TEf2FyZV+78rHB7e8E4dfb+tcJ6lZUxmsec2XtMmS3c9kKbFzzXQvT2idf3vzyqwaoF2fF5DKQeCdG2nTN0+7rj7PyjgQv7+/po7d65mzZqldevWae3atapfv77i4+M1fvx4+fu793EMXtl4rVy5UqmphX9YpKSkyLIszZ49275/3LhxnioNBjRqmq1W151nqB74Dfu/D9Yn7xb/4eTYoao6dqjwlvjwhrnFGq/t/66pPVuKDw0nbQ9R0vb//v7XD8y++4F0XRmdrRWv19cXq0N18aJNDa7I0ai/HNPQsccVWNXL4xCUXwX+V5ycnOxwe/Xq1TV16lRNnTrVcEWSzbIsr/uWxsXFKSEhwel+Z9/I0jh2IF0jmzpf4wNA+XycusvTJQCVlq3epsJfqzQycr6U4xkaPGW+28+z/PnRigoLdft5vIFXJl6LFy/2dAkAAEDeP+NV0XAfMAAAgCFemXgBAAAv4X0TSRUaiRcAAIAhJF4AAMApZrxci8QLAADAEBIvAADgmCUz63j5UKpG4gUAAGAIiRcAAHDKVuDpCioXEi8AAABDaLwAAAAM4VIjAABwzocG300g8QIAADCExAsAADjFAqquReIFAABgCIkXAABwjodkuxSJFwAAgCEkXgAAwCGbzMx42dx/Cq9B4gUAAGAIiRcAAHCMh2S7HIkXAACAISReAADAKdbxci0SLwAAAENIvAAAgHOs4+VSJF4AAACGkHgBAACnmPFyLRIvAAAAQ0i8AACAcyReLkXiBQAAYAiNFwAAgCFcagQAAE4xXO9aJF4AAACGkHgBAADnCoi8XInECwAAwBASLwAA4JglM8tJ+FCoRuIFAABgCIkXAABwirsaXYvECwAAwBASLwAA4JxF5OVKJF4AAACGkHgBAACnmPFyLRIvAAAAQ0i8AACAcyReLkXiBQAAYAiJFwAAcMrGXY0uReIFAABgCIkXAABwzJJUYOg8PoLECwAAwBASLwAA4IRlaMbLdyIvEi8AAABDaLwAAAAM4VIjAABwzneuAhpB4gUAAGAIiRcAAHCOBVRdisQLAADAEBIvAADgkE2SzUDgZXP/KbwGiRcAAIAhJF4AAMA5ZrxcisQLAADAEBIvAADglM3EQ7J9CIkXAACAISReAADAMUtmZrx8aIyMxAsAAMAQEi8AAOCcD6VRJpB4AQAAGELiBQAAnLKxjpdLkXgBAAAYQuIFAACcI/FyKRIvAAAAQ2i8AAAADOFSIwAAcI5HBrkUiRcAAIAhJF4AAMAplpNwLRIvAAAAQ0i8AACAYzwk2+VIvAAAAAwh8QIAAE5YhhZQ9Z3Ii8QLAADAEBIvAADgHOt4uRSJFwAAgCEkXgAAwCnW8XItEi8AAABDSLwAAIBzJF4uReIFAABgCIkXAABwzssTr59//lkffvihvvrqKx05ckTnz59XZGSkbrjhBo0ZM0ZhYWHFjs/Ly9P8+fP1wQcfKCUlRaGhoerVq5cmT56s2rVru71eGi8AAFBhvf/++1q6dKl69Oihfv36KSgoSLt27dJbb72l1atXa9myZbr66qvtx0+bNk2rV69Wjx499Mc//lFHjx7VwoUL9e233+qdd95RtWrV3FovjRcAAHDOyxOvPn36aMyYMapZs6Z92x133KF27drpscce00svvaSZM2dKkrZs2aLVq1erZ8+emjNnjv34li1batKkSZo/f74mTJjg1nqZ8QIAABVW69atizVdRfr37y9JSk5Otm9btWqVJCk+Pr7YsX369FFUVJR9vzuReAEAAMcsmVm53pJSU1MVFxfn9JCNGzde1lump6dLkurVq2fftnv3bvn5+aldu3Yljm/fvr3WrFmjjIwMhYaGXta5LgeJFwAAqHSKLi8OHjzYvi0tLU21a9dWYGBgiePDw8Ptx7gTiRcAAPC4yMjIy061nHn11Vf18ccfq3fv3rrtttvs27Ozs1WrVi2Hr6latar9GHei8QIAAE5VtEcGLVy4UDNmzFDnzp313HPPyWaz2fcFBQUpNzfX4etycnLsx7gTlxoBAEClsGDBAj311FPq0qWL5s6dq+Dg4GL7IyIidPr0aYfNV9FMWEREhFtrpPECAADOWZb7v1xg7ty5mj59urp3767XXnutRNMlSW3atFFBQYF2795dYt/OnTvVuHFjtw7WSzReAACggnv11Vf1/PPPq0ePHpo9e7Z9XuvXYmNjJUnz588vtn3Dhg1KSUmx73cnZrwAAIATllRgYsar7OdYunSpZsyYoXr16unmm2/WunXriu2vXr26evfuLUm64YYbNGDAAK1Zs0Zjx45Vr169dPToUb355ptq2rRpifW93IHGCwAAVFiJiYmSpBMnTujhhx8usT8qKsreeEnS9OnT1axZMy1fvlx///vfFRoaqtjYWE2ePFnVq1d3e702y6pgtyuU07ED6RrZ1L2PAwB82cepuzxdAlBp2eptKvy1SiMj5zt25KRG9/qX288zf+NUNWhU1+3n8QbMeAEAABjCpUYAAOCcb10Yczufa7zqN6qrRT+94ukygErLVs/x4oQAXMC/gaQ8T1eBcvC5xqtKQBU1aBLu6TIAACgjg391WzKTePlQqMaMFwAAgCE+l3gBAIDLYGQdL99B4gUAAGAIiRcAAHDOKvB0BZUKiRcAAIAhJF4AAMA51vFyKRIvAAAAQ2i8AAAADOFSIwAAcMIytJyE71zOJPECAAAwhMQLAAA4xiODXI7ECwAAwBASLwAA4BzLSbgUiRcAAIAhJF7wOhs2bNAbb7yhH374QQEBAerYsaMeeOABNWvWzNOlARXe3LlzlZSUpKSkJB0+fFh+fn5KSkrydFnwZiReLkXiBa/y3nvvaeLEicrKytKDDz6osWPHKjk5WcOHD1dycrKnywMqvOeff15fffWVIiIiVK9ePU+XA/gcEi94jTNnzmj69OmKiIjQsmXLFBISIknq16+f+vfvryeffFKLFi3ycJVAxfbJJ5+ocePGkqS4uDidOnXKwxXB6xXwkGxXIvGC19i4caMyMzM1bNgwe9MlSZGRkerTp4+2bt2qY8eOebBCoOIraroAeAaNF7zG7t27JUnt27cvsa9oW2JiotGaAMDnWZb7v3wIjRe8Rnp6uiQpIiKixL6ibWlpaUZrAgDAlZjxgtfIysqSJAUGBpbYV7QtOzvbaE0A4PN8LJFyNxIveI3g4GBJUm5ubol9RduCgoKM1gQAgCvReMFrhIeHS3J8ObFom6PLkAAAN7EsqcDAlw+lajRe8Bpt2rSRJO3cubPEvl27dkmSWrdubbIkAABcisYLXqN3796qXr263nvvPWVmZtq3p6amav369ercubMaNGjgwQoBwPdYVoHbv3wJw/XwGrVq1dLUqVP1t7/9TXfeeafuuOMO5ebmasmSJZKkRx55xMMVAhXfypUrlZqaKklKSUmRZVmaPXu2ff+4ceM8VRrgE2yW5UMXVlEhrF+/XvPmzbM/q7FTp06aPHmyoqOjPV0aUOHFxcUpISHB6X4ezYX/deznXxTf4WG3n2fBt0+pwZX13X4eb0DjBQAAHKLxcj1mvAAAAAxhxgsAADjHhTGXIvECAAAwhMQLAAA4V+Bbyz24G4kXAACAISReAADAMcvQ43x8aI6MxAsAAMAQEi8AAOCUxYyXS5F4AZVE8+bNFRcXV2zbyy+/rObNm2vr1q0equryXG69Dz30kJo3b66jR4+W67xxcXFq3rx5ud7jt7iqVgAVG4kXcBl+/Zezn5+fatasqebNm2vYsGG69dZbPVSZ+zRv3lydO3fW4sWLPV0KAE/wofkrE2i8gDKYMGGCJCkvL08HDhzQxo0btXXrVn333XeaNm2ah6v7r7vuuku33HKLIiMjPV0KAEA0XkCZTJw4sdjvt2zZovj4eC1cuFBxcXFq2LChhyorrk6dOqpTp46nywBQkRWQeLkSM16AC3Tp0kVNmjSRZVlKTEyUVHxe6cMPP9SwYcPUvn179ezZ0/66rKwsvfbaa4qNjVW7du3Uvn173XHHHVqzZo3D8+Tm5mrWrFnq3bu3WrVqpZ49e2rGjBnKzc11ePylZqb279+vadOmqWfPnmrVqpW6dOmiESNG6K233pIkLV++3H5pNSEhQc2bN7d/vfzyy8Xea/fu3Zo0aZK6du2qVq1a6cYbb9Rjjz2m9PR0h3V99913+uMf/6j27durQ4cO+sMf/qCdO3f+xne59JYvX66JEyeqV69eatOmjTp06KDhw4dr1apVl3xdbm6uZsyYYf+e9O7dW6+88orT7+/+/fv10EMP6cYbb1SrVq10ww03aMqUKTpw4IDLPguAyoXEC3AR6z9zEDabrdj2BQsW6KuvvlKPHj103XXX6dy5c5Kks2fPatSoUUpKSlLLli01ZMgQFRQUaPPmzZoyZYp+/PFH3X///cXef/Lkydq4caMaN26su+++WxcvXtQHH3ygH3744bJq/fe//6377rtPubm56t69u/r376+zZ88qOTlZb7zxhkaMGKEWLVpowoQJeuWVVxQVFaXbbrvN/vrOnTvb//n999/XY489psDAQPXs2VMRERE6dOiQ3nvvPW3atEnvvvtusUud3377reLj43Xx4kXdfPPNuuKKK7R3717FxcXp+uuvv6zP4czjjz+upk2bKiYmRvXr11dGRoY+//xzTZ06VQcPHtTkyZMdvu6+++5TYmKi+vbtqypVqmjjxo16+eWX9d1332nOnDnF/t1+8cUXmjhxovLy8tSjRw81btxY6enp2rBhg/79739r0aJFatmypUs+D+BRFnc1uhKNF+ACX3/9tQ4ePCibzabWrVsX2/fNN9/onXfe0bXXXlts+1NPPaWkpCQ9+OCDuueee+zbc3JyNG7cOL322mvq27evWrRoIUlas2aNNm7cqHbt2mnRokWqWrWqpMLLnkOHDi11radOndKUKVOUn5+vhQsXFmuiJCktLU2S1KJFC7Vo0cLeeP368qokHTx4UI8//riioqK0ZMkShYeH2/dt2bJFo0eP1pNPPqlZs2ZJKmweH374YWVnZ9uTuyILFy7UU089VerPcSlr1qxR48aNi23Lzc3VPffco9dff1133nlnsVqLHDhwQGvXrlWtWrUkSffff79Gjhypzz77TKtWrdKgQYMkSWfOnNGUKVMUFBSkpUuXqmnTpvb3+OGHH3THHXfo0Ucf1YoVK1zyeQBUHlxqBMrg5Zdf1ssvv6wZM2Zo0qRJ+tOf/iTLsjRq1ChFRUUVO/b2228v0XSdPn1aq1evVqtWrYo1XZJUtWpV/fnPf5ZlWfrwww/t25cvXy6psBkoarokKTQ0VOPGjSt17StXrlRmZqaGDx9eoumSpIiIiFK/17Jly3Tx4kU98sgjJRqZLl26qGfPnvrss8+UmZkpqTDtOnjwoGJiYoo1XZJ09913l2iWysrR+wQGBuquu+5SXl6etmzZ4vB1//d//2dvuqTCfxcPPPCAJOmDDz6wb1+5cqXOnj2rSZMmFWu6JKlZs2YaNmyYkpKS9NNPP7ni4wCeY0lWgeX2L/nQGBmJF1AGr7zyiqTCy4o1a9ZUx44dNXToUMXGxpY4tk2bNiW2JSYmKj8/XzabrcS8lFR4t6SkYrNCSUlJ8vPzU8eOHUsc76iBcmbXrl2SpN/97nelfs1vvVdCQoJ9tu1/nTx5Uvn5+fr555/VqlUrJSUlSZJiYmJKHOvv76+OHTvq8OHD5a4rNTVVr7/+urZs2aJjx44pOzu72H5ns2eOvo8dO3aUv7+/9u7da99W9Ln37dvn8N/fzz//LKlwBuzXjRkA30bjBZRBcnJyqY+tV69eiW0ZGRmSChswRw1LkfPnz9v/+dy5c6pVq5YCAgJKHFe/fv1S11M0Y+boUtvlKvoc8+bNu+RxFy5cKHZuR9+TS22/HEeOHNHQoUN19uxZderUSd26dVNISIj8/f2VkpKiFStWOB2Wd3T+KlWqqHbt2jp58qR9W9Hnfvfddy9ZS9HnBiouy9CMl+9EXjRegJv9ethekmrUqCFJ+sMf/lDqdb9q1KihM2fO6OLFiyWar19++aXU9RSdOz09vdyrtYeEhEiSduzYYf/n0pz7xIkTDvc72345FixYoIyMDD399NMaPHhwsX1r1qy55NzViRMnSqx5lpeXp9OnTxf7fEWfY9WqVYqOji53zQB8BzNegAe0adNGfn5+2r59e6lfc+2116qgoEA7duwosS8hIaHU79OuXTtJhXfllYafn5/y8/Mv+V6l/RxFs27btm0rsS8/P9/hZ7tchw4dkiT9/ve/L7Hvt75Pjvbv2LFD+fn59pscJKlt27b2fQBwOWi8AA+oW7eubr31Vn333XeaNWuWw8bm8OHDOnLkiP33RenNiy++qJycHPv2jIwMzZkzp9TnHjRokEJCQvT22287bICK7mosEhoaWmJbkbvuuksBAQF6+umndfDgwRL7c3NzizVlHTp00FVXXaVt27bp008/LXbskiVLXDLfVXRzw6+bqC+//FLvv//+JV87Z84cnTlzxv77nJwcvfDCC5KkIUOG2LcPHjxYNWvW1CuvvKI9e/aUeJ+CgoIK83xM4LcYGa73IVxqBDzkscce06FDh/TSSy9p9erV6tChg+rVq6fjx49r//79SkxM1AsvvKBGjRpJkgYMGKCPPvpImzZt0oABA9SrVy/l5eVp/fr1at26dambljp16uj555/XpEmTNHLkSP3ud79T8+bNlZmZqeTkZB07dkybNm2yH9+lSxetXbtWY8eO1bXXXqsqVaooJiZGMTExuvrqq/Xkk0/qkUce0YABA9S9e3ddeeWVysvLU2pqqnbs2KHatWtr/fr1kgovuz755JMaPXq0Jk2aVGwdry1btqh79+768ssvy/V9HTFihJYvX6777rtPffr0UVhYmH788Ud9+eWX6tevnz766COnr23SpIn69+9fbB2vw4cP66abbip240Tt2rX10ksvafz48br99tvVpUsXNW3aVDabTWlpadq5c6cyMjIuOb8HwDfReAEeEhISosWLF+vdd9/VmjVrtGHDBuXk5KhevXq64oorNG3aNN1www324202m2bOnKm5c+dqxYoVWrJkicLCwjRkyBCNHz++xPphl3LTTTfpgw8+sN/599VXX6lmzZpq0qSJ7r333mLHPvLII7LZbNqyZYs+//xzFRQUaMKECfY7E2NjYxUdHa0FCxZo69at2rx5s6pVq6awsDD16dNH/fr1K/Z+HTt21NKlSzVjxgz75c62bdtq8eLF2rx5c7kbr+joaC1atEgvvviiPv/8c+Xl5Sk6OlqvvPKKatSoccnGa+bMmZo1a5Y+/PBDHT9+XOHh4Zo4caLGjBlTYlavS5cuWr16tebPn6/Nmzdr+/btCggIUFhYmK6//nr16dOnXJ8D8BosoOpSNsviseMAAKCk/Lx8HT9c/ptefktY43ryr+Lv9vN4AxovAAAAQxiuBwAAMITGCwAAwBAaLwAAAENovAAAAAyh8QIAADCExgsAAMAQGi8AAABDaLwAAAAMofECAAAwhMYLAADAEBovAAAAQ2i8AAAADPl/J9PKTZ/+sP0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "conf_mat = confusion_matrix(y_true=y_true, y_pred=y_pred)\n", "\n", "conf_mat_disp = ConfusionMatrixDisplay(conf_mat, display_labels=np.unique(y_true))\n", "conf_mat_disp.plot()\n", "\n", "plt.gcf().set_size_inches(7, 7)\n", "\n", "# seaborn turns on grid by default ... looks best without it\n", "plt.grid(False)" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" } }, "nbformat": 4, "nbformat_minor": 4 }