{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### DS4420: Exercise on Dimensionality Reduction via PCA" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your name*: " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In general dimensionality reduction may be viewed as *encoding* data $x$ into latent (lower dimensional) representations $z$ that one can then use to recover, noisily, the original $x$'s. \n", "\n", "One may view this as an encoding-decoding process, where we encode $x$ into $z$, and then decode $z$ into a reconstruction $\\tilde{x}$. We saw that if our objective is to capture as much variance as possible in the compressed space, then we can use eigenvectors; namely the top $m$ eigenvectors (ranked by corresponding eigenvalues) may be used to create an $m$ dimensional representation of $x$. In this exercise, we see this in practice." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np \n", "import matplotlib.pyplot as plt\n", "\n", "import torch\n", "# conda install -c pytorch torchvision\n", "import torchvision\n", "\n", "# note: if you cannot get torchvision installed \n", "# using the above sequence, you can resort to \n", "# the colab version here: \n", "# -- just be sure to download and then upload\n", "# the notebook to blackboard when complete.\n", "fMNIST = torchvision.datasets.FashionMNIST(\n", " root = './data/FashionMNIST',\n", " train = True,\n", " download = True) " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([60000, 28, 28])" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fMNIST.data.shape" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOwUlEQVR4nO3df4xV9ZnH8c/jDPiDHwoKBMF12KKyG/8QJbphzcaNgbj8g2isYEzcbJOpWrUmm7ikS1ITNTG7WzfxnyY0NWU3XZsmamqazbYGm7V/VYFYQbAVG34MTBgBDYMO8uvZP+ZMd4pzvt/xnnvuufC8X8nkzpznnnueOcOHe+793nO+5u4CcOG7qOkGAHQGYQeCIOxAEIQdCIKwA0H0dnJjZsZb/0DN3N0mWl7pmd3M7jKz35nZbjNbX+WxANTLWh1nN7MeSb+XtELSgKR3JK1z952JdXhmB2pWxzP7rZJ2u/sf3P2kpJ9IWl3h8QDUqErYF0jaP+7ngWLZnzCzfjPbYmZbKmwLQEVV3qCb6FDhS4fp7r5R0kaJw3igSVWe2QckXTPu54WSDlZrB0BdqoT9HUnXmdkiM5sqaa2k19vTFoB2a/kw3t1Pm9ljkn4hqUfSS+7+fts6A9BWLQ+9tbQxXrMDtavlQzUAzh+EHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSBanp9dksxsj6RhSWcknXb3Ze1oCkD7VQp74W/d/XAbHgdAjTiMB4KoGnaX9Esz22pm/RPdwcz6zWyLmW2puC0AFZi7t76y2dXuftDM5kp6Q9Lj7v5W4v6tbwzApLi7TbS80jO7ux8sbockvSbp1iqPB6A+LYfdzKaZ2Yyx7yWtlLSjXY0BaK8q78bPk/SamY09zn+5+/+0pSsAbVfpNftX3hiv2YHa1fKaHcD5g7ADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSDaccFJVFScJtyy1JmLPT09yXXPnj3b8mNLUm9v+p/Q6dOnk/UqLroo/VyV+93qNGXKlGQ9tV/qOhOVZ3YgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKry17gmhwHr+qRRx5J1jds2JCsL1iwoJ3tnDe4uiwQHGEHgiDsQBCEHQiCsANBEHYgCMIOBME4+3kgd757J/+G51q3bl2yvnTp0tLafffdl1x3ZGQkWT916lSyvnPnztJaru+qpk6dmqw/9dRTpbVnn3220rZbHmc3s5fMbMjMdoxbNtvM3jCzD4vbWZW6A1C7yRzG/0jSXecsWy9ps7tfJ2lz8TOALpYNu7u/JenoOYtXS9pUfL9J0t1t7gtAm7V6Dbp57j4oSe4+aGZzy+5oZv2S+lvcDoA2qf2Ck+6+UdJGiTfogCa1OvR2yMzmS1JxO9S+lgDUodWwvy7poeL7hyT9rD3tAKhL9jDezF6WdIekq8xsQNJ3JT0v6adm9g1J+ySlB0yDqzpOXmUcffHixcl6bqx7+fLlyfrKlSuT9Y8++qi0NjAwkFz32LFjyXpfX1+yvmrVqmS9TmvXrk3Wb7vttg518v+yYXf3sk8f3NnmXgDUiI/LAkEQdiAIwg4EQdiBIAg7EMQFM2Vz1el7c6cknjx58iv3NKbqKahXXHFFsv7cc8+V1u6///7kup9//nmyPjg4mKy//fbbyXpq6uJLL700ue4HH3yQrC9cuDBZf+aZZ5L1lLlzSz8BLim/X1944YVkfcmSJaW1W265Jbnu1q1bk/UyPLMDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBDn1aWkU6eK5sbZz5w5U2XTldx5Z/oEwXvvvTdZf+CBB5L1I0eOlNYOHDiQXDc3ZfPMmTOT9csuuyxZT10OOjfGP3369GR9aCh9zZQ5c+aU1nLTOecuY719+/ZkffXq1cn6JZdcUlqr+nszZTMQHGEHgiDsQBCEHQiCsANBEHYgCMIOBHFejbM36YknniitPfzww8l1582bl6xXvaRy6jMEuW3n5K4DkJO6DkDu9/riiy+S9RkzZiTrF198cWlt7969yXXXrFmTrOds2LAhWX/00UdLa/v27Uuu++CDD5bW9u/frxMnTjDODkRG2IEgCDsQBGEHgiDsQBCEHQiCsANBdNU4+80335xcf8WKFaW1G264Iblu6vxhSbr66quT9dQ5xrlryn/22WfJ+uWXX56s56Z8Tv1uPT09yXVz55Snrvsu5XtL7ZvcOHpuv+b+7aZ6u/7661teV8qfc5777MTHH39cWstdI+DNN98srb344osaGBhobZzdzF4ysyEz2zFu2dNmdsDM3i2+mpsIG8CkTOYw/keS7ppg+b+7+03F13+3ty0A7ZYNu7u/JeloB3oBUKMqb9A9ZmbvFYf5s8ruZGb9ZrbFzLZU2BaAiloN+/clfU3STZIGJX2v7I7uvtHdl7n7sha3BaANWgq7ux9y9zPuflbSDyTd2t62ALRbS2E3s/njflwjaUfZfQF0h+z87Gb2sqQ7JF1lZgOSvivpDjO7SZJL2iPpm5PZ2Jw5c5LzWt9zzz3J9VPzeVcZ75Xy48mp8eiqY7K5c8Zz4/Sffvppaa23N/0nzm079/mE3O+eOqc89xmA3Pztud5Sf9PcufS56+l/8sknldZP/W658/RblQ27u6+bYPEPa+gFQI34uCwQBGEHgiDsQBCEHQiCsANBdPQU197eXk8NQy1ZsiS5/vLly0trN954Y3Lda6+9Nlm/8sork/XUUElueCs3XXRuuulcPTU1cW5oLTf8NXXq1GQ997vnek85fvx4sp4bkkwNt+aGxnK5OHHiRLKe+71TQ3fTpk1Lrvv444+X1rZt26bh4WEuJQ1ERtiBIAg7EARhB4Ig7EAQhB0IgrADQXR8nD11+t6pU6eS6+fGVVNSp1pK0qJFi5L1xYsXl9b6+vqS6+YuU131NNLUmG5unP3w4cPJem6s+8iRI8l66vTbVG0y9ZGRkWQ9d5nslNznC3J/k5zUfs/9O89l1t0ZZwciI+xAEIQdCIKwA0EQdiAIwg4EQdiBILpqyubcebwzZ85MPXZrTRVy5zenxmxz4+S5zw/k5M45T/0Nc+dV53qv83z33LZz9dylplP/XnKXDs/9zXLn8eemXR4eHm5523v37i2t7d69WyMjI4yzA5ERdiAIwg4EQdiBIAg7EARhB4Ig7EAQXTXO3qTcmG1qXDY3Fp0bc82da58bE07J9ZYbh899/qDK9quOdafGqqX0Zy9y4+S53nL7Jff4qfVz5+EfPHgwWW/5fHYzu8bMfmVmu8zsfTP7drF8tpm9YWYfFrezco8FoDmTOYw/Lekf3f0vJP2VpG+Z2V9KWi9ps7tfJ2lz8TOALpUNu7sPuvu24vthSbskLZC0WtKm4m6bJN1dV5MAqku/sDiHmfVJWirpN5LmufugNPofgpnNLVmnX1J/tTYBVDXpsJvZdEmvSHrS3Y9N9sQTd98oaWPxGF37Bh1woZvU0JuZTdFo0H/s7q8Wiw+Z2fyiPl/SUD0tAmiH7NCbjT6Fb5J01N2fHLf8XyUdcffnzWy9pNnu/lTmsXhmB2pWNvQ2mbDfLunXkrZLGrsI+Xc0+rr9p5L+TNI+Sfe5+9HMYxF2oGYth72dCDtQPyaJAIIj7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IIhs2M3sGjP7lZntMrP3zezbxfKnzeyAmb1bfK2qv10ArZrM/OzzJc13921mNkPSVkl3S/q6pOPu/m+T3hhTNgO1K5uyuXcSKw5KGiy+HzazXZIWtLc9AHX7Sq/ZzaxP0lJJvykWPWZm75nZS2Y2q2SdfjPbYmZbKnUKoJLsYfwf72g2XdL/SnrO3V81s3mSDktySc9o9FD/HzKPwWE8ULOyw/hJhd3Mpkj6uaRfuPsLE9T7JP3c3W/MPA5hB2pWFvbJvBtvkn4oadf4oBdv3I1ZI2lH1SYB1Gcy78bfLunXkrZLOlss/o6kdZJu0uhh/B5J3yzezEs9Fs/sQM0qHca3C2EH6tfyYTyACwNhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgiOwFJ9vssKS9436+qljWjbq1t27tS6K3VrWzt2vLCh09n/1LGzfb4u7LGmsgoVt769a+JHprVad64zAeCIKwA0E0HfaNDW8/pVt769a+JHprVUd6a/Q1O4DOafqZHUCHEHYgiEbCbmZ3mdnvzGy3ma1voocyZrbHzLYX01A3Oj9dMYfekJntGLdstpm9YWYfFrcTzrHXUG9dMY13YprxRvdd09Ofd/w1u5n1SPq9pBWSBiS9I2mdu+/saCMlzGyPpGXu3vgHMMzsbyQdl/QfY1Nrmdm/SDrq7s8X/1HOcvd/6pLentZXnMa7pt7Kphn/ezW479o5/Xkrmnhmv1XSbnf/g7uflPQTSasb6KPruftbko6es3i1pE3F95s0+o+l40p66wruPuju24rvhyWNTTPe6L5L9NURTYR9gaT9434eUHfN9+6SfmlmW82sv+lmJjBvbJqt4nZuw/2cKzuNdyedM8141+y7VqY/r6qJsE80NU03jf/9tbvfLOnvJH2rOFzF5Hxf0tc0OgfgoKTvNdlMMc34K5KedPdjTfYy3gR9dWS/NRH2AUnXjPt5oaSDDfQxIXc/WNwOSXpNoy87usmhsRl0i9uhhvv5I3c/5O5n3P2spB+owX1XTDP+iqQfu/urxeLG991EfXVqvzUR9nckXWdmi8xsqqS1kl5voI8vMbNpxRsnMrNpklaq+6aifl3SQ8X3D0n6WYO9/Ilumca7bJpxNbzvGp/+3N07/iVplUbfkf9I0j830UNJX38u6bfF1/tN9ybpZY0e1p3S6BHRNyRdKWmzpA+L29ld1Nt/anRq7/c0Gqz5DfV2u0ZfGr4n6d3ia1XT+y7RV0f2Gx+XBYLgE3RAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EMT/AUGd8+8h8/s/AAAAAElFTkSuQmCC\n", "text/plain": [ "