{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b11d1db5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2025 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e661697d",
   "metadata": {},
   "source": [
    "## BigQuery DataFrame Visualization Tutorials\n",
    "\n",
    "<table align=\"left\">\n",
    "\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/googleapis/python-bigquery-dataframes/blob/main/notebooks/visualization/tutorial.ipynb\">\n",
    "      <img src=\"https://raw.githubusercontent.com/googleapis/python-bigquery-dataframes/refs/heads/main/third_party/logo/colab-logo.png\" alt=\"Colab logo\"> Run in Colab\n",
    "    </a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://github.com/googleapis/python-bigquery-dataframes/tree/main/notebooks/visualization/tutorial.ipynb\">\n",
    "      <img src=\"https://raw.githubusercontent.com/googleapis/python-bigquery-dataframes/refs/heads/main/third_party/logo/github-logo.png\" width=\"32\" alt=\"GitHub logo\">\n",
    "      View on GitHub\n",
    "    </a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://console.cloud.google.com/bigquery/import?url=https://github.com/googleapis/python-bigquery-dataframes/tree/main/notebooks/visualization/tutorial.ipynb\">\n",
    "      <img src=\"https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTW1gvOovVlbZAIZylUtf5Iu8-693qS1w5NJw&s\" alt=\"BQ logo\" width=\"35\">\n",
    "      Open in BQ Studio\n",
    "    </a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e93c4c1",
   "metadata": {},
   "source": [
    "This notebook provides tutorials for all plotting methods that BigQuery DataFrame offers. You will visualize different datasets with histograms, line charts, area charts, bar charts, and scatter plots."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96c47f7",
   "metadata": {},
   "source": [
    "# Before you begin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8dd598a",
   "metadata": {},
   "source": [
    "## Set up your project ID and region"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d442ab74",
   "metadata": {},
   "source": [
    "This step makes sure that you will access the target dataset with the correct auth profile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7cc6237d",
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT_ID = \"bigframes-dev\"  # @param {type:\"string\"}\n",
    "REGION = \"US\"  # @param {type: \"string\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bf96593a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import bigframes.pandas as bpd\n",
    "\n",
    "bpd.options.bigquery.project = PROJECT_ID\n",
    "bpd.options.bigquery.location = REGION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "165fedc6",
   "metadata": {},
   "source": [
    "You can also turn on the partial ordering mode for faster data processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ac5a1722",
   "metadata": {},
   "outputs": [],
   "source": [
    "bpd.options.bigquery.ordering_mode = 'partial'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ed45ca7",
   "metadata": {},
   "source": [
    "# Histogram"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88837be7",
   "metadata": {},
   "source": [
    "You will use the penguins public dataset in this example. First, you take a look at the shape of this data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fb595a8f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>species</th>\n",
       "      <th>island</th>\n",
       "      <th>culmen_length_mm</th>\n",
       "      <th>culmen_depth_mm</th>\n",
       "      <th>flipper_length_mm</th>\n",
       "      <th>body_mass_g</th>\n",
       "      <th>sex</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Adelie Penguin (Pygoscelis adeliae)</td>\n",
       "      <td>Dream</td>\n",
       "      <td>36.6</td>\n",
       "      <td>18.4</td>\n",
       "      <td>184.0</td>\n",
       "      <td>3475.0</td>\n",
       "      <td>FEMALE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Adelie Penguin (Pygoscelis adeliae)</td>\n",
       "      <td>Dream</td>\n",
       "      <td>39.8</td>\n",
       "      <td>19.1</td>\n",
       "      <td>184.0</td>\n",
       "      <td>4650.0</td>\n",
       "      <td>MALE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Adelie Penguin (Pygoscelis adeliae)</td>\n",
       "      <td>Dream</td>\n",
       "      <td>40.9</td>\n",
       "      <td>18.9</td>\n",
       "      <td>184.0</td>\n",
       "      <td>3900.0</td>\n",
       "      <td>MALE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Chinstrap penguin (Pygoscelis antarctica)</td>\n",
       "      <td>Dream</td>\n",
       "      <td>46.5</td>\n",
       "      <td>17.9</td>\n",
       "      <td>192.0</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>FEMALE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Adelie Penguin (Pygoscelis adeliae)</td>\n",
       "      <td>Dream</td>\n",
       "      <td>37.3</td>\n",
       "      <td>16.8</td>\n",
       "      <td>192.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>FEMALE</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     species island  culmen_length_mm  \\\n",
       "0        Adelie Penguin (Pygoscelis adeliae)  Dream              36.6   \n",
       "1        Adelie Penguin (Pygoscelis adeliae)  Dream              39.8   \n",
       "2        Adelie Penguin (Pygoscelis adeliae)  Dream              40.9   \n",
       "3  Chinstrap penguin (Pygoscelis antarctica)  Dream              46.5   \n",
       "4        Adelie Penguin (Pygoscelis adeliae)  Dream              37.3   \n",
       "\n",
       "   culmen_depth_mm  flipper_length_mm  body_mass_g     sex  \n",
       "0             18.4              184.0       3475.0  FEMALE  \n",
       "1             19.1              184.0       4650.0    MALE  \n",
       "2             18.9              184.0       3900.0    MALE  \n",
       "3             17.9              192.0       3500.0  FEMALE  \n",
       "4             16.8              192.0       3000.0  FEMALE  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "penguins = bpd.read_gbq('bigquery-public-data.ml_datasets.penguins')\n",
    "penguins.peek()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "176c12f8",
   "metadata": {},
   "source": [
    "You want to draw a histogram about the distribution of culmen lengths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "333e88a3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: ylabel='Frequency'>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 13488 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "penguins['culmen_depth_mm'].plot.hist(bins=40)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e0aa359",
   "metadata": {},
   "source": [
    "# Line Chart"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0f37913",
   "metadata": {},
   "source": [
    "In this example you will use the NOAA public dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "49ed2417",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>stn</th>\n",
       "      <th>wban</th>\n",
       "      <th>date</th>\n",
       "      <th>year</th>\n",
       "      <th>mo</th>\n",
       "      <th>da</th>\n",
       "      <th>temp</th>\n",
       "      <th>count_temp</th>\n",
       "      <th>dewp</th>\n",
       "      <th>count_dewp</th>\n",
       "      <th>...</th>\n",
       "      <th>flag_min</th>\n",
       "      <th>prcp</th>\n",
       "      <th>flag_prcp</th>\n",
       "      <th>sndp</th>\n",
       "      <th>fog</th>\n",
       "      <th>rain_drizzle</th>\n",
       "      <th>snow_ice_pellets</th>\n",
       "      <th>hail</th>\n",
       "      <th>thunder</th>\n",
       "      <th>tornado_funnel_cloud</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>010030</td>\n",
       "      <td>99999</td>\n",
       "      <td>2021-11-10</td>\n",
       "      <td>2021</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>26.4</td>\n",
       "      <td>4</td>\n",
       "      <td>17.9</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.0</td>\n",
       "      <td>I</td>\n",
       "      <td>999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>010030</td>\n",
       "      <td>99999</td>\n",
       "      <td>2021-02-01</td>\n",
       "      <td>2021</td>\n",
       "      <td>02</td>\n",
       "      <td>01</td>\n",
       "      <td>8.9</td>\n",
       "      <td>4</td>\n",
       "      <td>0.5</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>2.76</td>\n",
       "      <td>G</td>\n",
       "      <td>999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>010060</td>\n",
       "      <td>99999</td>\n",
       "      <td>2021-07-22</td>\n",
       "      <td>2021</td>\n",
       "      <td>07</td>\n",
       "      <td>22</td>\n",
       "      <td>34.4</td>\n",
       "      <td>4</td>\n",
       "      <td>9999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.0</td>\n",
       "      <td>I</td>\n",
       "      <td>999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>010070</td>\n",
       "      <td>99999</td>\n",
       "      <td>2021-04-05</td>\n",
       "      <td>2021</td>\n",
       "      <td>04</td>\n",
       "      <td>05</td>\n",
       "      <td>17.9</td>\n",
       "      <td>4</td>\n",
       "      <td>6.4</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.0</td>\n",
       "      <td>I</td>\n",
       "      <td>999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>010070</td>\n",
       "      <td>99999</td>\n",
       "      <td>2021-02-04</td>\n",
       "      <td>2021</td>\n",
       "      <td>02</td>\n",
       "      <td>04</td>\n",
       "      <td>19.1</td>\n",
       "      <td>4</td>\n",
       "      <td>9999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.0</td>\n",
       "      <td>I</td>\n",
       "      <td>999.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      stn   wban        date  year  mo  da  temp  count_temp    dewp  \\\n",
       "0  010030  99999  2021-11-10  2021  11  10  26.4           4    17.9   \n",
       "1  010030  99999  2021-02-01  2021  02  01   8.9           4     0.5   \n",
       "2  010060  99999  2021-07-22  2021  07  22  34.4           4  9999.9   \n",
       "3  010070  99999  2021-04-05  2021  04  05  17.9           4     6.4   \n",
       "4  010070  99999  2021-02-04  2021  02  04  19.1           4  9999.9   \n",
       "\n",
       "   count_dewp  ...  flag_min  prcp  flag_prcp   sndp  fog  rain_drizzle  \\\n",
       "0           4  ...      <NA>   0.0          I  999.9    0             0   \n",
       "1           4  ...      <NA>  2.76          G  999.9    0             0   \n",
       "2           0  ...      <NA>   0.0          I  999.9    0             0   \n",
       "3           4  ...      <NA>   0.0          I  999.9    0             0   \n",
       "4           0  ...      <NA>   0.0          I  999.9    0             0   \n",
       "\n",
       "  snow_ice_pellets hail thunder  tornado_funnel_cloud  \n",
       "0                0    0       0                     0  \n",
       "1                0    0       0                     0  \n",
       "2                0    0       0                     0  \n",
       "3                0    0       0                     0  \n",
       "4                0    0       0                     0  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "noaa_surface = bpd.read_gbq(\"bigquery-public-data.noaa_gsod.gsod2021\")\n",
    "noaa_surface.peek()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "239ec3d1",
   "metadata": {},
   "source": [
    "You are going to plot a line chart of temperatures by date. The original dataset contains many rows for a single date, and you wan to coalesce them with their median values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e06afd00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Query job a2cee421-0f51-49a8-918a-68177b3199dc is DONE. 64.4 MB processed. <a target=\"_blank\" href=\"https://console.cloud.google.com/bigquery?project=bigframes-dev&j=bq:US:a2cee421-0f51-49a8-918a-68177b3199dc&page=queryresults\">Open Job</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>temp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2021-02-12</th>\n",
       "      <td>24.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-02-11</th>\n",
       "      <td>25.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-02-13</th>\n",
       "      <td>30.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-02-14</th>\n",
       "      <td>32.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-01-09</th>\n",
       "      <td>32.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            temp\n",
       "date            \n",
       "2021-02-12  24.6\n",
       "2021-02-11  25.9\n",
       "2021-02-13  30.4\n",
       "2021-02-14  32.1\n",
       "2021-01-09  32.9"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "noaa_surface_median_temps=noaa_surface[['date', 'temp']].groupby('date').median()\n",
    "noaa_surface_median_temps.peek()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "68324aaf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='date'>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 39496 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "noaa_surface_median_temps.plot.line()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b1e75df",
   "metadata": {},
   "source": [
    "# Area Chart"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "544f5605",
   "metadata": {},
   "source": [
    "In this example you will use the table that tracks the popularity of names in the USA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0c8f9726",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>gender</th>\n",
       "      <th>year</th>\n",
       "      <th>name</th>\n",
       "      <th>number</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AL</td>\n",
       "      <td>F</td>\n",
       "      <td>1910</td>\n",
       "      <td>Sadie</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AL</td>\n",
       "      <td>F</td>\n",
       "      <td>1910</td>\n",
       "      <td>Mary</td>\n",
       "      <td>875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AR</td>\n",
       "      <td>F</td>\n",
       "      <td>1910</td>\n",
       "      <td>Vera</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AR</td>\n",
       "      <td>F</td>\n",
       "      <td>1910</td>\n",
       "      <td>Marie</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AR</td>\n",
       "      <td>F</td>\n",
       "      <td>1910</td>\n",
       "      <td>Lucille</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  state gender  year     name  number\n",
       "0    AL      F  1910    Sadie      40\n",
       "1    AL      F  1910     Mary     875\n",
       "2    AR      F  1910     Vera      39\n",
       "3    AR      F  1910    Marie      78\n",
       "4    AR      F  1910  Lucille      66"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "usa_names = bpd.read_gbq(\"bigquery-public-data.usa_names.usa_1910_2013\")\n",
    "usa_names.peek()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be525493",
   "metadata": {},
   "source": [
    "You want to visualize the trends of the popularities of three names in US history: Mary, Emily and Lisa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a12cd1f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Query job d1fc606b-18b0-4e7e-a669-0e505a95aa5a is DONE. 132.6 MB processed. <a target=\"_blank\" href=\"https://console.cloud.google.com/bigquery?project=bigframes-dev&j=bq:US:d1fc606b-18b0-4e7e-a669-0e505a95aa5a&page=queryresults\">Open Job</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>name</th>\n",
       "      <th>Emily</th>\n",
       "      <th>Lisa</th>\n",
       "      <th>Mary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1927</th>\n",
       "      <td>1631</td>\n",
       "      <td>0</td>\n",
       "      <td>70864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1918</th>\n",
       "      <td>2353</td>\n",
       "      <td>0</td>\n",
       "      <td>67492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1912</th>\n",
       "      <td>1126</td>\n",
       "      <td>0</td>\n",
       "      <td>32375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1923</th>\n",
       "      <td>2047</td>\n",
       "      <td>0</td>\n",
       "      <td>71799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1933</th>\n",
       "      <td>1036</td>\n",
       "      <td>0</td>\n",
       "      <td>55769</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "name  Emily  Lisa   Mary\n",
       "year                    \n",
       "1927   1631     0  70864\n",
       "1918   2353     0  67492\n",
       "1912   1126     0  32375\n",
       "1923   2047     0  71799\n",
       "1933   1036     0  55769"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "name_counts = usa_names[usa_names['name'].isin(('Mary', 'Emily', 'Lisa'))].groupby(('year', 'name'))['number'].sum()\n",
    "name_counts = name_counts.unstack(level=1).fillna(0)\n",
    "name_counts.peek()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4af287bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='year'>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 60696 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "name_counts.plot.area(stacked=False, alpha=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26d14b7e",
   "metadata": {},
   "source": [
    "You can also use set `subplots` to `True` to draw separate graphs for each column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "531e20b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([<Axes: xlabel='year'>, <Axes: xlabel='year'>,\n",
       "       <Axes: xlabel='year'>], dtype=object)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 44900 characters]",
      "text/plain": [
       "<Figure size 640x480 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "name_counts.plot.area(subplots=True, alpha=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89d576a9",
   "metadata": {},
   "source": [
    "# Bar Chart"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e4c6864",
   "metadata": {},
   "source": [
    "Bar Charts are suitable for analyzing categorical data. For example, you are going to check the sex distribution of the penguin data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e4aef1a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='sex'>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 13604 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "penguin_count_by_sex = penguins[penguins['sex'].isin((\"MALE\", \"FEMALE\"))].groupby('sex')['species'].count()\n",
    "penguin_count_by_sex.plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41f5f621",
   "metadata": {},
   "source": [
    "# Scatter Plot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d79c527a",
   "metadata": {},
   "source": [
    "In this example, you will explore the relationship between NYC taxi fares and trip distances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b6bf3f2a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vendor_id</th>\n",
       "      <th>pickup_datetime</th>\n",
       "      <th>dropoff_datetime</th>\n",
       "      <th>passenger_count</th>\n",
       "      <th>trip_distance</th>\n",
       "      <th>rate_code</th>\n",
       "      <th>store_and_fwd_flag</th>\n",
       "      <th>payment_type</th>\n",
       "      <th>fare_amount</th>\n",
       "      <th>extra</th>\n",
       "      <th>mta_tax</th>\n",
       "      <th>tip_amount</th>\n",
       "      <th>tolls_amount</th>\n",
       "      <th>imp_surcharge</th>\n",
       "      <th>airport_fee</th>\n",
       "      <th>total_amount</th>\n",
       "      <th>pickup_location_id</th>\n",
       "      <th>dropoff_location_id</th>\n",
       "      <th>data_file_year</th>\n",
       "      <th>data_file_month</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>2021-09-19 10:25:05+00:00</td>\n",
       "      <td>2021-09-19 10:25:10+00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>N</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>264</td>\n",
       "      <td>264</td>\n",
       "      <td>2021</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2021-09-20 14:53:02+00:00</td>\n",
       "      <td>2021-09-20 14:53:23+00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>N</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>193</td>\n",
       "      <td>193</td>\n",
       "      <td>2021</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>2021-09-14 12:01:02+00:00</td>\n",
       "      <td>2021-09-14 12:07:19+00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>N</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>170</td>\n",
       "      <td>170</td>\n",
       "      <td>2021</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>2021-09-12 10:40:32+00:00</td>\n",
       "      <td>2021-09-12 10:41:26+00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>N</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>193</td>\n",
       "      <td>193</td>\n",
       "      <td>2021</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>2021-09-25 11:57:21+00:00</td>\n",
       "      <td>2021-09-25 11:58:32+00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>N</td>\n",
       "      <td>1</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>0E-9</td>\n",
       "      <td>95</td>\n",
       "      <td>95</td>\n",
       "      <td>2021</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  vendor_id            pickup_datetime           dropoff_datetime  \\\n",
       "0         2  2021-09-19 10:25:05+00:00  2021-09-19 10:25:10+00:00   \n",
       "1         2  2021-09-20 14:53:02+00:00  2021-09-20 14:53:23+00:00   \n",
       "2         1  2021-09-14 12:01:02+00:00  2021-09-14 12:07:19+00:00   \n",
       "3         2  2021-09-12 10:40:32+00:00  2021-09-12 10:41:26+00:00   \n",
       "4         1  2021-09-25 11:57:21+00:00  2021-09-25 11:58:32+00:00   \n",
       "\n",
       "   passenger_count  trip_distance rate_code store_and_fwd_flag payment_type  \\\n",
       "0                1           0E-9       1.0                  N            1   \n",
       "1                1           0E-9       1.0                  N            1   \n",
       "2                1           0E-9       1.0                  N            1   \n",
       "3                1           0E-9       1.0                  N            1   \n",
       "4                1           0E-9       1.0                  N            1   \n",
       "\n",
       "   fare_amount  extra  mta_tax  tip_amount  tolls_amount  imp_surcharge  \\\n",
       "0         0E-9   0E-9     0E-9        0E-9          0E-9           0E-9   \n",
       "1         0E-9   0E-9     0E-9        0E-9          0E-9           0E-9   \n",
       "2         0E-9   0E-9     0E-9        0E-9          0E-9           0E-9   \n",
       "3         0E-9   0E-9     0E-9        0E-9          0E-9           0E-9   \n",
       "4         0E-9   0E-9     0E-9        0E-9          0E-9           0E-9   \n",
       "\n",
       "   airport_fee  total_amount pickup_location_id dropoff_location_id  \\\n",
       "0         0E-9          0E-9                264                 264   \n",
       "1         0E-9          0E-9                193                 193   \n",
       "2         0E-9          0E-9                170                 170   \n",
       "3         0E-9          0E-9                193                 193   \n",
       "4         0E-9          0E-9                 95                  95   \n",
       "\n",
       "   data_file_year  data_file_month  \n",
       "0            2021                9  \n",
       "1            2021                9  \n",
       "2            2021                9  \n",
       "3            2021                9  \n",
       "4            2021                9  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "taxi_trips = bpd.read_gbq('bigquery-public-data.new_york_taxi_trips.tlc_yellow_trips_2021').dropna()\n",
    "taxi_trips.peek()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "413c0f91",
   "metadata": {},
   "source": [
    "First, you santize the data a bit by remove outliers and pathological datapoints:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "d4876b08",
   "metadata": {},
   "outputs": [],
   "source": [
    "taxi_trips = taxi_trips[taxi_trips['trip_distance'].between(0, 10, inclusive='right')]\n",
    "taxi_trips = taxi_trips[taxi_trips['fare_amount'].between(0, 50, inclusive='right')]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1ed53f7",
   "metadata": {},
   "source": [
    "You also need to sort the data before plotting if you have turned on the partial ordering mode during the setup stage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "e9ddad9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "taxi_trips = taxi_trips.sort_values('pickup_datetime')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e34ab06d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='trip_distance', ylabel='fare_amount'>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 50100 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "taxi_trips.plot.scatter(x='trip_distance', y='fare_amount', alpha=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ab4ded3",
   "metadata": {},
   "source": [
    "# Advacned Plotting with Pandas/Matplotlib Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51e3b044",
   "metadata": {},
   "source": [
    "Because BigQuery DataFrame's plotting library is powered by Matplotlib and Pandas, you are able to pass in more parameters to fine tune your graph like what you do with Pandas. \n",
    "\n",
    "In the following example, you will resuse the taxi trips dataset, except that you will rename the labels for X-axis and Y-axis, use `passenger_count` for point sizes, color points with `tip_amount`, and resize the figure. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "51c4dfc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='trip distance (miles)', ylabel='fare amount (usd)'>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 155832 characters]",
      "text/plain": [
       "<Figure size 1500x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "taxi_trips['passenger_count_scaled'] = taxi_trips['passenger_count'] * 30\n",
    "\n",
    "taxi_trips.plot.scatter(\n",
    "    x='trip_distance', \n",
    "    xlabel='trip distance (miles)',\n",
    "    y='fare_amount', \n",
    "    ylabel ='fare amount (usd)',\n",
    "    alpha=0.5, \n",
    "    s='passenger_count_scaled', \n",
    "    label='passenger_count',\n",
    "    c='tip_amount',\n",
    "    cmap='jet',\n",
    "    colorbar=True,\n",
    "    legend=True,\n",
    "    figsize=(15,7),\n",
    "    sampling_n=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6356cdab",
   "metadata": {},
   "source": [
    "# Visualize Large Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fce79ba0",
   "metadata": {},
   "source": [
    "BigQuery DataFrame downloads data to your local machine for visualization. The amount of datapoints to be downloaded is capped at 1000 by default. If the amount of datapoints exceeds the cap, BigQuery DataFrame will randomly sample the amount of datapoints equal to the cap.\n",
    "\n",
    "You can override this cap by setting the `sampling_n` parameter when plotting graphs. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "3d0ef911",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='date'>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 29488 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "noaa_surface_median_temps.plot.line(sampling_n=40)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64d6f86d",
   "metadata": {},
   "source": [
    "Note: `sampling_n` has no effect on histograms. This is because BigQuery DataFrame bucketizes the data on the server side for histograms. If your amount of bins is very large, you may encounter a \"Query too large\" error instead."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
