{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "\n",
    "<a href='http://www.pieriandata.com'><img src='../Pierian_Data_Logo.png'/></a>\n",
    "___\n",
    "<center><em>Copyright Pierian Data</em></center>\n",
    "<center><em>For more information, visit us at <a href='http://www.pieriandata.com'>www.pieriandata.com</a></em></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Time Shifting\n",
    "\n",
    "Sometimes you may need to shift all your data up or down along the time series index. In fact, a lot of pandas built-in methods do this under the hood. This isn't something we'll do often in the course, but it's definitely good to know about this anyways!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('../Data/starbucks.csv',index_col='Date',parse_dates=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>38.0061</td>\n",
       "      <td>6906098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>37.2781</td>\n",
       "      <td>11623796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>36.9748</td>\n",
       "      <td>7664340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-07</th>\n",
       "      <td>37.8848</td>\n",
       "      <td>9732554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08</th>\n",
       "      <td>38.4961</td>\n",
       "      <td>13170548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Close    Volume\n",
       "Date                         \n",
       "2015-01-02  38.0061   6906098\n",
       "2015-01-05  37.2781  11623796\n",
       "2015-01-06  36.9748   7664340\n",
       "2015-01-07  37.8848   9732554\n",
       "2015-01-08  38.4961  13170548"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-12-24</th>\n",
       "      <td>60.56</td>\n",
       "      <td>6323252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-26</th>\n",
       "      <td>63.08</td>\n",
       "      <td>16646238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-27</th>\n",
       "      <td>63.20</td>\n",
       "      <td>11308081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-28</th>\n",
       "      <td>63.39</td>\n",
       "      <td>7712127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-31</th>\n",
       "      <td>64.40</td>\n",
       "      <td>7690183</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Close    Volume\n",
       "Date                       \n",
       "2018-12-24  60.56   6323252\n",
       "2018-12-26  63.08  16646238\n",
       "2018-12-27  63.20  11308081\n",
       "2018-12-28  63.39   7712127\n",
       "2018-12-31  64.40   7690183"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## .shift() forward\n",
    "This method shifts the entire date index a given number of rows, without regard for time periods (months & years).<br>It returns a modified copy of the original DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>38.0061</td>\n",
       "      <td>6906098.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>37.2781</td>\n",
       "      <td>11623796.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-07</th>\n",
       "      <td>36.9748</td>\n",
       "      <td>7664340.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08</th>\n",
       "      <td>37.8848</td>\n",
       "      <td>9732554.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Close      Volume\n",
       "Date                           \n",
       "2015-01-02      NaN         NaN\n",
       "2015-01-05  38.0061   6906098.0\n",
       "2015-01-06  37.2781  11623796.0\n",
       "2015-01-07  36.9748   7664340.0\n",
       "2015-01-08  37.8848   9732554.0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shift(1).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-12-24</th>\n",
       "      <td>61.39</td>\n",
       "      <td>23524888.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-26</th>\n",
       "      <td>60.56</td>\n",
       "      <td>6323252.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-27</th>\n",
       "      <td>63.08</td>\n",
       "      <td>16646238.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-28</th>\n",
       "      <td>63.20</td>\n",
       "      <td>11308081.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-31</th>\n",
       "      <td>63.39</td>\n",
       "      <td>7712127.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Close      Volume\n",
       "Date                         \n",
       "2018-12-24  61.39  23524888.0\n",
       "2018-12-26  60.56   6323252.0\n",
       "2018-12-27  63.08  16646238.0\n",
       "2018-12-28  63.20  11308081.0\n",
       "2018-12-31  63.39   7712127.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# NOTE: You will lose that last piece of data that no longer has an index!\n",
    "df.shift(1).tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## .shift() backwards"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>37.2781</td>\n",
       "      <td>11623796.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>36.9748</td>\n",
       "      <td>7664340.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>37.8848</td>\n",
       "      <td>9732554.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-07</th>\n",
       "      <td>38.4961</td>\n",
       "      <td>13170548.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08</th>\n",
       "      <td>37.2361</td>\n",
       "      <td>27556706.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Close      Volume\n",
       "Date                           \n",
       "2015-01-02  37.2781  11623796.0\n",
       "2015-01-05  36.9748   7664340.0\n",
       "2015-01-06  37.8848   9732554.0\n",
       "2015-01-07  38.4961  13170548.0\n",
       "2015-01-08  37.2361  27556706.0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shift(-1).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-12-24</th>\n",
       "      <td>63.08</td>\n",
       "      <td>16646238.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-26</th>\n",
       "      <td>63.20</td>\n",
       "      <td>11308081.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-27</th>\n",
       "      <td>63.39</td>\n",
       "      <td>7712127.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-28</th>\n",
       "      <td>64.40</td>\n",
       "      <td>7690183.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-31</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Close      Volume\n",
       "Date                         \n",
       "2018-12-24  63.08  16646238.0\n",
       "2018-12-26  63.20  11308081.0\n",
       "2018-12-27  63.39   7712127.0\n",
       "2018-12-28  64.40   7690183.0\n",
       "2018-12-31    NaN         NaN"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shift(-1).tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shifting based on Time Series Frequency Code\n",
    "\n",
    "We can choose to shift <em>index values</em> up or down without realigning the data by passing in a <strong>freq</strong> argument.<br>\n",
    "This method shifts dates to the next period based on a frequency code. Common codes are 'M' for month-end and 'A' for year-end. <br>Refer to the <em>Time Series Offset Aliases</em> table from the Time Resampling lecture for a full list of values, or click <a href='http://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases'>here</a>.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-31</th>\n",
       "      <td>38.0061</td>\n",
       "      <td>6906098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-31</th>\n",
       "      <td>37.2781</td>\n",
       "      <td>11623796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-31</th>\n",
       "      <td>36.9748</td>\n",
       "      <td>7664340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-31</th>\n",
       "      <td>37.8848</td>\n",
       "      <td>9732554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-31</th>\n",
       "      <td>38.4961</td>\n",
       "      <td>13170548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Close    Volume\n",
       "Date                         \n",
       "2015-01-31  38.0061   6906098\n",
       "2015-01-31  37.2781  11623796\n",
       "2015-01-31  36.9748   7664340\n",
       "2015-01-31  37.8848   9732554\n",
       "2015-01-31  38.4961  13170548"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Shift everything forward one month\n",
    "df.shift(periods=1, freq='M').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more info on time shifting visit http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.shift.html<br>\n",
    "Up next we'll look at rolling and expanding!"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}