6.24. DataFrame Pivot

pd.pivot_table()
Create a spreadsheet-style pivot table as a DataFrame
Levels in the pivot table will be stored in MultiIndex objects

Create a spreadsheet-style pivot table as a DataFrame. The levels in the pivot table will be stored in MultiIndex objects (hierarchical indexes) on the index and columns of the result DataFrame.

6.24.1. Parameters

values : column to aggregate, optional

indexcolumn, Grouper, array, or list of the previous: If an array is passed, it must be the same length as the data. The list can contain any of the other types (except list). Keys to group by on the pivot table index. If an array is passed, it is being used as the same manner as column values.
columnscolumn, Grouper, array, or list of the previous: If an array is passed, it must be the same length as the data. The list can contain any of the other types (except list). Keys to group by on the pivot table column. If an array is passed, it is being used as the same manner as column values.
aggfuncfunction, list of functions, dict, default numpy.mean: If list of functions passed, the resulting pivot table will have hierarchical columns whose top level are the function names (inferred from the function objects themselves) If dict is passed, the key is column to aggregate and value is function or list of functions.
fill_valuescalar, default None: Value to replace missing values with (in the resulting pivot table, after aggregation).
marginsbool, default False: Add all row / columns (e.g. for subtotal / grand totals).
dropnabool, default True: Do not include columns whose entries are all NaN.
margins_namestr, default 'All': Name of the row / column that will contain the totals when margins is True.
observedbool, default False: This only applies if any of the groupers are Categoricals. If True: only show observed values for categorical groupers. If False: show all values for categorical groupers.
sortbool, default True: Specifies if the result should be sorted.

6.24.2. Returns

DataFrame: An Excel style pivot table.

6.24.3. See Also

DataFrame.pivot: Pivot without aggregation that can handle non-numeric data.
DataFrame.melt: Unpivot a DataFrame from wide to long format, optionally leaving identifiers set.
wide_to_long: Wide panel to long format. Less flexible but more user-friendly than melt.

6.24.4. SetUp

>>> import pandas as pd
>>>
>>>
>>> df = pd.DataFrame([
...     {'firstname': 'Alice', 'lastname': 'Apricot', 'age': 30, 'gender': 'female'},
...     {'firstname': 'Bob', 'lastname': 'Blackthorn', 'age': 31, 'gender': 'male'},
...     {'firstname': 'Carol', 'lastname': 'Corn', 'age': 32, 'gender': 'female'},
...     {'firstname': 'Dave', 'lastname': 'Durian', 'age': 33, 'gender': 'male'},
...     {'firstname': 'Eve', 'lastname': 'Elderberry', 'age': 34, 'gender': 'female'},
...     {'firstname': 'Mallory', 'lastname': 'Melon', 'age': 15, 'gender': 'male'},
... ], index=['a', 'b', 'c', 'd', 'e', 'm'])

>>> df
  firstname    lastname  age  gender
a     Alice     Apricot   30  female
b       Bob  Blackthorn   31    male
c     Carol        Corn   32  female
d      Dave      Durian   33    male
e       Eve  Elderberry   34  female
m   Mallory       Melon   15    male

6.24.5. Pivot Table

By default, the pivot_table function will compute a mean of the values

>>> df.pivot_table(
...     values='age',
...     index='gender'
... )
              age
gender
female  32.000000
male    26.333333

>>> df.pivot_table(
...     values='age',
...     index='gender',
...     aggfunc='mean'
... )
              age
gender
female  32.000000
male    26.333333

>>> df.pivot_table(
...     values='age',
...     index='gender',
...     aggfunc='median'
... )
         age
gender
female  32.0
male    31.0

6.24.6. Aggregation Function

You can specify the aggregation function to use with the aggfunc parameter

>>> df.pivot_table(
...     values='firstname',
...     index='gender',
...     aggfunc='count'
... )
        firstname
gender
female          3
male            3

>>> df['gender'].value_counts()
gender
female    3
male      3
Name: count, dtype: int64

6.24.7. Multiple Aggregations

You can specify multiple aggregation functions to use with the aggfunc parameter

>>> df.pivot_table(
...     values='age',
...     index='gender',
...     aggfunc=['mean', 'min', 'max']
... )
             mean min max
              age age age
gender
female  32.000000  30  34
male    26.333333  15  33

6.24.8. Multiple Value Columns

You can specify named aggregations to use with the aggfunc parameter

>>> df.pivot_table(
...     values=['age', 'firstname'],
...     index='gender',
...     aggfunc={
...         'age': 'mean',
...         'firstname': 'count'
...     }
... )
              age  firstname
gender
female  32.000000          3
male    26.333333          3

>>> df.pivot_table(
...     index='gender',
...     values=['age', 'firstname'],
...     aggfunc={
...         'age': ['mean', 'min', 'max'],
...         'firstname': 'count'
...     }
... )
       age                firstname
       max       mean min     count
gender
female  34  32.000000  30         3
male    33  26.333333  15         3

6.24.9. Margins

You can add margins to the pivot table with the margins parameter

Without margins:

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     aggfunc='mean',
... )
              age
gender
female  32.000000
male    26.333333

With margins:

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     aggfunc='mean',
...     margins=True,
... )
              age
gender
female  32.000000
male    26.333333
All     29.166667

6.24.10. Group by Multiple Columns

This is one of the most common pivot table layouts.

>>> df
  firstname    lastname  age  gender
a     Alice     Apricot   30  female
b       Bob  Blackthorn   31    male
c     Carol        Corn   32  female
d      Dave      Durian   33    male
e       Eve  Elderberry   34  female
m   Mallory       Melon   15    male

>>> df['adult'] = df['age'] >= 18
>>>
>>> df
  firstname    lastname  age  gender  adult
a     Alice     Apricot   30  female   True
b       Bob  Blackthorn   31    male   True
c     Carol        Corn   32  female   True
d      Dave      Durian   33    male   True
e       Eve  Elderberry   34  female   True
m   Mallory       Melon   15    male  False

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     columns='adult',
...     aggfunc='count',
... )
adult   False  True
gender
female    NaN    3.0
male      1.0    2.0

6.24.11. Fill Missing Values

You can use multiple index levels in the pivot table.

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     columns='adult',
...     aggfunc='count',
...     fill_value=0,
... )
adult   False  True
gender
female      0      3
male        1      2

6.24.12. Using Multiple Index Levels

>>> df.pivot_table(
...     index=['gender', 'adult'],
...     values='age',
...     aggfunc='mean'
... )
               age
gender adult
female True   32.0
male   False  15.0
       True   32.0

6.24.13. Several Aggregations at Once

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     aggfunc=['count', 'mean', 'std']
... )
       count       mean       std
         age        age       age
gender
female     3  32.000000  2.000000
male       3  26.333333  9.865766

6.24.14. Equivalent to SQL GROUP BY

You can use pivot_table to perform SQL-like GROUP BY operations.

>>> df.pivot_table(
...     index='gender',
...     values='age',
...     aggfunc='mean',
... )
              age
gender
female  32.000000
male    26.333333

>>> df.groupby('gender')['age'].mean()
gender
female    32.000000
male      26.333333
Name: age, dtype: float64