Beginner Intermediate 21 min read

Chapter 32: Data Visualization & Storytelling

Data visualization sits at the intersection of statistics, design, and communication. A model that cannot be explained is a model that cannot be trusted, and a finding that cannot be shown is a finding that cannot be acted upon. Before the first dashboard ever loads in a stakeholder meeting, the practitioner must answer a sequence of questions: What does my audience need to understand? What is the honest shape of the data? Which visual encoding preserves that shape most faithfully? This chapter treats visualization not as decoration but as an analytical instrument — a means of making patterns legible to both the analyst exploring raw data and the executive deciding whether to ship a product.

The craft of visualization draws on a small number of well-studied perceptual principles. Cleveland and McGill's landmark ranking of visual encodings, Tufte's concept of data-ink ratio, and Colin Ware's work on pre-attentive processing all point toward the same practical advice: encode the most important variable on the most accurate channel (position on a common scale), eliminate visual noise ruthlessly, and guide the viewer's attention with deliberate color and annotation. We will formalize these principles before turning to code, because no library API can substitute for the judgment that precedes it.

The chapter is organized as a practitioner's progression. We begin with perceptual theory and chart selection, then build fluency with Python's matplotlib and seaborn stack, add interactivity with Plotly, introduce R's ggplot2 grammar for readers working in that ecosystem, and close with dashboard design and narrative storytelling techniques for real stakeholder communication. Throughout, we call out accessibility requirements and anti-patterns — the ways that even well-intentioned charts mislead — because visualization ethics is not optional.

Runnable companion notebook for this chapter. Download the Jupyter notebook (.ipynb)

Learning Objectives

  • Identify the correct visual encoding for a given data type and analytical goal, using the Cleveland-McGill accuracy hierarchy.
  • Apply Tufte's data-ink ratio and pre-attentive processing principles to reduce chart clutter and guide viewer attention.
  • Build publication-quality static charts with matplotlib and seaborn, including fine-grained layout control and custom theming.
  • Construct interactive charts and dashboards with Plotly Express and Plotly Dash that support filtering, hover, and drill-down.
  • Use ggplot2's grammar of graphics in R to compose layered visualizations with consistent aesthetic mappings.
  • Recognize and correct common chart anti-patterns: truncated axes, dual-axis confusion, misleading area encodings, and rainbow color maps.
  • Structure a data story with a clear narrative arc — from insight framing through evidence to a concrete recommendation — suitable for a non-technical audience.

32.1 Perceptual Principles and Chart Selection Beginner

Why Perception Matters Before Code

Every chart is a mapping from data values to visual properties. The quality of that mapping determines whether a viewer perceives the truth or a distortion. Cleveland and McGill (1984) conducted controlled experiments asking people to estimate ratios from different visual encodings. Their results, later replicated and extended, produced an accuracy ranking that every visualization practitioner should internalize.

The Encoding Accuracy Hierarchy

From most to least accurate for quantitative variables:

  1. Position on a common scale (bar chart, scatter plot)
  2. Position on non-aligned scales (small multiples)
  3. Length (bar chart with a shifted baseline — still good)
  4. Angle and slope (pie charts, line slope)
  5. Area (bubble chart, treemap)
  6. Volume (3-D bar — avoid)
  7. Color saturation and hue (heatmap)

This ranking carries a direct design prescription: when your comparison is the primary message, use position. Reserve area and color for secondary variables or for when spatial layout genuinely matters (geographic maps, matrix overviews).

For categorical variables the preferred encodings are: position (grouped bars), color hue (scatter plot categories), shape (scatter plot, use sparingly with ≤6 categories), and — least preferred — size or texture.

Pre-Attentive Processing

Certain visual features are processed by the human visual system in under 200 ms, before conscious attention is directed. These pre-attentive attributes include color hue, color intensity, size, orientation, and motion. A single red dot among fifty blue ones is found instantly; a single bold number in a table is not found instantly because weight is not pre-attentive.

Practical consequence: use a single pre-attentive attribute to highlight the one thing you want the viewer to notice first. Using three different colors plus bold text plus a callout arrow produces visual noise, not emphasis.

Tufte's Data-Ink Ratio

Edward Tufte defines the data-ink ratio as the proportion of a chart's ink devoted to non-redundant data information. The corollary principle: maximize the data-ink ratio by erasing non-data ink and redundant data ink.

Concrete examples of non-data ink to eliminate:

  • Default grey background (seaborn's darkgrid style adds gridlines that compete with data)
  • 3-D effects, drop shadows, gradient fills
  • Redundant axis labels when the title already states the unit
  • Legends that merely repeat the axis label

Tufte's ideal is not a spartan chart with no context — it is a chart where every pixel earns its presence.

$$\text{data-ink ratio} = \frac{\text{ink used to display data}}{\text{total ink used in the chart}}$$

Chart Selection Guide

The right chart type follows from three questions: What is the data type? What is the comparison type? How many variables?

Distribution of one quantitative variable: histogram, KDE plot, violin plot. Use a histogram when bin counts matter; KDE when the shape of the distribution is the message; violin when comparing distributions across categories.

Relationship between two quantitative variables: scatter plot. Add a regression line when trend is the message. Use hexbin or 2-D KDE when point density obscures the scatter.

Comparison of a quantitative variable across categories: bar chart (avoid pie charts for more than ~4 categories). If you need to show variance, use a box plot or violin. If individual points matter, use a strip plot or beeswarm.

Change over time: line chart. Use bars only when the time series is sparse (fewer than ~8 points) or when discrete periods matter more than continuity.

Part-to-whole: stacked bar (for 2-4 parts), treemap (for hierarchical parts). Avoid pie charts when slices are similar in size — human angle judgment is weak.

Geographic distribution: choropleth for aggregated rates; dot map for individual events. Never encode count with a choropleth of unequal areas without normalizing to a rate.

Common Anti-Patterns

Truncated y-axis. Starting a bar chart's y-axis at a non-zero value exaggerates differences. Rule: bar charts must start at zero because bars encode length from a baseline. Line charts may use a zoomed range when the absolute baseline is irrelevant and the trend is the message — but you must say so explicitly.

Dual y-axes. Two different scales on the same axes create spurious correlation or mask it. The perceived relationship between the two series changes arbitrarily depending on scale choice. Use small multiples instead.

Rainbow color maps. Jet and similar rainbow gradients introduce perceptual artifacts — false boundaries appear where the hue shifts rapidly. For sequential data use perceptually uniform maps: viridis, plasma, cividis. For diverging data use RdBu or coolwarm with a meaningful midpoint.

Unlabeled axes and missing units. A chart without axis labels or units cannot be verified or reproduced. This is not a style choice; it is a factual omission.

Code Examples

Encoding comparison: bar vs. pie for the same data

Demonstrates why position (bar) produces more accurate comparisons than angle (pie) for closely-valued categories.

PYTHON
import matplotlib.pyplot as plt
import numpy as np

categories = ['Product A', 'Product B', 'Product C', 'Product D', 'Product E']
values = [23, 18, 21, 20, 18]

fig, axes = plt.subplots(1, 2, figsize=(12, 4))

# Pie chart — hard to rank B, C, D, E
axes[0].pie(values, labels=categories, autopct='%1.0f%%',
            colors=plt.cm.Set2.colors)
axes[0].set_title('Pie Chart\n(try to rank B, C, D, E)')

# Bar chart — ranking is immediate
bars = axes[1].bar(categories, values, color='steelblue', edgecolor='white')
axes[1].set_ylim(0, 30)
axes[1].set_ylabel('Count')
axes[1].set_title('Bar Chart\n(ranking is immediate)')
for bar, val in zip(bars, values):
    axes[1].text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.5,
                 str(val), ha='center', va='bottom', fontsize=9)

plt.tight_layout()
plt.savefig('bar_vs_pie.png', dpi=150, bbox_inches='tight')
plt.show()
Output
Two side-by-side charts saved to bar_vs_pie.png. The pie chart makes it nearly impossible to distinguish B (18), D (20), and E (18), while the bar chart shows the ranking immediately.

Perceptually uniform vs. rainbow color map on the same data

Shows how the Jet color map introduces false visual boundaries that viridis avoids.

PYTHON
import matplotlib.pyplot as plt
import numpy as np

# Smooth 2D gradient — no real structure
rng = np.random.default_rng(42)
x = np.linspace(0, 1, 100)
y = np.linspace(0, 1, 100)
X, Y = np.meshgrid(x, y)
Z = np.sin(2 * np.pi * X) * np.cos(2 * np.pi * Y)

fig, axes = plt.subplots(1, 2, figsize=(11, 4))
for ax, cmap, title in zip(axes,
                            ['jet', 'viridis'],
                            ['Jet (misleading)', 'Viridis (perceptually uniform)']):
    im = ax.imshow(Z, cmap=cmap, aspect='auto')
    ax.set_title(title)
    plt.colorbar(im, ax=ax)

plt.tight_layout()
plt.savefig('colormap_comparison.png', dpi=150, bbox_inches='tight')
plt.show()
Output
Jet shows false red/green/blue banding on a smooth gradient; viridis shows a smooth perceptual progression.

32.2 Static Visualization with Matplotlib and Seaborn Beginner

The Matplotlib Object Model

Matplotlib exposes two APIs: a stateful pyplot interface modeled after MATLAB, and an object-oriented interface built on Figure and Axes objects. For anything beyond a single throwaway chart, use the object-oriented API. It makes multi-panel layouts, reusable styling, and fine-grained control tractable.

The hierarchy is: Figure (the whole image) → one or more Axes (each individual plot panel) → Artist objects (lines, patches, text, collections). Every visual element is an Artist that can be retrieved and modified after creation.

PYTHON
import matplotlib.pyplot as plt
import numpy as np

rng = np.random.default_rng(0)
x = rng.normal(size=200)
y = 0.8 * x + rng.normal(scale=0.5, size=200)

fig, ax = plt.subplots(figsize=(6, 4))  # returns Figure and Axes
ax.scatter(x, y, alpha=0.4, s=20, color='steelblue', label='observations')

# Add regression line manually
m, b = np.polyfit(x, y, 1)
xline = np.linspace(x.min(), x.max(), 200)
ax.plot(xline, m * xline + b, color='crimson', lw=2, label=f'OLS: y={m:.2f}x+{b:.2f}')

ax.set_xlabel('Feature X')
ax.set_ylabel('Target Y')
ax.set_title('Scatter Plot with OLS Fit')
ax.legend(frameon=False)
ax.spines[['top', 'right']].set_visible(False)  # Tufte: remove chartjunk

plt.tight_layout()
plt.savefig('scatter_ols.png', dpi=150, bbox_inches='tight')

The call ax.spines[[&#039;top&#039;, &#039;right&#039;]].set<em>visible(False) is a single-line application of the data-ink principle: the top and right spines carry no information.

Multi-Panel Layouts

plt.subplots(nrows, ncols) returns a 2-D array of Axes. For irregular grids use GridSpec or subplot</em>mosaic, the latter of which accepts an ASCII-art layout string.

PYTHON
fig, axs = plt.subplot_mosaic(
    [['hist', 'hist'],
     ['box',  'strip']],
    figsize=(10, 7)
)

This creates a wide histogram spanning the top row and two narrower panels below — without manually specifying colspan.

Seaborn: Statistical Charts with Sensible Defaults

Seaborn is built on matplotlib and provides chart types suited to statistical analysis: histplot, kdeplot, boxplot, violinplot, stripplot, pairplot, heatmap, and lmplot (scatter with regression). Its key design choice is the figure-level / axes-level split.

Axes-level functions (e.g., sns.histplot, sns.boxplot) accept an ax argument and draw into an existing matplotlib Axes. They integrate seamlessly with multi-panel layouts.

Figure-level functions (e.g., sns.displot, sns.catplot, sns.relplot) create their own Figure and return a FacetGrid object that supports automatic faceting by a categorical variable. They cannot be dropped into an existing Axes.

PYTHON
import seaborn as sns
import pandas as pd

tips = sns.load_dataset('tips')

# Axes-level: integrate into existing layout
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
sns.violinplot(data=tips, x='day', y='total_bill', hue='sex',
               split=True, ax=axes[0], palette='Set2')
axes[0].set_title('Total Bill by Day and Sex')
axes[0].spines[['top', 'right']].set_visible(False)

sns.scatterplot(data=tips, x='total_bill', y='tip',
                hue='time', style='smoker', ax=axes[1],
                alpha=0.7, s=60)
axes[1].set_title('Tip vs. Bill')
axes[1].spines[['top', 'right']].set_visible(False)

plt.tight_layout()
plt.savefig('seaborn_panels.png', dpi=150, bbox_inches='tight')

Theming and Style

Seaborn ships five built-in themes: darkgrid, whitegrid, dark, white, ticks. For publications, white or ticks with manually removed spines gives the cleanest result. Set the theme once at the top of a script:

PYTHON
sns.set_theme(style='ticks', context='paper', font_scale=1.2)

The context parameter (paper, notebook, talk, poster) scales all line widths, marker sizes, and font sizes proportionally. Use talk or poster for presentation slides — never copy a paper-context chart into a slide.

Color and Accessibility

About 8% of males and 0.5% of females have color vision deficiency (CVD). The most common form (deuteranomaly) reduces discrimination between red and green. Practical rules:

  • Use colorblind-safe palettes: colorblind (seaborn), tab10 with care, or explicitly choose from the Okabe-Ito set.
  • Never rely on color alone to convey categorical differences — reinforce with shape, pattern, or direct labeling.
  • Ensure sufficient contrast ratio for text overlaid on chart elements (WCAG 2.1 requires at least 4.5:1 for normal text).

PYTHON
# Okabe-Ito palette — safe for all forms of CVD
okabe_ito = ['#E69F00', '#56B4E9', '#009E73',
             '#F0E442', '#0072B2', '#D55E00', '#CC79A7']
sns.set_palette(okabe_ito)

Annotations and Text

Annotations transform a chart from a display of data into an argument. Use ax.annotate() for callout arrows and ax.text() for inline labels. A chart that requires the viewer to cross-reference a legend is always weaker than one with direct labels.

PYTHON
# Direct labeling instead of a legend
for line, label, color in zip(lines, labels, colors):
    ax.text(x[-1] + 0.2, line[-1], label, va='center',
            color=color, fontsize=9)
ax.get_legend().remove()

This technique — attributing labels at the line endpoints rather than in a legend — reduces the eye travel required to decode the chart and is consistently rated as more readable in user studies.

Code Examples

Faceted distribution with seaborn displot

Uses a figure-level function to show distributions of a variable split by two categorical variables simultaneously.

PYTHON
import seaborn as sns
import matplotlib.pyplot as plt

sns.set_theme(style='ticks', context='notebook')
tips = sns.load_dataset('tips')

g = sns.displot(
    data=tips,
    x='total_bill',
    col='time',
    hue='sex',
    kind='kde',
    fill=True,
    height=4,
    aspect=1.2,
    palette='colorblind'
)
g.set_axis_labels('Total Bill ($)', 'Density')
g.set_titles(col_template='{col_name} Service')
g.figure.suptitle('Bill Distribution by Time of Day and Sex', y=1.02)
plt.savefig('displot_faceted.png', dpi=150, bbox_inches='tight')
plt.show()
Output
Two KDE panels (Lunch | Dinner) side by side, each showing overlapping filled distributions for Male and Female, saved to displot_faceted.png.

Annotated time series with direct line labeling

Builds a multi-line time series chart with direct endpoint labels replacing the legend, and an event annotation.

PYTHON
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(7)
dates = pd.date_range('2022-01', periods=24, freq='ME')
products = {
    'Product A': np.cumsum(rng.normal(5, 3, 24)) + 100,
    'Product B': np.cumsum(rng.normal(3, 4, 24)) + 80,
    'Product C': np.cumsum(rng.normal(2, 2, 24)) + 60,
}
colors = ['#0072B2', '#D55E00', '#009E73']

fig, ax = plt.subplots(figsize=(10, 5))
for (name, vals), color in zip(products.items(), colors):
    ax.plot(dates, vals, color=color, lw=2)
    # Direct label at end of line
    ax.text(dates[-1] + pd.Timedelta(days=10), vals[-1],
            name, color=color, va='center', fontsize=9, fontweight='bold')

# Event annotation
event_date = pd.Timestamp('2022-09-30')
ax.axvline(event_date, color='gray', lw=1, linestyle='--')
ax.annotate('Relaunch', xy=(event_date, ax.get_ylim()[1] * 0.95),
            ha='center', color='gray', fontsize=8)

ax.set_xlim(dates[0], dates[-1] + pd.Timedelta(days=60))
ax.set_ylabel('Monthly Revenue ($k)')
ax.set_title('Revenue by Product — 2022–2023')
ax.spines[['top', 'right']].set_visible(False)
plt.tight_layout()
plt.savefig('timeseries_labeled.png', dpi=150, bbox_inches='tight')
plt.show()
Output
A 24-month line chart with three directly labeled lines and a vertical dashed line annotated 'Relaunch', saved to timeseries_labeled.png.

Correlation heatmap with masked upper triangle

Produces a clean lower-triangular correlation heatmap — removing the redundant upper half reduces visual noise.

PYTHON
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(1)
df = pd.DataFrame(rng.standard_normal((200, 6)),
                  columns=[f'Feature_{c}' for c in 'ABCDEF'])
corr = df.corr()

mask = np.triu(np.ones_like(corr, dtype=bool))  # hide upper triangle

fig, ax = plt.subplots(figsize=(7, 6))
sns.heatmap(corr, mask=mask, annot=True, fmt='.2f',
            cmap='RdBu_r', center=0, vmin=-1, vmax=1,
            linewidths=0.5, ax=ax,
            cbar_kws={'shrink': 0.8, 'label': 'Pearson r'})
ax.set_title('Feature Correlation Matrix')
plt.tight_layout()
plt.savefig('corr_heatmap.png', dpi=150, bbox_inches='tight')
plt.show()
Output
A lower-triangular heatmap with annotated correlation values, using a diverging red-blue palette centered at zero.

32.3 Interactive Visualization with Plotly Intermediate

When Static Charts Are Not Enough

A static chart forces the designer to make every analytical decision on behalf of the viewer. An interactive chart delegates some decisions to the viewer: which region to zoom, which category to filter, which outlier to inspect. Interactivity is not always better — for a printed report or a slide deck it is a liability — but for exploratory analysis and web-based reporting it substantially increases the chart's analytical bandwidth.

Plotly is the dominant Python library for browser-rendered interactive charts. It produces charts as JSON-specified Vega-Lite-style specifications that are rendered by a JavaScript engine in the browser. The plotly.express (px) module provides a high-level API modeled on seaborn; plotly.graph<em>objects (go) gives full control over every trace and layout property.

Plotly Express: The Fast Path

PYTHON
import plotly.express as px
import pandas as pd

df = px.data.gapminder()
fig = px.scatter(
    df.query('year == 2007'),
    x='gdpPercap',
    y='lifeExp',
    size='pop',
    color='continent',
    hover_name='country',
    log_x=True,
    size_max=55,
    title='GDP per Capita vs. Life Expectancy (2007)',
    labels={'gdpPercap': 'GDP per Capita (log)', 'lifeExp': 'Life Expectancy (years)'}
)
fig.update_layout(legend_title_text='Continent')
fig.show()  # opens in browser or Jupyter
fig.write_html('gapminder_2007.html')  # standalone file with no server needed

This single call produces a fully interactive chart: hover tooltips, legend click-to-filter, box-select, lasso-select, and zoom. The write</em>html method bundles the Plotly JavaScript library into a single self-contained HTML file that can be emailed or hosted on any static server.

Animated Charts

Adding animation<em>frame turns a static chart into a time-lapse:

PYTHON
fig = px.scatter(
    df,
    x='gdpPercap', y='lifeExp',
    size='pop', color='continent',
    hover_name='country',
    animation_frame='year',
    animation_group='country',
    log_x=True, size_max=45,
    range_x=[200, 120000], range_y=[25, 90],
    title='GDP vs. Life Expectancy Over Time'
)
fig.write_html('gapminder_animated.html')

Fix range</em>x and range<em>y explicitly when animating; otherwise the axes rescale per frame, making trends unreadable.

Graph Objects: Full Control

When px defaults are insufficient — non-standard traces, multi-axis layouts, custom buttons — drop down to plotly.graph</em>objects:

PYTHON
import plotly.graph_objects as go
import numpy as np

x = np.linspace(0, 4 * np.pi, 300)

fig = go.Figure()
fig.add_trace(go.Scatter(x=x, y=np.sin(x), name='sin(x)',
                          line=dict(color='royalblue', width=2)))
fig.add_trace(go.Scatter(x=x, y=np.cos(x), name='cos(x)',
                          line=dict(color='firebrick', width=2, dash='dash')))
fig.update_layout(
    title='Trigonometric Functions',
    xaxis_title='x (radians)',
    yaxis_title='f(x)',
    template='plotly_white',
    hovermode='x unified'  # single tooltip for all traces at same x
)
fig.show()

The hovermode=&#039;x unified&#039; setting is one of the most useful Plotly details: it renders a single tooltip with all series values at the cursor's x position, which is far more readable for time series comparisons than separate per-trace tooltips.

Subplots with make<em>subplots

PYTHON
from plotly.subplots import make_subplots
import plotly.graph_objects as go

fig = make_subplots(
    rows=2, cols=2,
    subplot_titles=('Volume', 'Price', 'Returns', 'Drawdown'),
    shared_xaxes=True
)
# add traces with row=, col= arguments
fig.add_trace(go.Bar(x=dates, y=volumes, name='Volume'), row=1, col=1)
fig.add_trace(go.Scatter(x=dates, y=prices, name='Price'), row=1, col=2)
# ... etc.
fig.update_layout(height=600, showlegend=False)

shared</em>xaxes=True links the zoom and pan controls across all panels in the same column, which is essential for financial or time-series dashboards where synchronized panning is the primary interaction.

Building a Minimal Dashboard with Dash

Plotly Dash wraps Plotly charts in a Flask-based web application with reactive callbacks. A minimal Dash app that connects a dropdown to a chart:

PYTHON
from dash import Dash, dcc, html, Input, Output
import plotly.express as px

app = Dash(__name__)
df = px.data.gapminder()
all_continents = df['continent'].unique()

app.layout = html.Div([
    html.H2('GDP Explorer'),
    dcc.Dropdown(
        id='continent-dropdown',
        options=[{'label': c, 'value': c} for c in all_continents],
        value='Europe',
        clearable=False
    ),
    dcc.Graph(id='scatter-chart')
])

@app.callback(
    Output('scatter-chart', 'figure'),
    Input('continent-dropdown', 'value')
)
def update_chart(continent):
    filtered = df[df['continent'] == continent]
    fig = px.scatter(
        filtered, x='gdpPercap', y='lifeExp',
        size='pop', color='country',
        animation_frame='year', log_x=True,
        size_max=40, title=f'GDP vs. Life Expectancy — {continent}'
    )
    return fig

if __name__ == '__main__':
    app.run(debug=True)

The @app.callback decorator defines a reactive dependency graph. Every time the dropdown value changes, update_chart is called and the figure is re-rendered. This pattern — Input triggers Output through a pure function — scales to arbitrarily complex multi-input dashboards.

Performance Considerations

Plotly charts serialize all data to JSON and send it to the browser. Charts with more than ~50,000 points will be sluggish. Mitigations:

  • Downsample time series with pandas.DataFrame.resample before plotting.
  • Use go.Scattergl (WebGL-based) instead of go.Scatter for large scatter plots — it handles millions of points.
  • For Dash, use server-side callbacks with caching (e.g., Flask-Caching) to avoid recomputing expensive aggregations on each interaction.

Code Examples

Interactive histogram with marginal rug and box

Uses Plotly Express to overlay a histogram with marginal distribution plots — revealing outliers that a pure histogram hides.

PYTHON
import plotly.express as px
import numpy as np

rng = np.random.default_rng(42)
data = np.concatenate([
    rng.normal(loc=50, scale=10, size=400),
    rng.normal(loc=80, scale=5, size=100)  # minority cluster
])

fig = px.histogram(
    x=data,
    nbins=40,
    marginal='box',     # 'rug', 'violin', or 'box'
    title='Bimodal Distribution — Histogram + Box',
    labels={'x': 'Value', 'y': 'Count'},
    color_discrete_sequence=['steelblue']
)
fig.update_layout(template='plotly_white', bargap=0.05)
fig.write_html('histogram_marginal.html')
fig.show()
Output
An interactive histogram with a marginal box plot above it; the box plot's outliers and whiskers reveal the minority cluster near 80 that blends into the right tail of the histogram.

Choropleth map of a normalized rate

Creates a choropleth using a normalized rate (per 100k) rather than raw counts, following best-practice for geographic visualization.

PYTHON
import plotly.express as px

# Built-in dataset: US unemployment by county FIPS
df = px.data.election()  # use gapminder country-level as proxy

# Use world choropleth with gapminder life expectancy (2007)
df_2007 = px.data.gapminder().query('year == 2007')
fig = px.choropleth(
    df_2007,
    locations='iso_alpha',
    color='lifeExp',
    hover_name='country',
    color_continuous_scale='Viridis',
    range_color=(40, 85),
    title='Life Expectancy by Country (2007)',
    labels={'lifeExp': 'Life Expectancy (years)'}
)
fig.update_layout(geo=dict(showframe=False, showcoastlines=True),
                  coloraxis_colorbar=dict(title='Years'))
fig.write_html('choropleth_lifeexp.html')
fig.show()
Output
A world choropleth with a Viridis color scale showing life expectancy from ~40 (dark purple) to ~85 (yellow), saved to choropleth_lifeexp.html.

32.4 Grammar of Graphics with ggplot2 (R) Intermediate

The Grammar of Graphics

Leland Wilkinson's The Grammar of Graphics (1999) proposed that every statistical chart can be described as a composition of independent components: data, aesthetic mappings, geometric objects, scales, coordinate systems, and facets. Hadley Wickham implemented this grammar in the R package ggplot2 (2005, updated 2016), and it remains the most influential visualization API in data science. Understanding the grammar makes chart construction feel inevitable rather than arbitrary.

The seven components of a ggplot2 specification:

  1. Data — a tidy data frame (one observation per row, one variable per column)
  2. Aesthetics (aes) — a mapping from data variables to visual properties: x, y, color, fill, size, shape, alpha, linetype
  3. Geometries (geom<em><em>) — the geometric object that represents data: point, line, bar, histogram, violin, tile, ...
  4. Statistics (stat</em></em>) — a transformation applied before rendering: binning, smoothing, summarizing
  5. Scales (scale<em><em>) — mapping from data space to aesthetic space, with axis labels, breaks, and color palettes
  6. Coordinate system (coord</em></em>) — Cartesian, polar, flipped, map projections
  7. Facets (facet<em>wrap, facet</em>grid) — small multiples by categorical variable
  8. A Minimal Example

R
library(ggplot2)

ggplot(data = mpg, aes(x = displ, y = hwy, color = class)) +
  geom_point(alpha = 0.7, size = 2) +
  geom_smooth(method = 'lm', se = FALSE, color = 'black', linewidth = 0.7) +
  scale_color_brewer(palette = 'Dark2') +
  labs(
    title = 'Engine Displacement vs. Highway MPG',
    x = 'Engine Displacement (L)',
    y = 'Highway MPG',
    color = 'Vehicle Class'
  ) +
  theme_minimal(base_size = 12) +
  theme(
    legend.position = 'bottom',
    panel.grid.minor = element_blank()
  )

Each layer is added with +. This additive structure means you can build a chart incrementally: start with ggplot() and aes(), add a geom, inspect the result, add a scale, inspect again. The grammar enforces a clean separation between what data to show (aes) and how to show it (geom).

Aesthetic Mapping vs. Aesthetic Setting

This is the single most common source of ggplot2 bugs. When an aesthetic is inside aes() it is mapped from data. When it is outside aes() it is set to a fixed value.

R
# MAPPED: color varies by cylinder count
ggplot(mtcars, aes(x = wt, y = mpg, color = factor(cyl))) +
  geom_point(size = 3)

# SET: all points are blue regardless of data
ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(color = 'steelblue', size = 3)

# BUG: color='red' inside aes() maps the string 'red' to a default color
# scale — all points get the same color, but it won't be red:
ggplot(mtcars, aes(x = wt, y = mpg, color = 'red')) +  # WRONG
  geom_point(size = 3)

Faceting: Small Multiples

Small multiples are among the most powerful techniques in exploratory visualization. They use position — the most accurate encoding — to represent an additional categorical variable. ggplot2 makes this effortless.

R
ggplot(diamonds, aes(x = carat, y = price)) +
  geom_hex(bins = 40) +  # 2D density via hexagonal binning
  scale_fill_viridis_c(trans = 'log10', name = 'Count (log10)') +
  facet_wrap(~cut, nrow = 2) +
  labs(title = 'Diamond Price vs. Carat by Cut',
       x = 'Carat', y = 'Price ($)') +
  theme_bw()

The ~cut formula tells facet<em>wrap to create one panel per level of the cut variable. All panels share the same x and y scales by default, making comparisons between panels honest.

Building a Polished Bar Chart

The canonical ggplot2 bar chart with error bars, ordered bars (a common requirement), and direct value labeling:

R
library(ggplot2)
library(dplyr)

set.seed(42)
df <- data.frame(
  group = LETTERS[1:6],
  mean  = c(42, 55, 37, 61, 48, 53),
  se    = c(3, 4, 2.5, 5, 3.5, 4.2)
)

df <- df |> mutate(group = reorder(group, mean))  # order by value

ggplot(df, aes(x = mean, y = group)) +
  geom_col(fill = '#0072B2', width = 0.65) +
  geom_errorbarh(
    aes(xmin = mean - 1.96 * se, xmax = mean + 1.96 * se),
    height = 0.25, color = 'white', linewidth = 0.8
  ) +
  geom_text(aes(label = mean), hjust = -0.3, size = 3.5, color = 'grey30') +
  scale_x_continuous(expand = expansion(mult = c(0, 0.12))) +
  labs(title = 'Mean Score by Group (95% CI)',
       x = 'Mean Score', y = NULL) +
  theme_minimal(base_size = 12) +
  theme(
    panel.grid.major.y = element_blank(),
    axis.ticks = element_blank()
  )

Key techniques: reorder(group, mean) sorts bars by value so the viewer immediately sees the ranking; hjust = -0.3 places value labels just outside the bar end; expansion(mult = c(0, 0.12)) adds right-side padding for the labels without adding padding on the left (which would separate the bars from the y-axis).

Custom Themes

For a consistent house style, define a custom theme function that wraps theme</em>minimal:

R
theme_corporate <- function(base_size = 12) {
  theme_minimal(base_size = base_size) %+replace%
    theme(
      text           = element_text(family = 'sans', color = '#2C3E50'),
      plot.title     = element_text(face = 'bold', size = base_size * 1.2),
      plot.subtitle  = element_text(color = '#7F8C8D', margin = margin(b = 10)),
      panel.grid.minor = element_blank(),
      panel.grid.major = element_line(color = '#ECF0F1'),
      axis.ticks     = element_blank(),
      legend.position = 'bottom'
    )
}

# Use it like any other theme:
ggplot(mpg, aes(displ, hwy)) +
  geom_point() +
  theme_corporate()

Store theme_corporate in a shared R package or a utils.R file sourced by all scripts in a project. This ensures every chart in a report has identical typography and grid styling without repeating theme code.

Code Examples

Raincloud plot: combining violin, box, and raw data

A raincloud plot shows the distribution (violin), summary statistics (box), and individual points simultaneously — more honest than a bar chart with error bars.

R
library(ggplot2)
library(dplyr)

set.seed(99)
df <- data.frame(
  group = rep(c('Control', 'Treatment A', 'Treatment B'), each = 80),
  value = c(rnorm(80, 50, 10), rnorm(80, 58, 12), rnorm(80, 65, 8))
)

ggplot(df, aes(x = group, y = value, fill = group, color = group)) +
  # Half violin (density)
  geom_violin(trim = FALSE, alpha = 0.4, width = 0.8) +
  # Box plot inside
  geom_boxplot(width = 0.15, outlier.shape = NA, alpha = 0.8, color = 'white') +
  # Jittered points
  geom_jitter(width = 0.08, alpha = 0.3, size = 1.2) +
  scale_fill_manual(values  = c('#0072B2', '#D55E00', '#009E73')) +
  scale_color_manual(values = c('#0072B2', '#D55E00', '#009E73')) +
  labs(title = 'Response by Treatment Group',
       x = NULL, y = 'Response Value') +
  theme_minimal(base_size = 13) +
  theme(legend.position = 'none',
        panel.grid.major.x = element_blank())
Output
Three overlapping violin+box+jitter panels in blue, orange, and green, showing distributional shape alongside individual data points.

Heatmap calendar of daily values

Produces a GitHub-style contribution calendar heatmap using geom_tile and facet_wrap by month.

R
library(ggplot2)
library(dplyr)
library(lubridate)

set.seed(5)
dates <- seq.Date(as.Date('2023-01-01'), as.Date('2023-12-31'), by = 'day')
df <- data.frame(
  date  = dates,
  value = pmax(0, rnorm(length(dates), mean = 5, sd = 3))
) |>
  mutate(
    week_of_month = ceiling(day(date) / 7),
    dow           = wday(date, label = TRUE, week_start = 1),
    month         = month(date, label = TRUE)
  )

ggplot(df, aes(x = week_of_month, y = fct_rev(dow), fill = value)) +
  geom_tile(color = 'white', linewidth = 0.4, width = 0.9, height = 0.9) +
  facet_wrap(~month, nrow = 3) +
  scale_fill_viridis_c(option = 'plasma', name = 'Value') +
  scale_x_continuous(breaks = NULL) +
  labs(title = 'Daily Activity Calendar — 2023',
       x = NULL, y = NULL) +
  theme_minimal(base_size = 10) +
  theme(panel.grid = element_blank(),
        strip.text = element_text(face = 'bold'))
Output
A 12-panel calendar heatmap with days-of-week on the y-axis and weeks on the x-axis, colored by plasma palette intensity.

32.5 Narrative Structure and Communicating to Stakeholders Intermediate

From Analysis to Argument

A technically correct visualization can fail completely in a stakeholder meeting. The failure mode is not inaccuracy — it is irrelevance. The analyst shows a chart; the stakeholder asks "so what?"; the analyst has no answer. Storytelling with data means structuring a presentation so that every visualization builds toward a specific, actionable conclusion.

The SCR (Situation-Complication-Resolution) framework from management consulting is directly applicable to data presentations:

  • Situation: The current state, agreed upon by all parties. "We have 2M monthly active users and $40M ARR."
  • Complication: The event, trend, or question that disrupts the stable situation. "Retention among users acquired after the Q3 product change has dropped 18 percentage points."
  • Resolution: The recommended action, supported by evidence. "Reverting the onboarding flow to the pre-Q3 version is projected to recover 12pp of retention based on A/B test data from a 5% holdout group."

Every chart in the deck should serve one of these three roles. Charts that are interesting but do not advance the SCR narrative should be moved to an appendix.

The Assertion-Evidence Slide Design

The most common slide error is a chart title that describes the chart rather than asserts its conclusion. Compare:

  • Descriptive title (weak): "Retention Rate by Cohort, Q1–Q4"
  • Assertive title (strong): "Post-Q3 Cohorts Retain 18pp Fewer Users at 90 Days"

The assertive title is the insight. The chart is the evidence for that insight. When the viewer can read the slide title and understand the conclusion, the chart confirms and quantifies it. When the title is only a description, the viewer must derive the conclusion themselves — and may reach the wrong one.

Designing for Rapid Comprehension

Stakeholder attention is limited. Design charts that can be decoded in 10 seconds or fewer:

Reduce cognitive load. Remove legends when direct labeling is possible. Annotate the single most important data point. Use a subtitle to state the takeaway in one sentence.

Direct the eye. Use a single pre-attentive attribute (typically color) to highlight the relevant element. Gray out everything else.

PYTHON
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(3)
quarters = ['Q1', 'Q2', 'Q3', 'Q4']
cohorts  = ['Pre-Q3', 'Post-Q3']
retention = {
    'Pre-Q3':  [72, 70, 71, 69],
    'Post-Q3': [73, 71, 54, 51],
}

fig, ax = plt.subplots(figsize=(8, 4.5))
for cohort, vals, color, lw, alpha in [
    ('Pre-Q3',  retention['Pre-Q3'],  '#AAAAAA', 1.5, 0.8),
    ('Post-Q3', retention['Post-Q3'], '#D55E00', 2.5, 1.0),
]:
    ax.plot(quarters, vals, color=color, lw=lw, alpha=alpha, marker='o', ms=6)
    ax.text(quarters[-1], vals[-1] + 0.8, cohort, color=color,
            fontsize=10, fontweight='bold' if cohort == 'Post-Q3' else 'normal')

ax.annotate('18pp drop after\nproduct change',
            xy=('Q3', 54), xytext=('Q3', 44),
            arrowprops=dict(arrowstyle='->', color='#D55E00'),
            color='#D55E00', fontsize=9, ha='center')

ax.set_ylim(35, 80)
ax.set_xlabel('Quarter')
ax.set_ylabel('90-Day Retention (%)')
ax.set_title('Post-Q3 Cohorts Retain 18pp Fewer Users at 90 Days',
             fontsize=12, fontweight='bold')
ax.spines[['top', 'right']].set_visible(False)
plt.tight_layout()
plt.savefig('retention_narrative.png', dpi=150, bbox_inches='tight')

The Post-Q3 line is highlighted in orange; the Pre-Q3 line is grayed out. The annotation drives the eye to the Q3 inflection point. The assertive title states the conclusion. A stakeholder can process this chart in five seconds.

Dashboard Design Principles

Dashboards differ from individual charts: they must answer multiple questions simultaneously, support different levels of detail, and be useful to viewers with varying expertise. Core principles:

Hierarchy of information. The most important KPIs (revenue, active users, error rate) belong in the top-left — the area the Western-language eye reads first. Supporting breakdowns and trend charts go below and to the right.

Context for every metric. A number without context is meaningless. Every KPI should show a comparison: versus prior period, versus target, versus plan. Use sparklines or small trend arrows to convey direction without consuming full chart space.

Consistent color semantics. Choose one color to mean "good" (green or blue), one to mean "alert" (orange), one to mean "bad" (red), and use them consistently across the entire dashboard. Never repurpose a semantic color for a non-semantic use.

Progressive disclosure. The top-level dashboard shows aggregated summary metrics. Clicking through reveals breakdowns and raw data. This preserves cognitive bandwidth for the viewer who only needs the summary while preserving detail for the analyst who needs to investigate.

Accessibility in Dashboards

Accessibility is a legal requirement in many jurisdictions (WCAG 2.1 for web content) and a quality requirement everywhere else. For dashboards:

  • All charts must have text alternatives (alt text or ARIA labels) for screen readers.
  • Color must not be the only differentiator between categories — use shape, pattern, or direct labeling as a secondary encoding.
  • Minimum font size of 14px for body text in charts; chart elements must have sufficient contrast against their background.
  • Interactive controls (dropdowns, date pickers) must be keyboard navigable.
  • Test with a CVD simulation tool (e.g., Coblis or the colorblindcheck R package) before publishing.
  • Choosing the Right Channel for Insights

Not every insight belongs in a dashboard. Consider the audience and urgency:

  • Operational dashboards (engineers, operations): real-time or near-real-time, sparse, focused on a few critical metrics, optimized for speed of reading.
  • Analytical reports (managers, directors): weekly or monthly, richer narrative context, conclusions stated explicitly in prose accompanying the charts.
  • Executive decks (C-suite, board): quarterly, five or fewer key charts, assertive titles, conclusion-first ordering, no exploration required.
  • Self-service BI tools (broad employee population): flexible filtering, contextual tooltips explaining metric definitions, documentation embedded in the UI.

The chart that is appropriate in an analytical report may be inappropriate in an executive deck. The same finding requires different visual treatments for different channels — not because the data changes, but because the question the viewer is trying to answer changes.

Code Examples

Annotated 'before vs. after' comparison chart

Implements a stakeholder-ready before/after bar chart with highlighting and an assertive title.

PYTHON
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import numpy as np

metrics = ['Activation\nRate', 'Day-7\nRetention', 'Day-30\nRetention', 'NPS']
before  = [42, 31, 18, 24]
after   = [49, 40, 29, 31]
x = np.arange(len(metrics))
w = 0.35

fig, ax = plt.subplots(figsize=(9, 5))
bars_b = ax.bar(x - w/2, before, w, label='Before (Q2)', color='#BBBBBB')
bars_a = ax.bar(x + w/2, after,  w, label='After  (Q3)', color='#0072B2')

# Value labels
for bar in bars_b:
    ax.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.5,
            f'{int(bar.get_height())}', ha='center', va='bottom', fontsize=9, color='#666')
for bar in bars_a:
    ax.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.5,
            f'{int(bar.get_height())}', ha='center', va='bottom', fontsize=9,
            color='#0072B2', fontweight='bold')

# Delta annotations
for i, (b, a) in enumerate(zip(before, after)):
    ax.annotate(f'+{a-b}', xy=(x[i] + w/2, a + 3), ha='center',
                fontsize=9, color='#009E73', fontweight='bold')

ax.set_xticks(x)
ax.set_xticklabels(metrics)
ax.set_ylabel('Rate / Score')
ax.set_title('New Onboarding Flow Improves All Key Metrics',
             fontsize=13, fontweight='bold')
ax.legend(frameon=False)
ax.spines[['top', 'right']].set_visible(False)
ax.set_ylim(0, 55)
plt.tight_layout()
plt.savefig('before_after_metrics.png', dpi=150, bbox_inches='tight')
plt.show()
Output
A grouped bar chart with gray 'Before' and blue 'After' bars, green delta annotations above each pair, and an assertive title.

Minimal KPI scorecard with sparklines

Renders a text-and-sparkline KPI summary panel using matplotlib — a common top-of-dashboard pattern.

PYTHON
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import numpy as np

rng = np.random.default_rng(10)

kpis = [
    {'label': 'Monthly Revenue', 'value': '$4.2M', 'delta': '+8%',
     'color': '#009E73', 'trend': np.cumsum(rng.normal(0.5, 1, 12)) + 40},
    {'label': 'Active Users',    'value': '183K',  'delta': '+3%',
     'color': '#009E73', 'trend': np.cumsum(rng.normal(0.3, 1, 12)) + 170},
    {'label': 'Churn Rate',      'value': '2.4%',  'delta': '-0.3pp',
     'color': '#009E73', 'trend': np.cumsum(rng.normal(-0.05, 0.1, 12)) + 3},
    {'label': 'P95 Latency',     'value': '210ms', 'delta': '+40ms',
     'color': '#D55E00', 'trend': np.cumsum(rng.normal(1, 2, 12)) + 170},
]

fig = plt.figure(figsize=(14, 3))
gs = gridspec.GridSpec(1, len(kpis), figure=fig)

for i, kpi in enumerate(kpis):
    ax = fig.add_subplot(gs[i])
    ax.plot(kpi['trend'], color=kpi['color'], lw=2)
    ax.fill_between(range(12), kpi['trend'], alpha=0.15, color=kpi['color'])
    ax.axis('off')
    ax.set_title(kpi['label'], fontsize=10, color='#555', pad=4)
    ax.text(0.5, -0.1, kpi['value'], transform=ax.transAxes,
            fontsize=18, fontweight='bold', ha='center', color='#222')
    ax.text(0.5, -0.28, kpi['delta'], transform=ax.transAxes,
            fontsize=11, ha='center', color=kpi['color'], fontweight='bold')

fig.suptitle('Executive Dashboard — June 2025', fontsize=13,
             fontweight='bold', y=1.05, color='#2C3E50')
plt.tight_layout()
plt.savefig('kpi_scorecard.png', dpi=150, bbox_inches='tight')
plt.show()
Output
Four KPI tiles in a row, each showing a metric name, large current value, delta vs. prior period in semantic color, and a sparkline trend.