Show code
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.express as px
from _plot import show
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import confusion_matrix, accuracy_scoreA classification tree and a Monte Carlo simulation
Arsy Nas (group coursework, UBC Sauder COMM 414)
2022
COMM 414 Assignment 5, a group assignment from 2022. It has two separate parts.
Part A: ticket resale alerts (classification tree)
Part B: cinema screen plan (Monte Carlo simulation)
A baseball team watches the resale market for its tickets. A high share of resale tickets can show high demand, a low price or resellers at work. The brief defines an alert level for each game from the resale ratio.
baseball = pd.read_csv('../baseball.csv')
baseball['resale_ratio'] = baseball['tickets_resale'] / baseball['tickets_sold']
def alert_level(resale_ratio):
if resale_ratio <= 0.075:
return 'low'
elif resale_ratio <= 0.1:
return 'medium'
return 'high'
baseball['alert'] = baseball['resale_ratio'].apply(alert_level)
alert_counts = baseball['alert'].value_counts().reindex(['low', 'medium', 'high'])
print("Games in each alert level:")
print(alert_counts.to_string())
print(f"\nMean resale ratio: {baseball['resale_ratio'].mean():.3f}")Games in each alert level:
alert
low 209
medium 161
high 91
Mean resale ratio: 0.083
fig = px.bar(x=alert_counts.index, y=alert_counts.values, color=alert_counts.index,
color_discrete_map={'low': '#3DDA84', 'medium': '#F4B942', 'high': '#E84855'},
labels={'x': 'Alert level', 'y': 'Games'}, title='Games in each resale alert level')
fig.update_layout(showlegend=False, height=380)
show(fig)train_bb = baseball.query('2011 <= season <= 2015')
test_bb = baseball.query('season == 2016')
features = ['attendance', 'tickets_sold', 'tickets_resale', 'opponent_payroll', 'opponent_rank', 'opponent_win_pcnt']
x_train, y_train = train_bb[features], train_bb['alert']
x_test, y_test = test_bb[features], test_bb['alert']
print(f"Training set: {len(x_train)} games (2011 to 2015)")
print(f"Test set: {len(x_test)} games (2016)")Training set: 405 games (2011 to 2015)
Test set: 56 games (2016)
Training accuracy: 100.0%
Test accuracy: 87.5%
The tree learns the training games fully. The gap between the two values shows that it fits the training seasons more closely than the test season.
depths = list(range(1, 15))
train_scores, test_scores = [], []
for d in depths:
clf_d = DecisionTreeClassifier(criterion='entropy', random_state=0, max_depth=d).fit(x_train, y_train)
train_scores.append(clf_d.score(x_train, y_train))
test_scores.append(clf_d.score(x_test, y_test))
fig = go.Figure()
fig.add_trace(go.Scatter(x=depths, y=train_scores, mode='lines+markers', name='Training', line=dict(color='#2E86AB', width=2)))
fig.add_trace(go.Scatter(x=depths, y=test_scores, mode='lines+markers', name='Test', line=dict(color='#3DDA84', width=2)))
fig.add_vline(x=6, line_dash='dash', line_color='#E84855', annotation_text='Depth that we chose', annotation_position='top right')
fig.update_layout(title='Accuracy for each tree depth', xaxis_title='Maximum depth', yaxis_title='Accuracy',
yaxis=dict(tickformat='.0%'), height=420)
show(fig)clf_opt = DecisionTreeClassifier(criterion='entropy', random_state=0, max_depth=6).fit(x_train, y_train)
y_pred = clf_opt.predict(x_test)
cm = confusion_matrix(y_test, y_pred, labels=['low', 'medium', 'high'])
acc = accuracy_score(y_test, y_pred)
fig = go.Figure(data=go.Heatmap(
z=cm, x=['Predicted: low', 'Predicted: medium', 'Predicted: high'], y=['Actual: low', 'Actual: medium', 'Actual: high'],
text=cm, texttemplate='%{text}', colorscale=[[0, '#EEF2F7'], [1, '#2E86AB']], showscale=False,
))
fig.update_layout(title=f'Confusion matrix, depth 6 (accuracy {acc:.1%})', height=400)
show(fig)baseline = y_test.value_counts(normalize=True).max()
results = pd.DataFrame({
'Model': ['Tree with no depth limit', 'Tree with depth 6 (submitted)', 'Most frequent level (baseline)'],
'Training accuracy': [f'{train_acc_full:.1%}', f'{clf_opt.score(x_train, y_train):.1%}', '-'],
'Test accuracy': [f'{test_acc_full:.1%}', f'{acc:.1%}', f'{baseline:.1%}'],
})
results.style.hide(axis='index')| Model | Training accuracy | Test accuracy |
|---|---|---|
| Tree with no depth limit | 100.0% | 87.5% |
| Tree with depth 6 (submitted) | 94.8% | 82.1% |
| Most frequent level (baseline) | - | 58.9% |
For: fans can buy and sell tickets easily and can change games. A lower resale price can bring more guests, who also buy food and merchandise.
Against: resellers make a profit on the product of the team. The team has less control of prices, season tickets can lose value, and the market is difficult to monitor.
A cinema has five screens and shows three films on one night:
The manager gives each screen a film before she knows the demand. Her revenue target for the night is $6,000.
lam = 400 # mean number of guests in one night
sims = 100_000 # simulated nights for each plan
np.random.seed(42)
p_avatar, p_black_adam, p_the_fabelmans = 0.4, 0.4, 0.2
price_imax, price_reg = 19.99, 12.99
cap_A, cap_B, cap_C, cap_D, cap_E = 120, 120, 80, 80, 40 # 440 seats
# As in the submitted notebook: one Poisson draw for each film
exp_avatar = np.random.poisson(lam, sims) * p_avatar
exp_black_adam = np.random.poisson(lam, sims) * p_black_adam
exp_fabelmans = np.random.poisson(lam, sims) * p_the_fabelmans
print(f"Simulated nights: {sims:,}")
print(f"Mean demand: Avatar {exp_avatar.mean():.0f} | Black Adam {exp_black_adam.mean():.0f} | The Fabelmans {exp_fabelmans.mean():.0f}")Simulated nights: 100,000
Mean demand: Avatar 160 | Black Adam 160 | The Fabelmans 80
| Screen | Film | Type | Seats |
|---|---|---|---|
| A | Avatar | IMAX | 120 |
| B | Avatar | IMAX | 120 |
| C | Black Adam | Regular | 80 |
| D | Black Adam | Regular | 80 |
| E | The Fabelmans | Regular | 40 |
s_A = np.minimum(exp_avatar, cap_A)
s_B = np.minimum(exp_avatar - s_A, cap_B)
s_C = np.minimum(exp_black_adam, cap_C)
s_D = np.minimum(exp_black_adam - s_C, cap_D)
s_E = np.minimum(exp_fabelmans, cap_E)
s_rev = (s_A + s_B) * price_imax + (s_C + s_D + s_E) * price_reg
sarah_avg = s_rev.mean()
print(f"Expected revenue: ${sarah_avg:,.2f}")
print(f"Mean seats filled: {(s_A + s_B + s_C + s_D + s_E).mean():.0f} of 440")
for name, s, cap in [('A, Avatar IMAX', s_A, cap_A), ('B, Avatar IMAX', s_B, cap_B), ('C, Black Adam', s_C, cap_C),
('D, Black Adam', s_D, cap_D), ('E, The Fabelmans', s_E, cap_E)]:
print(f" Screen {name}: {s.mean():.0f} of {cap}")Expected revenue: $5,754.86
Mean seats filled: 357 of 440
Screen A, Avatar IMAX: 120 of 120
Screen B, Avatar IMAX: 40 of 120
Screen C, Black Adam: 80 of 80
Screen D, Black Adam: 77 of 80
Screen E, The Fabelmans: 40 of 40
This plan gives 240 seats to Avatar for about 160 guests, and 40 seats to The Fabelmans for about 80 guests.
| Screen | Film | Type | Seats |
|---|---|---|---|
| A | Avatar | IMAX | 120 |
| B | Black Adam | Regular | 120 |
| C | Avatar | Regular | 80 |
| D | The Fabelmans | Regular | 80 |
| E | Black Adam | Regular | 40 |
Avatar gets 200 seats, Black Adam gets 160 and The Fabelmans gets 80.
o_A = np.minimum(exp_avatar, cap_A)
o_B = np.minimum(exp_black_adam, cap_B)
o_C = np.minimum(exp_avatar - o_A, cap_C)
o_D = np.minimum(exp_fabelmans, cap_D)
o_E = np.minimum(exp_black_adam - o_B, cap_E)
o_rev = o_A * price_imax + (o_B + o_C + o_D + o_E) * price_reg
opt_avg = o_rev.mean()
print(f"Expected revenue: ${opt_avg:,.2f}")
print(f"Mean seats filled: {(o_A + o_B + o_C + o_D + o_E).mean():.0f} of 440")
for name, s, cap in [('A, Avatar IMAX', o_A, cap_A), ('B, Black Adam', o_B, cap_B), ('C, Avatar', o_C, cap_C),
('D, The Fabelmans', o_D, cap_D), ('E, Black Adam', o_E, cap_E)]:
print(f" Screen {name}: {s.mean():.0f} of {cap}")Expected revenue: $5,973.75
Mean seats filled: 395 of 440
Screen A, Avatar IMAX: 120 of 120
Screen B, Black Adam: 120 of 120
Screen C, Avatar: 40 of 80
Screen D, The Fabelmans: 78 of 80
Screen E, Black Adam: 37 of 40
bins = np.linspace(min(s_rev.min(), o_rev.min()), max(s_rev.max(), o_rev.max()), 61)
mid = (bins[:-1] + bins[1:]) / 2
fig = go.Figure()
fig.add_trace(go.Bar(x=mid, y=np.histogram(s_rev, bins)[0], name='Plan of the manager', opacity=0.65, marker_color='#E84855'))
fig.add_trace(go.Bar(x=mid, y=np.histogram(o_rev, bins)[0], name='Our plan', opacity=0.65, marker_color='#3DDA84'))
fig.add_vline(x=6000, line_dash='dash', line_color='#2E86AB', line_width=2,
annotation_text='Target', annotation_position='top right')
fig.update_layout(barmode='overlay', bargap=0, title='Revenue in one night: the two plans',
xaxis_title='Revenue in one night ($)', yaxis_title='Simulated nights', height=440,
legend=dict(yanchor='top', y=0.99, xanchor='left', x=0.01))
show(fig)| Plan | Expected revenue |
|---|---|
| Plan of the manager | $5,754.86 |
| Our plan | $5,973.75 |
| Difference | +$218.88 |
The submitted 2022 run was not seeded. It gave $5,756.26 and $5,974.61. This page uses a fixed seed, so its values differ by about one dollar.
The chart shows that our plan passes the $6,000 target on more nights. The size of that change depends on how the demand for the three films is drawn: one draw for each film (as here and in the submission) or one shared draw for the night. The revenue gain is the same in the two models, so it is the result on this page.