Ticket resale alerts with a classification tree

A group assignment. We trained a classification tree that gives each baseball game a resale alert level: low, medium or high.

Role
Group member (coursework)
Period
2022
Stack
Python, scikit-learn, Decision trees
Evidence
Published
Question
Can a tree model put each home game in the correct resale alert level?
Data
Atlanta Braves home games, 2011 to 2016. The alert level comes from the ratio of resale tickets to tickets sold: low to 0.075, medium to 0.10, high above 0.10.
Method
We trained on the 2011 to 2015 seasons and tested on the 56 games of the 2016 season. We tested tree depths from 1 to 14 and compared the accuracy on the two sets.
ResultPublished
A depth of 6 gave 82.1% accuracy on the test season and 94.8% on the training seasons. A tree with no depth limit gave 87.5% on the test season. A model that always gives the most frequent level is correct for 58.9% of the test games.

Coursework output from a group assignment. The inputs include the two ticket counts that define the alert level, so the tree learns a known rule. The result shows the method. It is not a forecast of resale activity.

82.1%Test accuracy, depth 6
94.8%Training accuracy, depth 6
58.9%Baseline: most frequent level
56Test games (2016 season)

1 · Business context

A baseball team watches the resale market for its tickets. A high share of resale tickets can show high demand, a low price or resellers at work. The assignment gives each game an alert level: low, medium or high.

2 · Data preparation

  • Resale ratio = resale tickets ÷ tickets sold. The mean over the 461 games is 0.083.
  • Alert level: low to 0.075, medium to 0.10, high above 0.10.
  • Games in each level: 209 low, 161 medium and 91 high.
  • Training set: 405 games of the 2011 to 2015 seasons. Test set: 56 games of the 2016 season.
  • Inputs: attendance, tickets sold, resale tickets, and the payroll, rank and win percentage of the opponent.

3 · Tree with no depth limit

A tree with no depth limit is correct for 100% of the training games and 87.5% of the test games. The gap of 12.5 points shows that the tree learned the training games fully.

4 · Depth tuning

We fitted one tree for each depth from 1 to 14 and read the two accuracy values side by side. We chose a depth of 6: 94.8% on the training seasons and 82.1% on the test season.

The tree with no limit has a higher test accuracy than the depth-6 tree. The depth-6 tree is smaller and easier to read, but it is not more accurate on this test season.

5 · The resale market: for and against

  • For: fans can buy and sell tickets easily, and can change games or sections.
  • For: a lower resale price can bring more guests, who also buy food and merchandise.
  • Against: resellers make a profit on the product of the team.
  • Against: the team has less control of prices, and season tickets can lose value.
  • Against: the market is difficult to monitor.

6 · The limit of this result

The alert level is calculated from resale tickets and tickets sold. The two counts are also inputs to the tree. Thus the tree can find the rule again from its inputs, and a high accuracy is the expected result.

A useful alert model would predict the level before the resale count is known, from the opponent, the date and the early ticket sales.

Accuracy of the alert level

Accuracy of the alert level
ItemValue (%)
Depth 6, training seasons94.8
Depth 6, test season82.1
No depth limit, training seasons100
No depth limit, test season87.5
Most frequent level, test season58.9
Alert level and depth tuning (from the assignment notebook)
baseball['resale_ratio'] = baseball['tickets_resale'] / baseball['tickets_sold']

def alert_level(resale_ratio):
    if resale_ratio <= 0.075:
        return 'low'
    elif resale_ratio <= 0.1:
        return 'medium'
    return 'high'

baseball['alert'] = baseball['resale_ratio'].apply(alert_level)
train_bb = baseball.query('2011 <= season <= 2015')
test_bb = baseball.query('season == 2016')

for depth in range(1, 15):
    clf = DecisionTreeClassifier(criterion='entropy', random_state=0, max_depth=depth).fit(x_train, y_train)
    print(depth, clf.score(x_train, y_train), clf.score(x_test, y_test))