The problem with MAT
The 2023 Summer MAT happened last week. Our contestant count, as does the count for every other contest (including the AMCs) is dropping, though it was still decently high. That is mildly worrying, but it is not the problem we discuss today.
Background
The MAT’s tiebreaker format is as following: there is one tiebreaker set, and ties are broken first on points then on time. Another way to think about each tiebreaker problem is worth of a point.
To understand the problems with tiebreaker formats, I have to first present the Fundamental Theorem of Tiebreakers:
If two people have identical performances, then there is no way to rank them differently.
It is not necessarily true that it is always possible for two people to have identical performances; it depends how you measure it. For an in person contest, you might use “submission order” as a tiebreaking method. (Indeed, we use “time” as a proxy for this.) However, for our purposes,
It is always possible for two people to have an identical performance.
In fact, for MAT it is quite feasible. For two people to score the same on the main sets and the tiebreakers, and for them both to forget to submit until they’ve used all 30 minutes, is not impossible. Though we avoided it the last two years, this year the jig is up: we had two ties (3rd and 7th).
“Submit earlier, stupid”
This was our initial reaction. If only someone decided “hey I’m not going to get the problem in the last 10 seconds, let’s submit at 29:50”. But unfortunately that is not how real life works; people are not always optimal strategists. Even more so in a contest like MAT.
The MAT was designed not to require much strategy. That is why we have split the contest into several sets. You do not need to wonder, “hey should I check questions 1-5 or take a stab at question 9 which I left for later or reach for questions 11-15”? All you can do is solve what is in front of you.
Contests like HMMT Guts require a lot of strategy, and that can be fun. But focusing on solving problems is much better when it comes to developing logical skills in the long run, and personally I think it is more enjoyable. We’re not the only ones who think this way: top contestants have directly contacted us to express the same sentiment.
So tiebreakers requiring a strategic decision (time or problems?) is already a jarring break from our philosophy. Realize it or not, but the first three sets lower peoples’ guard. It gets them thinking, “thank god for a contest I can just relax in”. Which is a great mentality to have, but the downside is it increases the risk of ties after tiebreakers.
However, I think a reminder on the contest page during the tiebreaker set (“Remember that ties are broken on time if scores are identical.”) might help with that, since people have told me they forgot tiebreakers was even time based.
Estimathon
When originally designing the MAT, I considered using an estimathon (or even just one estimation question) as a tiebreaking method. In theory, if you make the upper bound for the answer around (“how many primes are less than 10000?”), then it will be very unlikely people guess the same thing. Add multiple such problems and add contrived formulas for the estimation score and bam. No more ties.
There’s a couple of problems with that. One: the more complicated your scoring formula, the more complicated the optimal strategy. Which means that contestants will either have to perform suboptimally, or shift their focus away from solving problems. Two: people gravitate towards certain numbers. Whether they are seen as funny, or look nice (multiples of 5, multiples of large powers of 10, etc). So someone is more likely to guess 2500 than 2583. Although with multiple problems, the second point is more moot.
Furthermore, estimation problems tend to have less clean solutions, if any at all. The skills we are trying to reinforce do not include “throw out wild guesses at random questions and guess less badly than everyone else”. It would be such a stark departure from our contest format too, and it would drastically reduce the problem quality (even if it is quarantined in tiebreakers). An estimathon would be jarring and feel out of place. So we bite the bullet and accept that our tiebreaker format might lead to ties.
Breaking less ties
I floated an idea around recently, though it is one we are unlikely to adopt: who cares about ties? After biting the bullet once, it seems much more acceptable to think that way now. Instead of making a compromise on our philosophy with the tiebreaks format — ties being broken by time — why not just ignore time?
Because it could lead to massive ties, too large to be acceptable. Last year (not this year), we had a four-way tie for 1st if one only considers main set scores and tiebreaker score (i.e. if one ignores time). And looking back, it was a disaster narowly averted. If people decided to submit a couple minutes later, we may have had ties with a tiebreak time of 30:00.
But think of it another way: four people all score 8 points on the main sets and ace the tiebreakers. The only difference is, their submission times are 25, 26, 27, and 28 minutes. Are their performances really qualitatively different? No. We only acknowledge this difference because they all came in with the shared expectation, “time matters”. Yet that mostly awards strategic skills, not mathematical ones — and strategic skills based on extrospection, rather than introspection. The role of a math contest is to develop problem-solving introspection. So a contest format with an optimal strategy based on “what would other people likely do here”, even if very tangentially, is not desirable.
Yet it too is a bullet we must bite for prize reasons. Many of our prizes are discrete items that cannot just be “split” between massive ties. And on some level, it is good to have a winner — the adrenaline of winning a math contest is part of the fun. So we will likely continue using time as a tiebreaker method. It is imperfect. Indeed, it is the worst tiebreaker method, besides all the others that have been tried.